Shipping Culture Is Hurting Us

Preface:

I started writing this post with the intention of discussing Gary Bernhardt’s conclusions in his talk, A Whole New World. Gary makes what I think are some really good points about infrastructure, tooling, and the paralysis we seem to sometimes have around it. I wanted to expand on that a bit, while also tying in my frustrations with what I personally see as techs with much more sizzle than steak, such as Node.js and MongoDB.

As feedback continues to roll in, I begin to think that perhaps I stretched this thesis too thin. The point I was trying to get across is that I feel like people choose technologies that make initial strides simple, but fail to address real challenges that I think other tools handle better. I certainly could have argued this better, and many of you have made very valid criticisms of what I have said below. My crystal ball tells me that next week’s post may be a follow-up saying as much.

At any rate, many thanks to everyone who offered feedback. It has been quite the discussion. Now, back to our regular scheduled programming.

We have a shipping culture. For the last several years, we have been told that the best way to put out good software is to iterate quickly. Get something, anything, out to your customers or users, because the best way to build software they want is to get it into their hands and let them tell you everything they like and everything they hate about it. Jeff Atwood echoes this sentiment with the Better Off Dead quote,

Go that way, really fast. If something gets in your way… turn.

Seth Godin says,

Ship often. Ship lousy stuff, but ship. Ship constantly.

You’ll find people everywhere saying the same thing.

Half-assing it

This isn’t bad advice. Quickly getting something in front of the people that will actually use it is a great idea. It means you waste less time building something they don’t actually want. But I look around the industry today and I get worried. Don’t get me wrong – I see brilliant people shipping brilliant, innovative software. But I also see a lot of us using half-baked technologies to shove half-assed software out the door.

I see an insane amount of the industry doing very serious business with a language designed in ten days for the purpose of “mak[ing] the monkey dance when you moused over it” in a 1995 web browser.

Since it was made for such simple tasks, the language lacks basic features one might expect from any programming language, like integers or a sane type system. Instead we get some bizarro-world where you can call functions with the wrong arity, where NaN !== NaN,¹ and a chart like this exists for something as simple as comparing two values.

I see an insane amount of people using a database that has become infamous for unreliability. Even if those concerns have been addressed, it’s hilariously insecure by default and the recent announcement of the newest version gives no indication that its creators care at all about ACID transactions, a traditionally desired property of any database.

I am not the first to offer such complaints. Many developers (or at least many developers on Reddit) love to hate JavaScript and MongoDB. But I’m not just here to rant. I have a bigger point to make.

Why does anyone who writes code professionally use these technologies instead of laughing them out of the room? You can’t just categorically dismiss every single developer who employs them as an idiot. That’s too easy — to do so would be a cop-out. They aren’t all idiots, and some software built with these technologies is half decent. Why then, are Node.js and MongoDB popular? It is (at least in part) because they appeal to the shipping culture. Both systems give you a lot out of the box, or make it easy to learn the basics. They are easy to set up and start using for your next project. “Who cares if people give them shit on the internet?”, you say. “These tools are working for me right now. They get the job done, and fast. Ship it.”

Anti-intellectual bandwagons

The problem is that these technologies, being so beginner-friendly and aggressively marketed, rapidly pick up steam and become the “cool” things to use, regardless of actual merit or lack thereof. Nothing illustrated this more to me than when I went to my first (and only) hackathon last year. I went with the assumption that I would see a wild variety of projects using a wild variety of technologies. Instead, I found the vast majority of contestants there writing some web app, usually with Node.js and MongoDB. It certainly didn’t help that MongoDB people were there, at the hackathon, marketing their wares.

Computer science and software engineering are ludicrously young fields when compared to most other sciences and schools of engineering. Engineers have been using an understanding of physics to build bridges for centuries, if not millennia, but we have only been doing things with computers in the last handful of decades. In this short time, however, we have made fantastic strides. People like Turing and Dijkstra and Knuth and countless others have made wonderful discoveries and invented brilliant algorithms. But so many of these hyped technologies seem to reinvent the wheel, to shun what precious knowledge we have gained so far. You can offer excuses of industry professionals having different goals and constraints than those in the ivory tower, but I think it goes deeper than that.

Systems stopped using cooperative multitasking at least 20 years ago because it sucked compared to the alternative of automatic, preemptive multitasking. And yet Node.js harks back to those dark days with its callback-based concurrency, all running in a single thread.

Smart minds suggested ACID transactions might be a good idea all the way back in the late 1970s. Database schemas were developed as a feature, not a liability, because organizing massive amounts of data is a very complicated task.² But this doesn’t seem to bother users of MongoDB one bit.

Your father’s technology, today!

But even if you disregard everything I’ve said above, there is another large concern that is a consequence of our shipping culture: infrastructure paralysis.

If you have never had the privilege of watching it, I warmly recommend you take 20 minutes to enjoy A Whole New World by Gary Bernhardt. He’s the same gentleman who gave the famed (and hilarious) wat talk.

Gary discusses the oddities in our choices of tooling. It is 2015, and yet our terminals use escape codes designed for a machine released in 1978.

Actual 24-bit color is a rarity, and displaying images is just out of the question. This system is woefully outdated, and it isn’t the only example of ancient, creaky tech we use on a daily basis. Why are we still using these antiques? Most developers don’t even think of this as a problem. Many would see any effort to improve or replace them as unneeded or even masturbatory. There’s no technological reason to not have better infrastructure, but we just keep pushing on with what we have. We’re too busy shipping.

Gary quips,

[Shipping] is a great way to achieve short-term business gains and a great way to discover what your customer wants. But when you’re a programmer, and you are the customer, and you’re writing the system, and you’ve been using these tools for 20 years, you don’t need customer discovery. You need to go off and sit in a hammock for a couple of years and think hard.

I don’t know what the solutions to these challenges are. I don’t have a good answer. And shipping often isn’t a bad thing. But we need to not get so wrapped up in shipping that we stop caring about what we have learned so far and how we could make a better future.

Shipping culture is hurting us. Or at the very least, it is stopping us short of a better tomorrow.

Sometimes a schema-less setup may be the better choice, but there are NoSQL solutions that provide consistency much better than MongoDB.