This post has been half-finished in a buffer for about a month, and I’m tired of seeing it, so here goes.
Dilettantes, Philistines, Autodidacts, and Renaissance Men
Growth in volume and ease of access to information makes it easier than ever to be a polymath. Becoming knowledgeable about many topics is possible without a lot of effort. We’re living at a pretty exciting in the free world, in that regard.
Of course, as much as there’s change on this front, there’s the same old problems. Becoming exceptional requires exceptional effort. Becoming exceptionally good at something requires exceptional effort. Focus is the key to developing this sort of expertise, as any other approach has scaling problems or involves electroshock.
I want to analogize a bit, because that’s a comfortable crutch, and then I’m going to try to make some sort of point.
I like listening to music. A moderate investment in audio equipment substantially increases my experience. I perceive a pretty dramatic increase in the quality of the listening experience. Like so many things, there’s a diminishing marginal return, but it’s easy for most to observe, if not appreciate, the difference at the entry level.
One of the first lessons I learned when it came to audio equipment is in the value of discrete components. I bought an inexpensive receiver as a teenager. Mated to some speakers I borrowed from my dad, it sounded pretty okay. I picked up a nice (for the time) CD player, which turned out to sound an awful lot better than the portable unit I’d hooked to the receiver prior to that point. Switching the receiver out for a well-made pre-amplifier and amplifier produced further improvements that were not subtle. There was more musicality, more control, more bass, pretty much more everything.
The funny thing was that these new components were not much different in price than the things they replaced. Rather, they did fewer and fewer things each. It was clear that for the money, the components that did one thing well tended to shine.
There’s something a bit obvious about this: it’s not surprising that a $200 amplifier sounds better than a $200 device that provides pre-amplification, amplification, radio tuning, phono pre-amplification, and video switching. It gets a little more interesting, however. A $600 combination tuner receiver often doesn’t sound as good as a $200 tuner plus $200 pre-amplifier plus $200 amplifier.
There’s a hidden tax, of course. Discrete components require more cables, more space, and more research, consideration and evaluation to get things right. Beware, this post can devolve into infinite regress.
I’m not really trying to rant about the banalities of high-end audio. Rather, my point is one of focus. Discrete audio components that focus on doing one thing well sound better. Working in concert, several components that focus produce a system that sounds better. This is the case because the individual components don’t have to compromise.
The Liquor Store
Here’s my premise: I’m calling bullshit on a liquor store that “specializes in fine wine, revivalist cocktails, craft beer, and associated miscellany.”
To specialize in something requires, well, concentration and expertise. Unless a store has dedicated staff for each of these pursuits, the likelihood that they’re specializing in all of them is slim. It’s not that it’s impossible that a fanatic store owner or curator could exist, but I’m not buying it.
Let’s pick on wine, in specific, because just specializing in wine is a bit of a lark. Do they specialize in wine under $25? High-end wine? Wine from uncommon varietals, locations, or methods?
I would want to shop at a wine shop that specializes in high-end wine if I’m shopping for high-end wine. They’ll know my purchases, they know about a much larger universe of high-end wine than I ever will, and on and on. Seems like a dumb place to pick up beer, though, unless it’s out of convenience. If I want somebody mediocre at high-end wine that’s also mediocre at half a dozen other things, I have no reason to visit them versus some anonymous website that sells alcoholic beverages on the Internet.
So, when I walk into a shop that specializes in inexpensive “value” wines and they offer to procure expensive wines if I’m interested, I don’t really see the point.
So, we get the big box stores that sell everything. One of these rolled in just down the street from me. Their model was to call up all the distributors in the state and say “give me a case of everything you have.” When I asked about a particular craft-method grape distillate they went ahead and ordered a case “just because they didn’t have it yet.” The mission of this store is to literally stock everything. Their specialty is nothing but minimum price margins and having the largest variety out there.
Now, they can’t really stock everything, because a lot of the more interesting parcels are limited production or obscure. But, they stock a whole bunch of stuff, and some of it is clearly interesting. It would take a lot of study on my part to know if any of it is worth my time. I have to develop the expertise and focus if I want to use a store like this. That’s not really my cup of tea for fine wine, but it’s not a bad way to go if I need a bottle of vodka to pour into a melon.
So, I don’t mind these sorts of stores. They are mediocre at everything, but they fill in the gaps that aren’t otherwise covered, and they provide cheap access to stuff for which the market doesn’t need focus. Anybody paying more that the state minimum price on a case of Budweiser is an idiot. I guess that’s where I’m the opposite of Matt Kramer. Buy online or in anonymous big box stores for the cheap stuff that everybody has. Buy in local focused shops for the interesting stuff. I don’t mind if these stores exist, and I don’t mind if they disappear.
The funny thing is that I also don’t mind when these big box stores push little stores out of business. That little store clearly lacked focus and specialization enough to command a market and survive. I guess that’s the capitalist in me. The stores that distinguish themselves with expertise and focus will survive and do well.
The Software Engineer
A decent software engineer these days has access to a billion different tools to solve a billion billion different problems. Given any sort of problem, there’s a bunch of tools out there that can collectively solve it. Even better, there’s a bunch of specialized glues that magically make all of those tools talk to each other. This isn’t your Brooksian silver bullet. This is an entire damn armory made of silver. Think Oak Ridge’s 13,300 tonnes of silver!
Too many analogies and I digress. At the end of the day, an engineer is given a problem. The engineer pulls a bunch of tools and abstraction leak-proof epoxy out of their toolbox, and out pops a finished solution. Maybe it’s not perfect, but perfect is the enemy of good and all. Well, what about just barely good enough? Ship early and often, or something like that, the kinks can be ironed out later. The problem, as I see it, is that the kinks are intrinsic. The way to fix a system built via bolt-ons is by adding more bolt-ons. Server crashes? Add a server monitoring and crashing bolt-on. Communication channels prone to failure? Add a multiplexer bolt-on and more communication channels. Success!
Something doesn’t smell right.
But wait, this is just the punchline to the classic hacker joke, right? While everybody else is arguing about what programming language to use, they just pull a few tools together via a pipe and the problem is solved. Maybe I’m missing something. I’m not sure what’s different, but I’m pretty sure something is. Maybe it’s that the tools that are being reached for are not generic, simple, time-tested, and robust.
I see more of these systems being built, and they end up being unmaintainable, poorly thought out, and complicated. Often the original problem was relatively simple and easily specified, but the solution ends up involving seventeen libraries, four services, eight programming languages, fourteen servers, and two operating systems. “It’s really simple,” they argue. That’s crazy! There’s nothing simple about it. There’s no reason for it, and yet it’s seen as the simplest possible solution because the only conception this engineer has is of solving problems with generic problem solving building blocks.
The point I’m doing a poor job of getting to is that the engineer that’s a generalist builds these sorts of systems. They know a bit about everything, and have the ability to build systems from the silver armory in rapid fashion. The problem is these systems suck, and they suck because the engineer isn’t really great at anything except for plugging stuff together and shipping it. They lack focus and specialization.
Is this advocating for depth above breadth and extremism in software engineering? Maybe. Again with the infinite regress, Voltaire!
I don’t have the original material and have failed to track it down, but I was told of the results of some organizational behavior studies recently. These studies aimed to analyze the characteristics of exceptional leaders in organizations. It’s no surprise that most leaders are at least moderately effective at everything they do. What was interesting to me, however, is that the best leaders tend to be exceptionally talented in one or two specific areas. The rest of the pack fail to specialize and differentiate from the pack of “generally good at most things.” This is somewhat unintuitive, but it agrees with my premise.
It’s clearly not pragmatic to have an engineer that specializes in every thing that needs to be done. With that said, engineers that have focused their development and skills in certain directions tend to find simple solutions to complicated problems. The end result is often nothing like what the person doling out problems imagined, but that’s perfectly okay. I think this is one of the key challenges of engineering management, in fact: It’s important for this to be perfectly okay. What’s important is delivering an elegant, supportable, and reliable solution. The odds of this seem far higher when something is crafted by a specialist rather than assembled by a generalist.
I suppose there’s the argument that somebody could specialize in being an exceptionally good generalist. They could be really good at picking only what’s necessary to get the job done and making as robust an outcome as possible. I guess these sorts of people exist, but I think most specialists already know when to reach for something off the shelf to fill a need in their solution. I think this is the key difference: They’re grabbing a gemstone to complete a ring, and they accept that they’re going to have to cut facets on it and adjust the mounting to make it all work.
I have this other belief that people who really know their stuff get pretty passionate about it. I want passionate arguments from people that know their thing inside and out, and I want the bigger systems built by these specialists working together. Compare this with big systems being built by generalists who lack focus: All the cooperation and argument is done at the interconnection layer, rather than the architectural layer. These systems are doomed.
As the last little hand-waving proof of my premise, here’s a quick mental exercise. When stuff gets sticky, what sort of engineer comes to mind? I’m willing to bet it’s the person that really has focused in on something and has differentiated themselves from their peers greatly. On paper, when something isn’t right, getting somebody that knows a bit of everything sounds like a good idea. In reality, though, it’s often that person who is pretty good at everything but has insanely deep knowledge in a couple of areas that figures out the key insights. What’s weird is they figure things out really indirectly, but they still figure it out.
It’s not enough just to be good, it’s crucial to have the focus to develop expertise and specialization in a way that’s singular, uncommon, and special. Engineers that really do this are way more interesting than people that can just do everything well enough.
Updated 2012-07-08: Fixed some English issues. Emphasized specialization more, focus per se less. My clarified assumption is that focus develops specialization, but a lot of what I’ve written above talks more about the symptoms of focus, not the process or state of it. Reworked most of the second half of the software engineer section to align it better with what I’m really trying to get at: This isn’t meant to be a rant against Lego block programming, and it was devolving into that. It’s meant to be a rant about the importance and value of singular excellence achieved through focus. I think this result is closer, but it’s still not quite right.