Aaron N. Tubbs bio photo

Aaron N. Tubbs

Dragon chaser.

Twitter Facebook Google+ LinkedIn Github

I’ve been meditating a lot lately on what it is that makes a good technical interview. I’ve been trying to enumerate the differences between good technical interviews and bad ones. I wrote up a bunch of notes. I generated a lot of assumptions. I found myself entangled in internal inconsistency. Here’s the universal truth:

Good technical interviews are a product of good technical interviewers.

That’s it. I can’t specify how to create a good technical interview in more detail. There’s no formula for good technical interviewing. There are not a lot of hard rights and wrongs. Interviewing is an art. It is not science. It is not a process. There are no instructions to follow. There’s no flowchart.

I have a few anecdotes to share; these represent atypical interviews for me, but then the only thing typical about my interviews is that each is quite different.

Anecdote One

Here’s how the best technical interview I ever gave went down. Details changed slightly to protect the indecent.

Me: “So I had some questions, but first off, this project on your resume sounds really neat. Tell me a bit about it.”

Candidate: “Oh, I’m sort of embarrassed about that; I was writing some code to do network communication between two agents in the system, and it ended up being a really crappy and slow implementation. But it satisfied the needs they had.”

Me: “Tell me more!”

Candidate: “I’ll tell you some more!”

Me: “Well, hell, let’s make this faster.”

The candidate and I spent the next 40 minutes rewriting his project on the board. We figured out better data structures and algorithms. We wrote better code. We found bottlenecks in the old implementation. It was Candidate and me on the white board, having a conversation, doing what the best engineers do.

It was awesome. I didn’t ask any other questions, really, it was just 40 minutes of engineering. The candidate drove the problem as much as I did. I usually have a large body of prepared material to ask a candidate (several hours worth of questions). I pick stuff from it to match the situation of the interview thus far. I make sure I cover certain things. I didn’t get to any of it. We covered a lot of core topics that I usually want to hit (data structures, algorithms, complexity, code writing, and a bunch of other stuff), but it wasn’t at all “according to plan.”

My strongest recommendation for the guy; everybody else liked him too. The guy got an offer, he didn’t accept. Bummer. The guy was an engineer. He could do shit.

Here’s the funny thing. I barely ever ask people about the projects on their resume. I usually would ask a question like “tell me a story about the nastiest bug you ever had to track down” or “talk to me about the most challenging project you ever worked on” if anything. This time, however, I asked the candidate about a specific project. I have no idea why I opened with that, and can’t explain how it evolved into what it did.

If I tried to do the same thing 99 more times I would put money down that the same thing wouldn’t happen. I just knew I should ask the guy about that project based on the first few seconds of talking to him.

Anecdote Two

I had a candidate do very well at my technical interview. It was a certain type of technical interview that gives some very detailed analysis of a very narrow part of an engineer’s job. So I was happy. In the wrap-up, somebody doing a more general technical interview was pretty cold on the guy. The interviewer didn’t get the answer they expected to a question they’d posed, and got something completely different.

The candidate’s solution combined with the interviewer’s inability to comprehend genius (and pragmatism) when he saw it quickly thrust me into a “hire this guy or I quit” state. The candidate accepted the offer, so this one has a happy ending. The candidate’s “completely wrong” answers told me everything I needed to know about both the candidate and the interviewer. I didn’t really care whether we hired or not based on my interview, because my interview was a sanity check at best, but somebody else’s bad interview turned me into a strong advocate. Nothing according to plan, but it worked out.

Anecdote Three

I’ve had great success opening with asking somebody what their top programming language is. I then ask them to tell me where they are on a scale of 1 to 10, with 10 being as good as it gets.

You would not believe how many people give themselves a “9 or 10.”

I’ve never talked to a 9 or 10. At this point or later in the conversation usually it’s easy enough to sanity check their self-rating. I’d estimate that most of the people I’ve talked to that were strong in C++ self-graded themselves in the 4-6 range!

Anyhow, the next step is finding out what the candidate’s second-strongest language is.

Most often, I end up with a mix of something like C or C++ and Java or C# as the two languages. Sometimes I get slightly more interesting aspects like C as their strongest and C++ as their second-strongest.

Every rare once in a while I get something weird like Haskell as #1 and x86 assembly as #2. But let’s ignore those situations from now, beyond hinting that it’s exceptionally useful to understand at least 1% of everything: You should know at least enough to ask things like “give me a haiku that describes a Monad” or “tell me what your favorite undocumented x86 instruction is!”

So, off the cuff again, but about 80% of the time I get some sort of static typed language with manual memory management and a stack versus something garbage collected that may or may not be interpreted in one way or another and may or may not be duck typed.

There are a few attack vectors here, but the point is to get the candidate to compare and contrast their languages in which they are most competent. There are a lot of neat things that happen here. We get to play with a lot of neat things like typing, garbage collection, programming styles, frameworks, APIs, trojan horses, time to market, interpreters, bytecode, compilers, optimizers, linkers, performance considerations, parallelism, abstractions, and god knows what else.

Let’s get to the specific example.

I had an interview once where this was the only question I asked! The candidate had Java as their strongest language, and C++ as a distant second. They talked a good high-level talk, and I was pretty positive on the candidate based on our initial discussions. Good conversationalist, engaged, saying the right things. The candidate tossed out some statements comparing the two languages, and citing garbage collection as a chief difference between the two. So, I went with a softball to ease into some puzzles that I expected would be easy. We never got to the puzzles.

“Describe how a garbage collector works.” That was it; interview over. The basic description was there, so I asked him how he might implement a simplistic garbage collector for a language. We kept getting nowhere.

At this point I could do two things: I could accept that the candidate just doesn’t know how to answer this stuff and move on, or I could keep digging deeper and try to arrive at some universal truth. I realized there was a vehicle here to explore what I needed to know, so I went with the latter approach. The problem kept getting more and more simplified. At some point we reached:

“Give me any sort of data structure that allows me to insert, look up, and remove integers quickly.”

Still nothing. We’re not really talking garbage collection anymore, I’d just be happy to talk about the trade-offs of storing integers in an array at this point.

A few minutes later it was:

“All I want is a data structure that can store more than one integer at a time.”

This was another interview that, like the first anecdote, had “one question.” It took a meandering path in much the same way, and resulted in as strongly negative a response as the first case was strongly positive. Nothing went according to plan. It was terrible in terms of conforming to any particular specifics.

So What

The preceding anecdotes all describe atypical interviews that resulted in very strong reactions from me, in both directions. These are radical departures; they are nothing at all like a typical technical interview, positive or negative. That said, these were some of the easiest decisions with regards to recruiting that I’ve ever made, and I am confident that the result was correct.

I have a lot of interviews that go according to plan, more or less. I have positive and negative reactions to candidates under these conditions. Even within the scope of a planned discussion, the variety in how an interview plays out is pretty crazy.

How I Interview

This post is incoherent, but bear with me. I want to talk a bit about how I interview. The only consistent thing is this: My goal is to push somebody until they run out of knowledge, and then see what they come up with.

Upon reaching the point of uncertainty, good candidates should be able to use the tools they have to get further. They should have intuition about a problem. They should show aptitude to take what they’ve learned previously in a discussion and apply it to a current problem. They should be creative. They should be confident, but at the same time they should know when to admit that they “don’t know.”

Bad candidates make shit up. They lie. They toss out buzzwords and can’t substantiate them. They refuse to dive deeper. They don’t understand abstractions. They can’t think “outside the box.” They can’t find new ways to use the tools they have, and they can’t come to understand new tools and paradigm shifts on the fly.

Here’s a fun exercise for the interviewer: Next time you’re talking to somebody, ask them to explain something that you don’t understand. Keep asking them more detailed, specific, and difficult questions. It doesn’t matter if you know the answer; keep pushing well beyond where you know the answer. Some really neat things happen here, even though you have no fucking clue what you’re asking. Trust me. Just try it once.