Why you should stop making developer candidates do tests

Dan Draper
Expert360 Engineering
5 min readJan 7, 2019

--

If you’re a developer with a job, you’ve probably done a coding test but the take home test isn’t as valuable as you think.

I remember interviewing with a company in the US (I live in Sydney, Australia) a number of years ago who gave me a coding challenge. As I stumbled through it at some ungodly hour, 5 (yes, FIVE!) other engineers watched me type every line of code as I screencast my work on Google Hangouts. Even for an experienced engineer like myself, this was nerve racking and I absolutely did not perform at my best.

The issue with this sort of approach is quite simply that it’s unrealistic. If your engineers are as nervous as I was during a test like that on a daily basis you have bigger problems. When validating candidates you need to challenge them without having them turn into useless stress balls like I was during that early morning coding test.

Another common approach — especially to reduce the pressure on the engineer — is to give them a take home test. The issue here is that you only get to see the result and not the process the candidate took to tackle the test. Not to mention the fact that take-home tests tend to be more involved and many developers will baulk at the idea of spending many unpaid hours to do them.

Several years after that not-so-impressive coding test I was once again in a position where I was hiring other engineers myself. I decided I wanted to take a vastly different approach than the one that had been taken with me. One that would give the interviewer a sense of the candidate’s ability, problem solving and communication style without putting them under duress.

On a trip to San Francisco back in 2011, I was introduced to Rob Mee, the CEO at Pivotal Labs (now Pivotal). At the time I was researching how great companies in the bay area went about hiring engineers and Rob was kind enough to meet with me for almost an hour.

When I asked Rob how he validated candidates at Pivotal Labs he told me he personally paired with every single candidate. I was taken aback! Rob was (and still is) the CEO. Firstly, I thought it was unusual for the CEO to be technical but secondly, the coding “test” was actually a pairing session.

Noticing my surprise, Rob suggested we pair on something right then and there! “Do you know Java?” he asked. Having made the switch to Ruby full-time a couple of years earlier I said “Yes, but I’m a little rusty”.

“That’s OK, I’ll drive.” Rob said nonchalantly.

And so began my pairing session with Rob Mee, the CEO of Pivotal. The whole thing was quite a blur really and I doubt I impressed him with my coding skills. But what I was there to do was learn how great hiring managers hire great engineers — and I came away from the session with an entirely new perspective.

The Pairing Interview

Over the years I’ve incorporated the pairing interview style into my own hiring process albeit with a few tweaks. Significantly, I don’t tend to pair with candidates myself (except for tech leads or architects). This isn’t for any reason other than to reduce the feeling of nervousness for prospective team members. I imagine a junior engineer being asked to pair with the VP Engineering might feel a little apprehensive. Besides, it’s not a daily activity — pairing with a colleague or their tech lead is. So that’s who does the pairing session.

Most members of my team have been coached on running a pairing interview. This way as candidates come through the process we can share the load and candidates get to meet a variety of people.

At Expert360, our pairing interviews consist of solving two problems — one simple and one more involved. The interviewer has solved them both several times over so doesn’t have the cognitive burden of thinking through the solution. Instead she focusses her effort on guiding and assessing the candidate.

Juggle Pairing

I recommend using an approach I call “Juggle Pairing”, a concept inspired by Uncle Bob’s Transformation Priority Premise.

When you conduct a pairing interview start the session by writing a single unit test (say in Ruby or JavaScript). This is just one assertion; nothing complex at all. Then pass the keyboard to the candidate and ask her to make the test pass. Once she’s done so she passes it back to you so that you can write the next test.

In these first few juggles it’s important to start very simple (even for experienced developers). It’s like stretching before going for a run. Just a warm up to get the coding synapses firing. Your job is to make the candidate feel at ease.

As the interview progresses you can start to move faster. Inevitably there will be moments where the candidate must make an insight or “cognitive leap” in order to solve the problem. It’s these moments you must pay attention to. How does she handle the challenge? Does she ask questions or talk through her thinking? Does she come up with a good solution? If not, does she recognise a poor solution and change tack?

After doing a few of these interviews you’ll soon realise that they are not really about the solution. Instead they give you insight into how the developer goes about solving a problem and how they behave during the process.

When I do a pairing interview I look for signs of ego (I actively hire for low ego and have passed on some great engineers because of it). Is the candidate willing to change her mind or let go of an idea when it doesn’t work?

I also look for communication style: can the candidate explain to me the problem they are solving? Does she get frustrated with me if she struggles with the problem?

And finally, I look at how the candidate codes. Does she take a methodical approach or is the style more sporadic? Of course everyone’s style is different but you can tell a lot about an engineer by working through a problem together.

Having used the pairing interview in the hiring process for several years now (in my current, and previous role) I reflect on how effective it has been for my teams. At Expert360, we have a highly skilled, collaborative team of low ego developers who all communicate exceptionally well. While many factors contribute to the selection of a great candidate, I believe that the pairing interview is at the heart of it.

If you give pairing interviews a try let me know how it works for you!

:wq

--

--

VPE/CTO, Nerd, Coder and Producer of the forthcoming film, Debugging Diversity.