Technical Tests: You’re Doing it Wrong, Part 2

Alternatives and suggestions for candidates and companies to avoid tests

In part one we covered types of technical tests—their relative costs, and why organizations need to understand the costs to the candidates if they want to attract the right type of candidates, and at what point in the process to test.

We haven’t covered alternatives yet. Larger organizations have the benefit of HR or recruitment divisions to bear the brunt of the early cost of the recruitment process—they can call candidates individually to check out interpersonal skills and to make the candidates feel wanted. Smaller organizations don’t necessarily have this, but they do have the benefit of being more flexible. If the brunt of the recruitment process falls on developers (or tech leads, CTOs, or other people in the technical organization) then obviously these organizations are trying to keep the time costs down—every hour invested in recruitment is an hour not spent on coding the company’s money-making product. But these techies are also in a much better position to be able to judge a candidate, and don’t always need to rely on one channel (the technical test, for example) to perform this judgment. They have other alternatives.

For example, if a candidate has a GitHub account on their CV, check it out. Sometimes there’s enough on there that you get a feel for their coding style without having to set them a contrived test: do they write unit tests? Do they write comments? How do they structure their code? Are their method/variable names descriptive? What sort of problems do they solve in their projects? Are they contributing to other open source projects, or starting their own? There are no right or wrong answers to these; the candidate needs to be judged against the standards and styles of the project or team they’ll be on.

If a candidate has written the book on, e.g., Java 8, and it’s there in Amazon with their name on, you probably don’t need to ask them what a lambda is. If they’ve given a deep technical presentation on a subject and it’s on YouTube, you probably don’t need to probe too deeply on that subject. And setting a technical test asking them to navigate an imaginary robot around an imaginary room is probably not going to demonstrate their ability to do the job any better than their book or blog or GitHub account can. An organization can save a lot of time, and be more inclusive of different candidates, by being a bit more flexible around how they judge tech ability.

If you’re desperate to have a technical test, then take a look at the matrix in part one and see if there’s something that will give you the value you want with less of a time investment from your people or from the candidates.

Finally, the TL;DR.

Technical tests during the interview process serve a specific purpose: to determine if the candidate actually has the coding ability they claim via their CV. It’s very easy to learn the words to say, to read a book and sound good in an interview. It’s supposedly more difficult to fake good coding style (although we did not cover the topic of possibility of cheating in this post, since it was quite long enough as it was).

As a recruiting organization, it’s tempting to try to minimize the costs to your company by choosing technical tests that require less of a time investment from your people and to perform the tests as early in the process as possible. Pushing the cost of proving technical ability onto the candidate, however, comes with its own dangers: those who have already proven themselves through other means don’t feel compelled to prove themselves to you, and unless you are an extremely desirable company to work for (a topic for another blog post), your best candidates won’t even make it into the pipeline, removing themselves from your process.

As a candidate, the amount of time you need to invest in getting the right job is too high if you have to spend hours on different types of tests for every organization you apply for before you even get a feel for who they are, what they want, and if you’d like to work there. Finding a new job is a full-time job in itself, and it’s sometimes easier to stick with the devil you know, or take the job with the lowest-cost process, without knowing if you’ve found the best position you could have.

But in both cases you have options.

As a candidate you can:

  • Have a GitHub profile. I’m not talking about starting something that’s The Most Amazing, Most Unique Project Ever. And I’m not talking about committing code to the core of OpenJDK. Simply having a profile with a couple of pet projects, or having a couple of one-line pull requests accepted on an open source project (I’ve got one for you if you like), is miles ahead of 90% of the other developers going for the same jobs as you.
  • Start a blog. I know, you think you’ve got nothing to talk about. But even talking about the stuff you’ve had trouble solving is a start, and there’s always someone out there that’s had the same problem as you. I started mine off nothing more than things I’d learned, and it’s changed the course of my career.
  • Spend 10 minutes a day / an hour a week on StackOverflow. Whether you ask questions, answer questions, or correct formatting so things are easier to read, there are ways to pick up points. Don’t worry about looking stupid (said the girl who’s never asked a question); even making constructive comments asking for clarification on what a question means adds to the overall value of the community.
  • Get involved in your local User Group. Many of the places I’ve worked have hired people their developers have met at User Group events. Imagine if you can use your sparkling personality to get a job interview instead of having to do a 4 hour coding exercise.
  • You don’t have to go the whole way and speak at international conferences to be recognized. You don’t even need to present lightning talks at your local user group (although that’s a really great start). You can do webcasts/YouTube videos/Slideshares to present technical content.

Any one of these things might get you out of having to do technical tests. If you’re a graduate with very little work experience (or someone coming into programming from another type of role), it’s even more important to pick one or more of these to create your portfolio.

As a company you can:

  • Get feedback. You might not realize you’re losing candidates. If you’re working with a recruitment agent or internal HR, try and find out if and why candidates have dropped out of your process. Also, give feedback. People hate spending hours on a technical test only to be told “no.” They want to know what they can do to improve themselves. If you give feedback, you’re giving back to the industry by teaching as well as simply taking the good candidates.
  • Be realistic. If the project that you’re recruiting for is a maintenance nightmare, then testing candidates on their ability to critique and fix spaghetti code is probably going to give a better indication of their ability to do the job than whiteboarding a design for an imaginary online sports application. Sure, it’s not as sexy, but you’ll get candidates who are better at that job, and they don’t get a nasty surprise when they join.
  • Be flexible. If someone has thousands of lines of code on GitHub, do you really want to set them a technical test as well? If there is an area you think they haven’t demonstrated, then test them just on that bit; don’t force them to do all of your technical test when you only needed to check 10% of the output. If someone has proved they can explain something key to your business in a blog post, don’t make them repeat the same material again. Save everyone’s time.
  • Get your developers to go to user groups. The people there (speakers and attendees) are using their own time to better their careers, which is a good indication of their desire to learn and improve.
  • Better yet, get your developers to present at user groups and conferences. This way, some of the best developers will come to you. Or at the very least, people will have heard of you, so when they’re faced with your 12 hour technical test, they actually bother to do it.

Basically respect the candidates and treat them like people. Your interview process is also an advert for your company, and you want people to go away and say positive things about you.

By the way, MongoDB is hiring Java developers right now. So if you want to find out what our interview process is like, then please do apply. And say Trisha sent you.

Related

Sign up for the O'Reilly Programming Newsletter to get weekly insight from industry insiders.