Code Critique as an Interview Tool
One of a software engineer’s most critical skills is the ability to read code. In my experience developers spend far more time and effort reading code than writing it. Others agree (Joel On Software, Jeff Atwood). It stands to reason that this is a skill that should be looked for during the interview process. I’ve found that code critiques can be a valuable tool in this regard.
They answer questions such as: Can this person see bugs? Can they comprehend an unfamiliar piece of code? Are they opinionated about what good code should look like?
For this purpose I came up with a stack-like class that fits on a single sheet of paper. This class has a variety of flaws ranging from the painfully obvious to the subtle. The candidate takes a few minutes to look over the sheet and then we spend 10-15 minutes talking about code. The issues include:
- Poor method naming (”doIt“)
- Poor time complexity (O(n) when O(1) would do).
- Duplicated code
- Handling of error cases (does a pop on an empty stack corrupt the datastructure?)
- Memory leaks
- (Java-specific) Misimplemented Object#equals method
- Input validation (does pushing a null onto the stack cause later failures in the contains method?)
- Inconsistent and/or non-existent comments
What I like about this process is that it give a very fine-grained scale to understand how someone thinks. If they don’t mention something you can start a conversation on that topic. “So what do you think the time complexity of pushing N items would be? Could it be better?” “What happens if multiple threads use the class at once?”
So far I’ve tried this on a small sample of candidates and it has been a reliable discriminator. One person failed miserably, one person more or less aced it, and a few more landed somewhere in the middle. With more time and experience I hope to refine the process further. I would love to hear from others who have thoughts or experience with this technique.
July 16th, 2007 at 2:57 am
We currently use this technique to screen all our development hires. This has been an invaluable tool in sorting out the guy rating himself 9/10, but just read the J2SE 1.5 certification book, and the guy who rates himself 7/10 but has indepth knowledge of production quality code.
Our test has about 30 issues on it. Its reasonably difficult, so if someone gets 8-10 of them; we’re happy. For a senior developer or architect; they should be getting between 10 and 20 issues. If I learn something from them doing the review; they’re definitely hired!
We’ve found code reviews don’t work on grad’s. They don’t have the real world experience to see code in that light.
If you’re new to this, start by taking some code that you’ve found in your codebase that you’re not happy with. It has to be genuinely a diversion from best practice (not a personal style difference). Find a couple of those, refine them to make them generic/abstract/non-proprietary, and make a couple up. We found http://www.javapractices.com/index.cjp to be a good source of potential inclusions (or a good place to cross-check).
Best of luck in your hiring!
July 19th, 2007 at 5:18 pm
I’ve used code reviews as an interview tool and I’m a fan of them. They definitely give you a lot of information relative to the amount of time that they take compared to other technical tasks/questions I’ve tried. Deliberately doctoring some of your code is quite fun too. I did once have one good candidate ask very worried if it was real code from our codebase though!
July 30th, 2007 at 7:31 am
When I interviewed for my current job my boss gave me a question like this. He gave the source to some home brewed string replace function. The function was omitted and the code was atrocious. All he was looking for was if the candidate could read the code and discern what the function should be named.
Ever since that I’ve used that same method on candidates but I would often ask them to critique the code as well. Works great as it is, but it really shines when you ask them to take you through the code and explain what’s happening. This is where you really get to see how they think and communicate.
July 30th, 2007 at 1:37 pm
What I did before, which I hope to install at my current workplace, is to do what you have described (not as many items to check off though, not that it is not a good idea), then give the candidate a new requirement for the existing code to implement. I will pair with him (we required TDD and pairing) and look at his coding behavior.
August 28th, 2007 at 4:59 pm
[…] rococo - then rubble » Blog Archive » Code Critique as an Interview Tool i like this approach over asking someone to write a simplistic application from scratch (tags: programming interview) […]