The Value of Performance Tests in Recruiting

Evaluating potential new team members for skills and capabilities fit is always challenging due to inherent information disparities in the recruiting process. Performance tests can provide pivotal insight.

When recruiting new team members, you seek a match of both skills and personal characteristics for the open position. The cost of getting the match wrong is tremendous in terms of wasted time, energy, and productivity, so these are always high-stakes activities for a startup. 

As you seek to learn about different candidates, one of the challenges is that they are telling you about their experiences, and you rarely have a way to verify what you are told independently. The candidate always knows what they did and what the situation was, and they share with you the best possible face on the story of their experiences. This asymmetry of information is challenging for the hiring manager seeking to evaluate the skills and personality of the potential team member. It is difficult to verify what candidates say, and often it is hard to differentiate the relative abilities of different candidates who are claiming to meet the job requirements.

One of my favorite recruiting tools – the performance test — seeks to flip that dynamic by putting the hiring manager in a position to evaluate the skills and approach of the candidate directly. While certainly, I am not the only one to ask for performance tests, I have been applying this technique to every recruiting process I have initiated in the past thirty years. While it is only one component in evaluating a candidate, for every job opening, I insist that we put careful thought into designing a performance test that focuses on the most essential skills for success in the job. When done well, the performance test often reveals surprising strengths in some candidates and shocking weaknesses in others, thus providing a powerful differentiating tool in recruiting great team members.

To illustrate, here are examples of some of the performance tests I have used over the years and spanning different startups and what happened when we applied them.

• Productivity Consultant:  Early in my career, I worked for a management consulting firm and rose through the vice president level. Our boutique firm had tried using a recruiting firm to fill out our ranks – and the experiment was an abysmal failure, with over half of the new recruits failing on the job even though their resumes and interviews seemed to indicate they had the skills to do the job. Determined to reverse the pattern, our CEO commissioned a 5-Vice President recruiting committee, empowering those who owned delivering services to clients with absolute recruiting responsibility and a no-justification needed veto power. We wrestled with how to improve the process and decided we needed a tool to see how the candidates handled an actual consulting situation. I developed a case study based on the real work we did. We had the candidates spend about 30 minutes reading the case study (which we knew because it was based on reality) and then stand at a whiteboard with a VP demonstrating how they would tackle the case in real-time and on-stage (which mimicked our real-world working situation). The VP would assess how the candidate led the discussion, problem-solved on the fly, and handled objections and questions. What a differentiator! Some candidates just fell apart. Some excelled. It became the most differentiating interview in our whole process. Plenty of candidates seemed to be good fits until the case study interview. And using this new tool took our success rate up above 90%. 

• Lab Technician:  We had a role where we needed lab technicians to assist with building electromechanical prototypes. A critical part of the job was handling electronics boards and other mechanical tools effectively, so we instituted a series of skills demonstrations in front of the hiring mechanical engineer where the technician candidates would solder, saw, and assemble for about half an hour. The skill requirements were spelled out in the job description. However, many candidates talked a good game, but when it came to laying hands-on and knowing how to use a soldering iron, table saw, and other tools, they clearly did not have the skills they claimed. Others demonstrated excellence – and they were the ones we hired.

• Data Scientist:  Hiring a data scientist is fraught with buzzwords. After an initial screening interview, candidates who described the right skills, culture alignment, and aptitude were invited to do a “take-home” evaluation that tested their ability to manipulate and harmonize complex data from two different sources and use statistics and visualization to explore the relationship between continuous outputs and intermittent values. The candidate we ultimately hired was effective with her first submission and then persevered to untangle a particularly thorny bit prior to the performance test review interview, where she presented and defended her results to the rest of our data science team. Those who failed made mistakes, missed key points, and generally did not demonstrate the mastery of the tools and approaches we were looking for.

• Sales / Implementation Manager: While a track record of success in similar roles is essential for customer-facing roles like salespeople and implementation managers, the ability to extract what is important from a conversation with a third party and translate that into effective, move-the-ball-forward communication is essential. For these roles, at the end of the screening interview, we don’t even name our request as a “performance test” (even though that is what it is, and we have a plan to give it). We ask the candidate to follow up with us with an email outlining what they think are the most important keys to success in our job and how they think they fit that. We give no deadline. And then we wait. How do they address the question? How clearly do they articulate their case? How quickly do they get back to us? And then, when we don’t respond, how long do they wait to follow up, and how effectively do they do so? Not doing the test was an automatic fail. And, the content was revealing of how each candidate was likely to interact with our customers.
  
  
•  Mechanical Engineer:  We were hiring a mechanical engineer to do design work using Solidworks CAD software. We had narrowed the field to three candidates we thought would all be excellent fits. The hiring engineer was uncomfortable with the idea of a performance test, convinced that he knew who could do what based on the interviews. I insisted on a simple test of having each candidate design a simple device in Solidworks while we watched. One candidate (who we hired) was a pro, navigating the tool with confidence and effectiveness. One candidate clearly was unfamiliar with the tool’s interface and kept hunting around trying to figure out how to use it, despite having claimed Solidworks experience on his resume and in his interviews. The final candidate had used the tool before but struggled with the basics of mechanical design. That was an eye-opener for the hiring manager – and I never had to convince him again of the value of a performance test.

Hopefully, those examples illustrate the power of performance tests to help the hiring manager make good decisions. When designing a performance test, keep these guideposts in mind:

• Think hard about what the job is really all about. Is it a technical position requiring strong technical expertise? Is it a sales position requiring strong communication skills and effective followthrough? Is it a specialty position requiring particular expertise? Develop the performance test for the job based on the most essential elements of success in that specific role.
  
  
• Be reasonable in the time and effort required to complete the performance test. The test is meant to demonstrate the essential skills and should not require excessive time to complete or evaluate. You should know what a good answer looks like.
  
  
• Look for signs that the candidate went above and beyond. When you evaluate a performance test, you can often see signs of a candidate’s strengths in the way they approach the task. Sometimes extra credit is earned by taking the work just a step farther or doing it flawlessly. Think about what the approach taken reveals about the candidate’s comfort zones and development opportunities.
  
  
• Be consistent.Do not waive the performance test. I have been surprised and saved from a hiring mistake so many times by a failing performance test when everything else looked reasonable.