I run a program called “The science of recruitment”, where we take participants through evidence based approaches to candidate attraction and candidate selection. We look at an extremely wide range of recruitment techniques, reviewing the validity of each as identified from research from the world of organisational psychology.
I find this research very interesting. There are some findings which are well worth knowing:
- The results are consistent across cultures. Things which predict job success in New York are surprisingly similar to what predicts job success in Beijing.
- The results are consistent across job categories. We all know that a university lecturer is different from a sales rep, but if you add up the psychological characteristics which predict success in the two professions the similarities vastly outstrip the differences. It’s a bit like wine. Everyone can tell the difference between the sour Sauvignon Blanc your neighbour bought at the cellar door of a vineyard you never heard of before and a $3,000 bottle of Chateau Petrus, but the bit that makes the difference is only about 3-5% of the total volume of the wine. Most of the wine is identical – water and alcohol.
- There are massive differences between the best and worst of the commonly used recruitment methods in their ability to predict success and failure at work. I am going to write further on this in future, so watch this space.
At the risk of being labelled a serial debunker, I will share with you the results of research into the accuracy of ref checks. Over a range of studies reference checks show validities of around 0.26 in predicting success at work (Robertson and Smith, 2001). I will destatisticate that for you. A validity of 0.26 means that reference checks are accurate to the tune of just under 7%.
I am writing from Melbourne and soon we will be heading into the Spring Racing Carnival – a time when people shake off the gloom of winter, dress in cheerful clothes and lose their money.
Here is a racing tip. Do not bet on horses with a 7% probability of success.
I worked for many years in executive search, recruiting senior executives for client organisations. Many of those clients love reference checks, because they perceive them as “objective”. Reference checks can’t be objective, or they would have a better accuracy rate than 7%. So this raises the fascinating question of why they are so inaccurate. The most common fault of ref checks is that they are overly positive. I think this is because:
- No matter how bad a job someone did the referee really doesn’t want to hurt that person’s career.
- Maybe they give them a good reference in order to get them out of their company.
- The candidate is “proximal” (close to the referee), you are “distal” (removed from the referee). As I have mentioned in other blog posts, proximal works, distal doesn’t. Even if the referee is not overly enamoured of their previous colleague, you are not even on their radar screen. Social psychologists certainly have no difficulty in explaining what is going on here. We tend to support people we are close to.
On the other hand I have also experienced unfairly negative reference checks, where the referee said scandalous things about the candidate, with the candidate being blissfully unaware of the whole sad saga.
Here are a couple of tips:
- Never accept the candidate’s reference list. You tell them who you want to reference check them with – their last three leaders is a good place to start. Ask them for permission to call and for contact details. If they don’t want you to contact one or some of these then that is important information in its own right.
- Ask the referees the same behavioural interview questions you asked the candidate. For instance, if the job has a client satisfaction responsibility, asking the referee, “Can you give an example of where X made clients happy?” is a powerful question and tends to draw deeper answers than the usual, “What was he like? Would you rehire him?” questions.
Even so, you should rely on more valid methods such as psych testing above and beyond the answers you will get from ref checks. Every so often I have stumbled across a gem of information in ref checks which I would not have found using any other method, so I believe you should ref check with every hire, just see them for the somewhat flawed source of information they demonstrably are.
Robertson, I. T. and Smith, M. (2001), Personnel selection. Journal of Occupational and Organizational Psychology, 74: 441–472.
