Testing Methods to Evaluate Candidates

Algorithmic methods to optimise your recruitment efforts

We deep dive into the benefits of incorporating algorithmic testing into your recruitment practices.

Discover how you can increase the accuracy of your recruitment decisions by up to 40% using psychometric assessments which measure cognitive ability, integrity, values and counterproductive behaviours.

Also learn about work sample testing which is a great predictor of a candidate’s future performance.

Watch the next video in this series here:

Part 9 – Evaluating Candidates Using High Performance Modelling

And watch the previous video here:

Part 7 – How to Conduct a Reference Check

And if you know of anyone who would benefit from this video, please share it with them.

Video Transcript

Hi, Andrew from SACS and welcome to video number eight on our sequence about candidate attraction and candidate evaluation.

In the earlier videos, one, two, and three, we talked about candidate attraction methods.

And so far in candidate evaluation methods we’ve talked about ranking the accuracy of recruitment techniques, biodata scoring, which is a method of scoring the applications you receive.

We talked about behavioural interviewing and a thing called behavioural reference checking.

This one is all about testing.

Earlier in the sequence I talked about algorithmic decision making. And research into decision making shows that human beings are sadly prey to a range of biases, blind spots, priming.

Priming being where a previous experience primes you to see candidates in a certain way.

Humans are poor decision makers

We are poor decision makers, we human beings, and it’s sometimes said that the aircraft of the future will be flown by a human being and a dog.

And the job of the dog will be to bite the human being if that human being touches anything.

It’s also said that in particularly places like Europe and the United States of America there are techniques now for algorithmic diagnosis of illness.

In other words, you go to a doctor and you tell the doctor your experiences.

Well, if that doctor has seen similar cases that had a certain diagnosis, they are apt to presume that this illness that you have will be that diagnosis.

There’s been research into radiology, people interpreting radiology.

In fact, Daniel Kahneman in his book “Thinking Fast and Slow” gives the example of where radiologists at a conference were asked to look at certain shots of, I don’t know, let’s say a bone or something like that, and conclude what’s right or wrong.

And they came up with very little convergence as to what was right or wrong with those particular radiological photographs.

So, look, human beings are not great at making decisions in an unstructured fashion.

The moment we bring an algorithm, and testing is an algorithm, we get way more accurate. And this has been demonstrated in a range of different ways.

Nature vs nurture

So that’s what this video is about.

The characteristics that cause a person to be able to do a job, to be able to achieve the outcomes that a job requires, and so the quality and quantity of work that they produce, and also the quality and quantity of the relationships that they have at work, is determined by things that could be generally categorised into nature or nurture.

And so nature, you’ll see on this side (refer to video), we’ve got things like cognitive ability, verbal reasoning, which is about words, numerical reasoning, which is about numbers, and abstract reasoning, which is all about the sequence of ideas.

So abstract reasoning is the idea that I can see this is likely to cause this is likely to cause this.

Or another abstract reasoning capability is, all right, well, if I do this, or if this happens, there are four potential consequences and I think this one is the most likely.

That’s classic abstract reasoning capability.

So that appears to be extremely heavily genetically determined because identical twins which are separated at birth and never meet each other have higher correlations in their cognitive ability scores than non-identical twins which grow up in the same family, hence the conclusion that it is heavily genetically determined.

And estimates can be anywhere from between 60 and 80%.

I’m probably more at the 70 or 80% mark myself based on recent research, but we know that that’s strongly genetically determined.

Then there’s integrity. And integrity is really the probability of doing good or bad things. And certainly that appears to be something like 50% genetically determined, which is why you’ll get what we call a black sheep in the middle of a very morally upright family and vice versa.

Why you get very honest people that are born to a family that might be involved in criminal activities.

So there is a genetic component to that and recent research from researchers like Adrian Raine put it somewhere around 50, 60%.0

And then there’s a thing called personality. And personality is kind of like the way you’re hardwired to respond to things, to like certain things, to find certain things easy, to find other things difficult.

And we’ll talk you through some of the components of this.

And again, the estimates are as low as 50% for that, or as high as about 70% for that. And once more, I’m probably at the 70% mark.

Recent research over the past five years at the University of Edinburgh, amongst other places, suggest that the big chunks of personality seem to be quite heavily genetically determined.

Then there are things that we acquire in the course of life.

So people’s skills, people’s experience, and people’s attributes, including their values.

Now, you can test values psychometrically like you can test personality, integrity, and intelligence, but values can change in the course of a person’s life and do change in the course of a person’s life.

We’ll mention later on Shalom Schwartz, who’s one of the most important researchers in the world of values.

And he has demonstrated in his research that, for instance, a value which is to do with caution, conservatism in the sense of taking risks, wanting everybody to be protected, that seems to grow significantly as you age and seems to correlate about 0.3 with your age in years.

So your values can, will, and do change.

Whereas, let’s say if you’re an introvert at age three, chances are you’re an introvert at age 63.

Cognitive ability testing

So let’s talk about cognitive ability testing.

Now cognitive ability testing of itself appears to give you about 25% accuracy in predicting the performance of people at work. 25%.

I mean, that is remarkably high.

The absolute best interviews ever constructed get to 25% and most interviews are somewhere around 10, 15, 20%. So that’s really accurate.

In fact it’s, as a single entity, it’s one of the most accurate predictors of work performance that we’ve ever seen.

And as I mentioned in an earlier video, it doesn’t seem to matter that much as to whether the job is an intellectual kind of a job or a more basic job.

Smarter people tend to be better employees.

Why? Well, cognitive ability gives you four things.

First is speed. People who are smarter do things more quickly.

Secondly it gives you power, not political power, but problem solving power. I can crack a problem better if I am smarter.

Thirdly, trainability. Trainability is all about learning new stuff. People with higher cognitive ability tend to learn things more quickly.

And we do know that there’s a strong memory component in cognitive ability.

Smarter people are literally able to simply hold more chunks of information in the forefront of their mind at any one time.

So you can imagine if you’re trying to solve a problem and you can hold two chunks of information in your mind, whereas another person can hold seven chunks of information in their mind, that simply gives them a statistical advantage about the various solutions that they can draw on to solve a problem.

So cognitive ability determines, or has a big impact, on trainability.

And then CWB stands for counterproductive work behaviours.

Counterproductive work behaviours occur more commonly amongst people with low cognitive ability.

Violence occurs more commonly amongst people with low cognitive ability.

Theft occurs more commonly amongst people with low cognitive ability.

Why? It appears that people with higher cognitive ability have better capacity to self regulate.

In other words, they see the implications of what they’re about to do and they can cause themselves not to do this by virtue of saying, “Okay, well, that’s not a good decision.”

So cognitive ability correlates with counterproductive work behaviours.

Now, so does dollars.

So if you have a lowly paid workforce, very often you get more counterproductive work behaviours, things like bullying, harassment, theft, those kinds of things, than in a highly paid workforce, because correlation between cognitive ability and dollars is quite strong. And you can find it’s out by simply Googling it.

Google correlation between cognitive ability and income and you’ll see that there’s pretty much a line as income increases cognitive ability increases.

So that’s why lowly paid workforces can be more of a challenge from a bad behaviour point of view.

So let’s talk about the three forms of cognitive ability that are most important.

Verbal, numerical and abstract reasoning

First is verbal reasoning. Well, most jobs in the world require communication.

So verbal reasoning is all about words and that’s why it’s important.

Second is numerical reasoning.

And some jobs require numerical reasoning, the ability to use numbers, but not all.

And so for certain jobs it will be very important, for other jobs you may not even test it because if the job doesn’t have a numerical component, or if you anticipate that it will never have a numerical component, then you may not use numerical assessment.

And the third is a thing called abstract reasoning.

So we call this the capacity to solve abstract problems.

But as I mentioned earlier, it’s really the ability to see sequences, probabilities, what will happen.

Now, you know that there are some people who seem to be able to anticipate what’s likely to go wrong or go right and other people who seem to be constantly surprised, “Well, who would’ve thought that that happened?” This may well be abstract reasoning, but certainly abstract reasoning, verbal reasoning, numerical reasoning, if you average those three scores to get what’s called a general cognitive ability measure, that’s a really good predictor of work performance.

HEXACO model

And then we move on to personality.

Everybody that you’ve ever met has had a personality and it is uniquely their own. It has many components.

So the most contemporary model of personality is a thing called the HEXACO model of personality is taking over from a thing called the Five-Factor Model of Personality, largely because it explains more about the way people behave, either at work or at home.

HEXACO was developed by Kibeom Lee and Mike Ashton from Canada.

And we at SACS developed an instrument called the SACS 6, which is actually a HEXACO, but the difference is that every question is work worded.

In other words, there are not questions in the way that the Lee and Ashton HEXACO is worded.

Often the questions are worded about general life, like I like to go to parties, or I like to socialise, or I’m a shy person, or whatever.

In the SACS 6 all of the questions are about work. They’re targeted specifically to work.

Integrity and modesty

Integrity & modesty.

Now, you’ll see that that gets an asterisk (refer to video).

And the reason for the asterisk is where there’s an asterisk in this case it means that this is important for any job and people who are high in integrity/modesty are better employees than people who are low in integrity/modesty.

Integrity/modesty is the characteristic of being truthful, straightforward, and not arrogant.

That’s what the modesty bit is.

So if a person is low in this, there is evidence from across the world that people who are like that are much more likely to steal, bully, sexually harass. It’s a correlate of negative characteristics. Also people who score low are much more likely to go to jail.

Now, this, one’s more, is quite heavily genetically determined and so it’s a good idea to recruit people who are intrinsically and naturally honest, and truthful, and straightforward.

Emotionality

Secondly, emotionality, and that has an asterisk (refer to video).

Emotionality is the characteristic of being emotionally stable or emotionally unstable.

And people who are emotionally stable make good employees, but as well as that they experience personally much less stress.

Now, if you think of this, it’s really kind of the unkindness of nature, isn’t it? Some people are lucky enough to be born relatively relaxed, relatively calm, relatively peaceful in their internal world and other people tend naturally to be highly emotional, where relatively small things set them off and make them uncomfortable.

Well, clearly, that’s very bad fortune, because people who are born calm and relaxed are likely to have better health outcomes and just have a happier life by virtue of not having this characteristic.

But in any event, if the job is stressful, you’re not being kind to a person who’s high in emotionality by putting them into that job without lots of support.

Extraversion

Then you have extraversion.

And extraversion doesn’t have an asterisk because of course there are certain jobs where it’s good to be an extrovert and there are certain jobs that are good to be an introvert.

If I’m sitting in front of a computer for hours on end doing analytical work, well, frankly, that suits an introvert better than it suits an extrovert.

So that’s worth considering.

On the other hand, if the job is a sales rep, or a sales manager, or something like that, well, extroverts tend to more naturally do that kind of work.

Now, later on in the very next video we’ll be talking about a thing called high performance modelling, where you take people who do a job well and you analyse their capabilities and you analyse their psychometric characteristics to come up with a profile that you’ll recruit against in the future.

Now, that’s the optimum method of recruitment.

You may well find that there are jobs where, let’s say, you assume that it’s an extroversion related job and you find some really good introverts doing the job.

That’s why it’s best not to presume.

It’s better actually to collect the data and to find out what is the truth of each of these situations.

Absence of anger

Then we move on to a thing called absence of anger.

And you’ll realise that has an asterisk (refer to video).

Well, anger is not a good thing in a workplace, is it?

And we know that when people are angry they tend to bully people more, they tend to provide worse services if they’re in direct service type roles, so try to recruit people who are low in anger.

Conscientiousness

Then there’s a thing called conscientiousness.

And you’ll see that that’s also got an asterisk (refer to video), because people are on a continuum generally from well organised and structured to disorganised and not very keen on achieving objectives.

So, clearly, the best people to recruit are people who are conscientious, hardworking, organised, structured in the way they go about things.

Now, how do we know this? Because there’s been a stack of research evidence to this effect.

And going back to earlier videos, in the very first video in this section about candidate evaluation, we talked about the research that’s been undertaken that shows that these things are in fact important determinants of performance at work.

Openness to experience

Then you have a thing called openness, otherwise known as openness to experience.

And people who are open to experience are the sort of people who like new things.

They’re broad-minded and they welcome change.

Now, for the right job, that’s important, but you’ll notice that that doesn’t have an asterisk either because of the fact that there are certain jobs where it’s important to be open-minded to new things, other jobs where it doesn’t matter so much.

SACS 6

So these are the six factors of a HEXACO model as described by this instrument here (refer to video), the SACS 6.

And here’s the diagram that fleshes this out in more detail. Integrity/modesty, emotionality, extroversion, absence of anger, conscientiousness, and openness to experience.

And you’ll see that under each of these six factors there are four facets.

And these facets are simply smaller factors, sub factors, that add up to integrity/modesty, for instance. So genuineness, rule favouring, absence of greed, and absence of arrogance.

If we combine those four psychometrically and statistically, they contribute to a thing called integrity/modesty.

So that’s about personality, an enduring characteristic.

Fellow by the name of Heroclitus, going back about 6,000 years ago, said character is destiny.

I think personality is character and personality, therefore, in a very real sense, is destiny. If you’re lucky enough to be born optimistic, hardworking, honest and truthful, and easy to get along with, boy, you’re going to have an easier life than if you’re unlucky enough to not be born like that.

Counterproductive work behaviours

Counterproductive work behaviours.

Counterproductive work behaviours are measures which are to do with negative behaviours.

So lateness, unpunctuality, not attending work when not too sick to do so, inability to get on with others, being distracted, incivility, impoliteness to others, theft, ignoring broader work policies or practises, and incivility, ignoring or snubbing other employees.

So the big article in this area, or one of the big articles in this area, was this Gruys Sackett article (refer to video), which really elevated the concept of counterproductive work behaviours to something which was very popular, all very well known in the world of psychology.

So we’ve developed instruments to measure this in workforces across Australia.

And, basically, it’s a good idea to measure these things because the probability of somebody undertaking these negative things is predicted by counterproductive work behaviour assessments, which are also called integrity assessments.

You may remember in the first slide in this video we talked about integrity.

Integrity testing is not a term that I personally like because it seems that you’re commenting on someone’s soul, where really what you’re commenting on is negative behaviours and there are ways of assessing this using an integrity test.

Integrity testing, or counterproductive work behaviour testing, is increasingly used in Australia and New Zealand and it should be more so.

Measuring values

And then there’s the question of values.

Now, we all talk about values. And values are the things, in a sense, that you have scrolling past in your sleep screen of your computer, particularly there’s a science of values. And what scientists mean when they talk about values of human beings, is that they mean that values are goals.

Goals which largely have been learned based on the social environment that you grew up in.

So as we grow up, we take certain values on board. Interestingly, not so much from our immediate family group.

We’re more influenced in our values by friends and by other people, other adults, like teachers and those sorts of things.

And in a way, I guess every parent knows this, but kids have evolved to kind of be more sensitive to values outside of the home than in the home.

And the reason for that is that it’s just better from a survival point of view.

If you all, in a certain family, see a thing a certain way, the probability of that family surviving is actually maximised by people saying, “Okay, well, no, there’s a different way of looking at this.”

Values make us fit in. So if we share the values of a work group that we belong to, we will be liked more by the people around us and we will like them more.

So values is all about what you might call culture match, there’s an element of culture match, anyway.

Do I fit in? Do I value the same things that other people do?

Schwartz is a very important researcher in this area. Probably one of the most cited researchers in the world and a very distinguished academic.

SACS Work Value Scale

We worked with the Schwartz Portrait Values Questionnaire. And in partnership with Simon Albrecht from Deakin University we built a thing called the SACS Work Value Scale.

So similar to the HEXACO, we were using a similar model, but what we wanted to do is to make all of these questions work worded.

And we also changed the name of some of the factors that are in this model largely to make them easier to understand.

And so authority is about I want to be the boss. Ambition is that I want to get ahead in my career. Enjoyment is that I want to have fun at work. Variety means I believe it’s important to get variety in my work. Autonomy. I want to make my own decisions. Social justice. I care about equality for people. Environmental sustainability. We know what that means. Helping and supporting means I want to help and support people. Rule respecting is that I believe that it’s important to support the rules of not just the organisation but social groups. Traditional values. I have respect for what came before me. And safety. Well, that’s pretty straightforward. I want a safe workplace.

The interesting thing about values at an individual level is that values can correspond with personality or it can oppose personality.

And I’ll give you a really practical example.

Talking about this concept of helping and supporting. Helping and supporting is a value.

So if a person’s high in this, and you’ll see that this person is high in this, 83rd percentile (refer to video). So that blue line is population average.

So this person’s well above average in helping and supporting. That means that they believe that it’s important to help and support other people. They’ve learned that in the course of their life.

Anger makes people not so helpful and supportive to other people. And what you’ll tend to find in a person that has that kind of mix is that if they are in a relatively comfortable situation, they will tend to live out their values. Their beliefs will come up trumps.

On the other hand, if they’re under stress, what will happen is that the genes will assert themselves and their anger will come out.

Now, I’ve seen many practical examples of this. So if you’re interpreting psych tests, it’s worthwhile knowing what is genetic versus what is acquired because values are largely acquired. And when it comes down to what really will show itself out in a stress situation, it’s much more likely to be the personality.

Now, if you want to predict bad behaviours, counterproductive work behaviours, in our research we found that we could predict counterproductive work behaviours about 40% accurately in respect of a person’s personality, but only about 16 or 17% accurately by their values. And that shows that the genes will out, as geneticists often say.

Work sample testing

Now I want to talk to you about a thing called work sample testing.

Work sample testing is very predictive. 0.54 in its own right, which makes it just over 25% accurate. 0.54 is a correlation and that adds up to about 21 to 22% accuracy.

So work sample testing is not role playing.

It’s where you take a slice of the job, as accurate as possible, as representative as possible, and you get the candidate to do it as though they’re already employed.

So it’s not some theoretical exercise.

So let’s give you an example. We did a exercise with a company that was a retail organisation and they wanted people to be able to run stores. And one of the things that they wanted to do is that they wanted the person to look at the store and have a look at the financial results that the store was achieving in its various categories and to decide what to do next. Now, this is not an imaginary thing. This is what retail managers do all the time.

So in a recruitment situation we created a work sample assessment where they actually did that. And then they would write a brief report and then they would sit it down in front of the panel and say, “Well, I think we should do this. And I think we should do that.” This proves to be very accurate. 0.54 is a very, very accurate method.

Now, an elongated version of a work sample assessment is a thing like summer clerkships for law firms or accounting firms, where the person comes and works with the organisation. And you get them, actually, doing the job that they would do in their first year. Well, after three or four months of this, you pretty much know whether they are suited to the work or not.

So that’s a really accurate method of recruitment.

So I guess the challenge for you, if you’re recruiting in this way, is to find out if there is some part of the job that can be taken out, particularly if it’s a very important part of the job, bundled up and given to the candidates before you’ve actually hired them.

That is a really powerful recruitment method.

So it could be analysing a set of financials, could be asking them to write a report similar to what they would do on the job, solve a problem from the workplace, make a presentation either by themselves or in a group, or internships.

They’re all work sample testing methods. It’s not role playing. It’s not an in-basket exercise.

Fortunately in-basket exercises have waned in their popularity. You get a whole bunch of emails and you say, “Okay, well, I’ll do this one first and I’ll do this one second. And I would do this about this one and.”

Look, too theoretical.

As I said earlier in this video, the best predictor of the future is the past and questions about what you’ve actually done are far better than questions about what you might do.

That leads us on to the last video in this sequence.

So we’ve covered a whole bunch of stuff about candidate attraction and candidate evaluation.

The final one is a thing called high performance modelling.

Have you ever thought, “Gee, this would be a better company if we had more Mary Smiths”? Well, this is a way of identifying your best people, finding out what makes them tick, and then using that intellectual property to make better recruitment decisions in future.

Join me for the next video to find out how to do that.

Watch the next video in this series to find out more about Candidate attraction & Evaluation:

Part 9 – Evaluating Candidates Using High Performance Modelling

And watch the previous video here:

Part 7 – Part 7 – How to Conduct a Reference Check

And if you’d like some help evaluating your next hire, contact us about our Psychometric Assessment Tools.