Recruitment for most employers is straightforward: you advertise, read through applications, invite the people you like in for an interview, think about it for a bit and then enter into negotiations with whoever you most want to employ. In education it’s different. Schools are weird. When I was first told how school recruitment works on my PGCE I couldn’t believe it, “They do what?”
For any non teachers, school recruitment works like this:
- All candidates for the job are invited in to the school on the same day.
- Candidates have to plan a lesson for a class they know almost nothing about beyond year group and ability.
- The interview day typically consists of a tour of the school, being observed teaching a lesson, being interviewed by students, with a final formal interview.
- The last interview question is always, “If you were offered the job would you accept it?”
- The interviewing panel discuss who they want to appoint and, later the same day, contact the successful candidate who has to make an immediate decision.
I’m not sure when this became standard practice, but precious few schools in the UK deviate from this approach. As such, it’s worth asking whether this is a system supported by research as the best way to recruit staff. As you might expect, it turns out that some of what schools do is for reasons of misguided expediency, some habit and, some based on evidence of what’s effective practice. First, I’ll briefly review the evidence that’s out there and then I’ll make a few suggestions for improving what we currently do.
Firstly, standardised application forms are, in the main a good idea, although there are a few pitfalls to be aware of. Ideally, you should have a way of comparing candidates’ qualification and employment history without being influenced by their personal details. I’m sure there are snazzy hi-tech ways to accomplish this but simply having the first page contain contact information so that it can be easily removed is a simple low-tech solution. Some sort of comparative judgement model with multiple evaluators will help to reduce unwanted biases. (Bohnet et al 2015)
Exam grades provide useful proxies both for intelligence and for motivation. If a candidate has a hatful of A grades this lets you know that as well as being clever, they probably work hard. These are both traits you should select for. There’s decades of research which suggests that IQ is important predictive power in determining whether a prospective employee is likely to work out. A high IQ not only gives you good information on a candidate’s ability to do a challenging job, it also predicts how conscientious they’re likely to be.
According to Kuncel & Hezlett, there are pretty strong correlations with performing in jobs designated as ‘high complexity’ and ‘medium complexity’ and intelligence even makes a difference when the job is relatively straightforward. IQ even shows correlations with leadership and creativity indicating that, on average, cleverer people will have more ideas and be better at getting others to implement them.
What’s probably pointless as far as application forms go is the personal statement. All this tells you is whether someone (not necessarily the applicant) was able to guess right as to what you wanted to hear. If you’re genuinely interested in appointing the best candidate you would probably be best to conduct an IQ test.
So, although application forms are, for the most part, worthwhile, there’s no evidence that unstructured interviews lead to better decisions and fairly weak evidence that even structured interviews will help us overcome our biases unless we’re very careful about the structures we put in place. This paper does a good job of showing that interviewers make better decision by just looking at applications rather than having their opinions influenced by interviews. The problem is that interviews generate too much information for us to properly evaluate. Because of this we are forced to rely on tacit decision making in which we often ignore important information and are influenced by irrelevancies such as how similar the candidate is to us. This is called sense making – the process whereby we give meaning to our experiences. If we hear irrelevancies or inconsistencies, we’re wired to try to make sense out of them and we will often arrive at a mental accommodation which feels satisfying even though, objectively, it shouldn’t be. This also leads to interviewers firmly believing in their powers of intuition and decision making acuity even when this is disputed by empirical evidence.
Despite these findings, interviews remain wildly popular – probably because they’re both convenient and a good way of assessing pertinent skills such as clear explanation and careful listening. Knowing that interviews should be as structured as possible is probably good news for most schools as, in my experience, interviewing panels go to some trouble to make sure each candidate is asked the same question in the same order. There is though more we can do than just this. Opinion is divided on the subject of what’s the best way to phrases questions – there’s evidence in support of the ‘tell us about a time when you…’ model as well as the ‘what would you do if…’ model – whatever you like most is probably good enough. The top tips for structured interviews are to commit to a series of questions you will ask each candidate, and make sure all questions are directly relevant to the job. Daniel Kahneman offers some useful suggestions in Thinking, Fast and Slow:
If you are serious about hiring the best possible person for the job, this is what you should do. First, select a few traits that are prerequisites for success in this position (technical proficiency, engaging personality, reliability, and so on). Don’t over do it – six dimensions is a good number. The traits you choose should be as independent as possible from each other, and you should feel that you can assess them reliably by asking a few factual questions. Next, make a list of the those questions for each trait and think about how you will score it, say on a 1 – 5 scale. You should have an idea of what you will call ‘very weak’ or ‘very strong’. (p. 232)
He offers one final piece of advice: “Firmly resolve that you will hire the candidate whose final score is the highest, even if there is another one whom you like better – try to resist your wish … to change the rankings.”
Once the interviews have been conducted, it’s important to get some fresh eyes involved. Ideally, a second set of independent evaluators would compare the candidates’ responses and offer an unbiased suggestion on which were best. Not only will anyone who wasn’t actually in the interview see things we might have missed, they’ll also avoid being distracted by irrelevancies.
So much for general advice, what about for schools in particular? The first thing we need to think carefully about is the interview lesson. Although we know that there is no reliable way to grade lessons, we still want to have some idea of how a prospective teacher is likely to perform in the classroom. The worst way to use a sample lesson in an interview is to send candidates a loose brief and expect them to guess what type of teaching you like. This not only creates undue stress and hours of pointless work, it also results in a performance which is unlikely to tell you much you couldn’t have found out in other ways. My advice is to send candidates a lesson plan you would like them to teach. Not only does this cut down on their preparation, they’re no longer having to guess about what you want to see. The benefit for the employer is that you get to see whether the candidate can adapt to fit your expectations. I think it’s helpful to debrief the lesson at some point, but the emphasis on this should not be to offer feedback – any such feedback in a highly artificial setting will serve no practical purpose – instead it should focus on what the candidate might have done differently and how they think the lesson might be improved.
I began writing this post with the idea that the interview day might not be a good idea, but, having reviewed the evidence (especially this document) I’ve concluded that having all the applicants in on the same day is probably the best way to reduce bias in appointing the best person. What I definitely would change is forcing everyone to make a snap decision. Instead, I would suggest announcing in advance that a decision will be made in, say, a week’s time. This will give time for both parties to really think things over and be sure that the right person gets the right job. I realise there are pressures which might prevent us from doing this – there’s often a fear that if we don’t snap someone up they may be lured away by another school – but we should, where possible seek to reduce the pressure on an immediate decision.
I’d also get rid of the walk of shame; the midday cut where the field is culled and those candidates we’ve taken against are sent packing. This practice is not just humiliating, it’s a poor way to combat bias and appoint the best person. Everyone who you decided was worth meeting on the basis of their application form should receive the same treatment and all go through the same process, most especially the structured interview.
Here then are my top tips:
- Anonymise application forms and have at least two assessors comparing applications to arrive at a rank order of best to worst applicants.
- Give all the applicants at the top of your rank order an IQ test. (This is probably impractical because of time and expenses but it is the Rolls Royce solution to recruitment.)
- Give prospective teachers a lesson plan you want them to teach and allow them an opportunity to reflect on how it went.
- Design an interview format around no more than six qualities or attributes and come up with a short list of questions for each attribute. Then score each interview on a scale of 1-5 for each of the metrics you’ve come up with.
- Allow time to make the best decision. Ideally you want to score the application form, IQ test, lesson observation and the interview questions and this shouldn’t be rushed.
In Part 2 I give particular attention to the formal interview and provide a template for effective interviewing, then, in Part 3, I discuss how we can improve interview lessons.
A lot of ground covered here in a thought provoking way.
Here are my takes on the suggestions:
1. Standardised application forms (which seems really to be about removing personal details); we understood the reasoning and tried this for a while, but a lot of the missing detail was relatively easy to guess
2. David knows I am not taken with IQ tests despite ‘decades of research’; we use an external personality testing service which has served us well.
3. Personal statements would address the elements in a detailed job description and paragraphed under the headings provided
4. Interviews (yes, everyone has the same questions and agreed, structured to elict attributes which can be gained no other way); we are a learning institution, so let’s find out about attitudes to learning; so we might ask this: what the best piece of advice you’ve been given and why?
5. A lesson plan: yes, a sound idea.
6. No ‘walk of shame’; everyone stays.
7. Time to decide: yes, again; always allow at least an overnight.
You’re not taken with IQ test but are happy to pay for personality testing? Forgive me but that’s like saying you don’t rate conventional medicine but are happy to pay for kinesiology.
This is a good primer on the importance of IQ to schools: http://www.aft.org/sites/default/files/periodicals/Nisbett.pdf
We wrote (briefly) about the problems with personality testing in What every teacher needs to know about psychology, but the best source is Annie Murphy Paul’s book https://www.amazon.co.uk/Cult-Personality-Testing-Miseducate-Misunderstand/dp/0743280725/ref=tmm_pap_swatch_0?_encoding=UTF8&qid=1494350424&sr=1-4
What IQ test would you suggest?
Either https the Stanford-Binet or the Wechsler
I think my last comments got lost. I would be very wary of using IQ assessments. They are sometimes used predictors of how a child manages school academically but for adults they are are even more decontextualised and you already know how they manage academia by exam results and achievements.
Many academics with high IQs would find it difficult to teach a child. The skills of teaching go far beyond imparting knowledge of a subject. Classroom management, relationship building and nurturing children along with motivating children to learn in a meaningful way can’t be detected by an IQ assessment. IQ assessments are only a snap shot of a persons actual general ability and the adult ones are a very narrow snap shot of particular skills which I do think have fairly poor prediction of how good a teacher will be at actual teaching.
1. No one’s suggesting teachers be given a job purely on the outcome of an IQ test.
2. IQ tests are not ‘snapshots’ they’ re very sophisticated battery of tests which have excellent predictive power for job performance, especially when the job is ‘high complexity’.
We have to agree to disagree on point 2. They are what a person can do on a given day at a given time for those particular activities within the sub tests within the assessments. They are used for predicting how kids might manage schools but I would question the use of them for job performance. You have to consider what extra information they can provide and are they normed for the population you are using them for? The Stanford Binet online seems to be American ? But possibly have uk norms. I don’t know it but I do know WISC and other weschler assessments.
Your line “They are what a person can do on a given day at a given time for those particular activities within the sub tests within the assessments”
applies to every method of judgment taken on one day, including interviews and sample lessons. They are not expressly designed to be used in predicting how kids might manage in school though they are often used for that in the same ways as predicting job performance because they have a fairly strong correlation. The normi issue is correct but correctly used tests already account for this limitation. MENSA uses two to account for some of these issues.
Personally the more pressing issue is how to get everyone to do a IQ test without causing score inflation, fraud etc: Though as a key indicator it is vastly more accurate then anything else.
But why would you need an IQ test when you have a candidates exam results in front of you? what added info would the IQ test provide? That. A candidate is good at solving a range of non verbal tasks and answer some rather old fashioned questions about vocabulary and possibly general knowledge depending on what test is used. The online Stanfor Binet one from Mensa has a very narrow range of activities. I would much rather use the information from exam results and any university degrees or post grads.
But a correlation of 0.5 can only be described as moderately strong at best. And at this level of correlation, only 25% of the variation in the dependant variable (job performance) can be attributed to the variation in the independent variable (IQ test). My gut feeling is that this doesn’t quite equate to “excellent predictive power”. A great set of articles, though.
The point is, statistically an IQ test is a much better predictor of success than an interview.
Here’s a tip for free (credit to an old colleague for this).
To ensure only lucky candidates are interviewed randomly throw away half the applications.
This is really thought provoking and useful David. I’d be really explicit about what you wanted the students to study and learn, and understand, rather than give a lesson plan just to see how they think (many candidates don’t think about learning much at all).
I’ve noticed in leadership interviews that SLTs look for reasons to like a candidate on the first day, cut the field for the second day, on which they then look for reasons to dislike the remaining candidates. The most frequent question seems to be: how will they fit in? Have you come across any research as to whether this is in any way relevant?
Yes, I’ve linked to some of it in the post. Consensus seems to be that ‘fitting in’ is a bad hiring strategy
In the States (at least Ohio), we follow a similar model of interviewing and selecting teacher candidates as you describe. We look at transcripts as we screen applications, and some schools will use screening instruments (i.e. Haberman Star Educator, Gallup TeacherInsight). Some schools will ask candidates to teach a lesson; but I have found that those forced lessons do not afford a candidate (or the school) an opportunity to to truly demonstrate his/her skills (it is a forced interaction for students, with a group of adults watching another adult perform). Strikingly different, our process typically takes several weeks – from screening, to initial interview, to follow-up interviews, to offer of employment. And interview questions should be focused around key characteristics and skills of teaching, and build from interview to interview.
The problem with waiting a week is that you’ve missed the other oddity of school recruitment – that you have to declare you are looking, ask (beg?) for a day off and get a reference for the school in advance – all of which is utterly bonkers in other industries. You would then give the person’s current school a week to make a counter offer – and chances are they would have another interview in that time which could throw a spanner in the works.
The real issue I think here is that when there’s a decent supply of teachers you’ll probably get a decent one by this or any method (most teachers imho are good) and you certainly won’t find schools doing any reflecting on their choice to disprove this. When supply is poor this process will put people off applying and leave you with perhaps a choice of 1 or 2 (or fewer if you want a Physicist). At this point what schools really need to do is to prospect – show how good you are as an employer, encourage people to apply at any time and create a talent bank, get rid of long application forms and three referees and just get people into the school. That’s the advice I gave blue chip companies in my previous career!
I think that’s fair
There have been some interesting longitudinal studies in medical schools trying to determine which aspects of the selection process correlate best with academic and clinical performance (and retention) down the line. That’s different to recruiting to a job but particularly pertinent to what I do. In medicine, academic but not clinical performance correlates well with previous academic performance (A-Levels). Multiple mini interviews seem to be promising for clinical performance (applicants seeing a number of separate interviewers, each asking a different question, and then the total score is used).
Somewhere I’ve seen discussion and possibly evidence that successful charter schools in the US very much recruit on the basis of fit, and this leads to improved sense of organisational purpose.
I haven’t done enough reading to know at all, but I think it might be important to look at that IQ research in relation to specific jobs and to look at it the other way round as well. If you look at Ree, Earles, Teachout (1994) note that Air Traffic Controllers have a lower correlation than the engineers, which I’m guessing might reflect the ‘real-time’ nature of their job. I’m happy to accept that a good IQ is important for teaching, but not that a high IQ is a consistently good predictor of teaching performance. I know you are not suggesting that but wonder whether any use of an IQ test might make more sense as a threshold criterion only.
Hi David. Great post as ever. What do you think about candidates having to send in a video of a lesson from their school? Should get a picture of how they interact with the younglings in a more secure environment. Can also spend more time on it and discuss with colleagues “live”..?
I think I would refuse to go to this trouble. Schools need to balance getting good information against annoying potential candidates. Also, a video could easily be staged.
Maybe. I think I would personally prefer to record a bog standard run-of-the-mill class with a class I know and am comfortable with than be chucked in with God knows what. How could it be easily staged? If there was an SLT member in the corner of the room that implies quite a high level of conspiracy (especially for a leaving member of staff). And if you’ve given them some crazy threat like an exclusion for whispering in class then you, as observer, can still see a lot of important things like the quality of explanation and practice activity..
[…] Part 1 I reviewed some of the research surround the best way to recruit and how this might apply to […]
As far as interviews go it is also important for the interviewee to see what they think of the school – you can tell a lot about a school by the questions that are asked and the way they are asked.
I learnt a lot from this post (and I can’t say that I’m a big fan of yours). The bit about high academic grades = ability to manage a complex job & conscientiousness was very interesting. I’d never thought of using grades as a proxy for conscientiousness. I might look back at the grades of my current staff and see how this stacks up!
[…] Part 1 of this series I reviewed some of the evidence on what makes for effective interviews, and in Part […]
‘Fitting in’ is so important. I have employed several teachers for a small school and when they don’t fit with the ethos and values they haven’t lasted, don’t work out as team players. Larger institutions may absorb them – the equilibrium in schools of less than 10 staff is completely changed.
I guess that depends on how rigid we are. Some people work very hard to prevent people with different values from fitting in.
[…] Can we improve school interviews?, by David […]
[…] know that David Didau has written a thought-provoking blog on the teacher recruitment process here and I have no desire to revisit the issue in detail. However, the reference templates I have read […]
[…] know that David Didau has written a thought-provoking blog on the teacher recruitment process here and I have no desire to revisit the issue in detail. However, the reference templates I have read […]
[…] know that David Didau has written a thought-provoking blog on the teacher recruitment process here and I have no desire to revisit the issue in detail. However, the reference templates I have read […]
[…] Part 1: A brief review of the evidence […]
[…] going into too much detail, I recommend you read David Didau’s blog series on the topic here. Given all that, if you aren’t ready to make massive overhauls to your recruitment process […]
[…] Can we improve school interviews? Part 1: A brief review of the research […]