“Works for me!” The problem with teachers’ judgement

One of the difficulties inherent in challenging teachers’ judgments is that when those judgements appear to be contradicted teachers sometimes say, “Well, it works for me and my students.” This is hard to challenge.

Anthony Radice made a similar point in a recent blog post about the debilitating nature of complacent certainty:

A clear example of this kind of complacency is contained in the words, ‘I know my pupils’. It’s the killer punch to an argument, because it is not falsifiable. There is no definitive evidence that can be presented to refute this statement. One can debate endlessly over the evidence for this approach or that, only to have the whole discussion closed down with these four short words. The implication tends to be that ‘you can pontificate all you like about cognitive science; you can use any logical argument you like; but I possess knowledge which supports my approach, knowledge to which you evidently have no access, you cold-hearted intellectual, you!’

Falsifiability is important because it’s the backbone of progress in science. A scientist states a theory, describes the evidence which supports that theory and then other scientists are able to test the theory and replicate the experiment looking for possible flaws. This is not how, say, religion works. Religious belief relies on faith. When faith is challenged by evidence, it’s possible to say things like, “God works in mysterious ways.” It’s not possible to disprove such a statement because it’s not falsifiable. How could you begin to attempt to show that God doesn’t work in mysterious ways? if a statement or a theory is true in all possible circumstances then it’s unfalsifiable: you can’t disprove it. If we can’t disprove something then no amount of evidence or reason could possibly sway us from our entrenched positions because those positions are not based on evidence or reason but are instead based on faith.

If, in the face of contradictory evidence, we make the claim that a particular practice ‘works for me and my students’, then we are in danger of adopting an unfalsifiable position. We are free to define ‘works’ however we please. If we’re told that students’ exam results might improve if we changed our practice we can say things like, “There’s more to education than exam results” and claim that our students are happier, better rounded, or have an excess of some other vague, unmeasurable trait. We can laugh at the idea of measurement and say, “Just because you can’t measure it, doesn’t mean it isn’t important.” We can insulate ourselves from logic and reason and instead trust to faith that we know what’s best for our students and who can prove us wrong?

The thing is, this sort of faith results in stagnation. Science has made tremendous leaps and bounds over the past 200 or so years because scientists learn from their mistakes. Religions have stayed pretty much the same (much to the satisfaction of their adherents as their beliefs represent eternal Truth.) If we as teachers adopt a position of faith, we cannot learn or improve. We cannot receive meaningful feedback about our mistakes if our response is to shift the parameters and respond with, “Yeah, but”. By testing our ideas and accepting where we have made mistakes, we learn.

As Daniel Kahneman and Gary Klein point out in Conditions for Intuitive Expertise: A Failure to Disagree, there are some professions – psychotherapy, clinical psychology, recruitment, radiology, stock broking, the judiciary – where practice and experience doesn’t seem to improve judgement. What all these professions have in common is that it’s very hard to learn from mistakes. If you work in recruitment it’s unlikely that you’ll track the long-term success of the personnel you recruit so you never know when your judgement was successful and when it was not. A psychotherapist takes feedback from the apparent progress of patients in the clinic but has no mechanism for knowing how they behave in the real world. All these professions rely on their subjective judgement without getting meaningful feedback on how successful their judgements might be. Teaching is, perhaps, a little like this. We get instant and meaningful feedback on some aspects of the job; if our behaviour management isn’t up to snuff, students are unambiguous in their feedback. But how do we know if our ability to get students to retain and transfer new skills and knowledge is good enough? Most of the time we rely on our subjective judgement and certain visible proxies such as whether children are working hard, solving problems and generally performing well in the classroom. It’s easy to look with satisfaction at a sea of happy, contented faces and conclude, “Well, it works for me.” As Kahneman and Klein put it, “human judgments are noisy to an extent that substantially impairs their validity.”

There are two main barriers to teacher improvement. One is that we often fail to notice whether our ability to teach is any good. The other is the way we are held to account. We are asked to justify and explain why students failed to make the grade; we are under pressure is to make excuses and conceal mistakes to avoid being blamed. Instead of admitting that what we’re doing doesn’t appear to be effective we shrug and say, “These things happen” and “What can you expect with kids like these?” Sometimes we blame specification changes or marking protocols: It must be the fault of the exam board, Ofqual, the DfE. I’ve heard failure blamed on timetabling, sports fixtures, room temperature and litany of equally plausible excuses. My favourite excuse is one I wrote about in my recent book:

Once in an exam analysis meeting, a school leader who taught in a particular department said that the reason the exam results of that department were so poor was because of their outstanding teaching. They concentrated on independent learning and refused to ‘spoon feed’. This obviously meant kids did less well in the test.

If we want to improve as teachers we have to acknowledge our errors. If school leaders want teachers to acknowledge their errors they must create a culture where this is safe and normal. One positive change to school culture would be to change the norm from using evidence to confirm our prejudices to using it explore how we might think and act differently.