TestsGuns don’t kill people, rappers do
Goldie Lookin Chain
I spent the day yesterday at the Department for Education thinking about how best to cut down on the “unnecessary workload” associated with marking. Today I spent far too much time bandying words with children’s writer, Michael Rosen about the value of testing over teacher assessment. It strikes me that both experiences offer an opportunity to set out my objections to teacher assessment and my support for standardised testing.
Let’s start with teacher assessment. My first concern is that any expectation on teachers to assess students’ work adds to their workload. If we’re going to ask teachers to work harder we ought to be pretty sure that the additional work we’re asking them to do is worthwhile. So, is it? Well, contrary to many people’s intuitive beliefs, teacher assessment is both less reliable and more unfair than standardised testing. This is difficult for some teachers to hear, but as Daisy Christodoulou puts it, “Teacher assessment is biased not because it is carried out by teachers, but because it is carried out by humans.”
There’s all kinds of evidence that humans are subject to predictable and unconscious bias. Firstly there’s the research into psychological concepts into heuristics and biases like the halo effect, confirmation bias, the anchoring effect, overconfidence bias, and many more. In Thinking, Fast and Slow, Daniel Kahneman relates how the halo effect led him to systematically mis-grade students’ essays. Quite reasonably, if a students’ first essay was awarded a high score, mistakes in later essays were ignored or excused. But Kahneman noticed a problem:
If a student had written two essays, one strong one weak, I would end up with a different final grade depending on which essay I read first. I had told students that the two essays had equal weight but this was not true: the first one had a much great impact on the final grade than the second. (p. 83)
In studies where teachers were told that a student had a learning disability, they rated that student’s performance as weaker than did other teachers who were told nothing at all about the student before the assessment began. There’s also evidence to suggest teachers are unconsciously biased against children from ethnic minorities. And here’s another study which investigates the bias against children due to race, gender and ability. Conversely, as Suskind & Rasmussen show, we also routinely assume “well-behaved students are also bright, diligent, and engaged.”
Daisy Christodoulou has unearthed further evidence here:
Both high and medium weight evidence indicated the following: there is bias in teachers’ assessment (TA) relating to student characteristics, including behaviour (for young children), gender and special educational needs; overall academic achievement and verbal ability may influence judgement when assessing specific skills. (Harlen, 2004)
Studies of the National Curriculum Assessment (NCA) for students aged 6 and 7 in England and Wales in the early 1990s, found considerable error and evidence of bias in relation to different groups of students (Shorrocks et al., 1993; Thomas et al., 1998). (Ibid)It is argued that pupils are subjected to too many written tests, and that some should be replaced by teacher assessments… The results here suggest that might be severely detrimental to the recorded achievements of children from poor families, and for children from some ethnic minorities. (Burgess and Greaves, 2009)
Teachers tended to perceive low-income children as less able than their higher income peers with equivalent scores on cognitive assessments. (Campbell 2015)
Understandably, you might not want to be bothered wading through all that lot. Instead, why not simply watch this video of Robert Coe explaining why teacher assessment is so problematic?
Frustratingly, teaching is not set up in such a way as to produce the conditions needed for reliable judgement to develop. Of course, we all believe we’re immune from these biases which affect everyone else. There’s a name for that too: the bias blindspot. In one study, only one out of 661 survey respondents admitted to being more biased than the average person! Claiming, “It works for me!” is just further evidence of bias.
All this suggests that adding to teachers’ workload by making them assess students’ work seems a bit of waste of time, especially when we can rely on standardised tests instead.
But what’s that you say? Standardised tests are evil? I beg to differ. Although there are certainly problems associated with testing these problems are not a function of their being standardised. Standardisation just means that we are better able to compare outcomes because we can be more certain of their reliability.
The problem isn’t tests themselves it’s the purpose to which they’re put. When tests are high-stakes we create all sorts of accountability problems which often result in schools and teachers pursuing perverse incentives. Not only that, high-stakes can result in crippling anxiety for children. We’re right to worry about the consequences of stress caused by exam pressures, but these pressure should only ever be transferred to students when they mean something tangible. A levels and GCSEs are qualifications which, whether you agree with them or not, have meaning. KS2 SATs are not. There’s nothing wrong with the government deciding to test children in KS1 and 2 to determine their attainment, but we must remember that while these tests assess students they are used to hold schools accountable. I really get that these results matter to schools, but they are (or should be) irrelevant to pupils.
What we really ought to object to is the way schools feel compelled to pass on these anxieties to children allow testing to warp the curriculum. Rather than attacking testing our time might be better spent attacking the foolish, ill-thought out accountability measures which create these incentives.