Sadly, I missed most of the Friday. I spent the morning speaking at a maths conference (I know, right?) on correcting the mistakes made in the name of ‘numeracy across the curriculum’. If you’re interested, I argued that whilst numeracy has a pretty superficial connection with much that goes on in other subjects, mathematical thinking would be a far more powerful way to explicitly teach pupils to filter how they viewed the curriculum. I may blog on this at some point in the future.

Then, channeling the spirit of the John Cleese film Clockwise I had to race across to Wellington in a 1979 VW Campervan. I arrived for my debate with the legendary Dylan Wiliam with about 5 minutes to spare and sweating profusely.


For those who don’t know, I wrote a post earlier in the year in which I set out my ideas on why AfL might be wrong. Generously, Dylan took the time to set out a defence and we subsequently agreed to thrash out our differences in front of a live audience. Not having met Dylan before I wasn’t sure what to expect. There is no doubt that he is a good deal wiser and better read than I – I suspect that if he’d wanted he could easily have made me look foolish. However, I found him as generous in person as he has been online.

This is my position: The ‘big idea’ of formative assessment, that teachers should “Use evidence about learning to adapt teaching and learning to meet student needs” is wrong. It assumes that you can assess what pupils have learned in an individual lesson, and then adjust future teaching based on this information. But “If nothing has changed in long-term memory, nothing has been learned.”

One of the biggest mistakes that I think we’ve made in teaching is that we can see learning. But, as Robert Bjork tells us, learning is distinct from performance. We cannot see learning because it takes places inside children’s heads; we can only see what they do. If we measure pupils’ performance we can only ever infer what might have been learned, but such inferences are highly problematic for two reasons. Firstly, performance in the classroom is highly dependent on the cues and stimuli provided by the teacher and secondly, performance is a very poor indicator of how well pupils might retain or be able to transfer knowledge or skills.

Dylan has suggested 5 ‘key strategies’ that are required to embed formative assessment. They are:

  1. Clarifying, sharing, and understanding learning intentions and success criteria
  2. Eliciting evidence of learners’ achievement
  3. Providing feedback that moves learning forward
  4. Activating students as instructional resources for one another
  5. Activating students as owners of their own learning

Broadly speaking I don’t really have a problem with numbers 1, 3 or 5, but 2 and 4 are more problematic.

The idea that we should spend lesson time on eliciting evidence of learners’ achievement misses the fact that this achievement is only evidence of what they can do in a particular lesson and in no way allows us to work out what they will still be able to do in future lessons because as learning occurs so does forgetting.

Dylan did point out that much of the research that cognitive psychologists conduct is carried out of psychology undergraduates and that we need to be cautious about implementing these findings wholesale in classrooms as children may well behave differently. But I’m with Carl Weiman on this: research conducted in laboratory conditions produces more empirically robust data. And whilst this data may be at odds with the reality of classrooms it will still allow us to make meaningful and measurable predictions on how pupils are likely to behave.

The eliciting that Dylan recommends is basically asking questions. The only useful information we can get from doing this when pupils don’t know the answer; this at least affords us an opportunity to do something about it. But if they answer our questions correctly, it means very little. Just because they know it now, doesn’t mean they’ll know it next lesson or in an exam. A correct answer to a question is, perhaps, the least useful student response we can hope for. Dylan pitched in with the observation that failing to answer a question correctly is more likely to lead to pupils retaining information than when they give a correct answer. My best advice would be to assume that what pupils appear to know at the end of a lesson provides little indication of what they will know next lesson.

So what is the point of asking questions? If questions make you think hard and grapple with difficult concepts then there’s a reasonable chance they’ll result in learning. But if they’re just used to capture evidence on what they’ve just been taught which we then use to “adapt teaching and learning to meet student needs”, they are, I think, largely pointless. If however they’re used to assess what you have retained from a previous lesson, then they are very useful for helping pupils remember the learning we determine to be important.

I hope I’m not caricaturing Dylan’s position to say that as far as I could ascertain, he agreed with me. The nearest we can to a disagreement was on the idea that formative assessment would allow teachers to develop a ‘nose for quality’. (The originator of this phrase, Guy Claxton, was in the audience, but sadly I didn’t get the opportunity to discuss any of this with him.) My contention is that our instincts are terrible. Most people only have a ‘nose’ for what they like. We do what we do because we like it and this has very little to do with the ‘quality’ of whatever we’re doing. My experience suggests that learning is deeply counter-intuitive and relying on an innate sense of ‘what’s right’ will lead us astray.

My other concern is the idea that activating students as instructional resources for one another is likely to result in learning. For me, the only argument for investing time on peer assessment is Nuthall’s observation that regardless of how teachers organise their classrooms, children will find opportunities to talk to each other, and most of the talk will go unnoticed by the teacher. In fact he was able to work out that about 80% of the feedback pupils get on their work is from each other. This might be cause for celebration except for the fact that about 80% of this feedback is wrong. If that’s true (and I think it might well be) it is probably worth our while to attempt to do something about it.

Here are some other things we agreed on:

  • There’s no substitute for knowledgeable, authoritative teacher talk and that denying this to children does not serve them well.
  • As a species we are terrible at self-assessment. Dylan gave the example that 85% of drivers in the US rate themselves as having above average driving ability. Similarly, try asking teachers how many believe they are better than average.
  • Difficulty is only desirable when it’s not too difficult.
  • Relying on ‘what kids like’ is a mistake.
  • Learning is most likely to occur when pupils are made to think hard, and that this is not necessarily something they will enjoy. If we only every do what kids like we will, inevitably, be dumbing down. I’d go further than this and suggest that the ideal state for a pupil to leave a lesson is one of struggle. If they understood that might be because they were asked to do something easy or produce in them the ‘illusion of knowing’. But if they get used to being confronted with troubling knowledge and difficult concepts they will leave thinking. And if learning happens when we think hard, and we remember what we think about, they are more likely to learn.

The reaction on Twitter was largely positive; although a few commentators seemed disappointed there was so much consensus. Maybe they were hoping we would strip to waste and wrestle? I have to admit I’m grateful we didn’t; for a man 15 years my senior, Dylan has a formidable physique.

All in all, I enjoyed the experience immensely and have a more complete understanding as a result of the discussion. We had a very civilised chat afterwards and I can say with complete sincerity that he is a thoroughly good egg.

The only other session I managed to attend on the Friday was Michael Gove’s finale in which he told us he “loved teachers” and admitted that policy was a matter of opinion. Love him or hate him, the one thing you can say about Gove is that he seems to sincerely believe in what he’s doing.

More on Day 2 soon…

Here’s an alternative reading of events from @MisterBHayes