This is #18 in my series on the Top 20 Principles From Psychology for Teaching and Learning and the first of three posts examining how to assess students’ progress: “Formative and summative assessments are both important and useful but require different approaches and interpretations.”

As I’m sure everyone knows, summative assessments are made to establish what students have learned and to provide a quantitative measurement of achievement. Formative assessments, on the other hand, are intended to establish how students are progressing and provide them with the support needed to arrive at their intended destination. Summative assessment takes place after instruction while formative assessment is conducted before and during instruction. But despite their clear differences, the Top 20 report says their functions are basically the same in that they aim “to produce valid, fair, useful, and reliable sources of information.”

Clearly that is the aim of summative assessment, but can the same be said for its formative cousin? The greater part of formative assessment (or Assessment for Learning as it’s usually dubbed) is made up by teachers on the fly. There are various classroom practices, from the gimmicky (traffic lights, no hands-up questioning, exit passes) to the more conventional (marking books) all designed to allow teachers to judge whether students are on track and to make decisions about future teaching. The problem with all these approaches is that teachers will be making judgements based on current performance, rather than on learning.

I’ve argued before that learning is invisible; we can only ever infer learning from performance. We want the skills and knowledge we teach to be both durable and flexible. For learning to be worth the name it has to have resulted in a change in long-term memory so that students will both remember and be able to transfer what they’ve learned between different contexts. Unfortunately we can’t see whether students know something later or can do something somewhere else unless we assess them at different times and locations.

Like any abstraction we make up metaphors to help us better imagine what we’re discussing. These metaphors are powerful; they can be enormously useful ways of thinking about the world but they can also fatally constrain our thinking. Language is cluttered with dead metaphors – ways of thinking about the world that were once fresh and vital but have, over long use, been trampled into cliché. Most people are unaware that the idea that learning can be seen even is metaphorical; repeated, unexamined usage has tricked us into believing we can literally see inside another’s head. We cannot. Any attempt to assessment learning is an attempt to map a mystery with a metaphor.

Why does this matter? There are two reasons. First, if we make judgements on what students appear to have learned then very often we will mistake mimicry for learning. New and troublesome concepts take time to integrate into patterns of prior knowledge. If we rush students to give to ‘prove’ what they have learned they will, often, simply give us the answers they think we want to hear. If they look hesitant or confused we prompt them with eager nods and points at relevant display or previous work in books. All too often they know the answer we want them to give with little understanding of what the answer means. But, grateful, we accept these meagre offerings as evidence that learning has taken place. Current performance is a poor proxy for learning. Our inferences are routinely wrong.

Second, and more importantly, there’s a compelling body of evidence that reducing current performance actually increases long-term retention and transfer. The better students do in the here and now, the less likely they are to do well six months later in the exam hall. This is because strong current performance produces the ‘illusion of knowing’. We remember that we could do something once and fail to notice that the substance of what we think we can do is forgotten. But when students have to struggle and dredge their memories for answers they know that they don’t know; there is no comforting sense of familiarity to lull them in a false sense of security. Instead of acting to shore up our ability to retrieve in the short-term, making the conditions of recall more difficult helps strengthen and embed items in long-term memory. So formative assessments which tell us that students seem to know what we’ve just taught them tell us little of value. As we saw in Principle 2, revealing and managing misconceptions is much more useful, but there’s even good evidence that getting a wrong answer now, helps reduce the likelihood of making the same mistake in the future.

Summative assessment, on the other hand, is a much more reliable indicator of learning as students’ performance is usually judged at a later date and in a different place. We’re still only able to see current performance, but that performance will better demonstrate with knowledge and skills have been retained and transferred.

None of this is to argue that formative assessment is useless, just that its goal should be explicitly different to that of summative assessment. Rather than pretending that it can give us “valid, fair, useful, and reliable sources of information” we should see its purpose as building students’ storage of the skills and knowledge we wish them to remember and apply. The most useful way of doing this is, perhaps, to make use of the testing effect which I discuss in Chapter 14 of my new book:

Testing can (and should) include some of the tricks and techniques we’ve been misusing and misunderstanding as Assessment for Learning. In fact, it doesn’t really matter how we test students as long as our emphasis changes. Testing should not be used primarily to assess the efficacy of your teaching and students’ learning; it should be used as a powerful tool in your pedagogical armoury to help them learn.

…
Studying material once and testing three times leads to about 80 per cent improved retention over studying three times and testing once. The research evidence suggests that it doesn’t matter whether people are asked to recall individual items or passages of text – testing beats restudying every time. Now, we all know that cramming for a test works. However, these studies show that testing leads to a much increased likelihood that information will be retained over the long term. This implies that if we want our students to learn whatever it is we’re trying to teach them then we should test them on it regularly. And by regularly I mean every lesson. What if every lesson began with a test of what students had studied the previous lesson? Far from finding it dull, most students actually seem to enjoy this kind of exercise. If you explain to them what you’re up to, and why, they get pretty excited at seeing whether the theory holds water.

This flawed understanding of what formative assessment should actually be for, infects the advice given in the Top 20 report. The advice they offer is all designed to improve students’ current performance and “collect evidence on student learning.”

Some of the advice is unambiguously contradicted by research. For instance: “Keep the length of time between the formative assessment and subsequent interventions relatively short; this is when effects on student learning will be strongest.” In fact, “Numerous studies—some of them dating back decades—have shown that frequent and immediate feedback can, contrary to intuition, degrade learning.” (Soderstrom & Bjork, 2013)

This section of the report concludes by stating that “Teachers can make better use of both formative and summative assessments when they understand basic concepts related to educational measurement.” This is dead right, but not in the way the report’s authors assume. To best understand how to use formative assessment we need to know about the psychology of memory and forgetting.

References cited in the report