Alex Quigley has just responded to my post Two Stars and a Bloody Wish! with the revelation that it works for him and others:

Using a ‘Two Stars and a Wish’ model ironically meant that many teachers were writing more concise comments and spending less time on marking than before. Rather than proving a waste of time as David Didau suggests, it was saving time for many (teachers weren’t beholden to two wishes each time and there was seldom ‘lavish praise’).

Well, good. If using a particular marking structure does actually save teachers time then who am I to criticise? Alex goes on to say that, “though flawed (there is no perfect method for marking), the two stars method at least provides consistency for students and it can better delineate feedback than some of the chunky paragraph responses I have seen given by teachers.”

OK. Although I’d like to suggest that teachers be dissuaded from producing “chunky paragraph responses”. Then he delivers his knock out blow: “Like most approaches in the classroom, it can be used well and used badly.” And, of course, this is true. Or at least, it’s become a truism. I’ve said it myself. In fact, I once watched a presentation on the EEF Toolkit where the presenter ended with the Fun Boy Three/Bananarama classic It Ain’t What You Do It’s The Way That You Do It (that’s what gets results, dontcha know?)

In one sense I have no argument with this point – you really can do anything badly or well, but as I explained here, the distribution of data matters. Doing some things well might not be as good as doing other things badly. For instance, the EEF toolkit helpfully tells us that the average progress you might expect from implementing a Learning Styles intervention is +2 months. Some studies have reported greater gains, some less, but the average has been rounded at +2 months. Compare that with the behemoths like Feedback & Metacognition which both report average gains of +8 months. You can implement a sub-standard feedback intervention and still out perform the very best efforts to use Learning Styles. It’s vital to understand the concept of Opportunity Cost when considering these issues.

The point of all this is that although the data gleaned from these studies might be suspect, it’s a hell of lot better than relying on teachers’ intuition about what works. I can say anything works. Consider Alex’s reflection on why he likes the two stars method:

I gave two stars to the above essay because that helped me recognise and record individual strengths. I also explained what any abstract fancy language meant when I talked to the individuals and the class. It is always essential? Certainly not all the time, but then I only do it when I think it is appropriate or serves a use. Sometimes students get just one star if the boot fits.

For me, it is important that students to reflect upon their work that has been assessed, if it is a substantive piece of work – like an essay in my English class. I want them to take the time to see if there is a pattern of wishes over time. Thinking about one key improvement that can be revisited is fine for students to handle and so the ‘Two Stars and a Wish’ model proves helpful here. I want students to think hard about how they can meaningfully ‘fix’ their wish. [my emphasis]

I’m not saying he’s wrong – how could I know? – but I am saying that he’s relying on his intuitive judgement on what he thinks is most likely to be effective for his students. What so wrong with that? Well, it’s exactly the same process as that used by doctors up until the late 19th century. Until then bloodletting was routine. Doctors intuitively understood that using leeches to relieve patients of bad humours would cure a wide range of illnesses. In the absence of other treatments for conditions like hypertension, bloodletting might sometimes have had a beneficial effect in temporarily reducing blood pressure by reducing blood volume, but since hypertension is often asymptomatic and thus undiagnosable without modern methods, this effect was down to luck. The fact that in the overwhelming majority of cases bloodletting was harmful to patients went undetected as the treatment was never subjected to a proper test. If patients recovered, then it was down to bloodletting, if they died that was just one of those things.

The alarming truth is that none of the marking practices routinely undertaken by teachers in UK schools have ever been subjected to any kind of fair test. Two stars and a wish, what went well/even better if, DIRT, triple impact marking, green pens and all the other marking fads which have come and done obver the past few years have gained traction purely on the basis of teachers’ intuition. The only practice for which there is any kind of empirical support is the finding that giving grdaes seems to act against students reading and acting on written comments.

I sincerely hope that the forthcoming EEF report on marking will reveal the full extent of the paucity of research in this crucial area and that researchers will respond by setting up and funding large-scale RCTs as soon as is practical. Obviously, using two starts and a wish isn’t going to kill anyone, but it might not be helping much either. But more important than this, it might, just might prove to be a massive waste of time. Maybe it’s not. Maybe we’ll run trials and discover that so-called ‘deep marking’ is the longed for magic bullet, but I for one would like to be a little more confident of my facts before encouraging any teacher to invest time in routinely writing extensive comments on students’ work.