The following is a guest post from the mastermind of Comparative Judgement, Dr Chris Wheadon.
The marking of English Language is likely to be extremely challenging this year. English Language has long form answer questions, typically with 8, 16 and 24 mark responses. Ofqual’s research suggests the following range of precision is normal across GCSE and A level:
8 mark items: +/- 3 marks
16 mark items: +/- 4 marks
24 mark items: +/- 6 marks
So, when an 8 mark item is marked, for the same response, it is normal for one marker to give 4 marks, while another will give 7 marks. So, referring to the AQA mark scheme for English Language, one marker will mark a response as a ‘Perceptive Summary’ (Level 4) while another will mark it as ‘Some attempts at summary’ (Level 2). Ofqual’s research show that this difference in opinion is a normal occurrence in marking of 8 mark items.
When a 16 mark item is marked, it is normal for one marker to give 12 marks, while another will give 16 marks, the difference between ‘Clear, relevant’ and ‘Some attempts’.
When a 24 mark item is marked it is normal for one marker to give 9 marks, and another to give 15 marks which is the difference between ‘Simple, limited’ and ‘Some success’.
To be clear, Ofqual’s research is based on the differences that are normal for established specifications, after marker standardisation, and with the use by exam boards of sophisticated statistical rules which stop poor marking as soon as it is detected.
Working in isolation, without access to the range of work being produced nationally, the marking in your school is likely to be considerably worse than these levels of precision. Further, when scripts are marked by exam boards the responses from each candidate are distributed to different markers. The distribution process ensures that severe marking by one examiner of one question is usually cancelled out by generous marking by another.
The conclusion is that unless you have sophisticated systems for distributing responses you won’t even be rank ordering your scripts correctly, let alone aligning your marking with the published mark schemes. And even the awarding bodies don’t suggest that you can go from marking to giving grades at this early stage.
So, what should you do? Not get students to sit mocks at all?
Mocks are obviously useful as examination practice, but it likely the most use comes from the sitting of them rather than the marking and grading that occurs afterwards. However, without the incentive of marks, some of this purpose may be lost.
There is an alternative…
The critical part of an exam is that candidates should be measured fairly in comparison with each other. Comparative Judgement allows you to distribute each candidate’s response amongst all your teachers, so any individual bias is cancelled out. At the end of a judging session your pupils will be measured fairly against each other. You can give them a mark from 0 to 40 if you like against this measure, or a level or grade if you want to take a punt!
And the feedback?
Presumably this is something you give students regularly already? Rather than wasting time on individual feedback on exam performance, use exemplars from the exams, annotate them and work on them in class.
But there’s a cost…
Yes, 25p per script if you use nomoremarking.com and take advantage of the bar coded answer sheets. For a cohort of 100 students this would come to grand total of £25. There is probably no better way to spend your departmental budget.
There’s also a saving…
Teachers estimate that the judging process represents a 75 per cent time saving compared to marking, and that includes admin. So, let’s say you normally take 5 hours across 10 teachers to mark your mocks. That is 50 hours. With Comparative Judgement, you are likely to take around 12.5 hours. And once you get the hang of the process you’ll be going it much more quickly than that! Even the worst case scenario will result in a week’s worth of time back for an English department.
If you’d like to get started judging your English mocks there is a step by step guide over on nomoremarking.com
Give it a try. Get a week back and do something more useful with it than marking.