The problem with book monitoring

//The problem with book monitoring

Stupidity has a knack of getting its way.

Albert Camus

Most schools these days routinely monitor students’ exercise books in an attempt to extrapolate the quality of teaching. In some ways this is positive and reflects the growing recognition that we can tell much less than we might believe about teaching quality by observing lessons. On the whole I’m in favour of looking at students’ work, but, predictably, book monitoring goes wrong for pretty much the same reasons lesson observation doesn’t work.

The thing is, there’s nothing wrong with observing lessons, work scrutiny or any of the other practices used to peer inside the black box of teaching quality, the problems stem from how the information gleaned is then used. If I observe a lesson with a checklist of criteria like the one below, I will be viewing the lesson through a set of predefined parameters which will inevitably distort what I see.

Screen Shot 2015-11-26 at 14.21.50

It’s not that I think Ross McGill’s approach is unhelpful per se – in fact, as he explains in his post, the idea that this monitoring should be accompanied by conversation with students is probably useful – but by using the pro forma he suggests then the best you can expect is to find what you’re looking for.

What, you might wonder, is wrong with finding what you’re looking for?

Let’s consider the Learning Policy Ross mentions in his blog:

1. Teachers must have a secure overview of the starting points, progress and context of all students.

2. Marking must be primarily formative including use of a yellow box which is clear about what students must act upon and selective marking, where relevant.

3. Marking and feedback must be regular.

4. The marking code must be used.

Number 1 is a clear statement of what the role of a teacher entails and as such seems an excellent way to hold teachers to account. But this is undermined by predetermining what good looks like in points 2-4. Why must a yellow box be used? Why can’t marking be irregular? What’s the reasoning for one marking code being superior to another? This sort of thing results in teachers marking books not for students’ benefit, but for the convenience of auditors. This isn’t a learning policy, it’s managerialism and it is to be resisted. Rather than creating unnecessary workload, it would be better to simply say, “We trust you to have a secure overview of the starting points, progress and context of all students and how you go about doing that is up to you.”

It comes down to whether you’re more interested in getting what you want or trusting people to do what is best. Instead of looking for items on a checklist we should be looking at what is there and asking questions about why it’s there and what it represents. As I’ve argued before, accountability only works if those being held to account are prompted to try to be their best instead of trying to look good. When teachers are told what good looks like, they know that anything that deviates from this expectation is likely to be viewed with suspicion and subject to misunderstanding. The safe option is to cover your back, give the observer what they want and regularly festoon your books with yellow boxes.

The point is, none of this matters. The only thing worth checking for is the quality of students’ work. As such, if you really feel you need a pro forma to fill in, I suggest it looks something like this:

Screen Shot 2015-11-26 at 15.17.28

Now, let’s consider the evidence. Teacher 1’s students have produced work which is untidy and lacking in quality. Teacher 2’s classes, on the other hand, have produced some great stuff, but it hasn’t been marked. Teacher 3’s classes are turning out rubbish work which is also going unmarked and the students of Teacher 4 are working well and their work is being marked. What does this tell you? Which outcome do you prefer? What assumptions are you in danger of making? What questions would you want to ask?

The last two cases present few difficulties. It seems reasonable to have a word with Teacher 3 and suggest her books need marking. Even if we charitably assume that there are other methods the teacher might be giving feedback, clearly it’s not working. In the case of Teacher 4, both teacher and student seem to be doing exactly what’s expected and required. Case closed.

But what about the first two teachers? What has our scrutiny actually revealed? I’d want to have a chat with Teacher 2  to find out how this magic is being worked. It would be interesting to compare students’ work across subjects to see whether they’re all simply highly motivated young chaps who do what’s required despite feckless teachers. I might want to speak to some of the students to ask about the conditions under which their work was produced and to find out whether they have been receiving feedback through means other than marking. But, if the work is good, the last thing I want to do is tell off the teacher.

Teacher 1 though is a cause for concern. Despite the work being marked it’s just not up to snuff. Is this because their students blithely ignore their teacher’s earnest efforts? Might it be that the presence of marking isn’t providing useful feedback? If the teacher is working hard to mark, but the quality of work isn’t improving, maybe the teacher needs some support? Or perhaps the situation will right itself given time and should just be earmarked for further monitoring. It should always be remembered that treating teachers equally is fundamentally unfair.

Both of these cases reveal circumstances where book monitoring could go wrong.It’s far harder to assess the quality of work than it is the quality of marking and so we have an entirely natural tendency to do what’s easier. If we’re just looking to see whether a marking policy has been followed Teacher 1 might get a gold star, despite the poor quality of work. And I can well imagine a scenario where Teacher 2 is forced to comply with a marking policy despite the successes of the students.

Another related point is about who’s doing the book monitoring. McGill makes the point in his post that it should be subject leaders and this is generally sound advice. The last thing we want is school leaders with a subject specialism in, say, DT, quality assuring maths books. I was once told by a PE teacher that the work my Year 7 class had been doing wasn’t challenging enough. When I asked why, he told me this was because they’d been studying The Lady of Shallot, a poem he’d seen being taught in a primary school he’d visited. “Hmm,” I replied. “That’s odd because Tennyson’s poetry is on the A level specification and I’m about to start studying it with my Year 13 class.”

Who cares if marking is regular or in line with a policy as long as the work students produce is of a fantastic quality? And if the work is ropey, only a fool would be happy if the marking meets expectations.

This should be the standard against which we hold teachers to account: Is the work great? If the answer’s no, then whatever they’re doing isn’t working. But if the answer’s yes, no other question need be asked.

Further reading

If Venus de Milo did feedback – what reach she could have had by Andy Day

Is book sampling valid? by Greg Ashman

Evidence? What evidence? by Toby French

2016-10-05T21:32:57+00:00

30 Comments

  1. costadelsolent November 26, 2015 at 5:59 pm - Reply

    “Teacher 1 though is a cause for concern. Despite the work being marked it’s just not up to snuff. Is this because their students blithely ignore their teacher’s earnest efforts? Might it be that the presence of marking isn’t providing useful feedback? If the teacher is working hard to mark, but the quality of work isn’t improving, maybe the teacher needs some support?”

    More likely that teacher 1 is spending so much time marking that they have little time to plan.

    • stowie75 November 26, 2015 at 11:08 pm - Reply

      This is the real issue – teacher time spent on marking …. Rather than careful planning. Hence shutting the door once horse has bolted

      • Andrew Lowery (@AndrewDLowery) November 27, 2015 at 12:21 am - Reply

        Marking (or, at least, checking) is part of good planning though. How can you plan without knowing what your students are thinking and writing?

        • David Didau November 27, 2015 at 6:03 pm - Reply

          Do you think you have to make marks in books to know what students are thinking or writing?

          • Gooner1981 September 25, 2016 at 2:20 pm

            It seems to be a fairly common view. And it’s wrong. I am heartily sick of the pseudoscientific crap that hard working teachers are being subjected to. And now there seems to be a whole army of edu-types ( the large majority of which don’t seem to teach much) on twitter leading discussions solely through having the loudest voices.

            If your teachers are working hard, have good subject knowledge and try to improve what they do, get off their backs and give them the time to teach well

  2. julietgreen November 26, 2015 at 6:37 pm - Reply

    Could there be other factors in the quality of work from teacher 1’s books? Good/bad/indifferent teaching is not the only thing that impacts on the quality of work. In addition, is there a common understanding of ‘quality’? Again – what looks good, may not be what is good.

  3. Anthony Radice November 26, 2015 at 8:14 pm - Reply

    A related point is that it is really only heads of department who can judge the quality of work. So why bother involving all of these other bored, overpaid managers in the process?

    • David Didau November 26, 2015 at 8:16 pm - Reply

      That’s not quite true – any subject specialist ought to be able to have an informed view on the quality of work in books but you’re right that preventing SLT PE teachers from quality assuring maths books is probably in everyone’s best interest.

      • Anthony Radice November 26, 2015 at 9:52 pm - Reply

        Yes, any subject specialist could do it. But a head of department still has hands on day to day experience too. In any case, most SLT are not qualified to make judgements of most books.

  4. joiningthedebate November 26, 2015 at 8:28 pm - Reply

    Anyone outside teaching would think that TTs form was perfectly reasonable.
    The problem comes when managers want to find fault. They use this type of paperwork as evidence in order to bully certain staff.
    Also I am not in favour of whole school policies (they only exist because otherwise OFSTED) will criticise the school for being inconsistent.
    Here’s another problem… Managers are judged on their ability to force everyone to comply with a whole school policy. It doesn’t matter what the policy is as long as everyone does it. Then that manager can wax lyrical at interviews about what what they’ve done for* the school and then get promoted.
    * or done to

  5. Richard Clutterbuck November 26, 2015 at 10:29 pm - Reply

    Enough of the Argumentum ad Seniorteamium, from Radice and Co. Focus on the topic. When one carries out a book look/work scrutiny/call it what you will, is one looking at the quality of the a student’s work, the quality of the teacher’s comments or the response of the student’s comments to the teacher’s comments? The list could go on. Whichever area you are looking at requires a different set of criteria and a different set of outcomes. The difference between a simple check that the marking policy is being followed compared to what the quality of work is showing is vast and complex. Perhaps a significant refinement is needed in this area of scrutiny and coaching teachers in to being able to provide feedback which truly enhances learning is something that needs as much effort as classroom teaching practice practise. Nevertheless, to David’s table; I would want to know of teacher 2 if he/her were to mark and if it were carried out in the right way* whether this would result in even better quality work coming from his/her students? Should teacher 2 be marking?

    *What is the right way to mark a piece of work?

    • David Didau November 26, 2015 at 10:32 pm - Reply

      I would be interested in that too Richard. But there’s always an opportunity cost: time is finite. If we were to make Teacher 2 mark, what would she have to stop doing? Maybe there are better things than marking. Certainly, giving feedback is not the same thing as marking: https://www.learningspy.co.uk/workload/is-marking-the-same-thing-as-feedback/

      • Richard Clutterbuck November 26, 2015 at 11:13 pm - Reply

        I agree and perhaps I ought to be trying to find out what it is teacher 2s do in their whole practice which produces such good work. I’ve posed it as a question on interview to aspiring middle and senior leaders, “You have a member of your team whose students produce great work and get great results, but he/she refuses to follow department procedures, specifically marking. What would you do?” I don’t think there’s an easy or simple answer to this and it’s as much of a challenge for those asking it as the candidate faced with it. The danger is amplified by my mother’s story from when she trained as a nurse back in the 60s; she was regularly instructed to wake the patients up in order to give them their sleeping pill.

    • joiningthedebate November 26, 2015 at 10:50 pm - Reply

      If I am part of the co you mention, then thankyou. I am focussing on the topic. Recently a manager failed to notice any of my marking during a lesson observation. I and my dept are being told how to mark by someone who feels they are above scrutiny. They are not able to demonstrate supposed good practice by showing a set of their own books.
      I did mark regularly and consistently and with the right colours, but it was never good enough. So I stopped using the colours as this was time consuming. Obviously the recent marking was not spotted because they are only looking for certain colours. The yellow box is an example of potentially lazy management (no specific offence to TTs yellow box). Use of colours is generally not for the kids – it is an easier way to monitor staff. Keep them busy, keep them tired, then they won’t have the energy to complain when they’re in capability.

  6. David Rogers (@davidErogers) November 28, 2015 at 12:41 pm - Reply

    Blimey – I’ve read something of yours David 🙂

    I agree that book monitoring done badly is not useful. As any form of quality control, leaders need to consider a range of evidence.

    Hope you’re enjoying the sun 🙂

    David

  7. […] response to various posts on book monitoring earlier in the week, Lee Donaghy asked what the role of school leaders ought to be. Now, some […]

  8. Andy Day November 28, 2015 at 5:22 pm - Reply

    I think the one area of ‘student written response’ assessment that would be of value by SLT is a study of a sample of students’ work from across the range of their subjects. While it would be less than easy to compare what they were completing in maths with RE, I would be happy to have photocopied a page of their work from another subject area and shown to me to enquire ‘they’re writing at this standard and sophistication in English – how come I’m not seeing the same depth and clarity of thinking in their Geography work?’ I could then either flip out their 3 sided briliant essay from their ‘assessment folder’ that wasn’t asked for – or enquire of the student why they are turning out majestic writing in one subject, but not for me? (Which should also ask me to pose a few questions of myself.) Charting the steepness of the Alpine profile of response across subjects ‘could’ be a valid use of the procedure, if done in a constructive way.

    (and thanks for referring to my post DD)

  9. […] the practices of other schools I know little about, I do pretty much agree with the points made by David Didau, Andy Day, Greg Ashman and Martin Robinson about work scrutiny being a crude tool of managerialism. […]

  10. […] of the criticisms of my post about book monitoring is that I have omitted checks to see whether students have responded to feedback. This omission is […]

  11. […] Again, this might be innocuous, but I’d argue it’s likely this kind of statement will end up driving unnecessary workload. Probably the only objective, reliable way to see if pupils are making progress and standards are improving is to aggregate comparative judgement as described here. Otherwise, book checks should sample the quality of the work, not the quality of the marking. […]

  12. […] which can take a range of forms. It seemed great… But then I read David Didau’s blog, ‘The Problem with Book Monitoring’ and was momentarily seduced by its radicalism. If a school insists on prescriptive written […]

  13. PMyson December 10, 2015 at 9:57 am - Reply

    “Who cares if marking is regular or in line with a policy as long as the work students produce is of a fantastic quality?”

    – Yes but you could say anything in the place of ‘if marking is regular’ as long as it is postfaced with ‘as long as the work students produce is of a fantastic quality’:

    Who cares if students turn up as long as the work students produce is of a fantastic quality?

    Who cares if teachers don’t plan for the curriculum as long as the work students produce is of a fantastic quality?

    Who cares if teachers don’t pay attention to their students as long as the work students produce is of a fantastic quality?

    The answer to any of these questions (from a students’ performance-based perspective) would probably be ‘well, yes, quite’ because the phrasing entirely begs the question.

    When facing a new task students very rarely produce work of a fantastic quality because they need time for reflection, guidance and support, much of which comes in the form of written feedback. The real issue has got to be how to get students to produce work of a fantastic quality in the first place.

    • David Didau December 10, 2015 at 10:39 am - Reply

      You make an excellent point 🙂
      How many things are done in schools for the same of compliance or simply because they’ve always been done? As you say, “The real issue has got to be how to get students to produce work of a fantastic quality in the first place.” Not, how to get teachers to follow a meaningless marking policy!

      • pops December 12, 2015 at 10:40 am - Reply

        if one set of books could deem me to be ’cause for concern’, could I show evidence of another set of books to demonstrate I am ‘gold star’. My year 10 produce inconsistent quality but my yr9 are fantastic. Is it the teacher who is a cause for concern or the individuals who fail to produce the quality expected? I work in an inner city school with difficult behaviour issues and high sen/pp/eal btw. I have been judged both ‘inconsistent’ and ‘outstanding’ this term. Perhaps the pupils are accountable? Inconsistent can also be caused by poor attendance and a whole host of other factors?

  14. Tina Courtenay-Thompson September 25, 2016 at 5:52 pm - Reply

    Totally on the nail, David. In teaching I so often find myself muttering: Can’t you see the emperor has no clothes on!

    Book monitoring should have a specific and simple aim that is related to the students’ progress. For instance: are students able to transfer their literacy knowledge from one subject to another? If they aren’t, then what can we do about it in a constructive/fun etc way.

    There should be no blame or shame attached to it.

Constructive feedback is always appreciated

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: