“Faced with the choice between changing one’s mind and proving that there is no need to do so, almost everyone gets busy on the proof.” JK Galbraith, Economics, Peace and Laughter (1971), p. 50
Evidence is about being right, proving something, constructing an argument to support a belief. It’s legalistic and limiting. Lots of folk talk about ‘what works’ as if there could ever really be any agreement about that. But on the other hand, I’m increasing keen on research. The more research I read, the more questions I have. The more interesting the study, the more numerous and unexpected the questions it generates. Good research isn’t about finding answers – I’m pretty sure there are no definitive answers – it’s about finding out better questions.
You can find evidence to support any idea and prove pretty much anything. ED Hirsch Jr says this:
Almost every educational practice that has ever been pursued has been supported with data by somebody. I don’t know a single failed policy, ranging from the naturalistic teaching of reading, to the open classroom, to the teaching of abstract set-theory in third-grade math class that hasn’t been research-based. Experts have advocated almost every conceivable practice short of inflicting permanent bodily harm.
To research is to search and search again. To have a bloody good look. And just to be clear, I’m all for research being as rigorous, as fearless and as true to the principles of scientific method as it’s possible to be. While small-scale, personal inquiry might be useful or interesting for an individual it adds little or nothing to our collective pool of knowledge. That said, when we talk about teachers being ‘research literate’ this shouldn’t (just) be about empowering ourselves to spot bogus ideas, it should also provoke ideas we’d like to test out in our own classrooms. When someone tells us “the research says,” we shouldn’t (just) be able to respond with, “but it also shows…”, we should be able to ask, “Why does it say that?” and “Would it work like that with my students?” and “What if I tried it like this instead?”
I used to get a bit upset about people arguing against teaching becoming an ‘evidence-based’ or even an ‘evidenced-informed’ profession, but now I think I get it: I’d like teaching to be a ‘research-informed’ profession. I’d like us to ask, “What does the research say?” and then use that research to explore rather than simply confirm our biases.
Just a thought.
Is the danger at the moment not that we lurch towards the latest trend in “evidence” to frame our practice? Are we putting the cart before the horse and using evidence to explain why we do things the way we do instead of looking for contrary evidence as to why we should try something different.
That is a danger. Which is why I urge that we seek to explore instead of confirm.
Te key statement for me in this article is: “Good research isn’t about finding answers ….. it’s about finding out better questions.” I believe that some people use “evidence” as a way of avoiding asking questions. Not evidence from their own research but that which suits what they are trying to prove without ever asking the question.
I think that might be exactly what I think.
Interesting.
I agree in one sense that teachers should be research ‘literate’ and should be willing to “explore”.
But is evidence (be it good or bad; be it reliable or unreliable; be it valid or invalid) not a natural consequence of research, or at the very least a byproduct?
I’m very interested in this debate at the moment (evidence-based v evidence-informed) and I find the comparisons being drawn between Teaching and other areas like Medicine fascinating.
I guess it depends on your interpretation. We often interpret evidence differently. Of course it’s useful to see that there isn’t supporting evidence for some claims, but that can be due to measurement problems and also because so many educational claims are unfalsifiable – i.e. they’re bad questions. Too much of the research I look at is conceived of as providing evidence for ideological positions instead of investigating interesting phenomena – it’s all about how (and whether) you think.
I’d say we always interpret evidence differently, and the purpose of research and research evidence being to reduce uncertainty whether or not we accept a claim depends on judgement not the measurement of truth. p-scales are just quality-criteria chosen by someone, they don’t represent reality. Just because a claim isn’t unfalsifiable by scientific methodology – I guess you’re talking about this when you mention measurement – doesn’t necessarily the research question was a bad one to start with. As teachers we don’t usually base our day-to-day inquiry into our work, our teacher research, on positivist quantitative methodology (epistemology, ontology). Aren’t we more likely are doing qualitative, descriptive, narrative research, that might better seen as a reflective process that a method of production of verifiable claims? I agree, it’s about how and whether you think.
Ahhh! So after having a go at Hattie on April 22nd (https://www.learningspy.co.uk/featured/whose-research-is-it-anyway/) you now have come around to his point of view about teachers being evaluators. Interesting!
Oh, I still enjoy having a go at Hattie 🙂
But on reflection I do think there’s something to what he says but nothing is this post is supporting the idea that teachers cannot be involved in research. The particular way I think teachers’ involvement is actually necessary is that we are the ones best placed to formulate the questions to be researched. This is, I think, an important distinction.
Where to access the research can be the problem. My school have recently shortened lessons to 40 minutes. They haven’t based their decision on research, they’re just trying to cram more into the school day. Although I feel frazzled I think shorter lessons might be better for my students learning – in the long term. I just can’t find any studies to help me think about my intuition. Any ideas?
@Mel. This is a great example of how I think we need to take time and give thought in formulating the question. I find that when something goes against my intuition I tend to think defensively and this tends to limit the response. In your example here you may go looking for information/research about lesson length when in fact the lesson is actually a component in the school day and the day is a construct formed by the timetable. Now the question changes to what part does the arrangement of the school day play in learning. I explored this very question in an action research project and the findings were very interesting. I managed to show engagement and behaviour were directly linked to the structure of the school day and the how it was managed (length of lessons, number of lessons per day, time between lessons, length of lunch period, lesson changes and even lesson structures etc). There is an introduction and summary here: http://wp.me/p2LphS-5g
Feel free to e-mail me at kevin@ace-d.co.uk
From a classroom teacher’s point of view, ‘research’ can exactly be about ‘exploring’ practice just as David suggests. Great. My students on our ‘Action’ module (MA Goldsmiths) do this. I still think it’s good for them to come to some conclusions about what they think there research has or has not shown…and why not. However, there is the problem as suggested above with what authority and bureaucracies impose and change without any reference to any research anywhere. At that point, we do feel as if we can or should ask, ‘and where’s the evidence for this change?’ or if you don’t have any, please be honest and explain the ideological or logistical or administrative reasons for it.
Well,you can *always* find evidence to prove your ideological beliefs (cf. some of the things you write about the teaching of reading Michael) so evidence is a bit pointless. Better to ask for the research supporting people’s beliefs and interrogate it.
I think that if we move away from asking for evidence then we end up back in the ‘think world’ to where education research is back to education philosophy – which is why we are in a pickle at the moment with constructivist practices in education.
I think research (with the exception of philosophy which is a thought experiment!) needs to have evidence to support it. However, what we need is critical thinking about evidence and also a more neutral stance if our ideas are proven wrong.
When I was studying political science, there was nothing wrong with the evidence being contrary to what we predicted based on previous research and theories. We thought nothing of concluding that the evidence was not what we expected or only partially supported the theoretical framework we had set up.
Conclusions containing revision of ideas or acceptance that other theories may explain a phenomenon better are part and parcel of essays as well as research.
In education, there is so much attachment to ideas regardless of what the reality is and this tendency towards ideological commitment to an idea above the reality of the results it produces is what has done such harm.
Maybe we should start with the evidence found and ask questions of that for further research but good quality evidence should never be dismissed and poor quality evidence should be called out for what it is – especially when it is deliberately misleading.
@teachwell It may not be so much about ideological commitment when looking for reasons why people (leadership) is so attached to ideas. My theory is that it is safer to do what has been done (follow conventional wisdom) than to challenge it. Education, especially for leadership careers, is a high risk environment these days. It is much safer to do more of the conventional wisdom, even if it does not work, because it is then a case of being able to apportion blame not to the idea but to the implementation of the idea. In other words its the fault of the teacher!
I agree with that but that is why the evidence is more compelling but I agree that we need to move to more solid and valid research questions. There is also the one worded demon – OFSTED!!
And it would be handy if those in charge of policy were honest about their initiatives. Thinking of Academies for example – Nick Gibb admitted they aren’t necessarily better than other kinds of schools. So…er…why have we got them? Because, he believes in them. So, let’s have that conversation which would be on the battlefield of a certain kind of libertarianism versus a certain kind of municipalism….But the government – in spite of Gibb’s frankness – want to make out that academies are ‘better’. More like a moving battlefield.
It’s because of our preference for certainty over accuracy. I outline the problem here: https://www.learningspy.co.uk/featured/foxy-thinking-my-red15-slides/
As ED Hirsch says, “Almost every educational practice that has ever been pursued has been supported with data by somebody. I don’t know a single failed policy, ranging from the naturalistic teaching of reading, to the open classroom, to the teaching of abstract set-theory in third-grade math class that hasn’t been research-based. Experts have advocated almost every conceivable practice short of inflicting permanent bodily harm.”
You can prove ANYTHING with evidence. The problem is that evidence, by its very nature is intended as proof to confirm (and sometimes refute) our beliefs. Re*search* is a quest, an exploration, a seeking after truth in the full understanding that as our knowledge grows, so does our ignorance.
I like the idea of exploring. This is important and should enable us to learn from doing and reflecting. There are ideas that can be used directly i.e. can be adopted but often they need some kind of adaption either after or before adoption. There is no one size fits all and indeed no guarantees but we should persevere! I accept that some of us feel more able to do this given the contexts we work in and perhaps confidence or belief in what we do/aim for.
David, might there be a place for other than quasi-scientific research, for example ethnography? ‘Learning to labour: how working class boys get working class jobs’ – Paul Willis is an example that told an interesting truth at the time. Why so much dedication to the positivist paradigm when ‘interpretivism’ can be such a powerful source of knowledge.
The problem isn’t with evidence as such. Evidence can be used either to back up an opinion or argument or it can be evaluated. Researchers are supposed to evaluate the evidence and in some disciplines that’s usually what they do. Courts of law, HM Most Loyal Opposition and formal debates are supposed to do the same, but it does seem to be a practice that’s falling into disuse.
You’re right, the problem isn’t with the evidence ‘as such’ it’s the state of mind which goes looking for it.
I disagree with this quite strongly. There are many people out there pedalling approaches for which there is little evidence. If I cannot point this out because we don’t like use of the term ‘evidence’ then they have yet more opportunity to obfuscate.
Well now, just because I don’t think it’s a great idea to go about amassing proof in preference to investigating phenomena doesn’t mean we can’t point out bullshit when we see it. It’s not like I’m saying we need to believe in fairies or anything 🙂
Could it be there two mindsets at work when we talk of evidence and research, the closed and the open (fixed or growth – as per Dweck too perhaps). Having decided something do some people look for and also discount evidence or even fail to produce evidence when challenged in order to only support their view or ideas. Do beliefs find germination here that then go on to become “truths” I wonder. Truths that dare not be challenged! I think research is a different mindset, it starts with a question. Is this where ideas are born along with practices such as collaboration. Perhaps during research the original question may never get directly answered and so it is more about the journey than the destination. The second mindset may be more suited to education than the first.
Just a few thoughts.
I do think there has been too great reverence for evidence in education. To get “good” evidence (by which is usually meant repeatable evidence) it is usually necessary somehow to suppress or ignore the effect of “confounding variables” and all that. But people are largely composed of confounding variables, so this is often fairly obviously flawed. Evidence varies according to its subjects. The behaviour of metals, say, may be predictable and variables easily spotted and controlled for, so evidence may be genuinely robust. It’s horses for courses. ‘They say’ that some 50% of research turns out to be wrong, but, of course, ‘they’ could be wrong…
Research is really enquiry. Enquiry is an ideal. We should read it, but our sacred duty is scepticism. Somewhere, the late, absolutely great JK Galbraith writes that the purpose of education is the inculcation of deep, even raucous scepticism. As usual, he hits the nail on the head.
Thanks for this Hugo – love the Gilbraith quote 🙂
I thought the whole point of research was to find out something. If research does not answer a question, at least in part, then I’m not sure of the point of the research. And when we search and search again (research), what are we looking for, if we not the evidence?
I am unsure if we can be informed through research to ask any new or better questions, if the research being examined does not give at least some answers to the original question in the form of evidence. How we interpret the evidence might be fraught with difficulty but at least one can say in such a context, under such conditions, the observation or measurement yielded this evidence. It seems to me that only by accepting that evidence, but not necessarily the conclusions can we ask new or better questions. Therefore I’m am struggling to see how we can embrace research without embracing the evidence that it yields.
I see the point of research as being to understand the world. We find things out which reveal a new multiplicity of things we don’t know. What we’re looking for is Truth, a Theory of Everything which explains all there is in the knowledge that this is a best hugely improbable. There is, it seems to me, a difference in the process of looking for proof of one’s opinions and values, and the search for ever greater knowledge and understanding of the world. I’m not trying to ban the word ‘evidence’ or under the impression that facts don’t trump guesses; I’m arguing against a mindset, a way of seeing the world which is grasping for certainty. The fact that you’re struggling is an excellent start. So are we all.
whether we will ever get as far as truth I wonder. We presume we are working to a more exact understanding of reality, but, of course, will never be able directly to see it. All we can hope for is a coherent story which seems to fit with the rest of our existing story. Humility is highly desirable, therefore, and the best researchers seem to have it. Evidence will always be speculative and we should be excited and inspired by the possibilities it seems to open up, but scepticism should always inform our attitude to it. This is better understood inside science than in, say, the media or politics, where “the research” is often accorded too great a degree of finality. This applies most in education, it seems to me, which is grappling with such a plethora of influences as to make simplicity an unattainable chimera. There is no evidence which is absolute, or perfectly bankable, but there is a lot of evidence which is highly suggestive and particularly of new places, or ways, to look.
Excellent post!!! Blogging has certainly helped me to became a better researcher.. The idea out there are creative, unique & quiet brilliant
[…] read David Didau’s recent post on ‘Research vs evidence’ and it struck some chords with me about the difference between ‘evidence’ and […]
[…] Research vs evidence 9th September – Some thoughts on the misuses of evidence. […]
[…] Also, it might be worth reading this post on research vs evidence. […]