Here are the slides I use for my talk at researchED Malmö:
The following is the English text of an article I wrote for Pedagogiska magasinet on which the presentation was, in part, based.
What leads to success? Obviously, as teachers, we should be interested in children’s academic test scores, but what else is important? Are there certain skills, qualities of dispositions that the successful possess and everyone else lacks? If there is, can we identify these magic ingredients and teach them to our students?
An exciting range of possibilities have been dangled in front of teachers: self-control, conscientiousness, resilience, and, more recently, we’ve been offered Carol Dweck’s growth mindset and Angela Duckworth’s grit.
The theory of growth mindsets expands on the well-established idea that our perceptions about intelligence affect how we perform, and goes on to argue that what students attribute their successes and failures to affects how they respond to the challenges and obstacles they face when learning in schools. Some students have an ‘incremental theory’ of intelligence (what has become known as a ‘growth mindset’). This means they tend to frame the experience of school in terms of learning goals and see ability as something that can be increased with effort and time. Other students possess an ‘entity theory’ of intelligence (a ‘fixed mindset’) and frame school work in terms of performance goals; seeing ability as something that is static and inflexible. If we have a growth mindset, Dweck’s research indicates that having a growth mindset leads to better academic achievement, whereas having a fixed mindset leads to poorer academic achievement. If, she agues, students are given a growth mindset intervention (which focuses on explaining the neuroscience involved) students’ academic performance improves.
Grit is defined as ‘perseverance and passion for long-term goals’ and research by Angela Duckworth has suggested it is a factor underlying success for groups as disparate as Ivy league undergraduates, West Point military cadets, elite athletes and National Spelling Bee competitors.
Both of these proposals are optimistic: Dweck’s research indicates that ‘the brain is like a muscle’ and if children believe that by working harder they can become cleverer, then they will. Duckworth tells us that the achievement of difficult goals in life depends not upon talent alone, but ‘a sustained and focused application of talent over time’. What’s not to like?
Well, despite the enthusiasm for grit, a recent meta-analysis has cast doubts about whether the concept stands up to scrutiny. The research looks at results of studies from 88 different samples involving over 66,000 individuals. They appeared to find that grit was only modestly correlated (0.18) with performance and strongly correlated to the personality trait of conscientiousness. Given these problems, can teachers have any confidence that interventions or character education initiatives based on the idea of developing grit will have an effect on student outcomes? Certainly, there’s no robust evidence to suggest so. Even Duckworth has admitted her findings of the independent impact of grit are what personality psychologists would place in the ‘small-to-medium’ range.
With growth mindset interventions, the trouble is that despite the enormous effect sizes reported in research fails to replicate in classrooms. In the UK, the Education Endowment Foundation’s Changing Mindsets study found that growth mindset interventions resulted in no “statistically significant effect on attainment in either maths or English”. Carol Dweck’s explanation for why attempts to replicate growth mindset interventions don’t seem to work nearly as well as we might expect is down to what she calls the ‘false growth mindset’. Because we’ve unanimously agreed that having a fixed mindset is egregious and a growth mindset makes you a better all-round human being, no one wants to admit to being ‘fixed’. When asked, we tend to say, “Yes of course I have a growth mindset,” because the alternative is to say, “No, I’m afraid I’m a terrible person.” It seems reasonable to suggest teachers are at least as prone to this as anyone; we tend to know more about the perceived benefits of growth mindset than most other people and so there’s a huge social pressure to fall into line. But just saying you have a growth mindset obviously does not mean you have one. What you actually have is a false growth mindset.
However, the problem with a theory that explains away all the objections is that it becomes unfalsifiable. There are no conditions in which the claim could not be true. If no amount of data or evidence can prove Dweck’s claims false because she can just say, “Well, that’s a false growth mindset,” not a real one, then what’s the difference between her claims and those of pseudoscience?
For instance, after Yue Li and Timothy Bates attempted to painstakingly replicate Dweck’s laboratory studies they found “no support for the idea that fixed beliefs about basic ability are harmful, or that implicit theories of intelligence play any significant role in development of cognitive ability, response to challenge, or educational attainment.” Dweck’s disappointing response was to say that Li and Bates had probably bungled their attempts to replicate her careful research: “Replication is very important, but they have to be genuine replications and thoughtful replications done by skilled people. Very few studies will replicate done by an amateur in a willy-nilly way.”
If it’s true that replicating the effects of Dweck’s studies cannot be done by amateurs “in a willy-nilly way”, then what chance does your average teacher have? This 2016 report suggests that over 80% of teachers who have implemented Dweck’s suggestions have failed to make effective changes in their classrooms. Maybe they were all amateurs proceeding in a willy nilly way? It could be the case that Dweck’s research is completely robust and yet has no practical application in the classroom. Or it could be that we just don’t know enough about children’s beliefs about effort and ability to make reliable predictions about how they behave in lessons.
To shed some light on some of the mythology surrounding mindsets, it might help to look at the research of Gary McPherson and James Renwick. In 2001 they published A Longitudinal Study of Self-regulation in Children’s Musical Practice in which they took 27 children learning to play a variety of musical instruments and tried to unpick how and why some children improved more than others over a period of years. All of the children practised: they all put in effort, they were all motivated and had good attitudes, but not all of them got better at the same rate. It’s tempting to think that the difference must have been innate ability, but actually the researchers concluded that it was the type of practice in which the children engaged that made the most difference.
The researchers counted the number of mistakes children made on first playing a piece and then compared this to the number of mistakes made on a second performance. The lowest performing student made an average of 11 mistakes a minute on her first play-through and was still making 70% of the same mistakes the second time through. The best performing student made an average of 1.4 mistakes first time round and was able to correct 8 in 10 of the these mistakes in her second rendition. McPherson and Renwick decided that the difference was due to the fact that these students had better ‘mental representations‘ of what a good performance would sound like and were able to self-check and provide their own feedback to eliminate as many mistakes as possible.
Now, of course 27 students is a very small sample size so we should be rightly sceptical of making any generalisable claims from this research. Luckily, McPherson and Renwick’s findings are supported by a large-scale study on the development of practising strategies, also published in 2001. Instead of the time-consuming approach of videoing and analysing practice sessions undertaken by McPherson & Renwick, Hallan et al relied on self-report questionnaires to make studying a much larger sample feasible. As with earlier studies, researchers found that while the quantity of practice and attitudes to learning matter, they don’t make nearly as much difference as we might hope. It was the ability to recognise one’s mistakes and then improve independently which differentiated the most accomplished students. This depended on being able to visualise what a good performance would feel like as well as sound like.
Grit and a growth mindset is not enough. In fact, it seems likely that practising more without getting results will probably erode beliefs about self-efficacy. It’s small wonder that some children learn that they “can’t do maths” or that “French is impossible” if they’re practising in the wrong way. More worrying still, if teachers believe that the difference between successful and unsuccessful students is their mindset we could be adding to a potentially toxic cocktail. It’s much more likely that a growth mindset follows from experiencing success. If we get good early results then our self-confidence can become invincible, but if we don’t… well, only a fool continues to believe anything is possible in the face of increasingly contradictory evidence.
In order to help students develop healthy beliefs about effort and hard work, we need to help them improve how they practice. We could help children understand what good looks and feels like through effective modelling. Then, we should provide the scaffolding and support required to ensure they can be successful before removing that support as soon as possible to prevent them from becoming dependent on it. Once children have experienced success then it becomes reasonable to ask them to struggle. It doesn’t matter that their successes were only possible due the support of a teacher, the point of all the support is to enable them to learn what quality performance looks and feels like.
School results are more likely to be down to the quality of what and how children are taught rather than sweeping generalisations ideas about character and personality. Rather than trying to teach things we don’t really understand, we’d be a lot better off if we focussed on teaching those things we do understand as well as possible. Maybe the best way to teach resilience is to give students challenging work to do. Maybe the best way to teach children that struggle can lead to success is to give them an experience of success before we expect them to struggle. Maybe the best way to make sure children are prepared for an uncertain future is to give them a thorough grounding in the academic content that they can then think both with and about when trying to solve new problems.
When I talk to teachers about this issues (which I do a lot) I emphasize a number of points. The first is that grit—at least as defined by Duckworth as a combination of passion and perseverance—is not a particularly useful concept for teachers. Of course people who are passionate about something, and who persevere, are likely to do well. The problem is that, as teachers, we often need to get our students to become successful at things they are not particularly passionate about. The second, related to Duckworth’s claims about the magnitude of the impact of grit on outcomes, is that when you look at a highly restricted population, such as students at the US Military Academy at West Point, then you get misleading data. In particular, if you study a sample of individuals who have been selected on the basis of their general ability, then general ability isn’t going to account for much of the variation in outcomes, in the same way that if you look at players who play at the position of center in the National Basketball Association, then height accounts for little of variation in performance, but that is because all the centers are between 6’9″ and 7’3″ in height. Across the whole population, height would be a much stronger predictor of basketball success.
As for the work of Carol Dweck, I point out that how she defines mindset has changed over the years. Her earlier work focused on fixed and incremental views of ability, and this word means different things in different contexts: it can denote something relatively fixed, as it does in the UK, for example in the context of “mixed-ability teaching” or something much more fluid, such as how able one is at doing something. The problem is that when we define ability as something fluid, then growth mindset means little more than the trivial observation that one can learn to do things one cannot currently do. In this context, it is worth noting that all children have growth mindsets. I often ask 12-year-olds, “Can you drive a car?” Most (at least in cities) say, “No”. If I ask them, “Do you think you would be able to learn to drive a car if you wanted to?” every 12-year-old I have ever asked says, “Yes.” Most 12-year-olds therefore seem to have a growth mindset about car driving. The challenge, then, is to get students believing that they can learn to do things they cannot currently do, whether it is car-driving or mathematics.
The other thing that I think is really important in interpreting the meta-analysis that Victoria Sisk and her colleagues published earlier this year on growth mindset is that much of the research analyzed involved students over the age of 10, for whom one year’s growth is 0.4 standard deviations or less. An average effect size of 0.08 for such students would be an increase of 20% in the rate of learning, which seems to me to be an educationally important effect, especially since growth mindset interventions take only a few hours per year. More generally, anyone who classifies effect sizes on the basis of their magnitudes, independent of context—for example by using Cohen’s guidelines to call an effect size of 0.2 to 0.3 as small, those from 0.4 to 0.6 as medium and 0.7 and above as large (what Russell Lenth called “T-shirt effect sizes)—is committing an act of statistical illiteracy. Effect sizes need to be interpreted in context. Standardized effect sizes that start with a decimal point and a zero can be hugely important. There is no magic single solution to improving education. Improving educational outcomes is going to be achieved by consistently implementing a large number of things that, individually, may only have a small effect, but collectively, could transform how much our students learn in school.
David,
The 80% comment re: the 2016 report is misleading, and a misrepresentation of the evidence presented, as I’m sure you know. Notwithstanding the fact that 44% of those same respondents said they had had no training on growth mindset, so Dweck’s frustrated claim that people are not implementing her work correctly may well be more credible than you choose to present here.
In terms of the single study of the single class of children learning musical instruments, where the ones who made the fewest mistakes first time around progressed at a quicker rate second time around ……. because they are at a different level of ability, clearly and so can self regulate more readily. Clearly.
I have no personal allegiance to any of the mindset research, but I know you tweet and blog prolifically, and would hope that your attempts to devalue the work of others are based on sound evidence and unbiased presentation. You have not done yourself justice here, I’m afraid.
How do you think I’ve misrepresented the evidence? Why is the 80% figure misleading? The fact that 44% had no specific training just adds weight to my suggestion that it’s really hard to get the kinds of results Dweck and team get.
But anyway, let’s assume that single data point is misleading, what do you make of all the other evidence I’ve presented? Is it *all* misleading?
[…] feel successful. I don’t understand all there is to know about mindsets, either, but I saw this post this morning and it resonated with me. I think helping kids see things in new ways–and […]
David, your final paragraph pretty much mirrors my entire thoughts on Mindset theory.
As a teacher of ten years, I have been subjected to numerous “flavour of the month” harebrained schemes to improve results. Nearly all with very little thought given on how to measure the impact in your school, let alone look at the actual evidence available on whether they work or not.
Hopefully we can file this one alongside VAK and move on.