The Chartered College of Teaching has announced that as part of its subscription fee, it will offer access to around 1700 education research journals, similar to a deal Scottish teachers already receive. Let me say up front that I believe (nearly) all publicly funded research should be available to the public, and education research journals are no exception. There are clearly benefits to providing access to research for teachers, but there are also risks that I believe haven’t been considered. Let’s start with a little brainteaser.
The average heights of adult men and women in the UK are 5 ft 10 in. and 5 ft 4 in, respectively.
A scientist picks a random sample of either men or women from the population.
Which of these two options is more likely to be a sample of men?
- Sample A consisting of a single person whose height is 5 ft 10 in. or
- Sample B consisting of 6 people whose average height is 5 ft 8 in.
This is a variation on a brainteaser posed by the famous psychologists Daniel Kahneman and Amos Tversky. About 75% of the participants in the study thought that sample A is more likely to be a sample of men than sample B, and they thought on average sample A was about 3x as likely to be a sample of men than sample B. But actually, sample B is about twice as likely to be a sample of men than sample A.
Variations of this sort of statistical brainteaser pop up in education and psychology research all the time. And how many teachers are equipped with the mathematical and statistical tools necessary to understand it? That’s not a dig at teachers: when Kahneman and Tversky posed the same problem to statisticians they got it wrong, too. Indeed, they came up with this question because they had fallen for it themselves.
The point is, research is hard. Evaluating research is a skill gained from practice. In most cases it takes years to acquire it. Even when you are skilled, it’s easy to make mistakes. We cannot expect classroom teachers to possess this skill, nor insist they obtain it.
When I started my PhD, it took me a long time to understand what reading a research paper was all about. It took me time to realise that a paper is just an argument, and that every sentence could be either right of wrong; that every claim should be scrutinised rather than accepted; and that every author, no matter how famous or revered, is just as fallible as the next.
By definition, research is cutting-edge, on the limits of human knowledge. Each idea is tentative, waiting to be scrutinised for weaknesses and refined. If we don’t have the ability to scrutinise research properly and simply take it at face value, we are giving it more credit than it’s due.
A good example of this phenomena in action is Growth Mindset. How many schools now have growth mindset posters plastered all over the walls? How many minutes of each meeting are spent discussing how to instil a growth mindset in our pupils? How many ancillary, life skills, or PSHE lessons have been dedicated to growth mindset?
And yet, the wheels are falling off: Researchers who have tried to replicate Dweck and others’ work on growth mindset have failed to do so. Other researchers have found numerous statistical and mathematical errors in the original paper, uncovering irregularities in the reporting of some of the studies’ methods.
None of this is to say that growth mindset is wrong, or false. It may well turn out to be everything it promises, and more. But at this stage, we just don’t know. There are questions that need to be answered, and problems that need to be solved before growth mindset should be centrepiece on the agenda of any school.
Without properly scrutinising research, confirmation bias will reign supreme: given the primacy of education research, you can find a paper that supports practically any point of view, and it’s possible to find two papers that offer diametrically opposite viewpoints. People will simply cherry-pick the research that supports their point of view and ignore the rest.
If we believe that simply providing access to education research is the answer to education’s ills, rather than also providing the tools to interrogate that research, the result will simply be an escape from the educational fads of the past – men in suits bearing brain training products – into the clutches of educational fads of the future – academic articles dressed up in mathematics and statistics.
The Royal Society is the world’s oldest learned scientific society still in operation. It counts the likes of Isaac Newton, Charles Darwin, Albert Einstein, and Francis Crick amongst its Fellows. It’s a prestigious and exclusive club: there are only about 1600 living fellows today, and there have only been about 8000 since its formation in 1660. At times like these, I am reminded of its motto: nullius in verba; take nobody’s word for it.