While governments anxiously await the results of the latest international student assessments (Pisa), the inventor of the tests defends their growing influence
It’s normally parents, teachers and pupils who anxiously await exam results. But now it’s the turn of education ministers to bite their nails. On 3 December, the Organisation for Economic Co-operation and Development (OECD) releases the results of its latest triennial tests, carried out in 2012 on 510,000 15-year-olds in 66 countries. They will tell us which countries perform best in reading, maths and science. The previous tests, carried out in 2009, placed England 25th, 27th and 16th respectively, hardly a stellar performance.
Paradoxically, Michael Gove, the education secretary, probably won’t jump for joy if there’s a dramatic improvement. The cohort that took last year’s tests had received all but the latest two years of their schooling under Labour. If they did well, parents and teachers may ask, why does he need to reform schools, curriculum and exams so drastically?
The man marking the papers is Andreas Schleicher, an intense, white-haired, 49-year-old German, who speaks perfect English at bewildering speed. At the OECD, where he is special adviser on education policy, he invented and still runs the Programme for International Student Assessment (Pisa), as the international testing of 15-year-olds is called. Another of his babies, the first international survey of adult skills, has already delivered bad news for the English: their young adults, aged 16 to 24, rank among the lowest in the industrialised world for literacy and numeracy. He doesn’t mince his words. “Deficiencies in the UK’s school systems,” he wrote last year, will lead to £4.5tn in lost economic output over a lifetime, “the equivalent of a permanent recession”.
America’s Atlantic magazine calls him “the world’s schoolmaster”, while Gove has described him as “the most important man in English education” and “the father of more revolutions than any German since Karl Marx”. By some estimates, half the countries that have taken Pisa tests since they started in 2000 have reformed their education systems in light of the results. Germany’s poor performance in the first tests shocked the country to its core. All but one state introduced an academic school-leaving exam, several abandoned the traditional German school half-day and others modified the stratified secondary schooling system. The US embarked on a nationwide development of “common core standards”. Even Japan, generally a high performer, called crisis talks when it slipped somewhat in 2009.
Schleicher has no qualms about the influence he wields. He believes, with a passion, that hard evidence can improve education for all children. “Our statistical model,” he told me when we met at the OECD’s headquarters in Paris, “accounts for 85% of the performance variation between schools in the countries we study.” He recently started a series of reports on individual countries, issues frequent bulletins on what works and what doesn’t (“the disciplinary climate in schools is strongly related to student performance,” he states sternly but unsurprisingly) and plans next year to offer schools a test that allows them to measure themselves against others across the world. His catchphrase, displayed in a continuously rolling ticker as he ends his lectures, is “without data, you are just another person with an opinion”.
But does Pisa have sufficiently robust data to justify its growing weight? Many critics think not. For one thing, the tests don’t work as people think they work. You would expect all pupils to answer the same questions. In fact, according to an analysis by Copenhagen University in Denmark, only 10% of those who took part in Pisa 2006 were tested on all 28 reading questions, and about half weren’t tested on reading at all. The OECD feeds real scores into a statistical device called the Rasch model so that it can work out “plausible values” for children who weren’t tested. Schleicher says: “We want to test lots of different things but we have limited time. So we give the students different tests with overlapping content.” He says it’s a long-established statistical technique, enhanced by modern technology. But some statisticians insist it can’t work for Pisa, because different test items work differently in different countries. Pisa’s league tables, they say, are almost meaningless. In 2006, the UK could have finished anywhere between 14th and 30th on reading, Canada anywhere between second and 25th, Japan anywhere between eighth and 40th.
Schleicher says margins of error are shown with the league tables. “The most plausible inference from the 2009 results was that the UK had dropped, but there was too much uncertainty to say for sure.” That wasn’t how Gove put it, I said. Under Labour, Pisa scores had gone “down, down, down”, he told MPs. Given that politicians and journalists never pay attention to the caveats, shouldn’t the OECD stop publishing the tables? “We publish 800 to 1,000 pages of analysis and just 10 pages of tables,” Schleicher replies. “We don’t attach that much importance to them, but people want to see comparisons. If we don’t know which school systems are doing best, how can we learn from them?”
A second objection to Pisa is that it can’t take account of social, economic and cultural differences. Can it make sense to compare, say, Peru, which has high levels of child labour and limited internet access, with western European countries? Can UK schools really learn anything from east Asian countries, with their deeply ingrained respect for authority? Or from Finland (top of nearly all international league tables for educational performance) where, in living memory, people could not in effect marry without passing a literacy test set by the Lutheran church? Aren’t high levels of economic inequality the biggest obstacle for many schools in the UK and the US?
“Sure, culture makes a difference,” Schleicher says. “But look at Poland. In 2000, its performance was average to below-average. Now, it does very well. They didn’t change the culture, they changed what they did in education. They had vocational schools which were dumping grounds. They got rid of them and, surprise, the low-performing pupils did better. Even bigger surprise, the high-performing ones did better as well. Shanghai, in 2003, was mediocre. They didn’t change the culture. They changed teachers’ career paths so that, to get promotion, they had to work in tough, low-performing schools. In Finland, the Lutheran church has existed for a long time. But our survey of adult skills showed that, in complete contrast to the UK, their young people greatly outperformed older people.”
As for economic inequality, Schleicher says it’s too easy an excuse. “Some countries are very good at moderating the effects of inequality. Others, such as France, exacerbate the effects.”
Schleicher is a pacifist – “we should construct the world, not destroy it,” he says – who, against his parents’ strong opposition, refused military service and opted instead for social service, teaching handicapped children. That was his only experience of teaching, but much of his passion for improving schools is influenced by his own experiences. His father was a professor of education and his mother a doctor, but Schleicher did so badly during his early schooling in Hamburg that he didn’t even make it to grammar school. Only when he went to a private school, and then only in the last year, did he become a high performer, winning a national science prize. “Some teachers really engaged me, which hadn’t been the case before,” he says.
He went on to Hamburg University to take a physics degree, followed by an MSc in maths at Deakin University in Melbourne, Australia. His first job was in the medical industry, researching the applications of nuclear magnetic resonance. But his interest in educational research had been aroused by a British professor at Hamburg, Neville Postlethwaite, one of whose lectures he attended by chance. He moved to the Institute for Educational Research in Holland to work on the initial design of the Trends in International Mathematics and Science Study (Timms), which unlike Pisa – largely a measure of 15-year-olds’ ability to solve real-world problems – tests straightforward subject knowledge.
He moved to the OECD in 1994. At that time, it measured inputs, such as spending on schools, but not outputs. Schleicher persuaded the OECD to seek more rigorous data. Many education ministers were reluctant, but were told that, if they didn’t monitor the knowledge and skills of the next generation, their economic competitiveness would suffer. The biggest sceptic was probably Schleicher’s father, who believed education should be about human qualities, not things you could measure.
The first Pisa results caused a sensation in several countries, particularly the US which found itself below neighbouring Canada. Yet Schleicher, who lives in Paris with his Italian wife, also an educational researcher, reserves his most dismissive comments for the school system to which he entrusts his own children, aged 17, 15 and 11. Is he satisfied with the French state schools they attend, I ask. “No,” he replies. “It is one of the most backward school systems. It is very much rote learning.” He says he kept his children in the system because he didn’t want to isolate them.
Despite his enthusiasm for hard data, I suspect Schleicher is, in educational terms, a bit of a trendy, at least in his views on curriculum and teaching styles, if not on school governance. I asked how many marks out of 10 he would award to Gove. He predictably avoided a direct answer but, while praising the education secretary for “giving schools more levels of responsibility”, said “students need to apply knowledge and some areas of the curriculum have gone in the other direction, making it more focused on facts and figures”. He has also, in the past, opposed performance-related pay, reported that private schools do no better than state schools when social background is taken into account, and argued that “the future is about user-generated wisdom … and personalising educational experiences”.
But none of that is likely to mollify Pisa’s critics. As they see it, Schleicher’s work threatens a global standardisation of education, wiping out school systems that were embedded in diverse local cultures, values and traditions. “The very meaning of public education is being recast,” write the American academics Heinz-Dieter Meyer and Aaron Benavot, editors of Pisa, Power and Policy, a collection of learned papers published this year, “from a project aimed at forming national citizens and nurturing social solidarity to a project driven by economic demands.” Schools, they argue, are increasingly “subject to the imperatives of efficiency, calculability, predictability and control”.
Until recently, almost nobody questioned the merits of Pisa and its league tables, only their interpretation. Now, as the world’s schoolmaster announces the latest planetary exam results, we can expect the political arguments to intensify.
Link to article: feeds.theguardian.com/c/34708/f/663889/s/341b1dba/sc/7/l/0L0Stheguardian0N0Ceducation0C20A130Cnov0C260Cpisa0Einternational0Estudent0Etests0Eoecd/story01.htm