Writing about Educational Achievement Requires Basic Maths Skills

In NPR, Samuel A. Abrams writes about what the U.S. need to do to improve the performance of their schools (via). As is standard procedure, he looks at how the Finns do things differently. That's because the Finns outperform the U.S. (and almost everybody else) on the PISA test. Abrams' reliance on the test as a measure of educational performance is kinda weird, given that he also suggests standardized testing is a bad idea, but never mind. He discusses a number of ways in which the Finnish system differs from the U.S.'s and suggests making American more like Finnish education would improve U.S. performance.

Then he senses that this kind of analysis may not be all that trustworthy:

The reflexive critique of comparing the Finnish and U.S. educational systems is to say that Finland's PISA results are consequences of the country being a much smaller, more homogeneous nation (5.3 million people, only 4 percent of whom are foreign-born). How could it possibly offer lessons to a country the size of the United States? The answer is next door. Norway is also small (4.8 million people) and nearly as homogeneous (10 percent foreign-born), but it is more akin to the United States than to Finland in its approach to education: Teachers don't need master's degrees; high school teachers with 15 years of experience earn only 70 percent of what fellow university graduates make; and in 2006, authorities implemented a national system of standardized testing. The need for talent in the classroom is now so great that the Norwegian government is spending $3.3 million on an ad campaign to attract people to teaching and, last year, launched its own version of Teach for America in collaboration with Statoil — called Teach First Norway — to recruit teachers of math and science.

Moreover, much as in the United States, classes in Norway are typically too large and equipment too scarce to run science labs. A science teacher at a middle school in Oslo told me that labs are unfortunately the exception, not the rule, and that she couldn't recall doing any labs as a student a decade ago. Unsurprisingly, much as in 2000, 2003, and 2006, Norway in 2009 posted mediocre PISA scores, indicating that it is not necessarily size and homogeneity but, rather, policy choices that lead to a country's educational success.

I highlighted the bits about immigrants that I found particularly iffy. I found them particularly iffy because (i) here in Germany, it is received wisdom that a high number of immigrants in the classroom is going to drive down achievement (which is plausible on both compositional and contextual grounds), (ii) 10% is 2.5 times as much as 4% and (iii) Norway's foreign-born rate is much closer to the United States', which is 12.9%, so the "nearly as homogenous" argument doesn't hold water.

Let's run a simple linear bivariate regression to further assess the claim. I calculated overall PISA scores by adding the scores for the three fields reading, mathematics, and science, taken from Abrams' link, and regressed them on the rate of foreign-borns. This yields the following equation:

SCORE = 1691.1 -16.8*(% FOREIGN-BORN)

In this equation, each percentage point increase in the foreign-born population decreases the score by 16.8%. If we take this result at face value, we should expect, on the basis of differences in immigration alone, Finland's result to be 143.3 points better than the United States'. That's almost exactly what we observe (1631 vs. 1489 points). This nice fit is unsurprising given that the regression statistically explains almost 94% of the variance.

Of course, I would not want to present this regression as a good measure of the causal effect of immigration on test scores. I would like to point out, though, that if you cannot distinguish the numbers 4 and 10, or think that 10 is closer to 4 than to 12.9, perhaps you should not try to teach others about educational attainment. Abrams unfortunately is writing a book on the matter. I think I will pass.