A Weakness That May Be a Strength

Steve Sailer breaks down US results on the PISA test by race; perhaps more interestingly, he adds a useful discussion of the limitations of the test. Most notably (formatting omitted):

Third: how motivated students are to work hard on the test is a significant imponderable. Administrators and teachers are supposed to motivate the students, but not threaten them. So, at least in theory, PISA is a low-stakes test for students. Doing badly on it isn’t supposed to hurt them. Which means the temptation to slack off and "bubble in" some of the answers is always there.

This is, of course, a problem with many such tests. How much of a problem it is for PISA, however, depends in part on what you want to know. If motivation to do well on the test is a reasonabled measure of conscientiousness more generally, then the results may measure, in effect, an interaction between conscientiousness and academic ability. If your goal is to predict a nation's future economic well-being, then the actual results may be more useful than if the test measured academic ability only.

Of course, there are other factors influencing both test results and a nation's well-being above and beyond human capital.