The basic idea is simple, the test taker is asked to categorize a series of faces, hitting a right hand key for a white face and a left hand key for a black face. Then the taker must similarly categorize a series of words as good or bad, words like wonderful, nasty, peace, hate etc.In a new paper called "Strong Claims and Weak Evidence: Reassessing the Predictive Validity of the IAT"*, Hart Blanton and five coauthors look at the ability of measures of racial bias derived from the Implicit Association Test to predict people's discriminatory behaviour. The abstract:
Now here is where it gets interesting. The next list contains both faces and words and the test taker is asked to hit a right hand key if the word is either good or the face is white or to hit a left hand key if the word is either bad or the face black. Finally, the same task is performed but now the test taker must categorize together good words and black faces and bad words and white faces. The test taker is asked to do the test as fast as possible.
Bias is revealed, so the argument goes, if response time is faster when good words must be paired with white faces and bad words paired with black faces than the reverse.
The authors reanalyzed data from 2 influential studies—A. R. McConnell and J. M. Leibold (2001) and J. C. Ziegert and P. J. Hanges (2005)—that explore links between implicit bias and discriminatory behavior and that have been invoked to support strong claims about the predictive validity of the Implicit Association Test. In both of these studies, the inclusion of race Implicit Association Test scores in regression models reduced prediction errors by only tiny amounts, and Implicit Association Test scores did not permit prediction of individual-level behaviors. Furthermore, the results were not robust when the impact of rater reliability, statistical specifications, and/or outliers were taken into account, and reanalysis of A. R. McConnell & J. M. Leibold (2001) revealed a pattern of behavior consistent with a pro-Black behavioral bias, rather than the anti-Black bias suggested in the original study.Not mentioned in the abstract is the fact that the authors were, as the polite way of putting it goes, unable to obtain most of the data sets they asked for (see p. 570, Table 1), which should radically reduce our confidence in the conclusions of the studies in question. And that's even before getting to such lofty questions as what the IAT actually measures or whether mock employment experiments such as the one conducted by Ziegert & Hanes (2005) can claim any external validity in a country in which your organisation can be sued if you don't hire enough minority applicants.
As Blanton et al. briefly note (p. 580), we should not be surprised if the IAT does not predict behaviour well. It has been known since the 1970s that explicit measures of attitudes toward broad constructs (e.g., African Americans; nuclear power) poorly predict specific behaviours (e.g., agreeing to have your photograph taken with a particular African American on Sunday 14th, when your parents are present; giving money to an anti-nuclear power initiative in September); what you want to do to increase predictive validity is to match the attitude object to the behaviour you want to predict (i.e., if you want to predict whether people will give money to an anti-nuclear power initiative in September, ask about exactly that) (see Ajzen and Fishbein 2005: 174-85 [Google Books]). It is surprising that, whatever the IAT exactly measures, this lesson had been completely forgotten in the context of the IAT.
The pointer is from Tyler Cowen. Here is an earlier post of mine on the poor availability of data in psychology and what to do about it. You can take an IAT yourself here.
______
*The link is to a .pdf. You may have problems viewing this document in your browser. Downloading and opening should work, although you definitely need an Acrobat Reader version newer than 5.0.
No comments:
Post a Comment