"Violent Cultures" and Aggression on the Football Pitch

Remember the article showing that diplomats from more corrupt countries are less likely to pay their parking tickets? Now Dani Rodrik links to a paper by Edward Miguel, Sebastián M. Saiegh and Shanker Satyanath entitled "National Cultures and Soccer Violence", which aims at something similar. Here's the abstract:

Can some acts of violence be explained by a society’s “culture”? Scholars have found it hard to empirically disentangle the effects of culture, legal institutions, and poverty in driving violence. We address this problem by exploiting a natural experiment offered by the presence of thousands of international soccer (football) players in the European professional leagues. We find a strong relationship between the history of civil conflict in a player’s home country and his propensity to behave violently on the soccer field, as measured by yellow and red cards. This link is robust to region fixed effects, country characteristics, and player characteristics (e.g., age, field position, market value). Reinforcing our claim that we isolate cultures of violence rather than simple rule-breaking or something else entirely, there is no meaningful correlation between a player’s home country civil war history and regular (no-card) fouls earned or goals scored.

I see a number of problems, some of which were already mentioned by commenters at the blog post linked to above.

1. The authors do cite evidence showing that "civil conflict" is followed by more violent crime, but treating "years of conflict" as a measure of culture - a vague and broad concept - is taking it a bit far.

2. Generally, the authors seem to have a poor grasp of football. For example, they write: "A player who receives a yellow card continues to play in the match, yet the yellow card serves as the first and last warning." Incorrect; anyone who watches football regularly knows that players who are already booked will often be verbally cautioned by refs that they're close to being sent off. More importantly, they also write: "In some cases, a yellow card may be awarded for persistent fouling, or for non-violent forms of unsporting behavior, for instance, disobeying an explicit order given by the referee. However, in practice the vast majority of cards are granted for flagrantly hard fouls." I'd like to see some numbers on this claim; I would guess that about a fifth of bookings are handed out for "non-violent forms of unsporting behaviour". Note, however, that this biases the results against the authors' hypothesis.

3. The authors write: "As mentioned, actual crime rates are unsatisfactory as measures of a 'culture of violence', since individuals’ real-world actions plausibly reflect the combined influence of legal institutions and economic factors, in addition to cultural norms." (pp. 2-3) Is that not true of "civil conflict"? (They later give better reasons for not using crime rates, yet this point reinforces the argument that the authors' measure of culture is poor.)

4. This one's hard to believe. The authors write: "The magnitude [of the association] is quite large [...]. The predicted number of yellow cards for [an African player in the French league] increases by 3.6 percent when civil conflict prevalence in his home country increases by one standard deviation, or 4 years" (pp. 8-9). You're calling that "large"?

5. The authors do not control for the quality of the team the player is on. I have not crunched the numbers on this but am pretty sure that holding player quality (which the authors control for) constant, players on lower-quality teams commit more fouls due to a) weaker teams having possession of the ball less often combined with b) players being much more likely to commit a foul when their team is not in possession.

6. The authors do control for exposure time by including variables on matches started and matches come on as a substitue (minutes on the pitch would have been better), but if their hypothesis were right, we should expect an interaction effect between these and the "years of conflict" variable. (Whatever your "culture", you're unlikely to get booked when you're not on the pitch.) They do not test for this.

7. When Colombia (an outlier) is excluded from the analysis, the results are not significant at the conventional 5% level anymore (p. 10). In other words, the result isn't robust, despite the large sample size. This isn't surprising to anyone who has seen this graph depicting the bivariate association between the central variables.

8. The association between years of conflict and red cards received - a much better measure of violent behaviour (see pt. 2) - is not significant at the 5% level (p. 11). In fairness, red cards are rare, which makes it hard to find an association.

9. The authors find no association of years of conflict with fouls that do not lead to a booking (p. 11). Strangely, they interpret this as supporting their hypothesis; I draw the opposite conclusion.

Bottom line: A paper which operationalizes both the dependent and the main independent variable poorly, uses a questionable statistical model and finds a significant association in only one out of three tests of the hypothesis - an association which turns out not to be robust.