Appreciative Thinking and Cross-Sectional Studies

One of Seth Roberts’ favourite topics is that in scientific discourse there is too little appreciative thinking. As he uses the term, it’s about acknowledging what’s good about a paper/book/presentation/etc. and complements critical thinking, which is about finding weaknesses. (For more detail, see his recent post.)

I’ve written before that I think this is in part due to the way discourse in the sciences works; I believe the ratio of appreciative to critical thoughts is higher than the ratio of appreciative to critical utterances. Having said that, I think there is also an imbalance as far as thoughts are concerned.

Exhibit A is myself. More than once have I said about a cross-sectional study*: “Bah! This correlation can be explained by at least two competing hypotheses, and this finding cannot distinguish between the two.” For example, there are lots of studies which find that delinquent boys have more delinquent friends than nondelinquent boys. (Results for girls are mixed.) Leaving aside the problem of possible confounding factors, this finding can be explained by (at least) two hypotheses:

H1: A person’s friends’ behaviours influence the person’s behaviours.

H2: A person is more likely to become friends with others who show similar behaviours than with others who show dissimilar behaviours (“homophilic selection”).

But this doesn’t mean that the cross-sectional study is useless. After all, if no or even a negative correlation had been found, this would suggest that neither of the hypotheses is true.** Finding a positive correlation suggests at least one of them is correct. The logical next step is to design a bit of research which allows one to distinguish between the two, such as a longitudinal study.

There is an important point in this context that tends to get overlooked in textbooks: Not all kinds of research are equally expensive – and as a rule longitudinal studies are more expensive than cross-sectional ones. Hence it makes sense to first devise a cross-sectional study which is designed to see whether the correlation is there at all. If it’s not, you may want to use the money you had set aside for the more in-depth study for something else.

As they say, you live, you learn.

*Sometimes “correlational studies” is used as a synonym for “cross-sectional studies”. This is unfortunate because longitudinal studies, too, use correlations.

**suggest! It is quite possible to construct scenarios in which one or both hypotheses are true, yet you do not find a correlation.

(An earlier post about the correlation-not-causation problem and an addendum)