19/03/2009

p > .05

Aleks Jakulin links to the open-access psychology Journal of Articles in Support of the Null Hypothesis. From the table of contents:
Playing Video Games Does Not Make for Better Visual Attention Skills

False Recall Does Not Increase When Words are Presented in a Gender-Congruent Voice

G.I. Average Joe: The Clothes do Not Necessarily Make the Man

Inconsistent Mood Congruent Effects in Lexical Decision Experiments
And my favourite:
Fear Primes May not Affect Women's Implicit and Explicit Mate Preferences
His commenters link to similar journals in other disciplines.

I like it. At least when exploring new stuff, articles that don't reach conventional significance levels are less likely to get published. As scientists have a huge incentive to get published, this should lead them to play it safe and look at associations they can be reasonably sure will yield significant results. In many (by no means all) cases, however, the riskier topics to look at will also be more interesting. What such journals do is reduce the disincentive to look into matters which are interesting but risky in terms of publishability.

Also, if negative results are published, they may keep others from studying the same matter. Given scarce resources, that's a good thing.

I've long thought that there should be something like a Journal for Attempts to Exactly Replicate Results in the Social Sciences. In first semester they tell you that scientists produce some finding and then lots of others try to replicate it (or rather, try to fail to replicate it). In reality that rarely happens. In fact, replication is in such low regard that one of the few articles I read which tried to replicate a previous finding (abstract) contained lengthy justifications for its rationale.

Generally, originality is overrated in academia.

2 comments:

pj said...

I wonder whether there might now be a gap in the market opening for 'results which are a bit inconsistent, and while not completely null are certainly not resoundingly positive'?

Tor Hershman said...

Some of them thar womies sure do likez to mate real good like.