17/01/2013

The 'Nonexpert' Challenge

When there is a public debate about a (social) science topic, pitting proponents of view A against proponents of view B, you sometimes encounter a variant of the following statement by someone promoting view A: "The specific person who holds that B is correct has never published original analyses concerning the topic in question in the peer-reviewed literature."

Let's note in passing that people who know the peer-review process first hand are often not as impressed by this institution as many journalists. More importantly, I am not so sure that not having published extensively about a topic makes your view less credible. If you do have a history of publication on the topic, you will probably have a better grasp of the facts, including methodological challenges brought about by the kind of data that we're talking about.

On the other hand, if you don't have such a history, you have some advantages. First, you can look at the issues with a fresh eye. You may, for example, see issues that people trained in a certain field can't, because that is not germane to the way one looks at things in a that field. Second, you are less subject to pressures that come with being a researcher in a field. This comes in two flavours - social disapproval (no one wants to have a beer with you after the conference) and professional disadvantages (no grant money for you). Third, long-standing researchers went on the record long ago with their view on whether A or B is correct. It is painful to state in public that you were wrong. A newbie does not have this problem.

Given this, it's not clear to me that an author's lack of a history of publication should signal reduced credibility of his views. It seems quite clear, though, that if the only rebuttal to A that B-proponents can come up with is that A-proponents have no history of publication in the field, this is a strong signal that there is a lot to be said for A. If that's all B-proponents offer, this suggests they have no good replies to the actual arguments in question.

While I'm at it, let me note that stating (perhaps correctly) that a certain view is "divisive" says nothing about its veracity, just as the claim that being a Christian improves your behaviour says nothing about the existence of god. 

No comments: