There’s a new post on Watts Up With That (WUWT) called peer review – the rotten business model of modern science. Given that one of the arguments in the post is that the anonymity of the reviewer gives them power over the author, it is somewhat ironically written under what I assume is a pseudonym (Abzats). Far be it for me to criticise anyone for remaining anonymous though.
In fairness, Abzats has a point. There are indeed problems with peer-review. In fact I’d go further and say that there are issues across academia and across the research community. What we value and how we decide who to fund and who is excellent is becoming more and more based on simplistic metrics and vague assessments of impact. Abzats’s main issue, however, appears to be that anonymous reviewers can do immense damage to someone’s career and can prevent good bits of work from being published.
Well, maybe but I doubt it’s that severe. Editors are also involved and if you believe your work has value, you can request another reviewer or switch to a different journal. Not ideal maybe, but the idea that a single anonymous reviewer can absolutely stymie someone’s career seems a bit extreme.
So what is Abzats’s solution. It’s appear to be simply to ban peer review. Really? That’s it? Ban peer-review? I know that peer-review isn’t perfect, but is that really a solution. I’m sure there must be viable alternatives that maintain elements of the peer-review process but minimises the issues associated with the current system. To be honest, I don’t really have a good solution of my own, but I’m pretty convinced that banning it isn’t really a particularly good one.
There was, however, one comment that I found quite interesting. It was by someone called Lance Wallace, and I’ll quote most of the first two paragraphs
I’ve been reviewing papers in my field for about 30 years, probably about 5-10 per year. Most of the time I believe I either helped make the paper better or else I saved the journal from publishing a below-par paper. I have some evidence for the first case, since authors have sometimes given the anonymous peer reviewers credit for having improved their paper. The many people in my field that I know personally have much the same experience as I do.
However, the climate science field is different. Here we have rather naive generalist scientists who happened to strike it rich in terms of public awareness, savvy politicos sensing an opportunity, a corrupt UN seeking global power, powerful NGOs like WWF and Greenpeace riding the wave, administrator-heavy scientific organizations seeing ways to funnel money to their membership, poor island nations seeing a way to shame richer ones, and just in general a perfect storm of chance variables coming together. Peer review became pal review for this branch of science.
So I started reading this and found the first paragraph quite balanced and thoughtful. I then got to the second, and simply found a diatribe against climate scientists, the UN, small island nations, essentially anyone who has not denounced anthropogenic global warming. Although Abzats doesn’t actually apply his criticism of peer-review to climate science, I assume it is this link that motivated Anthony to publish this post. The issue I have is that the problems with peer-review, research funding and with research careers is not unique to climate science. It is a problem across almost all academic research areas.
There’s no evidence – as far as I’m aware – that there is any particular problems with climate science. If anything, being such a heavily scrutinised research area it may well be the least likely to have any major problems. I suspect climate scientist are now being very careful about how they review other papers and how they assess research proposals and job applications.
So, yes there are problems with peer-review. No, it’s not unique to climate science. Banning peer-review is almost certainly not a viable solution, especially if what you’re trying to do is fix imaginary problems in climate research.