Watts Up With That (WUWT) has another post about John Cook’s consensus paper. I must admit that I find this all a little odd. In my experience, if people think a paper is ridiculously wrong, they typically just ignore it. That this paper has generated so much interest in certain quarters certainly implies that it must have hit some kind of nerve. Also in my experience, a paper that had generated as much interest as Cook et al. would be regarded by most as a success, even if it turned out to be wrong. I’m not suggesting that people encourage work that’s wrong; simply that even if such a paper turned out to be wrong, the interest it generated would almost certainly have lead to an improved understanding of the subject and hence the contribution would be seen as positive. Of course, I don’t work in a field where people try to tear papers down simply because the results don’t suit their political ideology.
Anyway, the most recent WUWT post is from Bjorn Lomborg’s facebook page and is called Cook’s 97% consensus paper crumbles upon examination. So, what does Bjorn say? Well he starts with
Virtually everyone I know in the debate would automatically be included in the 97% (including me, but also many, much more skeptical).
Well, Bjorn, the Cook et al. paper surveyed abstracts of peer-reviewed, scientific papers. The 97% in the Cook et al. refers to the percentage of abstracts that stated a position with respect to anthropogenic global warming (AGW), that endorsed AGW. You can’t be part of Cook et als. 97% unless you’re an abstract. Given that I believe you’re actually a human being, you aren’t part of Cook et als. 97%.
Bjorn goes on to say
They put people who agree into three different bins — 1.6% that explicitly endorse global warming with numbers, 23% that explicitly endorse global warming without numbers and then 74% that “implicitly endorse” because they’re looking at other issues with global warming that must mean they agree with human-caused global warming.
Voila, you got about 97% (actually here 98%, but because the authors haven’t released the numbers themselves, we have to rely on other quantitative assessments).
As I mentioned above, it is abstracts not people, but let’s not go through that again. You seem confused about where the 97% comes from because supposedly the authors haven’t released the numbers. Let me see if the paper helps. The Cook et al. paper says that their search of the Web of Science database resulted in 12465 papers that satisfied their search criteria. They removed 521 because they either weren’t climate related, weren’t peer-reviewed, or didn’t have an abstract. They then rated the remaining 11944 according to whether the abstract endorsed AGW, rejected AGW, or stated no position with respect to AGW.
The results of the abstracts ratings were that 3896 abstracts endorsed AGW, 78 rejected AGW, 40 were uncertain about AGW, and 7930 had no position with respect to AGW. Therefore a total of 4014 abstracts made some statement with respect to AGW. The fraction that endorsed AGW was 3896/4014 = 0.971. If I multiply this by 100, I get 97.1 %. There you go. Straight from the paper and really not all that complicated.
Bjorn then adds that
Now, Richard Tol has tried to replicate their study and it turns out they have done pretty much everything wrong. And they don’t want to release the data so anyone else can check it. Outrageous.
No Richard hasn’t tried to replicate the study in any way whatsoever. What Richard has done is run various statistical tests to show that there are problems with the Cook et al. paper. I also wrote the previous sentence quite carefully as it does really seem as though Richard’s goal is not to establish if there are problems with the Cook et al. paper. It is to show that there are problems with the Cook et al. paper. I’ve written a number of times about Richard Tol’s attempt to discredit the Cook et al. survey (here and here, for example) and as far as I’m concerned the tests he’s running aren’t really suitable for what he is trying to do and he isn’t really showing anything of any particular significance.
Bjorn’s post finishes with a letter Richard Tol has written to the Vice Chancellor of Queensland University complaining about John Cook not releasing all the data when Richard wanted it (despite the fact that replication was quite possible simply with the information in the paper itself). I find this absolutely remarkable. It does seem, however, that Richard has form. Seems like Richard gets upset when he doesn’t get his own way and then writes letters to people’s “bosses” complaining that they haven’t done as he wanted. It’s a good thing that the people Richard Tol annoys don’t write to his university, as the mail room there would be rather inundated if they did.
Anyway, Bjorn, most of what you say appears to be largely nonsense. The Cook et al. paper certainly doesn’t crumble under the analysis you’ve done. Why don’t you try reading the paper before complaining that it’s all wrong.
Given that I have been accused of being one of the defenders of the Cook et al. paper, I have become a little concerned that maybe some fundamental flaw will be identified and that I will then never hear the end of it. I wasn’t involved in the paper in any way whatsoever, so I obviously don’t know if the analysis was done in a completely appropriate way. I’m defending it because it is a published paper that is consistent with earlier work and because it is technically possible to replicate it (despite what Richard might say) given the information in the paper. For example, it only took me a few minutes to extract the abstracts from the Web of Science database.
Given that I wasn’t involved, I’ve been doing some abstract rating of my own to see if I can replicate the Cook et al. results. So far I’ve rated 133 abstracts and the results are
Endorse – 50
No position – 83
Reject – 0
So my basic result is that 37.5% of the abstracts I’ve rated endorse AGW, 62.4% have no position and none reject AGW. The results in the paper were, 32.6%, 66.4%, and 1% (I’ve added reject and uncertain to get the 1%). So, maybe I should have found 1 reject/uncertain abstract by now but the fact that I haven’t isn’t all that surprising. Also, the other percentages aren’t exactly the same, but the difference isn’t surprising given that I’ve only rated 133 and the paper rates 11944. I should add that I haven’t done this to convince anyone but myself that the Cook et al. survey results seem reasonable. I’ll probably carry on for a little while longer, but I’m reasonably convinced, now, that the Cook et al. results are reasonable.