Arguments
The Cook et al. (2013) 97% consensus result is robust
What the science says...
The 97% consensus has been independently confirmed by a number of different approaches and lines of evidence.
Climate Myth...
97% consensus on human-caused global warming has been disproven
Cooks ’97% consensus’ disproven by a new peer reviewed paper showing major math errors (Anthony Watts)
Communicating the expert consensus is very important in terms of increasing public awareness of human-caused climate change and support for climate solutions. Thus it's perhaps not surprising that Cook et al. (2013) and its 97% consensus result have been the subject of extensive denial among the usual climate contrarian suspects. After all, the fossil fuel industry, right-wing think tanks, and climate contrarians have been engaged in a disinformation campaign regarding the expert climate consensus for over two decades. For example, Western Fuels Association conducted a half-million dollar campaign in 1991 designed to ‘reposition global warming as theory (not fact).’
The 97% Consensus is a Robust Result
Nevertheless, the existence of the expert consensus on human-caused global warming is a reality, as is clear from an examination of the full body of evidence. For example, Naomi Oreskes found no rejections of the consensus in a survey of 928 abstracts performed in 2004. Doran & Zimmerman (2009) found a 97% consensus among actively publishing climatologists. Anderegg et al. (2010) reviewed publicly signed declarations supporting or rejecting human-caused global warming, and again found over 97% consensus among climate experts. Cook et al. (2013) found the same 97% result through a survey of over 12,000 climate abstracts from peer-reviewed journals, as well as from over 2,000 scientist author self-ratings, among abstracts and papers taking a position on the causes of global warming.
In addition to these studies, we have the National Academies of Science from 33 different countries all endorsing the consensus. Dozens of scientific organizations have endorsed the consensus on human-caused global warming. Only one has ever rejected the consensus - the American Association of Petroleum Geologists - and even they shifted to a neutral position when members threatened to not renew their memberships due to its position of climate denial.
In short, the 97% consensus on human-caused global warming is a robust result, found using several different methods in various studies over the past decade. It really shouldn't be a surprise at this point, and denying it is, well, denial.
Quantifying the Human Global Warming Contribution
There have also been various studies quantifying the human contribution to global warming, as we have previously documented.
Figure 1: Net human and natural percent contributions to the observed global surface warming over the past 50-65 years according to Tett et al. 2000 (T00, dark blue), Meehl et al. 2004 (M04, red), Stone et al. 2007 (S07, light green), Lean and Rind 2008 (LR08, purple), Huber and Knutti 2011 (HK11, light blue), Gillett et al. 2012 (G12, orange), Wigley and Santer 2012 (WS12, dark green), and Jones et al. 2013 (J12, pink).
Again, there's very little controversy here. The scientific literature is quite clear that humans have caused most of the global surface warming over the past half century, as the 2013 IPCC report stated with 95% confidence.
In Cook et al. (2013), we broadened the focus beyond definitions that quantify the human contribution, because there's a consensus gap on the mere question of whether humans are causing global warming. Nevertheless, we used the 2007 IPCC position as one of our consensus position definitions:
"We examined a large sample of the scientific literature on global [climate change], published over a 21 year period, in order to determine the level of scientific consensus that human activity is very likely causing most of the current GW (anthropogenic global warming, or AGW)."
The IPCC position (humans causing most global warming) was represented in our categories 1 and 7, which include papers that explicitly endorse or reject/minimize human-caused global warming, and also quantify the human contribution. Among the relatively few abstracts (75 in total) falling in these two categories, 65 (87%) endorsed the consensus view. Among the larger sample size of author self-rated papers in categories 1 and 7 (237 in total), 228 (96%) endorsed the consensus view that humans are causing most of the current global warming.
The self-ratings offer a larger sample size on this quantification question because of the limited real estate in a paper's abstract. Most journals have strict word limits on their abstracts, so authors have to focus on the specifics of their research. On the other hand, the author self-ratings are based on the full papers, which have much more real estate and are thus more likely to both take a position on the cause of global warming, and quantify the human contribution.
Confused Contrarians Think they are Included in the 97%
There have been a number of contrarians claiming that they are part of the 97% consensus, which they believe is limited to the position that humans are causing some global warming. The first error in this argument is in ignoring the fact that the data collected in Cook et al. (2013) included categories that quantify the human contribution, as Andrew Montford and the GWPF recently did, for example.
The second error has been made by individuals claiming they're in the 97%, but failing to actually check the data. For example, Roy Spencer claimed in testimony to US Congress that he is included in the 97% consensus. Since we made all of our data available to the public, you can see our ratings of Spencer's abstracts here. Five of his papers were captured in our literature search; we categorized four as 'no opinion' on the cause of global warming, and one as implicitly minimizing the human influence. Thus Spencer's research was included in the fewer than 3 percent of papers that either rejected or minimized the human contribution to global warming. Bjorn Lomborg made a similar error, claiming:
"Virtually everyone I know in the debate would automatically be included in the 97% (including me, but also many, much more skeptical)."
In reality Lomborg is included neither in the 97+% nor the less than 3% because as far as we can tell, he has not published any peer-reviewed climate research, and thus none of his writings were captured in our literature search. The 97% is a consensus of climate science experts, and that, Lomborg is not.
Nir Shaviv took the opposite approach, claiming he was wrongly included in the 97%. Though Shaviv also admitted that Cook et al. correctly classified his abstracts based on their content, but claimed that he worded the text in a way to slip it past the journal reviewers and editors.
"I couldn’t write these things more explicitly in the paper because of the refereeing, however, you don’t have to be a genius to reach these conclusions from the paper."
However, Shaviv, Spencer, and all other authors were invited to participate in the self-ratings process that resulted in the sae 97% consensus conclusion.
Tol's Rejected Comment
Richard Tol has also advanced various criticisms of Cook et al. (2013). It's worth noting that Tol does not dispute the existence of the consensus, writing:
"There is no doubt in my mind that the literature on climate change overwhelmingly supports the hypothesis that climate change is caused by humans. I have very little reason to doubt that the consensus is indeed correct."
Tol has nevertheless criticized the methods applied during the Cook et al. survey. For example, he has argued that the literature search should have been conducted with Scopus rather than the Web of Science in order to capture more papers, but also that fewer papers should have been included in the survey in order to focus on those specifically researching the causes of global warming. Tol has also applied various statistical tests comparing the abstract ratings to the author self-ratings, but these tests are invalid because the two phases of the survey considered different information (abstracts only vs. full papers) and are thus not comparable.
In fact, when we released the self-rating data, we explicitly discussed the difference between the two datasets and how the difference was actually instructive. As John Cook wrote,
"That's not to say our ratings of abstracts exactly matched the self-ratings by the papers' authors. On the contrary, the two sets measure different things and not only are differences expected, they're instructive."
Ultimately Tol submitted his criticisms to Environmental Research Letters as a comment, but the submission was summarily rejected by the editor who described it as a speculative opinion piece that does not identify any clear errors that would call the paper's conclusions into question.
In short, the 97% consensus has passed peer-review, while Tol's criticisms have not. Moreover, all of Tol's criticisms only apply to the abstract ratings, while the self-ratings also found the same 97% consensus result, completely independent from the abstract ratings.
Taking Consensus Denial to the Extreme
One critique of the consensus has been published in a paper in the journal Science & Education. The argument made in the paper was first published by Christopher Monckton on a climate contrarian blog. Monckton has also suggested the conspiracy theory that the journal Environmental Research Letters was created (in 2006) specifically for the purpose of publishing Cook et al. (2013).
The Monckton paper takes the point about quantification above to the extreme. It focuses exclusively on the papers that quantified human-caused global warming, and takes these as a percentage of all 12,000 abstracts captured in the literature search, thus claiming the consensus is not 97%, but rather 0.3%. The logical flaws in this argument should be obvious, and thus should not have passed through the peer-review process.
Approximately two-thirds of abstracts did not take a position on the causes of global warming, for various reasons (e.g. the causes were simply not relevant to or a key component of their specific research paper). Thus in order to estimate the consensus on human-caused global warming, it's necessary to focus on the abstracts that actually stated a position on human-caused global warming.
When addressing the consensus regarding humans being responsible for the majority of recent global warming, the same argument holds true for abstracts that do not quantify the human contribution. We simply can't know their position on the issue - that doesn't mean they endorse or reject the consensus position; they simply don't provide that information, and thus must first be removed before estimating the quantified consensus.
As noted above, when we perform this calculation, the consensus position that humans are the main cause of global warming is endorsed in 87% of abstracts and 96% of full papers. Monckton's argument is very similar to the myth that CO2 can't cause significant global warming because it only comprises 0.04% of the atmosphere. 99% of the atmosphere is comprised of non-greenhouse gases, but these other gases are irrelevant to the question of the CO2 greenhouse effect. The percentage of CO2 as a fraction of all gases in the atmosphere is an irrelevant figure, as is the percentage of abstracts quantifying human-caused global warming as a percentage of all abstracts captured in our literature search.
It's also worth noting that based on Monckton's logic, only 0.08% of abstracts reject human-caused global warming.
Climate Consensus Denialism
Overall, the critiques of Cook et al. (2013) have all exhibited the characteristics of scientific denialism. Given the long history of consensus denial campaigns by fossil fuel interests and climate contrarians, continued resistance to the consensus is an expected result. Nevertheless, the 97% consensus is a robust result from several different studies taking a variety of approaches, including two independent methods used by Cook et al. (abstract ratings and author self-ratings). The criticisms of the paper have all exhibited the same few logical flaws, some more extreme than others, but all erroneous.
Intermediate rebuttal written by dana1981
Update July 2015:
Here is a related lecture-video from Denial101x - Making Sense of Climate Science Denial
Last updated on 13 June 2016 by pattimer. View Archives
Guys this is really bad. It would be better if you just focused on the facts.
This does not do anything to address the issue: "Thus it's perhaps not surprising that Cook et al. (2013) and its 97% consensus result have been the subject of extensive denial among the usual climate contrarian suspects. After all, the fossil fuel industry, right-wing think tanks, and climate contrarians have been engaged in a disinformation campaign regarding the expert climate consensus for over two decades. For example, Western Fuels Association conducted a half-million dollar campaign in 1991 designed to ‘reposition global warming as theory (not fact).’"
And under "The 97% Consensus Is a Robust Result", you start by citing a study that produce a 75% consensus level -- but you don't tell us that. You cite that study without even revealing the figure, which does not support the claim in your headline. Also, it was a study of abstracts, not scientists, so it's unclear why you think we can mix the two.
The other 97% finding you cite is based on 77 people, which you don't mention.
There is no survey of scientists that gives us 97%. The Cook study was about papers, and you make no mention that most of the papers included in the consensus were coded by human raters as implicitly endorsing AGW. Nor do you mention the substantial disagreement between the raters, or the fact that the disagreed upon observations were still included in the results.
Since the debate is mostly about severity and confidence levels, studies that use broad or ambiguous litmus tests of simple agreement with human caused warming are not very useful. Actual surveys are better, especially if they ask more useful, finer grained questions, like this: http://journalistsresource.org/studies/environment/climate-change/structure-scientific-opinion-climate-change/#
Try to find your 97% there. Severe outcomes are endorsed by less than half of the sample.
(-snip-).
[DB] Tone-trolling snipped.
Phronesis, the 2004 Oreskes study found 75% of papers explicitly or implicitly supporting the consensus, 25% taking no position, and 0% contradicting the consensus. Thus, amongst papers expressing any view that study found 100% support for the consensus. Your claim that it found only "a 75% consensus level" is a logic error on your part. If you count papers which don't address the issue at all you could bring in hundreds of thousands of papers on quantum physics, stellar cartography, economics, sociiology, et cetera and claim that since none of these take any position on global warming there is less than 1% support for it... or any other subject you want to dismiss via blatantly flawed logic.
Phronesis seems to have a problem with things not mentioned, but is carefull not to mention a few things himself. I'll come to that, but first lets look at Doran and Zimmerman (2009). Phronesis dismisses that as a survey of just 75 people, but that is false. It was a survey of 3,146 scientists. Among those scientists, overall 82% answered yes to question 2:
That question is closest to the proposition tested by Cook et al (2013). That low figure is because most scientists asked were not specialists in the relevant field, and consequently had little more knowledge on the topic than any non-scientist. Of the small portion of the those surveyed who were specialists in climate science, and actively publishing in the field so that they had up to date information, 97.4% agreed with question 2. There were only 77 of those, but they were part of a much larger sample.
Turning now to Farnsworth and Lichter (2012), we find that it is a survey of scientists, 50% of whom were members of the AMS, and 50% of whom were members of the AGU, and all of whom were listed in the American Men and Women of Science. Their 489 respondents are therefore comparable to Doran and Zimmerman's 3,146 respondents, not to the 77 publishing specialists in climatology. Further (the relevant fact Phronesis did not reveal), only 41% of those 489 actively research in any aspect of global climate science.
It is not clear how those 200 scientists compare to Doran and Zimmerman's classification. For Doran and Zimmerman, and active publisher must have published at least 50% "... of their recent peer-reviewed papers on the subject of climate change". That is, by a reasonable measure, at least 50% of their research must be on the topic. In contrast, the 200 scientists from Farnsworth and Lichter need only have actively researched on any aspect of climate science, ie, greater than 0% and need not have brought the research to publication. Nor need they be specialists in climatology. They are probably best equated with Doran and Zimmerman's active publishers. For the other 289, however, the closest category would be Doran and Zimmerman's non-publishers/non-climatologists.
From all of Farnsworth and Lichter's respondents, 84% responded yes to the question:
Assuming Farnsworth and Lichter's non-researchers resonded at the same rate as Doran and Zimmerman's non-publishers/non-climatologists (ie,76.6% affirmative), that represents 221 affirmative responses. That leaves only 190 affirmative responses to come from Farnsworth and Lichter's researchers, meaning that 95% of them responded affirmatively.
Given the low bar on expertise set by Farnsworth and Lichter, and the strong correlation between expertise and current topic knowledge and acceptance of AGW found by Doran and Zimmerman, that 95% is surprisingly high. It shows, however, that the results of Farnsworth and Lichter is entirely consistent with those of Doran and Zimmerman, and also of Cook et al, (2013).
Phronesis may rightly reject the assumption that Farnsworth and Lichter's non-researchers affirmed AGW at the same rate as Doran and Zimmerman's as speculative. The implication of that, however, is that no comparison can reasonably be made between their results and those of Doran and Zimmerman because they do not survey groups with the same demographics. Farnsworth and Lichter would then provide information about general scientific acceptance, but not specifically about acceptance by those with specialist knowledge in the field.
Finally, Phronesis misrepresents the results of Farnworth and Lichter on expected outcomes. The question put, and results were:
Thus, less than half (but only just less than half) think the results of current climate trends will be catastrophic, or near catastrophic. Slightly more, but still less than half think the results will be moderate, while only 13% think the results will be trivial or near trivial.
Hi guys. One thing at a time. We have no idea what the Oreskes study did. There is no methods section in the paper. The paper is only a page long, and the part that actually deals with the study is only 126 words long.
We're told that papers that were classified as explicitly endorsing, evaluating impacts, or mitigation proposals were lumped together as endorsing the consensus. That's all we know. Note that the fact that a paper evaluates impacts does not imply endorsement or might report small impacts. We just don't know. We can't do anything with a paper when we don't know how it was done.
We also can't be converting 75% to 100% based on a paper that includes unspecified implicit criteria for inclusion into that bucket. Your logic is sound except for the implicit part. I'm not going to infer unanimity via an implicit classification scheme, much less a complete unknown classification scheme. Is this controversial?
Also, maybe your logic isn't sound after all. Let's linger on the fact that these figures are converted into "75% of climate scientist agree", or as you would have it, "100% of climate scientists agree".
How did we get there? Papers. And we ignored papers -- and scientists -- who expressed no position (by the completely unknown filtering and coding protocol Oreskes used). The sample was much smaller than Cook -- I'm not sure why. There would technical papers, heads-down so to speak, that would address narrow issues -- not really at the level of abstraction of human caused climate change, which is pretty high level. Am I correct in assuming there'd be all manner papers related to climate mechanism that would not ever have occasion to use the relevant phrases? We don't know anything about what those scientists think. We only know about the set that were included and rated by methods we have not been apprised of.
That makes me even more reluctant to say anything definite, certainly not that 100% of climate scientist think x, y, or z.
Let me know if I've missed supplemental materials on Oreskes. I didn't see anything.
Lastly for now -- we need detailed questions, nuanced views. Endorsement of the proposition that humans impact the climate or contribute to warming or are the primary cause of warming is not useful. Not if we want to do something with this knowledge, like pass laws and so forth. We need severities, probability and confidence, scientists' views on the likely roles of different forcings and feedbacks, interpretations of the pause in mean global surface temps, estimates of consequences and benefits of warming, etc. Obviously a 1 F warming by 2100 probably doesn't justify major increases in the costs of energy, for most people. I know the lower bound of IPCC 5 is higher than that, but I'd like to know what scientists think. I'd also like to see studies they have the opportunity to just express their judgments in their own words.
And is it true that the raters in Cook et al were not independent? I saw it here: http://rankexploits.com/musings/2013/i-do-not-think-it-means-what-you-think-it-means/
Rater independence is crucial in a study based on rater data.
Phronesis @4 & 5, it is more than a little hypocritical to not respond to one issue on the basis of limited time "one thing at a time", and then to yourself introduce additional side topics. It strongly suggests your intention is a gish gallop, where you introduce topics that are rhetorically convenient, but plead time constraints to avoid having to answer on issues where you have been shown to have been both hypocritical (leaving out relevant information, while complaining about what you consider to have been relevant information having been left out) and to misrepresent the study you quote.
Phronesis,
Here I linked two very recent reports (both less than one month old) from the National Academy of Science (and the Royal Society of England) and the American Association for the Advancement of Science that state clearly what they think about AGW. Both of these reports state clearly that AGW is likely to be extremely severe and that strong actions need to be taken immediately. For your convienence I have linked them again below. These reports state exactly what you claim above you need to make decisions.
What is your issue with these clearly worded reports from the top science organizations in the USA and England? All other Academies of Science in the World agree with these three organizations. The problem is that you have not bothered to try to find out what scientists clearly say.
AAAS report
NAS/Royal Society report
Phronesis #1: "...you start by citing a [Oreskes] study that produce a 75% consensus level"
Phronesis #4: "We have no idea what the Oreskes study did."
So you've gone from claiming that the Oreskes study did something it provably did not, to claiming that we just can't know... even though you originally claimed we did know.
*plonk*
Phronesis:
With respect to your desire for information regarding the severity of global warming impacts, etc. - read the IPCC Assessment Reports. That is their purpose.
Phronesis... I don't want to pile on here but would just like to offer one suggestion.
When a scientist gets his hackles up over some published finding, usually the response is to test the results him/herself. This is something you can easily do yourself. You don't need to review 12,000 papers. Honestly, Cook et al was way overkill in regards to what is statistically necessary to establish the point.
There are two easy ways to do this. Here on the SkS site, you can rate abstracts yourself! If for some reason you don't trust the system that SkS has set up, that's fine. Just go to google scholar and start pulling up papers. Set up your own spreadsheet to check the results.
You can get a rational statistical sampling with a couple hundred papers. That might take a few hours to process. But then you know for yourself with data created by someone you trust. You!
Guys, has anyone responded substantively to anything I said in my last round? I don't see anything.
Tom Curtis offers an insight about my intentions, that it is a "gish gallop". I don't know what that means, but I assume it's not charitable.
Michael Sweet cites the AAAS report. My earlier posts undercut one of the three sources used in that report, so I'm not sure why the report would be a counterargument. (Or why any of these reports should be interesting, given that they don't use standard scientific methods for aggregating knowledge, e.g. meta-analyses, at least not the AAAS.)
Dunkerson says that I mentioned Oreskes' 75% figure and that I later mentioned how we have no idea what was done in that study (because the paper doesn't tell us). I take the argument to be that the number of problems I've discovered with Oreskes, or the order I in which I've presented them, somehow refutes me?
Honeycutt regrets piling on another irrelevant, non-substantive reply, and talks about Cook. I had asked a question about whether the Cook et al study used independent rates, as stated in the paper. Honeycutt doesn't answer that question, but invites me to start rating abstracts.
(-snip-).
( -snip-.)
[DB] Sloganeering snipped.
Phronesis @5&11:
With regard to Schollenberger's criticism of Cook et al, Victor Venema had this to say:
"if you scan the manuscript, you find that much of what you “discovered” in the stolen forum posts was already written into the article manuscript:
“Each abstract was categorized by two independent, anonymized raters. … Initially, 27% of category ratings and 33% of endorsement ratings disagreed. Raters were then allowed to compare and justify or update their rating through the web system, while maintaining anonymity. Following this, 11% of category ratings and 16% of endorsement ratings disagreed; these were then resolved by a third party.”
It would have been fair, if you would have told your readers that this was written in the article."
To that Schollenberger admited:
So, right from the start much of Schollenberger's criticism consists of describing processes described in the paper, carefully not mentioning that fact, and then going "tut-tut".
There is a small measure of validity in his criticism, however. Some (5-10) abstracts were discussed in the forum, and should not have been in order to preserve independence. To calculate the effect of this, I shall assume that 50 abstracts were discussed, and that all of those were rated as endorsing the consensus. Both of these assumptions are known to be false. However, they are conservative. That is, they will massively overstate the impact of the breakdown in proceedure on the paper.
To estimate that effect, I simply reduce the number of endorsing papers by 50. The result is that the level of endorsement from abstract ratings falls from 97.1% to 97%. That is, this breakdown in procedure at most overstated endorsements by 0.1%, and did not change the headline result. It is very probable that the impact was negligible, because I have significantly overstated the number of papers involved, because not all papers involved were rated as endorsing the concensus, and because even of those endorsing the concensus, there is a high probability that the rating would not have changed to neutral, or not endorsing the consensus without the discussion.
Further, this breakdown in proceedure cannot effect the author ratings which also show a 97% endorsement rate.
Now, the question for you is - did you realize these facts before raising the issue? If not, why did you raise it, given as you claim to be pressed for time?
Phronesis... My point is substantive if you care to entertain the idea. What I'm telling you is that you can cast stones all day long, never hitting a target and convince no one, including yourself, of anything.
If you have genuine questions about what the reality is regarding the consensus of the published research, test the results yourself. Just be ready to adjust your position on this issue when you do.
My guess is, much like everyone I've ever argued with who rejects AGW, you're not really interested in the reality of the situation. You're merely interested in testing your throwing arm.
Sloganeering, DB? More like Hubris ad Absurdum.
Anyone who thinks that the consensus is significantly less than 97% can demonstrate this relatively easily. All they have to do is to find, let's say, 5-10% or more papers in a given random sample that implicitly or explicitly reject humans as the main cause of global warming. They don't have to do a fine sub-classification like we did, or sort the no-position papers from the rest, just find the thumbs-down papers. (Hint, some doubters of AGW already have some long--and dubious--lists of papers that supposedly reject AGW, so you could just search for them in any given sample.)
A few hours of careful work requiring no special equipment and Cook et al could, in principle, be falsified. Several people who question our work seem to have put lots of effort into writing blogposts, examining our statistics, submitting formal rebuttals, making FOI requests and writing angry letters to all and sundry. Yet, if they were right, a convincing knock-out punch would be easy to deliver, just by looking through a random sample of scientific papers on climate change. But this work has not even been tried, or if it has, it hasn't been reported.
Phronesis,
It is interesting to learn that you know more about scientific opinion on AGW than the American Association for the Advancement of Science. I am sure other readers will make their own choice about whether they should believe the AAAS or a nameless voice on the internet.
This is a scientific board. People are expected to provide evidence to support their claims. Peer-reviewed evidence is best. You do not appear to care about evidence. You might find that your method of argument is better received at one of the skeptic sites where they do not care about evidence either.
Phronesis:
To guide the What We Know initiative, AAAS convened a group of prominent experts in climate science.
Mario Molina (Chair), U of California, San Diego and Scripps Institution of Oceanography
James McCarthy (Co-chair), Harvard University
Diana Wall (Co-chair), Colorado State University
Richard Alley, Pennsylvania State University
Kim Cobb, Georgia Institute of Technology
Julia Cole, University of Arizona
Sarah Das, Woods Hole Oceanographic Institution
Noah Diffenbaugh, Stanford University
Kerry Emanuel, Massachusetts Institute of Technology
Howard Frumkin, University of Washington
Katharine Hayhoe, Texas Tech University
Camille Parmesan, U of Texas, Austin and University of Plymouth, UK
Marshall Shepherd, U of Georgia
I went through that link in that statement and don't see anywhere Shaviv agreeing that Cook et al. correctly classified his abstract of his paper "On climate response to changes in the cosmic ray flux and radiative budget" . And neither, as far as I can see, does the following point Shaviv makes there regarding the wording of the abstract imply he agrees :
Is there some other statement by Shaviv I may have missed where he clearly says he agrees with the Cook et al rating of his abstract?
The main reason I ask is because I recently checked out the Schulte consensus paper (that overlapped with Cook et al), and saw that it had categorised the Shaviv paper as rejecting the consensus. I then checked its Cook et al rating and was surprised to see it was instead classified as endorsing the consensus.
I would have agreed witht the Schulte rating and although the abstract is rejecting the consensus. Or at least not supporting it. This last past part of the Shaviv absrract seems to put it outside.
Doesn't the above key passage clearly indicate the paper holds a thesis that there is a greater proportion (>50%) increase in recent temperature rise that is not explained by anthropogenic causes and so puts it clearly outside the consensus?
Uncomfortable as it makes me, I have to agree with tlitb1 on this one. In partuclar, in stating that he understated his claims in the paper to get past peer review, Shaviv did not state that the claims in their understated form did not reject AGW. Further, in the abstract, Shaviv clearly asserts a climate sensitivity of 0.35 K/W/M^2 (equivalent to 1.3 C per doubling of CO2), and asserts that solar effects including indirect effects through changes in cosmic ray intensity are responsible for 0.47 K of the increase in temperature over the twentieth century. Using AR4 figures, that is 0.47 out of 0.7 K. Both of these should have been sufficient to classify the abstract as rejecting AGW.
The Shaviv paper just happens to be one of the demonstrably small number of mistakes in classification in Cook et al. It is silly to think that a project like the Consensus Project would be mistake free. What is remarkable about the project is how few the mistakes were, and how little impact (if any) they have on the headline result.
@Tom Curtis at 16:39 PM on 15 May, 2014
I'm glad you agree with me about the Shaviv assessement. My attention was only drawn to Shaviv because I had just for the first time read the Schulte paper, which itemises 6 papers (including Shaviv) as examples rejecting the consensus, and I picked Shaviv to compare first.
I've now had a look at the remaining 5 papers picked out by Schulte as rejecting the consensus - with an aim to see how they stack up against Cook et al. - and found: 2 dont appear in Cook et al, 2 agree, and I think have found one other paper, Lai et al. (2005), which Cook et al rates as supporting the consensus but seems actually to be rejecting it. IMO the abstract certainly is not clearly stating a position on anthropogenic contribution being >50%
Global warming and the mining of oceanic methane hydrate Chung-Chieng A. Lai, David E. Dietrich, Malcolm J. Bowman
http://www.skepticalscience.com/tcp.php?t=search&s=&a=dietrich&c=&e=&yf=&yt
If you look at the abstract I think you will see there is nothing that puts it in the consensus. If I am wrong I would love to know what I missed. In fact I think these two statements alone
and
by rights should be enough to define this abstract as *not* being in the consensus.
I think the statement "we propose a new hypothesis for global warming" is as near enough to saying 'this paper is not in the consensus' as you can get! ;)
I certainly wouldn't expect Cook et al to be mistake free but I have to say finding a disagreement between 2 of the 4 overlapping examples here in the Schulte paper, a paper which Cook et al itself draws attention to, is quite interesting; interesting because I think the two examples also indicate an insight as to why they were mis-rated, and why more papers could be too.
I think the mis-rating of Lai et al. (2005) could be explained if the raters were slavishly only looking for quantification, and not really parsing any deeper, then at a push I could see it being put in the neutral category, however its being rated in the endorse category still seems wrong.
And again with Shaviv, there is some dispute about the 'wording' indicated in the article above, as if it wasn't playing by the rules, i.e. as Dana Nuccitelli implies above, "he worded the text in a way to slip it past the journal reviewers and editors" . But surely if Shaviv did not lie in his abstract, no matter how it was 'worded' , Cook et al's methodology should have been robust to correctly rate it?
It clearly seems not in this case, could that have happened in more cases?
tlitb1 @20, sorry, my computer ate my long response, so you will have to make do with the short.
1) I agree with you about Lai et al (2005), which I would rate as (6) "explicitly rejects but does not quantify".
2) From the full rating data, which has been released by John Cook, it can be determined that there is an initial rating error rate between "endorsement levels" of 12.9% on average, which results in an expected error rate between levels after dispute resolution of 4.7%. Given that distinctions between "implicit" and "explicit" are graduated rather than binary (things can be more, or less explicit) such an error rate is unsurprising. It is also overstated in that it treats a large number of ratings of "0" (= uncertain) as errors, which is not the case.
3) The error rate between endorsing or rejecting AGW falls to a low 0.04%. That is because the vast majority of errors are between categories just one level apart, and because nearly all errors (98%) for initial ratings of (4) involve mistakenly identifying an implicitly endorsing abstract as being neutral. Overall, adjusting for errors based on internal data appears to slightly increase the endorsement percentage. (That is a provisional result using simplifying assumptions. More accurate results may change teh sing of the result but will not significantly change the magnitude which is less than 1%. Note that Richard Tol's claim that this adjustment makes a large increase in the "dissensus rate" depends on the false assumption that the error rate and distribution for all rating values is the same.)
4) Despite the low overall error rates, the sheer number of endorsing abstracts means a small fraction will have been incorrectly rated, ie, should have been rated as rejecting the consensus. On the figures above, we would in fact expect two such abstracts. Those figures show only internally detectable errors, however, so the number may by slightly larger.
5) Because we expect some such errors as a matter of course, no amount of highlighting single abstracts being correctly rated will show the Cook et al results to be false. That is because such anecdotal evidence does not provide a basis for statistical analysis. At most the response required is to adjust the values reported for endorsement and rejection in Cook et al by the number of individual abstracts found to be in error. Thus, you have found two abstracts in error. Therefore we would adjust the Cook et al figures of 3896 endorsing and 78 rejecting to 3894 endorsing and 80 rejecting, which changes the percentages (as a percentage of papers endorsing or rejecting) to 97.99% endorsing, down from 98.04%. Even that ignores the probable existence of papers with errors in the other direction, and makes no difference worth mentioning. Even if you were to find a net error of 100 abstracts in favour of endorsement, after correction you still have an endorsement rate of 95.52%, which again is not a challenge to the Cook et al result.
6) Because of this, the only valid method to challenge Cook et al is to do another survey yourself, of at least 2000 abstracts (and prefferably more). Make sure you state your classification criterion clearly. You will find either a result within 5% of the Cook et al value, or your classification criteria will be transparently tendentious.
likeithot... Then Mike Hulme is welcome to replicate the research and show everyone how he believes it should be done. As it is, Cook13 confirms all the previous research on this topic, including Oreskes04, Doran/Zimmerman09 and Anderegg11.
He actually gives several examples of how the "study" could have been done.
1.For starters, hire some independent people to actually review the abstracts.
2. create a definition of AGW that is actally meaningful, like for a paper to count as supporting AGW it would need to actually state that, and state the degree to which they think humans are responsible, more than 50%, or less, for example. This is obviously crucial, as most scientists don't deny warming or that CO2 has some effect. The debate is about how much of an effect, as they undoubtedly knew and avoided in order to be able to forward their political adgenda.
Incidentally, where else in science is a number like 97% so religiously adhered to/miraculously arrived at in "independent" studies? <Snip>
[PS] Read the comments policy. Compliance is not optional. Future violations are likely to result in the comment deleted rather than snipped.
likeithot said... "He actually gives several examples of how the 'study' could have been done."
How about we change the scare quotes to "...how the study 'could' have been done."
To my point once again, if Hulme has ideas on how the study could be done better, then by all means, let him do what scientists actually do in these situations where they disagree. Replicate the study. If it can be done better, then do it better!
I would suggest that the vast majority of those critiquing Cook13 don't want to replicate because they know they're going to come up with the same result.
In fact, you too can replicate the study, likeithot. Right here on this website. We've created a tab at the top where you can rate abstracts yourself. So, spend a few hours doing it. Maybe you'll come up with different results. If so, you can let us all know.
Anything short of that is, as they say, just belly-aching.
You didn't respond to any of the points I made regarding how the study should have been done. In short, it is poorly done because it avoids the critical question of the whole debate: HOW MUCH of the warming that we have witnessed is becasue of human activity. Almost no one argues that humans have had no impact, so disproving that is just knocking down a straw man.
As to your suggestion that I re-do the "study", I find the whole point of the study to be political in nature, trying to prove a scientific point by some kind of opinion poll and then bragging all over the press that your opinions are right because you claim to have lots & lots of people on your side. I have no interest in popularity contests. It is a sign that your arguments are weak, and a distraction from the fact that most models are very inaccurate and didn't predict the pause in warming of the past 17 years.
(Rob P) - Readers will note that the strike-through text relates to other climate myths dealt with here (Climate Models Show Remarkable Agreement with Recent Surface Warming) & here (Nuccitelli et al. (2012) Show that Global Warming Continues), for instance.