Posted on 3 May 2017 by Andy Skuce

The 97% consensus on anthropogenic global warming (AGW) reported by Cook et al. (2013) (C13) is a robust estimate. Alternative methods, such as James Powell's, that identify only explicit rejections of AGW and assume that all other instances are endorsements, miss many implicit rejections and overestimate the consensus.

The C13 method can be modified and used to estimate the consensus on plate tectonics in a sample of the recent geological literature. This limited study—not surprisingly to anyone familiar with the field—produces a consensus estimate of 100%. However, among the abstracts that expressed an opinion. only implicit endorsements of the theory were found: this underlines the importance in consensus studies of identifying implicit statements. The majority of the abstracts expressed no position. Assuming that these cases suggest either uncertainty about, or rejection of, plate tectonics (as some critics of C13 have claimed about AGW) would lead to the absurd conclusion that the theory is not widely accepted among geologists.

Opinion surveys show that the public is misinformed about true state of consensus among climate scientists, with only a minority aware that it is greater than 90%. The difference between 97% and 99.99% is tiny compared to this gap. The 3% of published papers that reject AGW are contradictory and have invariably been debunked. They do not therefore provide a coherent alternative account of climate science. Despite this, dissenters are awarded disproportionate attention by ideologically and commercially motivated interests, as well as by false balance in the media. Countering this public confusion requires both communicating the consensus and debunking bad science.

For a shorter summary than the one below, read Dana Nuccitelli's Guardian piece: Is the climate consensus 97%, 99.9%, or is plate tectonics a hoax?

In 2016, James Powell published an article Climate Scientists Virtually Unanimous Anthropogenic Global Warming Is True in the Bulletin of Science, Technology & Society. The paper was critical of the 2013 paper by John Cook and members of the Skeptical Science team (C13): Quantifying the consensus on anthropogenic global warming in the scientific literature. Powell’s article is pay-walled, but readers can read the details of his argument on his blog and in a Skeptical Inquirer piece from 2015. There is also a free version of the BST&S article hosted here, found through the Unpaywall Browser Extension.

We have previously responded to Powell’s arguments. John Cook published an article for Skeptical Inquirer and I wrote a piece on my blog Critical Angle. Yesterday, a team of eight co-authors (Andy Skuce, John Cook, Mark Richardson, Bärbel Winkler, Ken Rice, Sarah Green, Peter Jacobs and Dana Nuccitelli) published a more formal, peer-reviewed rebuttal: Does it matter if the consensus on anthropogenic global warming is 97% or 99.99%? in the Bulletin of Science, Technology & Society. The article is also paywalled, but the publisher allows us to post a version of the revised submitted manuscript, which can be read here.

In his BST&S paper, Powell makes four main arguments against C13:

1. Consensus studies by Doran & Zimmerman (2009), the Pew Research Center (2015) and Anderegg et al. (2010) fail to reveal a near-unanimous consensus on AGW “…because of some combination of small sample size, reliance on fallible opinion, and inclusion of nonexperts…”
2. The C13 study of 11,944 articles in the peer-reviewed literature from 1991-2011 failed to adequately measure the consensus because it ignored the abstracts and papers that did not express an opinion on AGW.
3. Applying the C13 methodology to a universally accepted theory like plate tectonics would yield misleading or absurd results.
4. If it were true that 3% of scientists rejected or were uncertain about AGW, then the case for action to prevent global warming would be weakened, since, by comparison with now-settled scientific questions like continental drift and the origin of lunar craters, the existence of dissenters might suggest that AGW theory was about to be overthrown

We respond in detail to these objections in the paper. I will summarize the main points below. Complete references are provided in the paper.

James Powell has written a rejoinder to our article, which should be published in coming weeks. We have not seen it yet.

1. Did previous studies fail to reveal a near-unanimous consensus?

This point is best addressed by referring to the meta-study Consensus on consensus: a synthesis of consensus estimates on human-caused global warming published in 2016 by John Cook and 15 co-authors. (C16). The study found that different studies using different methods produced estimates of the consensus on anthropogenic global warming (AGW) that range from 90-100% for sampled populations with expertise in climate science. Within this range, the results differ because of: “variations in the study timing, definition of consensus, or differences in methodology including surveys of scientists, analyses of literature or of citation networks.”

Although the studies show that multiple lines of evidence exist for overwhelming support among climate experts for AGW, most studies do reveal a small percentage of dissenting opinions. The degree of consensus correlates with increasing expertise.

From C16. Level of consensus on AGW versus expertise across different studies. Right colour bar indicates posterior density of Bayesian 99% credible intervals. Only consensus estimates obtained over the last 10 years are included (see S2 for further details and tabulation of acronyms).

2. Did C13 underestimate consensus?

Powell has two main objections to the C13 methodology.

Firstly, he argues that all abstracts that do not contain an explicit rejection of AGW should be deemed to endorse AGW. There is circularity to this argument: the assumption of near-universal endorsement leads Powell to classify any abstract lacking explicit rejection wording as endorsement. We found that implicit rejections of AGW were much more common than explicit rejections. Indeed, in our supplementary survey of the original authors, 1.8% of those who responded to classified their own papers as rejecting the consensus on AGW.

Secondly Powell claims that C13 ignored the “no position” abstracts. In fact, we did plot the percentage of “no position” abstracts over time and noted that these showed an increasing trend. Although it is somewhat counter-intuitive, we would expect no-position abstracts to increase as the acceptance of AGW rises with time. Space is limited in abstracts and scientists may consider that an endorsement of AGW would be stating the obvious. As we will see in the next section, in geological journals in 2015, abstracts that do not take a position on plate tectonics range from 55-76%. This is not because geologists are hesitant about the reality of plate tectonics, but rather that making implicit or explicit endorsements is either superfluous or irrelevant to the specific research topic of the paper.

From C13, Figure 1 (b). Percentage of endorsement, rejection and no position/undecided abstracts. Uncertain comprise 0.5% of no position abstracts.

It is true that the “no position” abstracts were not used by C13 in the estimate of consensus. Because the authors provided no text that revealed their opinion, we didn’t try to guess it. Nevertheless, we accept that the authors of the great majority of “no position” abstracts most likely accept AGW—but not all of them.

3. Can the C13 method estimate the consensus on plate tectonics?

We performed a small study applying the C13 methodology to consensus on plate tectonics. We looked at the abstracts of papers published in 2015 in the journals Geology and the Journal of the Geological Society. These journals were chosen because they are widely cited and publish articles over the full gamut of solid-Earth geoscience.

We chose criteria for the implicit endorsement of plate tectonics to be uncritical mentions of key elements of the theory, such as sea-floor spreading, subduction or continental drift. References to global-scale paleogeographic features such as Pangaea or Tethys Ocean would also qualify as implicit endorsements.

We looked for explicit endorsements of plate tectonics, but found none. This is not surprising. It is very unlikely that any scientists would put a statement like “plate tectonics is true” into a research abstract in 2015. Nor did we find any rejections of any kind. This also is unsurprising: although there are extremely rare dissenting articles written, even in recent years, it is very unlikely that they would pass peer-review in a prominent journal.

Our rating results:

Of the 265 Geology abstracts, 65 (24.5%) were rated as “implicit endorsements” and 200 (75.5%) were judged to be “no position”. The Journal of the Geological Society yielded 30 (45.5%) “implicit endorsement” and 36 (54.5%) “no position” ratings. Using the C13 methodology for calculating consensus (i.e., the number of endorsements divided by the sum of endorsements and rejections) yields 100%.

The purpose here was not to establish a definitive consensus study of plate tectonics, but to explore whether a modified form of the C13 method could be applied outside the field of climate science. A much broader sample of journals would be required for a comprehensive consensus study  and there is a possibility that more obscure journals might publish a very rare paper skeptical of plate tectonics. A study of the growth of the consensus over time, particularly in the decades following the 1950s would be interesting, but would require a major effort.

Our main findings:

• Yes, the C13 methodology works for plate tectonics, because it counts implicit endorsements. Relying solely on explicit endorsements or rejections would not work because such statements, in abstracts in recent papers in prominent journals at least, are essentially non-existent.
• Most of the abstracts do not take a position on plate tectonics. This is the case even though plate tectonics provides the fundamental framework for modern geology. The reasons for this are partly because some of the articles didn’t require any consideration of plate tectonics processes in their specific research focus (e.g. in geomorphology, glaciology, mineralogy). Failure to endorse the theory should not be taken to imply any uncertainty among geologists. The differences in the percentages of “no position” abstract between the two sampled journals possibly reflects a difference in editorial preferences of subject matter, but this would need to be demonstrated by further study.
• Some critics of C13, for example Legates et al (2013), claim that only explicit, quantified endorsements of global warming being caused mostly by humans should count towards a consensus on AGW. Using this approach, they calculated a consensus on AGW of 0.3%. If they applied the same criteria to plate tectonics they would get a consensus of zero, which is equally absurd.
• We don’t expect (but never say never!) that our small study will attract the kinds of hostile commentary that the C13 study has received. Those criticisms included, for example: that the raters were biased (we certainly are not neutral on plate tectonics); or that the results are suspect because raters colluded privately (we did freely compare and discuss ratings in the plate tectonic case, but only in a handful of problematic instances for the C13 study); or failed to keep to-the-minute time stamps (we didn’t here, either). The reason we don’t expect controversy is because AGW is politicized and doubt about it is largely manufactured and exaggerated by ideological and commercial interests. Plate tectonics has never attracted this kind of controversy, motivated by concerns outside of science.
• Everyone can see the reasons for our ratings in this document. The abstracts are freely available online and our ratings can be checked and replicated.

4. Does a 97% consensus imply a significant level of doubt?

Does the existence of a small percentage of scientists who doubt AGW pose a threat to the mainstream view or indicate the beginning of a scientific revolution? It might, if the dissenters could agree on a single consistent alternative model that accounted for the majority of observations. The reality is that the contrarian objections are incoherent and they contradict each other. They have also been widely debunked. Science revolutions occur when comprehensive alternative theories with greater explanatory power emerge. There is no sign of this happening in climate science.

Attempts to draw parallels between scientific revolutions in the past and the current situation in climate science, as Senator Ted Cruz and others have proposed, are based on false history. It is rare that a well-established general scientific theory is overturned by a handful of plucky skeptics. 

For example, sustained resistance to the heliocentric ideas developed by Copernicus, Kepler, Galileo and Newton came not from scientific rivals, but from religion. It is ridiculous, pretentious even, for climate science contrarians to compare themselves to Galileo.

Modern climate science itself emerged gradually from the mid-nineteenth century. Although there were disputes and problems along the way—typical for a developing science—improved models and new observations allowed the current consensus to form. Organised opposition to the consensus emerged once it became clear that the science implied that policy measures were required to reduce emissions to limit warming.

The controversy over continental drift that raged in the early twentieth century was only partly about the evidence. The reality was that the key data about the nature and age of the crust underlying the oceans was almost completely lacking then, so both sides were speculating about the age and nature of a crucial two-thirds of the Earth’s crust.

A major factor fueling the debate was methodological. North American geologists favoured an inductive, multiple-working-hypothesis epistemological model, whereas Europeans were more comfortable with theory-first reasoning. This is all explained in Naomi Oreskes’ excellent book The rejection of continental drift: Theory and method in American earth science.

As new data came in and a unifying kinematic model emerged—plate tectonics—methodological objections disappeared and almost all geologists embraced the new theory within a decade. See also my blog post Consensus on plate tectonics and climate science for further discussion.

From Isacks, Oliver and Sykes, 1968. Like the Dude Lebowski’s rug, this diagram and this one really tied the geological room together.

Hardly anyone but a few cranks oppose heliocentrism or plate tectonics these days. Current objections to AGW were amplified after nearly all climate scientists became convinced by multiple lines of evidence that human emissions were driving global warming. These objections were not usually put forward by scientists convinced that they were in possession of a better model, but by people whose ideology or commercial interests were threatened by the implication that prudent policy requires rapid reduction in the use of fossil fuels.

Discussion

Why is communicating the consensus on AGW important? Mainly, it is because the public, the American public especially, has been deliberately misinformed about the level of scientific dissent. Most people do not have the time or the ability to assess the scientific evidence for themselves. They rely on heuristics, such as what the experts have concluded. People do this every time they accept a physician’s treatment recommendation and every time they drive across a bridge designed by professional engineers. None of us are polymaths in the modern world, able to go to primary sources and evaluate them as if we were specialists. We are obliged to rely on the opinions of credible experts. There was a day when smart people could live by a principle such as “taking nobody’s word for it”. But today is no longer that day. Modern science has become too vast and specialized for any individual to master it all. Most experiments and observations are now beyond the reach of amateurs.

In a world in which the objections to the mainstream conclusions on AGW have been deliberately inflated, communication of the overwhelming scientific consensus becomes vital to keep the public’s discourse on policy rooted in reality. Studies have shown that such communication leads to changes of test subjects’ opinions and reduces the biasing influence of ideologically motivated actors. Furthermore, it has been shown that consensus estimates act as a “gateway belief”. Manufactured low consensus estimates close the gate, inhibiting further consideration. On the other hand, acceptance of the true, high estimates of scientific consensus unlocks the gate, permitting further investigation of the science and, crucially, encourages necessary reality-based discussion of policy options.

Conclusions

There is a very small but real percentage of scientific opinion that rejects mainstream conclusions on anthropogenic global warming. Such opinion is incoherent, usually ideologically motivated and invariably debunked. Literature surveys that rely solely on finding explicit rejections of AGW will miss the more numerous implicit rejections that are implicitly expressed or are present in abstracts that express no clear opinion. James Powell’s approach, which counts only explicit rejections and assumes that the rest of the literature endorses AGW, exaggerates the consensus a little.

It ought not to matter whether the consensus is 97% or 99.99%. Either figure demonstrates a near-unanimous endorsement. Unfortunately, today’s political and media environment gives a disproportionate degree of attention to the small minority of dissenters. The public has become understandably confused and opinion surveys consistently reveal that the perception of the degree of scientific rejection of climate change is larger than it is.

This misconception needs to be tackled head-on, by repetitively asserting the overwhelming scientific support for AGW. In addition, mainstream scientists need to examine and publicly debunk scientific notions that reject the consensus view.

Many years ago, a geology professor gave me some sage advice: “Never exaggerate your case, it only makes it easier to argue against”. We don’t need to claim that 99.99% of scientists conclude that global warming is real and largely man made. It’s quite sufficient to demonstrate that dissenters make up only a very small percentage of scientists, that their theories provide no coherent alternative and that their scientific arguments are debunked.

Postscript: You can say that again

My colleagues and I talk a lot about consensus. Many of us would instead prefer to focus on the amazing new science being produced and on the daunting policy challenges that humanity faces in avoiding harmful climate change. Sometimes I worry that the repetitive messaging is becoming a bit boring. But, as Tony Leseirowitz argues in a short video clip, repetition is essential.

In the recent congressional hearings on science organised by Republican politicians, only one mainstream scientist, Michael Mann, was invited as a witness for the 97%. Everybody at the meeting kept bringing up consensus, either to affirm it or deny it.

Here’s a brief video supercut made by John Cook of the mentions of consensus at the meeting that he made for the excellent Evidence Squared podcast series that he presents with Peter Jacobs:

Evidently, consensus still matters to scientists and politicians alike and on both sides of the so-called debate. But even as we repeat the consensus message, we are faced with the reality of an American government that rejects climate science and is hell-bent on removing regulations to mitigate emissions. It’s all rather disheartening—but there are glimmers of movement. 

Even though most conservatives have expressed consistent opposition to climate science and mitigation regulations—perhaps motivated by the need to show adherence to their cultural groups, as Professor Dan Kahan and others have argued—recent surveys have shown that there are surprising levels of support for climate and pollution mitigation policies, even among President Trump supporters. 

“The main finding for the survey is that Trump voters are more supportive of climate action — and much more supportive of clean energy — than most people might have assumed,” said Edward Maibach, a professor at George Mason University and a co-author of the study. “I suspect even Mr. Trump would be surprised by this finding.”

In a recent paper, Larry Hamilton of the University of New Hampshire reports that there are initial signs of growing awareness of the scientific consensus, in his surveys of the public in New Hampshire. He notices an uptick in recent years, with liberal perceptions of consensus rising from the mid-80% to the mid-90% and among conservatives from 30% to 40%.

Percentage who think most scientists agree that human activities are changing the climate.

From Hamilton, 2016, figure 3.

Of course, it is not possible, using these data alone, to attribute this recent rise to any single factor. Nevertheless, Hamilton writes:

Growing awareness of the scientific consensus, whether from deliberate messaging or the cumulative impact of many studies and publicly engaged scientists, provides the most plausible explanation for this rise in both series [viz., public opinions on human-caused climate change and on perceptions of the degree of agreement among scientists].

Perhaps we can draw some encouragement from this. But there’s still a very long way to go. And we have to make haste: nature is not about to slow down its physical processes to provide more time for humans to get over their cognitive dissonance and wishful thinking.

Acknowledgement. We thank James Powell for his discussion of the C13 paper. His critique prompted us to provide elaboration of our methods.