Arguments
Database of peer-reviewed papers: classification problematics
Posted on 31 May 2011 by Ari Jokimäki
While adding papers to the peer-reviewed paper database here in Skeptical Science, many problematic issues have emerged. In the database, papers are entered under certain skeptic arguments. For any paper, one can enter several arguments that are relevant to the paper in question. Each paper gets a classification pro-agw/neutral/skeptic depending on how the paper sees the argument in question. It is not very straight-forward task to assign a classification for each paper.
First of all, many arguments of so called climate “skeptics” are based on logical fallacies, especially on the logical fallacy known as “straw man”. When a straw man argument is made against the theory of anthropogenic global warming (AGW), the theory is misrepresented (straw man is being built) and then this misrepresentation (the straw man) is attacked. A common example of this is the argument “climate’s changed before”. AGW is misconstrued as claiming that climate has not changed in the past, and then this straw man is attacked with examples of past natural climate change, with the erroneous conclusion then being that humans cannot be causing this climate change.
How should we classify papers for this argument then? As the argument is a straw-man, there really can’t be many papers that can be classified as “skeptic” papers in the context of this argument. After all, logical fallacies are avoided in peer reviewed literature. However, the peer-review process is not perfect, so it is not impossible that some paper might have gotten through saying “oh look, there was warming in medieval times, so mankind can’t be causing warming”. If there are such papers, they should be classified as “skeptic” for this argument.
But what papers are “neutral,” and what papers are “pro-agw,” for the “climate’s changed before” argument? This is not very easy question to answer. My opinion is that the papers discussing past climate changes and suggesting that carbon dioxide can have a strong role in climate (and maybe even suggesting explicitly that their results support AGW theory) are “pro-agw” papers. Papers discussing past climate without taking a stand on the role of carbon dioxide or mankind on climate should be classified as “neutral”. Of course, there might be other opinions. In the future, we should synchronize our thoughts on this and document them in order to make this database more comprehensive.
There are some difficult arguments from a classification point of view, such as the "it's the Sun" argument. AGW theory doesn't suggest that there's no role for the Sun in Earth's climate, so how do we classify papers for this argument? A paper suggesting that the Sun has no role in Earth's climate whatsoever certainly seems to go against this argument, and while this would also go against mainstream climate science, it would be classified as pro-agw. Fortunately there are not many papers suggesting no role for the Sun (I'm not aware of any such papers).
On the other end of the paper spectrum, a paper might suggest that recent climate change is due to the Sun only. Such papers are "skeptic" papers. The middle ground between these two is the most difficult. There are papers saying that the Sun has had a warming effect during last few decades. The size of this effect varies by paper, so how much warming should a paper suggest as the contribution of the Sun to the recent warming so that we could classify it "skeptic" paper? Perhaps over 50%? What about a paper that suggests that sun has had a strong contribution to recent warming but CO2 is just taking over and will dominate in the future, is that pro-agw or skeptic paper?
One class of "it's the Sun" papers are the papers dealing with past climate changes and suggesting large role for the Sun. This is compatible with AGW theory which says that only in recent decades has anthropogenic CO2 become dominant climate forcing. Such papers should go to the neutral bin unless they also suggest that CO2 doesn't affect climate (or affects only weakly) in which case they should go to the skeptic bin. If a paper suggests a large role for the Sun in past climate changes but also emphasizes that CO2 can have a large role too, then the paper should go to pro-agw bin.
Papers addressing only part of the whole
There are a lot of papers that discuss certain parts of an issue but not the whole issue. There are papers that discuss certain weather events but not their climatic significance; some papers report regional studies, while the "skeptic" argument on that issue relates to the global situation; studies on single species are rarely conclusive from the perspective of the whole ecosystem. These are just a few examples of papers that address only part of the whole that is relevant to certain "skeptic" arguments. As a general rule, such papers should be put to the neutral bin. However, an exception to this rule is a paper that addresses only one part of the whole but claims explicitly to be relevant for the whole also. For example, if a paper reports warming in Finland and claims that it means global warming is happening, then we should put that paper to the pro-agw bin.
The issue of local v. global studies arises when a certain study concludes either a predicted response from AGW is (is not) happening, or explicitly concludes that AGW is (is not) directly observable or having an effect; but such study is of a specific location, and not global trends. It is not necessarily safe to assume that any effect observed in one area can be extrapolated to the entire globe, and as AGW theory most often deals with global trends, or trends dealing with whole biomes, we have generally agreed that local studies be placed as “neutral,” regardless of main conclusion.
I also think this same logic should be applied to research dealing with the effects of AGW on certain species: studies that focus on one plant species and how it will respond to AGW, for instance, are classified as “neutral” as well, as generalizations among plant species (especially) are illogical to make.
Of course, one question that arises is, “Where can the line in number of local publications be drawn so that a trend can be extrapolated?” While it is a good question, it is my opinion that that is not the point of the classification system. Papers can be referenced collectively to identify a widespread effect that extends beyond the local observations of each paper, but the limited extent of the observations of each individual paper subsequently limits the conclusions of each individual. As this database is to categorize individual papers by the explicit conclusions of the authors, and not groups of papers by the implicit agreement among them, it is my opinion that “lines” as above not be considered, and local/species-specific papers still be categorized as “neutral.”
We would also like to remind those using this database, though, that papers are also associated with related skeptic arguments to AGW, so papers can still be de-facto grouped together for careful analysis. ?
??Here we should also point out that sometimes "skeptic" arguments deal with only part of the whole. If such arguments are not straw men, then we should classify papers according to those arguments. A prime example of this is the situation with polar bears. There is the argument "polar bears are increasing;" for this argument, papers suggesting that polar bear populations are declining are "pro-agw" and papers suggesting that populations are increasing are "skeptic". However, also for this argument studies on a single population only should be considered "neutral" as they don't deal with the global situation of polar bear populations.
Other issues
In many cases a paper falls under many "skeptic" arguments. In and of itself, it's not a problem as the database allows several arguments to be assigned per paper. However, a paper can be "skeptic" for some argument while being "pro-agw" for some other argument. The database currently allows only one classification per paper so this is a problem. How do we select which argument we choose as the "primary" argument for which we determine the classification? The most obvious method is to select the argument which the paper addresses most, but sometimes that doesn't help. This problem requires future development of the database - perhaps classification for each argument is necessary.
Of course, we might ask what is the point of classifying papers by skeptic arguments anyway. Perhaps it would be better to ignore what the papers say about some "skeptic" arguments and classify the papers based on their support of AGW theory generally. These are issues that need to be thought of. Discussion on these matters is invited, but it is clear that further development is needed, and we especially need to set common ground rules for the classification of papers in the future.
Thanks to Alex C. who contributed substantially to this article.
[DB] I think you'll find what you are looking for here:
http://www.skepticalscience.com/detailed-look-at-climate-sensitivity.html
Many other threads touch upon climate sensitivity as well. Using the Search function finds this list.