Arguments
Science: What it is, how it works, and why it matters
Posted on 30 August 2022 by Guest Author
This is a re-post from the Thinking is Power website maintained by Melanie Trecek-King where she regularly writes about many aspects of critical thinking in an effort to provide accessible and engaging critical thinking information to the general public. Please see this overview to find links to other reposts from Thinking is Power.
And why most of what you learned in science class is wrong
Think back to your last science class. You probably used a huge textbook full of facts that you had to memorize for exams. You likely learned that there’s a recipe-like “scientific method,” which starts with an observation and tests hypotheses with carefully controlled experiments. If you were lucky enough to have a lab, experiments generally had a “right” answer.
And you may or may not have enjoyed the experience.
There’s a good chance you’ve forgotten much of what you learned. You may have even wondered at the time why you had to take a science class. After all, you didn’t want to be a scientist when you “grew up”.
It’s unfortunate that science is often taught this way. Not only is it not fun, it’s also not what science is. Science is so much more than a collection of facts—it’s a way of thinking. There’s also no single “scientific method.” There are many ways to do science.
And most importantly: Science isn’t just for scientists. In a world built by science, scientific literacy is essential for making wise decisions about everything from our health to how to vote.
Why we need science
Humans have long sought to explain the world around us. Our ancestors often attributed natural events, like illnesses, storms, or famines, to the work of supernatural forces, such as witches, demons, angry gods, or the spirits of the dead. We notice patterns, even when they’re not real, and we jump to conclusions based on our biases, emotions, expectations, and desires.
While the human brain is capable of astonishing levels of genius, it’s also remarkably prone to errors. It’s adapted for survival and reproduction, not for helping us determine the efficacy and safety of a vaccine or determining long-term changes in global climate. Personal experiences and emotional anecdotes can easily fool us, despite how convincing they may seem.
Why science is reliable
Science is the best way we’ve found to keep from fooling ourselves because the process is designed to correct for our limited perceptions and flawed thinking.
Individual scientists try to be as objective as possible, but they’re still human and prone to the same errors as the rest of us.
Science’s “secret sauce” is that claims are never evaluated by only one scientist, but by dozens – or even hundreds – of experts, all trying to find flaws in each other’s work.
Scientists submit their results to peer-reviewed journals and present at conferences so the scientific community can scrutinize their assumptions, methodology, and reasoning. Findings are always subject to criticism, but new ones especially so. And findings that can’t be replicated are discarded: nature is consistent, so any inconsistencies are almost certainly the result of human error or fraud. (And the “f word” in science is a career-killer.)
This built-in system of checks and balances means that science self-corrects and progresses: bad ideas are weeded out and good ideas are built upon. Hence the astonishing scientific advancements in the last century or more.
What is science?
There are many misconceptions about science, starting with the definition. And the way science is taught is part of the problem.
Most classes teach science as a body of knowledge. (The aforementioned textbooks are filled with science’s findings.) However, as important and inspiring as those findings may be, they are the least important aspect of science literacy. Because it’s not just what we know that matters…it’s how we know.
Science is a community of experts using a variety of methods to gather evidence and scrutinize claims. It’s a way of learning about the physical world, of trying to get closer to the truth by testing our explanations against reality and critically scrutinizing the evidence.
An essential foundation of science is skepticism, which is simply insisting on evidence before accepting a claim. Scientists are open to all claims, but proportion their acceptance to the strength and quality of the evidence. As my graduate professor used to say: “Show me the evidence.”
In addition, there are three assumptions underpinning the process of science:
1. The world is real: While we all perceive the world differently, there is an objective reality that exists outside of our heads.
2. It’s possible to understand that reality: It’s within our ability to use the tools of science to understand and explain natural phenomena.
3. We can explain observations with natural processes: We don’t need to resort to supernatural forces to explain events.
It’s important to note that science can’t answer all questions. Science is limited to what it can test and potentially falsify, which means that evidence must be observable, measurable, and repeatable. For example, science can’t answer subjective questions, such as personal preferences or moral judgements. In addition, supernatural explanations, like gods, spirits, or vague “energy” forces, aren’t observable and therefore not testable. (There are exceptions, such as claims to control supernatural abilities and those that leave physical evidence.)
Taken together, these assumptions and limitations help us learn about and solve problems. For example, if we assumed we could never know what caused the plague, or that it was the result of witches casting spells, we never would’ve found its true cause – a bacterium – saving countless lives of both plague victims…and “witches.”
Building scientific knowledge
The goal of science is to understand and explain the natural world. Is the pattern we observed real? Is a correlation due to causation? Is our understanding robust enough to make reliable predictions?
These are difficult questions to answer, and one of science’s greatest strengths is that it has the humility to recognize that we can never be completely certain. Scientific knowledge is tentative: science doesn’t prove, it reduces uncertainty. There’s always a chance we’re wrong, so we leave ourselves open to changing our minds with evidence.
The terminology scientists use describe different types of knowledge, and our level of confidence. These terms are often a source of great confusion for non-scientists, as they have drastically different meanings in everyday usage.
- A hypothesis is a testable explanation for a fairly narrow set of phenomena. They’re based on current scientific knowledge and observation, not wild guesses.
- A scientific theory is a broad explanation for a wide range of phenomena. When evidence accumulates for multiple related hypotheses, they’re combined into a single, clear, and powerful explanation. Well-supported theories have been rigorously tested and have high predictive power. Examples include the theories of evolution, cells, germs, gravity, and relativity.
- A scientific law is adescription of natural phenomena, is usually mathematical in nature, and is well-supported by evidence. Importantly, theories and laws serve different purposes: laws describe what will happen and theories explain why. There is no hierarchy: theories will not become laws or vice versa. Examples include the laws of gravity, relativity, and electromagnetism. (Note that there are very few laws in the life sciences.)
- A scientific model is a representation of an idea, object, process, or system to make it easier to understand. It can be physical, mathematical, or computational. Models describe and explain current knowledge and are used to create testable predictions. Examples include models of cells, atoms, the solar system, Earth’s climate system, etc.
- A fact is an observation that has been repeatedly confirmed and is generally accepted as true. It’s worth noting that even scientific facts can change! For example, all living organisms used to be classified as either plants or animals…then the system expanded to 5 Kingdoms (Plants, Animals, Fungi, Protists, and Bacteria)…and today the classification system has expanded even further, to three Domains (Bacteria, Archaea, and Eukarya).
- A scientific consensus is the collective position of evidence and/or experts (which is based on evidence). Recall that science is a social process: evidence is collected and evaluated as a community. A scientific claim is never accepted as “true” until it has gone through a lengthy process of careful scrutiny by fellow experts. Establishing a consensus can take significant time and evidence, but it’s the most trustworthy knowledge available at any given moment. The more diverse the community, the stronger the consensus, as they’re more likely to find each other’s biases and blind spots.
Importantly, a scientific consensus is not the result of groupthink or democracy. Quite the opposite! (The phrase “herding cats” comes to mind.) Scientists are incentivized to find new discoveries, errors in each other’s work, or disconfirm existing knowledge, not to go with the flow. Individual scientists may dissent from the consensus, but the odds that a lone contrarian is right and everyone else is wrong are very small.
While science deniers like to think of themselves as Galileo, the analogy is a false one. First, it’s impossible to compare what science knows today versus what was known nearly 400 years ago. Second, denying a consensus doesn’t make your position correct. Most people, scientists included, who refuse to accept a strong scientific consensus are just wrong. Third, Galileo was suppressed by the Church, not the “scientific establishment”. And finally, the scientific community accepted Galileo’s evidence and changed their minds accordingly. In fact, those who deny a strong scientific consensus ironically bear more resemblance to the ideologically-motivated church than to the scientific community.
To clarify what was previously stated, science’s goal is to understand and explain natural phenomena with laws, theories, models, and facts, all of which are achieved through consensus.
Scientific knowledge progresses over time as we dig deeper into established knowledge and expand into new territory. Confidence in our conclusions grows as findings are replicated and lines of evidence converge. Science is always tentative to some degree, but well-established findings that have been repeatedly and independently confirmed are very unlikely to be completely overturned.
Testing is at the heart of science
At its core, science is a process of testing our expectations against reality. Importantly, there is no single way to “do” science. In fact, the more ways we test explanations, the better.
Single studies
Broadly speaking, there are two types of studies that provide different kinds of evidence.
In controlled experiments, researchers actively control variables, which helps to reduce the impact of the researcher’s biases and other factors that might impact the study’s outcome. Experiments that are carefully controlled can provide causational evidence.
For example, drug trials are often blinded, which means participants randomly receive either the treatment or a placebo. The researchers can only conclude the treatment is effective if the subjects who received the treatment show a significant improvement compared to the placebo group.
For ethical reasons, researchers don’t start testing on humans, but on cells or non-human animals. However, cells aren’t entire organisms and lab rats aren’t people. Therefore, before we can conclude the treatment is effective, we need to wait until sufficient testing is done on humans.
In observational studies, data is collected in the “real world,” with no manipulation of variables. Observational studies are used when it’s not possible, or ethical, to design controlled experiments.
For example, we don’t have another planet Earth to use as a control for global climate change studies. And while we could design experiments to see the developmental effects of pesticides on children, we shouldn’t. These are important issues, so we need to find other ways to test our explanations. But because observational studies aren’t as controlled as lab experiments, there’s a greater chance that confounding variables might impact the study’s outcomes. Therefore, observational studies provide correlational evidence. (It is possible to infer causation with observational studies, but it requires a significant amount of research to rule out the potential impact of confounding variables.)
Syntheses and summaries
However––and this is important!––observational studies and controlled experiments are all single studies, which in science is never sufficient evidence to establish confidence in any conclusion. The question is: what does the body of evidence say?
While individual studies are limited in scope, research syntheses, such as systematic reviews and meta-analyses, filter and combine the results of dozens or even hundreds of studies on any given research question and therefore provide higher quality evidence. While these “studies of studies” aren’t immune to biases, well-designed synthesis reports can help us avoid the trap of being misled by a single study.
Summaries filter and combine the related individual studies and research syntheses. (It’s very meta.) These can take various forms, such as consensus reports, position statements, or clinical practice guidelines, but because they evaluate and synthesize all related literature to determine what can be concluded with the most confidence, they provide the strongest evidence.
For example, every few years the Intergovernmental Panel on Climate Change selects top scientists from around the world to evaluate and synthesize thousands of climate-related studies to provide the most reliable and comprehensive consensus report on the causes, impacts, and future risks of a changing climate. (Their most recent assessment states, “It is unequivocal that human influence has warmed the atmosphere, ocean and land.”)
Finding reliable scientific information
While science is the most reliable method for learning about the natural world, it can’t help us if we don’t know how to be good consumers of scientific information.
Your most important line of defense is healthy skepticism, not only of news stories or studies, but of your own beliefs. No one can fool us like we can: We’re most likely to fall for misinformation when we want (or don’t want) something to be true.
The scientific literature is for experts
Scientists publish their findings in peer-reviewed journals, also known as the scientific literature. Recall that science is a social process, and these journals are vital tools for experts to communicate with other experts.
But experts are only experts in their area of expertise, and scientists often specialize in very narrow subfields. It requires significant education and experience to be able to evaluate the quality of a study, and to have the background knowledge to put it in context with the fuller body of evidence.
Peer-reviewed literature is theoretically the best source of scientific information, but only if you have the expertise to understand and evaluate it.
Therefore, be careful of using Google Scholar to find scientific information. Typing in desired keywords and finding a title or two that seems to support a desired conclusion is a great way to be misled. If you don’t know what you don’t know, it’s easy to fool yourself…including in the scientific literature.
Science in the news
After completing their last science class, most people only learn about science in the news. However, the “news” tends to focus on new and noteworthy findings, and established science isn’t “news.”
In addition, the news often overstates or sensationalizes the results of single studies, leaving consumers with the impression that scientists are always “flip-flopping” or “changing their minds”. (Not sure why that’s a bad thing?!?)
So the next time you hear about a “scientific breakthrough” that “changes everything”, remember that a single study is never sufficient to confirm or disprove any conclusion. It’s best to judge any study within the broader context of the existing literature, and if the findings really are groundbreaking, to withhold judgment until more evidence is available.
How to do your own research
The phrase “do your own research” is everywhere these days. On its face it seems legit: what can be wrong with wanting to seek out information and make up your own mind?
The problem is that access to information isn’t enough. Due to confirmation bias, we seek out information that supports what we already think is true. However, the danger is that we end up cherry picking individual studies or experts and miss the bigger picture, misleading ourselves in the process.
If the goal is to find the most accurate representation of scientific knowledge, use neutral (not leading or inflammatory) search terms and make sure to use reliable sources.
And your best bet is to look for the consensus, if one has emerged.
[Learn more: How to do your own research]
The take-home message
Science is the most reliable method of understanding objective reality because it recognizes and corrects for our individual limitations and biases. Finding consensus isn’t easy––or pretty––as each scientist is incentivized to find flaws or limitations in each other’s work, but claims that withstand the collective scrutiny of the scientific community become the facts, models, laws, and theories that are used to further expand the body of knowledge.
Science created our modern world. It touches every aspect of our lives, from our health to the environment. The key to making good decisions, to knowing when someone is full of poo and trying to take advantage of you, is science literacy.
Unfortunately, too many science classes focus on what science knows instead of how it knows, leaving too many unable to spot claims that seem scientific…but aren’t. Science literacy is more than memorizing facts – it’s understanding how the process of science works.
As the great Richard Feynman said, “Science is a way of trying not to fool yourself. The first principle is that you must not fool yourself, and you are the easiest person to fool.”
To learn more:
Oreskes, N. (2019). Why Trust Science? Princeton University Press.
Understanding Science: How Science works. University of California Berkeley.
Special thanks to John Cook for his guidance and feedback.
This is another excellent presentation by Melanie.
That should be a reasonable common sense understanding. I think the following are related reasonable common sense understandings:
Excellent commentary. I dont recall learning anything in school about the scientific method or the purpose of science or logical thinking skills. The cynic in me thinks this is because the education system didn't want young people learning analytical skills, and thinking too much for themselves, and therefore maybe challenging the teachers!
I really like this post, too.
One of the things that might raise eyebrows is the claim that with science "bad ideas are weeded out and good ideas are built upon". In this sentence, the words "bad" and "good" have a fairly specific context. As described later in the article, this is not about moral good or moral bad, or good/bad as personal preferences. It is about ideas that enable us to make good evaluations of the world around us - evaluations that help us understand and predict how the world works (whether we like it or not).
Good ideas also gives us clues to the uncertainty in our understanding. People who are dogmatically certain about their viewpoints are not being very scientific. (And, yes, the IPCC looks at uncertainties.)
One of the ways that science works is by looking at competing explanations from the point of view of "what is the difference in the predictions they make?" If two ideas result in no differences, then they are functionally identical. Only by their differences can you tell them apart - and then going out and observing what actually happens will tell you which idea is more likely to be correct. A track record of accurate predictions gives confidence.
And we prefer explanations that can make a lot of accurate predictions with fewer assumptions. If every prediction fails and requires adding new assumptions to fit the observations, then the explanation is not very good.
With good scientific ideas, we can get reliable predictions about things we have not yet seen or measured - what will happen if we add more CO2 to the atmosphere, where to dig to find gold, how to make a faster airplane, what will be most the likely outcome of a particular medical treatment, etc. We know that falling off a tall building is dangerous because we truly do understand that gravity still works.
This is an excellent article on what science should be, what science claims to be, what science at its very best must be, but the reality is that in this day and age and for much too long, this is not what the scientific system of enquiry is. That is the problem.
This essay is like the Church in centuries past claiming what it is, while ignoring the corruption and distortions inherent in its system.
Much of modern science is not science by any stretch of imagination and the vested agendas who use it as tool and weapon, make it impossible for it to be real science, good science. If only the scientific system of enquiry were as the writers so breathlessly describes. But it is not.
Rosross @4,
In a limited evidence based way I agree with you. A clear case is the ways that the 'science' of marketing is abused.
There are many harmful misleading marketers. Regarding climate science they have misled people to delay the rate of increased awareness and understanding of climate science. They want to delay learning among the general (voting) population because that learning would lead to more rapid corrections of what has become popular and profitable. Those corrections, and making amends for harm done, would be disadvantageous to the misleading marketers and their misled fans.
I am interested in the quote: "Scientists are incentivized to find new discoveries, errors in each other’s work, or disconfirm existing knowledge, not to go with the flow. Individual scientists may dissent from the consensus... ". In light of this why have scientists like van Wijngaarden and Happer found it difficult to have their work published? Should this section of the paper not elaborated on the role that politics plays in science?
Cowpuncher - dont know van Wijingaarden, but in Happer case, yep, it is really hard to get your politically-motivated junk published. Getting basic errors through peer-review is a tough process. If you have somehow missed Happer's problems - then try here skepticalscience.com/Evidence-Squared-10-Debunking-William-Happer-carbon-cycle-myth.html and here skepticalscience.com/William_Happer_arg.htm
Thanks scaddenp. I checked both your references and I didnt see "junk" in Happer's comment about CO2 being breathed out. Also the list of "myths" are points of view shared by many scientists. Some appear wrong, others sensible. I suggest checking out https://wvanwijngaarden.info.yorku.ca/publications/ It seems to me to include studies that deserves to be published and not shunned from a political perspective. As I commented on the original paper it seems only some "new discoveries" can get published.
[BL] Link activated.
The web software here does not automatically create links. You can do this when posting a comment by selecting the "insert" tab, selecting the text you want to use for the link, and clicking on the icon that looks like a chain link. Add the URL in the dialog box.
Note: the link does not seem problematic to my browser (later comment by eclectic notwithstanding).
Thank you Guest Author for this well written piece! I wish everyone on this planet had a copy to read!
Rosross @4 :
You are certainly correct - to some extent. ( I agree with "OPOF" on that. )
OPOF makes good points on the unhappy level of corruption/marketization of modern science. It is something which a cynic would regard as difficult to avoid in this modern commercial world.
# Nevertheless, Rosross, the modern science system is like modern democracy ~ far from perfect, yet better than any alternative so far tried. If you have a more perfect (and practical !) system in mind, then it would be most interesting to read your description of it. (Doubtless you know the old joke about the overly-critical voyeur.)
Cowpuncher @8 ~ sorry, but your vanWijngaarden publications link shows as "highly insecure" and my computer won't proceed. If you have some excellently salient points (from Wijngaarden & colleagues) then please summarize those points.
Happer and vanW have received some earlier attention here at SkS ~ and as far as I recall, they were not making any notable advance in climate science. Basically, theirs was a re-hash of already-understood material . . . plus a large dob of bizarre motivated reasoning (but not as extreme as Lindzen's stuff). ~Motivated reasoning strongly influenced by political extremism, I mean. In other words, very poor science.
@8 Cowpuncher
Happer has a long list of touted climate myths.
Climate Misinformation by Source: William Happer
@8 Cowpuncher
"By breathing out, we are simply returning to the air the same CO2 that was there to begin with".
Source: Does breathing contribute to CO2 buildup in the atmosphere?
Rosross @ 4:
Do not confuse "science" and "stuff that people pretend is science". Yes, a lot of crap that people do and push as part of their agenda is crap, and not deserving of being call science proper. But that is the trick - just because someone wraps up their crap in sciencey terms does not make it science. I can call my Ford Pinto a Lamborghini Countach, but that does not make it one.
The fact is that using the terms that come from science helps the shysters sell their swamp land in Florida. People are fooled, because of their lack of knowledge and background in science. It looks sciencey, and without the critical thinking skills that are discussed in this blog post, people get fooled.
And even "scientists" that have successful careers are sometimes fooled. A successful academic career can result from publishing a lot of poor quality work. Publish or perish. Quantity, not quality. Sometimes they just want to fool others to move their career along. Other times, they fool themselves into thinking their long list of publications in poor quality journals actually represents "good science".
Hopefully, the information in the blog post helps people recognize what really is good science from the boatloads of crap that are sold under the "science" sign.
Cowpuncher:
Both Happer and van Wijingaarden are physicists with no real background in climate science.
From van Wijingaarden's profile page at York University, his research area is:
I highlighted the "climate change" part. It is not really his area of expertise. His publication list shows several recent climate-related titles. Looking at the titles, some are simple data analysis papers. Looking at some of the "journals", I notice that two of the papers with Happer are listed as "Atmospheric and Oceanic Physics arXiv". As far as I can tell, this is not an actual journal - just a place for people to post "papers". The PDFs are hosted on van Wijingaarden's York U web site, and give no indication that they have actually been published anywhere. They did not show up when I searched on arxiv.org.
Another paper is listed as "accepted Open Atmospheric Journal (2016)". Also links to a PDF on his own web page. I can find a journal called "Open Atmospheric Science Journal", but that paper does not appear in a search for "Wijingaarden" on their web page. Downloading the PDF from the YorkU site shows that the full title of the journal really is "Open Atmospheric Science Journal", and it lists Bentham Open as the publisher. Bentham Science Publishers has a page on Wikipedia, which notes:
Some of the "publications" give no journal name at all.
To put it bluntly - that list of "publications" is padded to the extreme. You may wish to believe that these "papers" represent some radically-innovative evidence that the field of climate science is keep the truth hidden. It is much more likely that they are crap, and the only way that the authors can "publish" them is to place them in locations where literally any old crap is accepted.
A followup to my own comment @14:
Looking a bit further into arxiv.org, I was able to find the two papers that van Wijngaarden lists on his publication record. There are also two more there, also co-authored with Happer.
None of the four give an indication - on the main page for each providing the abstract, or in the linked PDF files - that they have been submitted to or accepted in any actual journals. I don't know if this is normal for arxiv.org or not.
Just to clarify what ArXiv is and isn't, here is the beginning of the Wikipedia entry for it:
So, it's a somewhat moderated archive but nothing close to a peer reviewed journal and having papers listed there, doesn't really tell you anything about their quality.
As it was mentioned upthread, there is a successor to or at least archive of Beall's list of potentially predatory publishers available at https://beallslist.net/
Hope this helps!
There was mention @10 of previous SkS words on van Wijngaarden and Happer. This 'mention' may refer to the treatment one of their un-published papers got in this thread from a year ago.
The top 3 listed 'publications' are the only ones that have these two authors van Wijngaarden and Happer, their first cooperation writing since they were doing physics back thirty years ago.
These 3 listed 'publications' seem rather odd to me. It is as though some other un-attributed authors have contributed to the work but who then had no input into the final version. I say that as many of the numbers presented are not entirely wrong) but the way the papers are written sets them out to give the wrong conclusion. And there are rather too many inconsistencies suggesting too many cooks.
Thus, for example, the third in the list tells us it is a "a summary of a more detailed paper on radiative forcing by greenhouse gases that the authors plan to publish in the near future." And while there are two different titles given for this "more detailed paper" of which there is no sign, they are presumably referring to the top two in the list, all three being pretty similar in their coverage but strangely different in how they say it (and none of which get published). And strangely this 'third' paper 'summary' gives an odd message in its abstract that doesn't really match that given the full account. I call the message in the abstract 'odd' as it tells us not to be scared by methane because it is adding a forcing only one-tenth the CO2 forcing (which agrees with the NOAA AGGI numbers of the last decade) and that together they are adding a climate forcing of +0.05Wm^-2/y (which is 50% higher than the NOAA AGGI numbers of the last decade) but this will apparently only increase global temperatures by +0.012ºC/y (this about half the warming rate of the last decade).
Within the full text, this message is lost with the message being that CO2 is far more powerful a GHG than methane but that the biggest power of a GHG is when it is at low concentration which is why small increases of methane have such a big effect molecule-for-molecule that the higher concentrations of CO2, this being entirely true. But so what?
Untangling the totality of all this strangeness would be quite a task but given the papers are evident garbage, such a debunking task isn't really merited.
Thanks, Baerbel, for that information on arXiv. It indeed seems to be simply a place for people to upload papers with little regard to quality. They may or may not be papers that are submitted elsewhere. Calling something a "publication" because one puts a copy on arXiv is hubris at the extreme.
On their "About" page, arXiv states the following (emphasis mine):
The main arXiv page actually has a warning related to Covid-19 submissions (again, emphasis mine):
I agree with MA Rodger's initial evaluation of the merit of these "publications" - not worth the time to look at.