This is a re-post from the Thinking is Power website maintained by Melanie Trecek-King where she regularly writes about many aspects of critical thinking in an effort to provide accessible and engaging critical thinking information to the general public. Please see this overview to find links to other reposts from Thinking is Power.
Think back to your last science class. You probably used a huge textbook full of facts that you had to memorize for exams. You likely learned that there’s a recipe-like “scientific method,” which starts with an observation and tests hypotheses with carefully controlled experiments. If you were lucky enough to have a lab, experiments generally had a “right” answer.
And you may or may not have enjoyed the experience.
There’s a good chance you’ve forgotten much of what you learned. You may have even wondered at the time why you had to take a science class. After all, you didn’t want to be a scientist when you “grew up”.
It’s unfortunate that science is often taught this way. Not only is it not fun, it’s also not what science is. Science is so much more than a collection of facts—it’s a way of thinking. There’s also no single “scientific method.” There are many ways to do science.
And most importantly: Science isn’t just for scientists. In a world built by science, scientific literacy is essential for making wise decisions about everything from our health to how to vote.
Humans have long sought to explain the world around us. Our ancestors often attributed natural events, like illnesses, storms, or famines, to the work of supernatural forces, such as witches, demons, angry gods, or the spirits of the dead. We notice patterns, even when they’re not real, and we jump to conclusions based on our biases, emotions, expectations, and desires.
While the human brain is capable of astonishing levels of genius, it’s also remarkably prone to errors. It’s adapted for survival and reproduction, not for helping us determine the efficacy and safety of a vaccine or determining long-term changes in global climate. Personal experiences and emotional anecdotes can easily fool us, despite how convincing they may seem.
Science is the best way we’ve found to keep from fooling ourselves because the process is designed to correct for our limited perceptions and flawed thinking.
Individual scientists try to be as objective as possible, but they’re still human and prone to the same errors as the rest of us.
Science’s “secret sauce” is that claims are never evaluated by only one scientist, but by dozens – or even hundreds – of experts, all trying to find flaws in each other’s work.
Scientists submit their results to peer-reviewed journals and present at conferences so the scientific community can scrutinize their assumptions, methodology, and reasoning. Findings are always subject to criticism, but new ones especially so. And findings that can’t be replicated are discarded: nature is consistent, so any inconsistencies are almost certainly the result of human error or fraud. (And the “f word” in science is a career-killer.)
This built-in system of checks and balances means that science self-corrects and progresses: bad ideas are weeded out and good ideas are built upon. Hence the astonishing scientific advancements in the last century or more.
There are many misconceptions about science, starting with the definition. And the way science is taught is part of the problem.
Most classes teach science as a body of knowledge. (The aforementioned textbooks are filled with science’s findings.) However, as important and inspiring as those findings may be, they are the least important aspect of science literacy. Because it’s not just what we know that matters…it’s how we know.
Science is a community of experts using a variety of methods to gather evidence and scrutinize claims. It’s a way of learning about the physical world, of trying to get closer to the truth by testing our explanations against reality and critically scrutinizing the evidence.
An essential foundation of science is skepticism, which is simply insisting on evidence before accepting a claim. Scientists are open to all claims, but proportion their acceptance to the strength and quality of the evidence. As my graduate professor used to say: “Show me the evidence.”
In addition, there are three assumptions underpinning the process of science:
1. The world is real: While we all perceive the world differently, there is an objective reality that exists outside of our heads.
2. It’s possible to understand that reality: It’s within our ability to use the tools of science to understand and explain natural phenomena.
3. We can explain observations with natural processes: We don’t need to resort to supernatural forces to explain events.
It’s important to note that science can’t answer all questions. Science is limited to what it can test and potentially falsify, which means that evidence must be observable, measurable, and repeatable. For example, science can’t answer subjective questions, such as personal preferences or moral judgements. In addition, supernatural explanations, like gods, spirits, or vague “energy” forces, aren’t observable and therefore not testable. (There are exceptions, such as claims to control supernatural abilities and those that leave physical evidence.)
Taken together, these assumptions and limitations help us learn about and solve problems. For example, if we assumed we could never know what caused the plague, or that it was the result of witches casting spells, we never would’ve found its true cause – a bacterium – saving countless lives of both plague victims…and “witches.”
The goal of science is to understand and explain the natural world. Is the pattern we observed real? Is a correlation due to causation? Is our understanding robust enough to make reliable predictions?
These are difficult questions to answer, and one of science’s greatest strengths is that it has the humility to recognize that we can never be completely certain. Scientific knowledge is tentative: science doesn’t prove, it reduces uncertainty. There’s always a chance we’re wrong, so we leave ourselves open to changing our minds with evidence.
The terminology scientists use describe different types of knowledge, and our level of confidence. These terms are often a source of great confusion for non-scientists, as they have drastically different meanings in everyday usage.
To clarify what was previously stated, science’s goal is to understand and explain natural phenomena with laws, theories, models, and facts, all of which are achieved through consensus.
Scientific knowledge progresses over time as we dig deeper into established knowledge and expand into new territory. Confidence in our conclusions grows as findings are replicated and lines of evidence converge. Science is always tentative to some degree, but well-established findings that have been repeatedly and independently confirmed are very unlikely to be completely overturned.
At its core, science is a process of testing our expectations against reality. Importantly, there is no single way to “do” science. In fact, the more ways we test explanations, the better.
Broadly speaking, there are two types of studies that provide different kinds of evidence.
In controlled experiments, researchers actively control variables, which helps to reduce the impact of the researcher’s biases and other factors that might impact the study’s outcome. Experiments that are carefully controlled can provide causational evidence.
For example, drug trials are often blinded, which means participants randomly receive either the treatment or a placebo. The researchers can only conclude the treatment is effective if the subjects who received the treatment show a significant improvement compared to the placebo group.
For ethical reasons, researchers don’t start testing on humans, but on cells or non-human animals. However, cells aren’t entire organisms and lab rats aren’t people. Therefore, before we can conclude the treatment is effective, we need to wait until sufficient testing is done on humans.
In observational studies, data is collected in the “real world,” with no manipulation of variables. Observational studies are used when it’s not possible, or ethical, to design controlled experiments.
For example, we don’t have another planet Earth to use as a control for global climate change studies. And while we could design experiments to see the developmental effects of pesticides on children, we shouldn’t. These are important issues, so we need to find other ways to test our explanations. But because observational studies aren’t as controlled as lab experiments, there’s a greater chance that confounding variables might impact the study’s outcomes. Therefore, observational studies provide correlational evidence. (It is possible to infer causation with observational studies, but it requires a significant amount of research to rule out the potential impact of confounding variables.)
However––and this is important!––observational studies and controlled experiments are all single studies, which in science is never sufficient evidence to establish confidence in any conclusion. The question is: what does the body of evidence say?
While individual studies are limited in scope, research syntheses, such as systematic reviews and meta-analyses, filter and combine the results of dozens or even hundreds of studies on any given research question and therefore provide higher quality evidence. While these “studies of studies” aren’t immune to biases, well-designed synthesis reports can help us avoid the trap of being misled by a single study.
Summaries filter and combine the related individual studies and research syntheses. (It’s very meta.) These can take various forms, such as consensus reports, position statements, or clinical practice guidelines, but because they evaluate and synthesize all related literature to determine what can be concluded with the most confidence, they provide the strongest evidence.
For example, every few years the Intergovernmental Panel on Climate Change selects top scientists from around the world to evaluate and synthesize thousands of climate-related studies to provide the most reliable and comprehensive consensus report on the causes, impacts, and future risks of a changing climate. (Their most recent assessment states, “It is unequivocal that human influence has warmed the atmosphere, ocean and land.”)
While science is the most reliable method for learning about the natural world, it can’t help us if we don’t know how to be good consumers of scientific information.
Your most important line of defense is healthy skepticism, not only of news stories or studies, but of your own beliefs. No one can fool us like we can: We’re most likely to fall for misinformation when we want (or don’t want) something to be true.
Scientists publish their findings in peer-reviewed journals, also known as the scientific literature. Recall that science is a social process, and these journals are vital tools for experts to communicate with other experts.
But experts are only experts in their area of expertise, and scientists often specialize in very narrow subfields. It requires significant education and experience to be able to evaluate the quality of a study, and to have the background knowledge to put it in context with the fuller body of evidence.
Peer-reviewed literature is theoretically the best source of scientific information, but only if you have the expertise to understand and evaluate it.
Therefore, be careful of using Google Scholar to find scientific information. Typing in desired keywords and finding a title or two that seems to support a desired conclusion is a great way to be misled. If you don’t know what you don’t know, it’s easy to fool yourself…including in the scientific literature.
After completing their last science class, most people only learn about science in the news. However, the “news” tends to focus on new and noteworthy findings, and established science isn’t “news.”
In addition, the news often overstates or sensationalizes the results of single studies, leaving consumers with the impression that scientists are always “flip-flopping” or “changing their minds”. (Not sure why that’s a bad thing?!?)
So the next time you hear about a “scientific breakthrough” that “changes everything”, remember that a single study is never sufficient to confirm or disprove any conclusion. It’s best to judge any study within the broader context of the existing literature, and if the findings really are groundbreaking, to withhold judgment until more evidence is available.
The phrase “do your own research” is everywhere these days. On its face it seems legit: what can be wrong with wanting to seek out information and make up your own mind?
The problem is that access to information isn’t enough. Due to confirmation bias, we seek out information that supports what we already think is true. However, the danger is that we end up cherry picking individual studies or experts and miss the bigger picture, misleading ourselves in the process.
If the goal is to find the most accurate representation of scientific knowledge, use neutral (not leading or inflammatory) search terms and make sure to use reliable sources.
And your best bet is to look for the consensus, if one has emerged.
[Learn more: How to do your own research]
Science is the most reliable method of understanding objective reality because it recognizes and corrects for our individual limitations and biases. Finding consensus isn’t easy––or pretty––as each scientist is incentivized to find flaws or limitations in each other’s work, but claims that withstand the collective scrutiny of the scientific community become the facts, models, laws, and theories that are used to further expand the body of knowledge.
Science created our modern world. It touches every aspect of our lives, from our health to the environment. The key to making good decisions, to knowing when someone is full of poo and trying to take advantage of you, is science literacy.
Unfortunately, too many science classes focus on what science knows instead of how it knows, leaving too many unable to spot claims that seem scientific…but aren’t. Science literacy is more than memorizing facts – it’s understanding how the process of science works.
As the great Richard Feynman said, “Science is a way of trying not to fool yourself. The first principle is that you must not fool yourself, and you are the easiest person to fool.”
Oreskes, N. (2019). Why Trust Science? Princeton University Press.
Understanding Science: How Science works. University of California Berkeley.
Special thanks to John Cook for his guidance and feedback.
Posted by Guest Author on Tuesday, 30 August, 2022
 |
The Skeptical Science website by Skeptical Science is licensed under a Creative Commons Attribution 3.0 Unported License. |