Understanding science as an activist
As an activist, have you ever tried to read a scientific article and been completely lost? To be an efficient and science-informed activist, you will need to understand what science is, what it means to be scientifically literate, what goes into making a scientific article, and how to read an article with an eye towards understanding and identifying bias. If you have a strong foundation in understanding the process of science, then you will become a more effective advocate who can utilize scientific data to understand the most pressing issues we face and develop evidence-based solutions.
What is this thing called science?
Science as knowledge, process, and institution
What do you think about this thing we call science? For many, science invokes images of lab-coats, beakers, and traveling through challenging climates to study strange yet beautiful creatures. Others think about asking big questions, creating a hypothesis, gathering data, analyzing it, and forming conclusions. Perhaps science is the knowledge: scientific journals that publish ground-breaking research or textbooks found in any high school across the United States. Another group may think about the institutions attached to science, swayed by political forces, like the Environmental Protection Agency or the Food and Drug Administration. Most understand science as being all these components: the process, the knowledge, and the institutions.
Activists play a key role in informing science and being informed by science. An urgent and important example currently playing out across the world is that of climate change. Activists must understand the science of how and why the climate is changing to advocate for policies that can reduce the most important contributors to climate change and be able to convert people skeptical of human-induced climate change. Further, the methods of science can be used to validate and check claims made for the sake of political gains. In politics, people may use their discourse to further their own political aims rather than relying on sound reasoning and methodology. While science is not completely objective or always factual, an activist who understands how to read a scientific article can be more objective in their discourse because science safeguards against some bias.
The trap of quick tips and tricks
We all have limited time and attention, which leads us to want quick tips and tricks on how to understand the world. However, these simplifications can prevent someone from building a strong understanding by replacing thinking with, potentially counter-productive, intuitions. Let’s discuss a few examples of potential ‘tips and tricks’ which can lead to adverse outcomes in understanding science:
a.) “Make sure you check the sources to determine if someone is using evidence.” At first glance, someone may write an article with a huge list of sources. Using this tip, you may be inclined to believe them based on having references. However, consider that people can cite an article with the knowledge that most people won’t check the source to see what it says. Alternatively, someone may be writing based on what they assume is standard knowledge. An environmental biologist, for instance, discussing the mechanisms of evolution, may entirely be correct without listing sources. Lastly, consider that anyone can create a journal with a reputable name and offer to publish papers if the author pays for the service. As such, I could create a journal called “International Environmental Science Concerns” and publish a wide range of articles without checking them.
b.) “Check the author's credentials” can mislead people in three ways. First, many for-profit schools can be unaccredited, offering credentials for money and limited work. Second, someone may have a credential but place making money above their duty to telling the truth – as such, they may be snake-oil salespeople who attempt to discredit other scientists. For instance, the cigarette companies found credentialed scientists to publish pieces casting doubt on the relationship between smoking and cancer. Third, people can be credentialled in a certain field such as astrophysics but be making claims in a different field such as health; in this case, their credential does not credit them to make claims as an expert in the field of health.
c.) “If they are selling something, don’t listen” may seem normal, but it could lead to a distrust in any professional scientist. For instance, despite the overwhelming scientific consensus about the importance of vaccination, some people could take the stance that physicians are ‘selling’ vaccines, which means they must be un-scientific.
All of these are examples of why, in this article, we will be exploring scientific literacy and process instead of offering quick tips.
Foundations of scientific literacy
Scientific literacy means that a person can ask questions, gather data, and create explanations about experiences and phenomena in the natural world (National Academies of Sciences et al., 2016). Researchers think of scientific literacy as an umbrella, which covers many different components.
1. Foundations: Scientific literacy depends on being able to analyze and interpret texts, a skill which includes visual literacy, textual literacy, numeracy, and understanding charts and graphs.
2. Content: A person needs to understand a set of knowledge about scientific terms, concepts, and facts often covered in K-12, such as biology, chemistry, physics, etc.
3. Process: A person needs to understand how scientists do science. In other words, they need to understand how scientists design experiments, collect data and analyze it. For instance, scientists communicate through peer-review, utilize different methods with varying generalizability, and attempt to manipulate variables to produce an outcome.
4. Credentials: A person must be able to evaluate whether a person is an expert on a particular subject based on their standing and prestige in the scientific community and whether their credentials make them an appropriate spokesperson for an issue.
5. Principles:A scientifically literate person understands the principles which allow us to believe in scientific evidence, and the role uncertainty plays in science. They can also recognize what types of questions science can answer, ethical issues in science, and the strengths/limitations of scientific inquiry.
6. Culture: A person understands that science can often reflect a researcher’s social position within society. In other words, a racist society can (though will not necessary) produce biased research based on a misguided cultural frame. However, people understand that scientific methods can also unearth these cultural biases over time.
7. Attitude: A scientifically literate person needs to be open-minded, value science as one method of understanding the world, and commit to utilizing scientific evidence to answer questions the best they can.
How to read and digest a scientific article as an activist
Reading a Scientific Article for Understanding
After you have found an article that seems to be published from a credible journal and written by a credible author, the next step is reviewing the article itself. If you are first developing skill in reading scholarly works, this may be a slower process as you get used to the standard format of scientific articles, the methods articles utilize, and the content of the field. Don’t get discouraged if you don’t understand an article right away –focus on what you don’t understand and why. Sometimes you don’t understand a foundational concept written about in the article, and other times the results of the article may be ground-breaking, but the author does not explain the concept in an accessible way.
Start with reading the title, which should get across the main point of the article. Then, you should read the abstract, which breaks down each part of the paper into small summaries. If you still believe the article to be interesting or relevant to what you’re trying to understand, then you can move on to reading the rest of the article based on what you want to know. Avoid the temptation to read the article from start to finish. You can move back and forth between sections to understand things. You can start with the introduction to understand why the research question is important, and the rationale for the study. Then, you can move onto the analysis and conclusion to understand the implications of what they found. Reading the methods and the results sections can give you an insight into what was going on under the hood to lead to their analysis. Since scholarly articles will be well-cited, you should look into relevant citations that interest you if you want more background on a particular claim or if you want a deeper understanding of a concept.
Understanding theory, peer-review process, and bias
Research involves an interplay between scientific theory and observational data. Theory informs us how to interpret data, which observations influence what theories we make. When data does not match established theories, a keen researcher would not simply throw out something established as a result of one outlier. Instead, a researcher should attempt to understand how the researchers got the results, assess for bias, and see if other interpretations of the data could lead to a different analysis. Very rarely, the results truly do disprove a theory, and, over time, the scientific community incorporates this new theory to reexplain past data. One must have an open mind reading any study, but also should exert caution in believing claims which would reject previous interpretations of the evidence.
Scientists communicate their research and findings through journals, social media, and public presentations. While the scope of this paper cannot address analyzing all of these communication mediums, there is a need to understand the peer review process, which regulates the claims scientists make. A researcher writes up their methods, results, and analysis of their experiment,a which they sent to a scientific journal. Reputable journals call on other scientists to read the report and find flaws with the reasoning, methods, and conclusions the article reaches to determine whether to publish the piece, sent it back for revisions, or reject the piece outright. This scrutiny, while it doesn’t catch everything, creates a filter to provide further dissemination of research. Over time, scientists may eventually reject ideas that were previously published. However, this does not mean that science is wrong – it means that science is working! Published papers can create a discussion of alternative interpretations, and dialog about theories as a whole. As such, the strength and rigor of a journal rely on the editorial board – with some journals publishing nearly all papers submitted with others, such as Nature, only publishing work which has passed expert scrutiny.
As such, the peer review process attempts to avoid bias. Published papers may not always be true, but they can generate important discussions. Over time, the scientific community seeks to incorporate the best evidence into theories, and broader understandings that scientists can communicate to the public. In this science, science could have been ‘wrong’ from our standpoint now, but we only get to our current standpoint by communicating research findings and building from previous works. However, publications may thus only focus on positive findings that change our understanding of a topic, instead of publishing results which do not find any difference. For example, even though women and men maintain incredible amounts of similarity in brain structure and function, publications usually only report on significant differences.
Theory and laws represent different ideas in science. A theory is an in-depth explanation of a phenonomen supported by empirical evidence which explains why a relationship exists. New evidence can support a theory or refute a theory, but can never prove or disprove it. Laws, such as Newton’s Law of Gravity, provide a description instead of an explanation without describing how something works. The focus of laws is explaining why the relationships exist. If a law is incorrect, then formulas and euqations may be adjusted to ensure the law presents a correct description. If a theory is incorrect, then the explanation must change to develop a more sound theory over time. Through the peer review process and dialoge around theories, the scientific community can develop deeper understanding about how the world works.
Bias and Errors in Interpreting Research
The Introduction
The introduction of an article has three purposes. First, it should provide a context for why the researchers decided to write or study the problem. Second, it should provide a rationale for why the researcher chose a method to study the problem. Third, it should clearly state the research question and hypothesis, which guides the rest of the paper.
An introduction can suffer from bias when there is a lack of theory on the topic. For instance, the researcher could be presenting findings without understanding how those findings relate to a larger theoretical understanding. In this case, the study may not answer an important question, or the study might not be representative of a larger set of scientific problems. The researcher could also create an overly broad research question, which the paper cannot sufficiently answer. If the research question seems too large for the data and method, then you may want to reinterpret their data within a narrower research question. When analyzing an article like this, keep in mind the author may be making claims outside of what their study can prove.
Methods
The methods section of the article should describe where the data came from, how each variable in the study was measured, where the study took place, whether there was ethical approval from an institutional review board to run the study, and how the researchers undertook the analysis. The methods section should give us enough detail to replicate their experiment to see if we can replicate their results.
The methods section could be biased in a few ways. The method may not match the aims of the study. For instance, if an article states that they were studying how cancer develops in humans, and then their experiment took place in rats, then the methods would not match the actual aims of the paper. While my method could inform human research, it would be potentially misleading to anyone who wasn’t taking a closer look at the methods and results of the subject. Another issue to watch out for is non-standard techniques. For instance, the Weschler Abbreviated Scale Intelligence test is the standard assessment for IQ in people; if a study created their own set of 10 questions to measure IQ, then you may be more skeptical of their results. Lastly, the methods may show a small sample size, which relates to how many unique individuals experienced the experiment. If I told you that coffee improved my headaches, then you may remain skeptical. However, if I told you that I enrolled 1 million people in my study and found that coffee improved headaches then the large sample may hedge against random effects.
Results
The results section should lay out what the researchers found. For research involving humans, this should include demographic characteristics of the population studied and what happened to everyone in the study.
The results are determined through the statistical analysis developed in the methods and the rationale developed in the introduction. However, statistics could be misinterpreted to show a larger effect than intended. For instance, my statistical analysis may indicate that I multiplied the number of headaches experienced per month by ten so that people who reported experiencing about one headache per ten months would be shown as experiencing a Standardized one headache per month, and those experiencing one headache per month would show a Standardized ten headaches per month. While this would likely not happen, it demonstrates why understanding the methods could lead to different interpretations of the results. A second issue is developing literacy in reading graphical displays of data, which can mislead people depending on the scales the axis use. For instance, if my Y-axis is scaled from 40-60, then the difference between 40 and 60 will look a lot larger than if my Y-axis is scaled from 0 to 100. Keeping in mind how statistics can be used to mislead you is important in understanding how to read a scientific article.
Analysis
The author then takes their results and analyses them concerning their hypothesis developed in the introduction and attempts to answer the research question. The main issue to look for in the analysis is a lack of limitations on the data. Some researchers may analyze their data, and make a claim expanding beyond what their results may show. In other words, they may overgeneralize and suspect their results will be similar for problems which are different.
Affiliation
Occasionally, the author of an article may be affiliated with an organization or receive funding from an organization related to the research, which can limit the author’s credibility or incentivize them to publish misleading articles. Claiming someone has committed fraud by not sharing all the evidence, or cherry-picking results is a serious accusation; however, scientists do not always act ethically in an attempt to produce knowledge or may have a bias to stretch their results to please who funds their research. However, affiliation does not necessarily mean research is biased. Instead, this should be evaluated in the wider context of their research career, what other researchers have found, and whether the funders are hands-on or hands-off in terms of publishing results.
Conclusion
Reading a scientific article is a life-long skill, which one can improve upon to be more effective activism. Science has allowed us to understand the world and has great power when appropriately analyzed with limitations in mind. To tap into the world of science, an activist needs to understand scientific literacy as a foundation and move towards gaining proficiency in seeking out bias in studies. With a critical mind, you will be able to improve your advocacy, and thus the world, by interpreting scientific findings.