In recent years, there has been a significant anti-science movement in popular culture. This has led to debate over important issues such as climate change, vaccine efficacy & safety, or the safety of genetically modified food. The issue has become so pervasive in our culture that it even made the March 2015 cover of National Geographic.
Those who hold the belief that science is a bunch of evil men in white lab coats are unaware of how the process science is actually conducted. Science is not scary, it is not hellbent on humanity’s destruction, and it is certainly not all funded by evil corporations. In fact, most of it is conducted at public institutions with open visibility to the research being conducted. I think that scientific literacy is incredibly lacking in today’s population, and it is important that a greater percentage of the population not only understand how the process of science actually works, but also learn how to sniff out bad science.
Here are 5 tips to help those who lack a formal scientific education better understand some basic principles upon which science is built and how to evaluate scientific claims. It is my hope that this article can help others become more scientifically literate and alleviate any concerns or fears of science they may currently hold.
1) Understand the principle steps of the scientific method, specifically, replication.
The scientific method: notice the replication step (reproduce the experiment)
The most common mistake made when referring to scientific findings is that people are unaware of the how the scientific method actually works. The scientific method is a guideline for how to produce logically sound conclusions from a series scientific experiments.
When a group of researchers performs an experiment, they publish their results in a scientific journal. Doing so makes their findings available to the public, and one of the main purposes of publishing your results is to allow other scientists to review, critique, and replicate and potentially improve on or modify your work. Replication is the step I would like to focus on, because an ignorance of that step has led to some very controversial conclusions, specifically that the MMR vaccine for measles causes autism and that GMOs cause cancerous tumours.
The studies I have referenced have in fact both been retracted from their respective scientific journals due to, among other things, erroneously reported data and poorly designed methods. A large part of how this data was discovered to be falsified was through the process of replication. If an experiment was designed properly, one scientist should be able to produce similar results as another scientist if they follow the method properly. Because scientists were unable to produce results that were similar to that of Dr. Wakefield or Dr. Seralini, questions were raised at the legitimacy of the findings. As a result, the experiments were investigated and determined to be irrelevant because of their lack of reproducibility.
Unfortunately, the results of one study that support a preconceived notion in a debate are often treated as irrefutable proof. This condition has been termed “single study syndrome” by Andrew Revkin of the New York Times, and this term describes the attitude that many individuals possess who are skeptical or mistrusting of science. However, as demonstrated by the principle of replication, one study is never sufficient evidence to support a notion in a debate, especially if you consider the journal that the study has been published in, which conveniently leads into the 2nd tip.
2) The results of one study do not “prove” anything; not all journals are created equal.
In scientific vernacular, the word prove is a dangerous term, because there are always sources of error, however miniscule they may be. Any quality research paper will use words like “demonstrate” or “suggest”, but any research paper that employs pervasive use of the words prove or proof is suspect at best. In recent years, scientists from Oxford and Harvard have had papers accepted for publication in open access online scientific journals that demonstrate the suspect publishing scrutinies of many online journals. Online scientific journals do not operate under the same constraints as traditional print journals do. If you consider a traditional print journal, such as Nature or Science, the editors of these journals get hundreds, if not thousands, of submissions for each issue. In order to maintain an excellent publishing record, only the top submissions are selected for publication.
This is the basic of what is known as “peer review”. A panel of researchers who are highly accomplished in their respective fields are responsible for reviewing submissions to determine if the methods are sound, the data are accurate, and the conclusions are logical. Peer review allows for a system of checks and balances that keeps science as honest and open-minded as possible. As a result of their excellent publishing reputation, studies that are published in top journals are more highly respected and prone to citation in future studies. Subscriptions are required to view these journals, but because of their excellent reputation, these journals never have an issue selling subscriptions to educational institutions or fellow researchers.
Most online open access journals operate under a different business model. Since these journals are open access, anyone can access their content, so they need a different source of income. If a group of authors wishes to publish their results in an online open access journal, they can simply pay the journal a publishing fee, and the journal will publish their work; there is no panel of submission reviewers to weed out the lower quality submissions.
The two links above provide an eye-opening account of how researchers from world-renowned universities purposely published studies that were completely fabricated (the Harvard paper was titled: “Cuckoo for Cocoa Puffs? The surgical and neoplastic role of cacao extract in breakfast cereals”). The online journals were prepared to publish these papers if the researchers would commit to paying a “processing fee” of $500. The Harvard paper contained a completely made up experiment authored by two fake authors: Pinkerton A. LeBrain and Orson Welles. Journals that operate under this model have legitimate sounding names, but the quality of the papers contained in these journals pales in comparison to their similarly-named cousins.
So if the names of these open access journals are very similar, how do you know if a journal’s contents are trustworthy? There are two useful metrics to determine the influence and prestige of of scientific journal: the SJR and the h-index, the latter of which can also help determine the influence and prestige of an individual researcher.
The SJR, or scientific journal ranking, is a ranking of the average number of weighted citations received by documents published in the journal in a given time period. Essentially, the more citations a journal garners, the more influential its contents – pretty straightforward reasoning. The h-index is a measure of publishing reputation first described by Jorge E. Hirsch, which is calculated using the following formula: A scientist has index h if h of his/her Np papers have at least h citations each, and the other (Np − h) papers have no more than h citations each.
In other words, if I’m a researcher and I have published 15 papers that contain at least 15 citations, I have an h-index of 15; if I have published 20 papers with at least 20 citations, my h-index will be 20, and so on. 
These two methods should be used in conjunction when evaluating the quality of a journal, as the SJR of many journals can be inflated with popular review articles or techniques. For example, the paper that first described the technique and application of Polymerase Chain Reaction, a very common technique in molecular biology, will be referenced quite often due to the popularity of the technique in DNA sequencing. A common technique used by anti-science websites is to cite studies to support a preconceived notion (GMOs are harmful, vaccines cause health complications).
For example, the website gmoevidence.com, a website dedicated to proving the harms of GMOs, uses studies from “independent researchers” to support their agenda. If we examine an article on the website, “Dr. Oraby: GM Soy and Maize Toxic to Rats”, we can break down the anti-scientific approach employed by these types of websites and find the faults in their reasoning. The first step to determining the legitimacy of a study is to investigate the quality of the journal it was published in.
The study in question was published in the Turkish Journal of Biology. If we search for the journal using the SJR database, we can see that the journal has a very low h-index ranking relative to top journals. The top two journals in the world, Nature and Science, have h-indices of 829 and 801, respectively, which means that in the last 3 years, Nature has published 829 articles that have been cited 829 times while Science has published 801 articles cited 801 times.
The Turkish Journal of Biology has an h-index of 18, which means that in the last three years, the journal has published 18 papers that have been published 18 times. As a result, the journal has a very low h-index ranking relative to other journals around the world. The Turkish Journal of Biology also contains a very high number of self-citations, which is indicative of intellectual inbreeding and an overall low quality of research.
Additionally, almost 40% of the journal’s documents have been uncited, which further supports the notion that documents published in the Turkish Journal of Biology are not very influential; the research methods are poorly designed and the results are not significant in the grand scheme of the scientific community. In a recent review of literature across all academic disciplines, researchers found that almost 90% of scientific papers were cited at least once. The fact that the journal has such a high number of uncited documents indicates that no other scientists are performing similar work, which further supports the notion that the quality of this study is suspect at best.
After examining the journal and drawing the appropriate conclusions from the h-index and SJR, we can take the investigation one step further by examining the authors of the paper. For simplicity, let’s examine the lead author on the paper, Dr. Hanaa Oraby. Google Scholar provides a citation search tool that enables anyone to search for an researcher’s scientific publications.
A search for Dr. Oraby reveals that she has published 17 papers since 1989, has received 113 total citations on her work, and has an h-index of 5. Not bad, but if we examine her paper mentioned in the GMO evidence article, we can see that it has not received any citations! In fact, Dr. Oraby has published a total of 8 papers – including one other one on the dangers of GMOs – that have received no citations. Based on this analysis, we can conclude that Dr. Oraby’s research on GMOs has not been sought out by her peers.
This is due to several reasons, the most obvious of which is that her research simply isn’t useful or influential enough to cite, which is a polite way of saying that it’s poor science. Simply put, just because one study reaches a conclusion is no reason to accept that conclusion as credible fact. The journal quality and the researcher’s publishing history can reveal a great deal about their research credibility.
3) Beware of circular referencing and intellectual inbreeding.
Another common tactic briefly discussed in the previous example was that of intellectual inbreeding, which is an impediment for producing quality research. A good researcher will examine multiple angles of their experiment to find many different layers of evidence for their argument. Breadth of coverage makes for sound reasoning and a more stable base of support. This doesn’t mean that more citations automatically makes a paper stronger, but a greater variety of evidence sources usually does.
Consider the case of many pseudoscientific websites: while many of their articles may include a good deal of references, these references often loop back to previous articles written by the same author, or they reference the same study multiple times. Many websites publish multiple articles reporting the results of the same study years apart. Circular referencing is not an adequate form of evidence, and is a common deceptive tactic employed by many pseudoscientific websites to dress up their articles under a guise of credibility.
4) Just because an author has appropriate credentials does not mean that their research is valid.
This point builds upon the concepts of peer review and replication. There are a growing number of individuals who are taking advantage of the public’s nature to trust single sources of authority; after all, we are raised to trust our doctor, our dentist, our veterinarian, our pharmacist, and, if we attend university, our professor. Unfortunately, as discussed previously, one study – or in this case one person – does not necessarily equate to a valid opinion. This is otherwise known as an “appeal to authority” – a common logical fallacy.
If your doctor tells you to take your regularly prescribed medication for a thoroughly researched ailment, such as taking an antibiotic like Penicillin for an infected cut, then there should be no doubt in your mind to trust them. What you should be wary of is methods touted as “new” or “revolutionary”. These words are used to dress up unsupported claims dreamed up by individuals with the proper credentials but lacking the proper evidence. Those who are most commonly guilty of this are physicians or other “doctors” who design new diets, lifestyles, or supplements; by and large these are all designed to profit off of unsuspecting patients, and these methods are dressed up in scientific jargon that doesn’t hold water with actual research.
What many of these unscientific claims take advantage of is the general public’s lack of scientific literacy. It is said that the average university biology course contains more new words than a university language course, so becoming scientifically literate is difficult enough given the large vocabulary required to navigate scientific concepts. By sprinkling some authentic-sounding science terms on a product or service that they are selling, anyone with the letters “Dr.” before their name can make a bogus product or service seem quite legitimate. This is often done to sell the individual’s brand to the public, whether it’s a book, seminars, health consultations, or other practices done outside the scope of their professional designation.
A classic example of a pseudoscientific book written by someone with illusory credentials
What much of the public doesn’t know is that even someone as educated as a physician may be just an unqualified as you or I to give advice on particular health topics. This may seem shocking, but medical school (including the residency process) is lengthy for a reason: the body is an incredibly complex system, and physicians are specialized in various areas of it for that same reason. This is why it is nonsense for a neurologist to give advice on the health of your heart; that is the area of expertise for a cardiologist. The same goes for the trend of celebrity doctors giving advice on the health of the gut as a function of overall health.
According to her CV, Dr. Amy Meyers is trained as an emergency physician – she doesn’t know more about the gut than you or I if we spent a few hours researching the matter online or if we read a basic physiology textbook. She has also been featured in the Huffington Post and on the Dr. Oz show – both of which are prone to promoting pseudoscience and health myths, so those should also pop up as red flags towards her health brand’s credibility. Unless these physicians are gastroenterologists or gastrointestinologists, they have no business posing as an expert on the health of your gut. We should also tie this back to the first point: much of what Dr. Meyers is selling hasn’t been replicated, which should raise some eyebrows as to what her true motivation is (hint: it’s money).
5) Doing your research takes a lot longer than using Google for half an hour 
Public distrust of science has certainly stemmed from the widespread availability of information online. It is a consequence of our arrogance as humans that we think we know better than anyone else. What we seem to forget is that there is a reason certain individuals are trained professionals and others are not.
Scientific professionals undergo years of training just to be granted their license to research or practice, and even then their education is never fully complete. There is a contingent of armchair scientists who seem to think that reading a few scientific papers or articles on a matter is a sufficient replacement for a decade of post-secondary education. If the information and training taught to medical school students or scientific researchers was widely available online, there wouldn’t be much point in enrolling in medical school or a Ph.D. in biochemistry.
What much of the public has to remember is that scientific research is still very much an artisanal pursuit, and we as the general public should give the collective of scientific professionals the respect and trust they deserve.
I hope this guide has provided you with some useful information on how to navigate the sea of scientific information available online. There is a lot of misinformation out there, and it’s important that the public learn how to identify the legitimate from the pseudoscientific. This guide was also designed to help you avoid being scammed by numerous peddlers of pseudoscience. In closing, remember that good science requires replication, no study can “prove” anything, not all journals are created equally, check the credentials of your scientific authorities, and put your trust in those who have studied science for decades.
