5 Tips to Help You Science Better on the Internet

In 2016, a Ph.D candidate at the University of Wollongong was awarded a doctorate after a successful defense of her dissertation. The candidate’s central thesis was that the Australian Government’s vaccination policy was not based on credible evidence. Instead, the candidate asserted that the policy was the result of a conspiracy between the World Health Organization and Big Pharma. The candidate, to the horror of Australia’s scientific and medical community, was awarded her doctorate. It later became known that none of the examiners had a scientific background.

In recent years, there has been a significant anti-science movement in popular culture. This has led to debate over important issues such as climate change, vaccine efficacy & safety, or the safety of genetically modified food. The issue has become so pervasive in our culture that it even made the March 2015 cover of National Geographic.

NGM2015_MAR_CV2

 

Those who hold the belief that science is a bunch of evil men in white lab coats are unaware of how the process science is actually conducted. Science is not scary, it is not hellbent on humanity’s destruction, and it is certainly not all funded by evil corporations.

I think that scientific literacy is incredibly lacking in today’s population, and it is important that a greater percentage of the population not only understand how the process of science actually works, but also learn how to sniff out bad science.

Here are 5 tips to help those who lack a formal scientific education better understand some basic principles upon which science is built and how to evaluate scientific claims.

 

1) Understand the principle steps of the scientific method, specifically, replication.

300px-ScientificMethodflowchart
The scientific method: notice the replication step (reproduce the experiment)

The most common mistake made when referring to scientific findings is that people are unaware of the how the scientific method actually works.

When a group of researchers performs an experiment, they publish their results in a scientific journal. Doing so makes their findings available to the public, and one of the main purposes of publishing your results is to allow other scientists to review, critique, and replicate and potentially improve on or modify your work.

Replication is the step I would like to focus on, because an ignorance of that step has led to some very controversial conclusions, specifically that the MMR vaccine for measles causes autism and that GMOs cause cancerous tumours.

The studies I have referenced have in fact both been retracted from their respective scientific journals due to, among other things, erroneously reported data and poorly designed methods. A large part of how this data was discovered to be falsified was through the process of replication.

Because scientists were unable to produce results that were similar to that of Dr. Wakefield or Dr. Seralini, questions were raised at the legitimacy of the findings. As a result, the experiments were investigated and determined to be irrelevant because of their lack of reproducibility.

Unfortunately, the results of one study that support a preconceived notion in a debate are often treated as irrefutable proof. This condition has been termed “single study syndrome” by Andrew Revkin of the New York Times, and this term describes the attitude that many individuals possess who are skeptical or mistrusting of science.

2) The results of one study do not “prove” anything; not all journals are created equal.

In scientific vernacular, the word prove is a dangerous term, because there are always sources of error, however minuscule they may be. Any quality research paper will use words like “demonstrate” or “suggest”. In recent years, scientists from Oxford and Harvard have had papers accepted for publication in open access online scientific journals that demonstrate the suspect publishing scrutinies of many online journals.

Online scientific journals do not operate under the same constraints as traditional print journals do. If you consider a traditional print journal, such as Nature or Science, the editors of these journals get hundreds, if not thousands, of submissions for each issue. In order to maintain an excellent publishing record, only the top submissions are reviewed and selected for publication.

This is what is known as “peer review”. A panel of researchers who are highly accomplished in their respective fields are responsible for reviewing submissions to determine if the methods are sound, the data are accurate, and the conclusions are logical. Peer review allows for a system of checks and balances that keeps science as honest and open-minded as possible. As a result of their excellent publishing reputation, studies that are published in top journals are more highly respected and prone to citation in future studies.

Most online open access journals operate under a different business model. Since these journals are open access, anyone can access their content, so they need a different source of income. If a group of authors wishes to publish their results in an online open access journal, they can simply pay the journal a publishing fee, and the journal will publish their work; there is no panel of submission reviewers to weed out the lower quality submissions.

So if the names of these open access journals are very similar, how do you know if a journal’s contents are trustworthy? There are two useful metrics to determine the influence and prestige of of scientific journal: the SJR and the h-index, the latter of which can also help determine the influence and prestige of an individual researcher.

The SJR, or scientific journal ranking, is a ranking of the average number of weighted citations received by documents published in the journal in a given time period. Essentially, the more citations a journal garners, the more influential its contents — pretty straightforward reasoning.

The h-index is a measure of publishing reputation first described by Jorge E. Hirsch, which is calculated using the following formula: A scientist has index h if h of his/her Np papers have at least h citations each, and the other (Np − h) papers have no more than h citations each.

In other words, if I’m a researcher and I have published 15 papers that contain at least 15 citations, I have an h-index of 15; if I have published 20 papers with at least 20 citations, my h-index will be 20, and so on.

300px-H-index-en.svg

These two methods should be used in conjunction when evaluating the quality of a journal, as the SJR of many journals can be inflated with popular review articles or techniques. For example, the paper that first described the technique and application of Polymerase Chain Reaction, a very common technique in molecular biology, will be referenced quite often due to the popularity of the technique in DNA sequencing.

3) Beware of circular referencing and intellectual inbreeding.

Another common tactic briefly discussed in the previous example was that of intellectual inbreeding, which is an impediment for producing quality research. A good researcher will examine multiple angles of their experiment to find many different layers of evidence for their argument. Breadth of coverage makes for sound reasoning and a more stable base of support. This doesn’t mean that more citations automatically makes a paper stronger, but a greater variety of evidence sources usually does.

Consider the case of many pseudoscientific websites: while many of their articles may include a good deal of references, these references often loop back to previous articles written by the same author, or they reference the same study multiple times. Many websites publish multiple articles reporting the results of the same study years apart. Circular referencing is not an adequate form of evidence, and is a common deceptive tactic employed by many pseudoscientific websites to dress up their articles under a guise of credibility.

4) Just because an author has letters after their name, it does not mean their research is valid.

This point builds upon the concepts of peer review and replication. There are a growing number of individuals who are taking advantage of the public’s nature to trust single sources of authority; after all, we are raised to trust our doctor, our dentist, our veterinarian, our pharmacist, and, if we attend university, our professor. Unfortunately, as discussed previously, one study — or in this case one person — does not necessarily equate to a valid opinion. This is otherwise known as an “appeal to authority” — a common logical fallacy.

If your doctor tells you to take your regularly prescribed medication for a thoroughly researched ailment, such as taking an antibiotic like Penicillin for an infected cut, then there should be no doubt in your mind to trust them. What you should be wary of is methods touted as “new” or “revolutionary”. These words are used to dress up unsupported claims dreamed up by individuals with the proper credentials but lacking the proper evidence.

What many of these unscientific claims take advantage of is the general public’s lack of scientific literacy. By sprinkling some authentic-sounding science terms on a product or service that they are selling, anyone with the letters “Dr.” before their name can make a bogus product or service seem quite legitimate.

A classic example of a pseudoscientific book written by someone with illusory credentials
A classic example of a pseudoscientific book written by someone with illusory credentials

What much of the public doesn’t know is that even someone as educated as a physician may be just an unqualified as you or I to give advice on particular health topics. This may seem shocking, but medical school (including the residency process) is lengthy for a reason: the body is an incredibly complex system, and physicians are specialized in various areas of it for that same reason.

According to her CV, Dr. Amy Meyers is trained as an emergency physician — she doesn’t know more about the gut than you or I if we spent a few hours researching the matter online or if we read a basic physiology textbook. She has also been featured in the Huffington Post and on the Dr. Oz show — both of which are prone to promoting pseudoscience and health myths. Unless these physicians are gastroenterologists, they have no business posing as an expert on the health of your gut.

 

5) Doing your research takes a lot longer than using Google for half an hour 7783365118_ceac1b72a7_z

Public distrust of science has certainly stemmed from the widespread availability of information online. This is indicative a larger cultural shift where experts are doubted more than ever. Tom Nichols’ 2017 book The Death of Expertise reviews this problem in great detail. Debate can happen anywhere, so the barrier to entry no longer requires a well-thought out submission letter or an invitation. You just log on to Facebook or Twitter and let loose the words of ignorance.

Scientific professionals undergo years of training just to be granted their license to research or practice, and even then their education is never fully complete. There is a contingent of armchair scientists who seem to think that reading a few scientific papers (let’s be honest, abstracts) or articles on a matter is a sufficient replacement for a decade of post-secondary education.

What much of the public has to remember is that scientific research is still very much an artisanal pursuit, and we as the general public should give the collective of scientific professionals the respect and trust they deserve. We should also understand that scientists, like all experts, do make mistakes from time-to-time, but that shouldn’t undermine their credibility. If anything, it should make them more accessible in the sense that it reinforces that experts are people just like us; they just happen to know a lot about one thing, and we should give them the respect they deserve.

 

Advertisements

8 Comments Add yours

  1. David says:

    YES!!!
    Although I might point out that this might be rather long and involved for those without adequate background.

  2. Lars Moksness says:

    Hi. It appears like you are describing what’s known as “predatory journals”, while giving the impression that this is how all Open Access works. This is not correct.

  3. Hi Lars, I’m aware of a few notable open access journals, such as PLoS ONE, but most open access journals, as described in the articles I cited, are predatory. I’ll edit the article to make sure that I don’t make a sweeping generalization. Thanks

    1. Nathaniel says:

      Also, most journals charge researchers to publish their work.

      I think a bigger problem here is that you are trying to explain how the process works, but all you do is support “the establishment” that “independent researchers” have to go outside of because their findings are so “controversial.” What I mean is, you are only reinforcing the preconceived beliefs that encourage science-doubters to turn away from real research. I admire you effort, but I think all you will do is preach to the choir. Science doubters believe they are fighting the elitist scientific establishment that id keeping “the truth” from the average citizen. If you want to help change their minds, it would help to not sound like an elitist supporter of “the establishment.”

  4. Beth says:

    “According to her CV, Dr. Amy Meyers is trained as an emergency physician – she doesn’t know more about the gut than you or I if we spent a few hours researching the matter online or we read a basic physiology textbook. … Unless these physicians are gastroenterologists or gastrointestinologists, they have no business posing as an expert on the health of your gut.”
    If so, many skeptics are guilty of that, in spades. The Science Based Medicine blog has featured a pharmacist blogging about what food allergies “really exist” – as if a pharmacist should know! I’ve wished they would have a researcher on food hypersensitivity writing instead. Orac blogs about many things outside his specialty as a cancer surgeon, and him being a doctor surely gives his opinions extra credibility.

  5. Beth says:

    Sometimes it’s hard to tell if popular health books do harm or good.
    For example, I suggested to the woman who cuts my hair that she see a doctor to get tested for celiac disease. That was because she has Graves disease, an autoimmune thyroid disease. That made it more likely she has celiac disease, since it’s an autoimmune disease. Also, celiac disease is more common in people with autoimmune thyroid disease.
    But it went in one ear and out the other.
    Then a few years later, I went to get my hair cut. This time, she was raving about how she’d gone on a gluten-free diet and how much better she felt. She’d read “Grain Brain”, a scientifically dubious book about gluten.
    I mentioned some problems that tend to be caused by celiac disease – osteoporosis, anemia, peripheral neuropathy. She kept on saying yes, she’d had those problems.
    So it seemed likely that she does have celiac disease.
    I was happy that she’d discovered that a gluten-free diet might be important for her. Likely she’ll be healthier now.
    But, she’s unlikely to get a proper celiac disease diagnosis now. She would have to do a “gluten challenge” for month(s), eating enough gluten to re-damage her intestines so that the standard diagnostic tests for celiac disease (blood tests, intestinal biopsy) will work again.
    It can be very unpleasant to do a gluten challenge. If you do have a gluten problem, you will have lost the tolerance you had to gluten, and doing a gluten challenge could make you very sick. So a lot of people don’t do it.
    This is a problem because people with celiac disease are told to avoid gluten completely, not just cut down. They avoid gluten in medications; they check for gluten in all sorts of non-obvious sources, etc.
    And people who’ve read a popular book about gluten tend to be more casual about it than that. They don’t take it that seriously – when maybe, avoiding gluten completely could save their life.
    There probably are a lot of people who are kidding themselves about having a gluten problem.
    But, there are many others who do have celiac disease, and they’re under-diagnosing, not over-diagnosing.

  6. bob peckham says:

    How do peer reviewers determine the accuracy of the data ?

    1. They’re experts in the field. A review of the results section of any scientific paper will shed light on the accuracy of the results

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s