10 Signs That Study Is Bogus


1
It's a Cool Story
It looks like Nature was right. Let's hope you have some flame-retardant outfits handy. Don Arnold/WireImage/Getty Images

The April 2015 edition of "Nature," one of the most prestigious and reputable scientific journals in the world, startled its readership with the extraordinary results of a new study. Dragons, it said, are not mythical products of the pre-modern imagination, but real creatures who have been subsisting in a dormant phase since the Middle Ages. One of the most worrisome findings was that there was evidence that these ancient beasts are on the verge of waking up.

The publication date is the obvious tip-off that this article was an elaborate April Fool's Day prank. But despite the farcical nature of the piece, it points to an important feature of science journalism — people like good stories. Scientists know this. Even getting a paper published in a science journal can require the creation of a "beautiful story" to explain the findings. And the better the story, the more likely it is that mainstream media will cover it [source: Firger].

But scientific inquiry doesn't always result in good stories. In fact, more often than not, it doesn't. Much important research is highly inconclusive. At best, it might give us a provisional window on a possible truth. As often as not, it tells us little or nothing at all. So when it comes to science, beware of good stories, dragons or no.

Author's Note: 10 Signs That Study Is Bogus

I'm as cynical as the next person, but the research required for this article shocked me more than once. Like most people, I associate science with rigor, so I was disheartened to learn about some of the shoddy, slapdash and compromised work being done. But I suppose I shouldn't be. Scientists aren't robots operating in hermetically sealed environments. They're as subject to venality and cultural trends as anybody else. And in the end I came away with maybe even greater respect for a mode of inquiry that polices itself to the degree that science does. There's a lot of bad science out there, but the people blowing the whistle are ... scientists!

Related Articles

More Great Links

Sources

  • Brookshire, Bethany. "Psychology Is WEIRD." Slate. May 8, 2013. (Sept. 15, 2015) http://www.slate.com/articles/health_and_science/science/2013/05/weird_psychology_social_science_researchers_rely_too_much_on_western_college.html
  • Button, Kate. "Unreliable Neuroscience? Why Power Matters." The Guardian. April 10, 2013. (Sept. 18, 2015) http://www.theguardian.com/science/sifting-the-evidence/2013/apr/10/unreliable-neuroscience-power-matters
  • Couzin-Frankel, Jennifer. "When Mice Mislead." Science Magazine. Vol. 342. Nov. 22, 2013. (Sept. 15, 2015) https://www.gwern.net/docs/dnb/2013-couzinfrankel.pdf
  • Crowe, Kelly. "It's News, But Is It True?" CBC News. Oct. 5, 2012. (Sept. 14, 2015) http://www.cbc.ca/news/health/it-s-news-but-is-it-true-1.1282472
  • The Economist. "How Science Goes Wrong." Oct. 19, 2013. (Sept. 12, 2015) http://www.economist.com/news/leaders/21588069-scientific-research-has-changed-world-now-it-needs-change-itself-how-science-goes-wrong
  • Ferguson, Cat et al. "Publishing: The Peer Review Scam." Nature. Vol. 515. Pages 480-482. Nov. 26, 2014. (Sept. 17, 2015) http://www.nature.com/news/publishing-the-peer-review-scam-1.16400
  • Firger, Jessica. "Science's Reproducibility Problem." Newsweek. Aug. 28, 2015. (Sept. 12, 2015) http://www.newsweek.com/reproducibility-science-psychology-studies-366744
  • Fischer, Douglas. "'Dark Money' Funds Climate Change Denial Effort." Scientific American. Dec. 23, 2013. (Sept. 17, 2015) http://www.scientificamerican.com/article/dark-money-funds-climate-change-denial-effort/
  • Goldenberg, Suzanne. "Work of Prominent Climate Change Denier Was Funded by Energy Industry." The Guardian. Feb. 21, 2015. (Sept. 17, 2015) http://www.theguardian.com/environment/2015/feb/21/climate-change-denier-willie-soon-funded-energy-industry
  • Hamilton, Andrew J. et al. "Here Be Dragons." Nature. Vol. 520. April 2, 2015. (Sept. 14, 2015) http://water-pire.uci.edu/wp-content/uploads/2015/04/Hamilton_Dragons.pdf
  • Hiltzik, Michael. "A Bogus Study of Chocolate and Diets – And the Media Swallowed It Whole." Los Angeles Times. May 29, 2015. (Sept. 14, 2015) http://www.latimes.com/business/hiltzik/la-fi-mh-a-bogus-study-of-chocolate-20150529-column.html
  • Horowitz, Evan. "Studies Show Many Studies are False." The Boston Globe. July 1, 2014. https://www.bostonglobe.com/lifestyle/2014/07/01/studies-show-many-studies-are-false/PP2NO6lKd7HMyTZa1iCHGP/story.html
  • Ioannidis, John P.A. "Why Most Published Research Findings Are False." PLoS Med. Vol. 2, No. 8. Aug. 30, 2005. (Sept. 12, 2015) http://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124
  • Kolata, Gina. "Scientific Articles Accepted (Personal Checks Too)." The New York Times. April 7, 2013. (Sept. 17, 2015) http://www.nytimes.com/2013/04/08/health/for-scientists-an-exploding-world-of-pseudo-academia.html
  • Marcus, Adam and Ivan Oransky. "Getting the Bogus Studies Out of Science." The Wall Street Journal. Aug. 19, 2015. (Sept. 14, 2015) http://www.wsj.com/articles/getting-the-bogus-studies-out-of-science-1440024409
  • Miller, Henry I. "The Trouble With 'Scientific' Research Today: A Lot That's Published Is Junk." Forbes. Jan. 8, 2014. (Sept. 12, 2015) http://www.forbes.com/sites/henrymiller/2014/01/08/the-trouble-with-scientific-research-today-a-lot-thats-published-is-junk/
  • Yong, Ed. "Neuroscience Cannae Do It Cap'n, It Doesn't Have the Power." National Geographic Phenomena. April 10, 2013. (Sept. 18, 2015) http://phenomena.nationalgeographic.com/2013/04/10/neuroscience-cannae-do-it-capn-it-doesnt-have-the-power/

UP NEXT

Is the Universe Just a Simulation?

Is the Universe Just a Simulation?

Physicist Daniel Whiteson and cartoonist Jorge Cham ask whether the universe is a simulation on the new podcast Daniel and Jorge Explain the Universe.


More to Explore