10 Types of Study Bias

File Drawer Bias
On the flip side, scientists may relegate negative or neutral findings from clinical trials to a file drawer. blackred/Getty Images

In some ways, this is the flip side of publication bias. Negative results from a study get shoved in a metaphorical file drawer instead of being published. Critics see it as a particular problem when it comes to studies of new medications, which these days often are sponsored by the companies that developed them [source: Pannucci and Wilkins].

File-drawer bias can be significant. A study published in the New England Journal of Medicine in 2008 compared the results of published studies on antidepressants to data from a U.S. Food and Drug Administration registry of research that included unpublished information. It found that 94 percent of the published studies reported drugs having positive effects. But when the unpublished studies were included, the number with positive results dropped to 51 percent [source: Turner, et al.].

In an effort to get more information into the public domain, Congress in 2007 passed a law requiring researchers to report results of many human studies of experimental treatments to ClinicalTrials.gov. In 2016, the U.S. Food and Drug Administration strengthened the rules, requiring more thorough reporting of clinical trials, including drugs and devices that were studied but never brought to market [source: Piller].

But some critics worry that the laws won't have much teeth since there is no increase in enforcement staffing.

Author's Note: 10 Types of Study Bias

This assignment was an interesting one for me, since over the years I've often had to write articles based upon scientific research. Journalists, I think, have to avoid the temptation to assume that the latest published study must be the definitive word on any subject.

Related Articles

More Great Links


  • Athanasiou, Thanos, etal. "Key Topics in Surgical Research and Methodology." Page 32. Springer, 2010. (Sept. 10, 2017) http://bit.ly/2vZ9rsn
  • Boston College. "Differential Misclassification of Exposure." Bu.edu. (Sept. 10, 2017) http://bit.ly/2vYFIQo
  • Burge, Sandra. "Bias in Research." Familymed.uthscsa.edu. (Sept. 9, 2017) http://bit.ly/2xXMRhl
  • Center for Innovation in Research and Teaching. "Sources of Error and Bias." Cirt.gcu.edu. (Sept. 8, 2017) http://bit.ly/2xXsLne
  • Cochrane Methods. "Assessing Risk of Bias in Included Studies." Cochrane.org. (Sept. 9, 2017) http://bit.ly/2xXyl8W
  • Delgado, M., etal. "Bias." Journal of Epidemiology and Health. August 2004. (Sept. 10, 2017) http://bit.ly/2vYAtQO
  • Dusheck, Jennie. "Studies of scientific bias targeting the right problems." Med.stanford.edu. March 20, 2017. (Sept. 9, 2017) http://stan.md/2xXcCyh
  • Dwan, Kerry, etal. "Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias — An Updated Review." PLOS ONE. July 5, 2013. (Sept. 9, 2017) http://bit.ly/2xX2a9J
  • Enserink, Martin. "Most animal research studies may not avoid key biases." Science. Oct. 13, 2015. (Sept. 9, 2017) http://bit.ly/2xWwhy6
  • Fanelli, Daniele. "Do Pressures to Publish Increase Scientists' Bias? An Empirical Support from US States Data." PLOS ONE. April 21, 2010. (Sept. 7, 2017) http://bit.ly/2xXgvD1
  • Fanelli, Daniele. ""Positive" Results Increase Down the Hierarchy of the Sciences." PLOS ONE. April 7, 2010. (Sept. 7, 2017) http://bit.ly/2xYmLKR
  • Fanelli, Daniele; Costas, Rodrigo; and Ioannidis, John P.A. "Meta-assessment of bias in science." PNAS. March 20, 2017. (Sept. 7, 2017) http://www.pnas.org/content/114/14/3714
  • Freedman, David H. "Why Scientific Studies Are So Often Wrong: The Streetlight Effect." Discover. Dec. 10, 2010. (Sept. 10, 2017) http://bit.ly/2xYJTJ4
  • Hampton, John. "Therapeutic fashion and publication bias: the case of anti-arrhythmic drugs in heart attack." JLL Bulletin. 2015. (Sept. 10, 2017) http://bit.ly/2xXUN1L
  • Henning, Jeffrey. "Order Bias Is a Larger Source of Error Than You Think." ResearchAccess.com. Aug. 1, 2014. (Sept. 10, 2017) http://bit.ly/2vZdWDb
  • Institute for Work & Health. "What researchers mean by...selection bias." Iwh.on.ca. (Sept. 10, 2017) http://bit.ly/2xYlxzk
  • Kicinski, Michal. "Publication Bias in Recent Meta-Analyses." PLOS ONE. Nov. 27, 2013. (Sept. 9, 2017) http://bit.ly/2xWKr29
  • Krishna, R.; Maithreyi, R.; Surapaneni, K.M. "Research Bias: A Review for Medical Students." Journal of Clinical and Diagnostic Research. April 5, 2010. (Sept. 9, 2017). http://bit.ly/2xWJiYp
  • Lee, Chris. "Confirmation bias in science: how to avoid it." ArsTechnica. July 13, 2010. (Sept. 9, 2017) http://bit.ly/2xYNmHO
  • McCook, Alison. "What leads to bias in the scientific literature? New study tries to answer." Retractionwatch.com. March 20, 2017. (Sept. 9, 2017) http://bit.ly/2xXBqGi
  • Mullane, Kevin and Williams, Michael. "Bias in research: the rule rather than the exception?" Elsevier.com. Sept. 17, 2013. (Sept. 9, 2017) http://bit.ly/2xXci2n
  • Oxford Math Center. "Famous Statistical Blunders in History." Oxfordmathcenter.edu. (Sept. 10, 2017) http://bit.ly/2xYi1VE
  • Pannucci, Christopher J., and Wilkins, Edwin G. "Identifying and Avoiding Bias in Research." Plastic Reconstructive Surgery. Aug. 2010. (Sept. 9, 2017) http://bit.ly/2xWIbbt
  • Pennwarden, Rick. "Don't Let Your Own Opinions Sneak Into Your Survey: 4 Ways to Avoid Researcher Bias." Surveymonkey.com. Jan. 1, 2015. (Sept. 9, 2017) http://bit.ly/2xWBTbP
  • Pew Research Center. "Questionnaire Design." Pewresearch.org. (Sept. 9, 2017) http://pewrsr.ch/2vYk0vD
  • Piller, Charles. "New federal rules target woeful public reporting of clinical trial results." Statnews.com. Sept. 16, 2016. (Sept. 9, 2017) http://bit.ly/2xYpCU5
  • Roberts, Ian. "Retraction of scientific papers for fraud or bias is just the tip of the iceberg." The Conversation. June 11, 2015. (Sept. 9, 2017) http://bit.ly/2xWTkZD
  • Sarniak, Rebecca. "9 types of research bias and how to avoid them." Quirks.com. August 2015. (Sept. 9, 2017) http://bit.ly/2vWV8EQ
  • Schupak, Amanda. "How Often Are Scientific Studies Retracted?" CBS News. May 26, 2015. (Sept. 9, 2017) http://cbsn.ws/2xXO8F9
  • Shuttleworth, Martyn. "Research Bias." Explorable.com. Feb. 5, 2009. (Sept. 9. 2017) http://bit.ly/2xXzDRk
  • Simon, Matt. "Fantastically Wrong: The Imaginary Radiation That Shocked Science and Ruined Its 'Discoverer.'" Wired. Sept. 3, 2014. (Sept. 10, 2017) http://bit.ly/2xYwHUS
  • Thase, Michael E. "Do antidepressants really work? A clinicians' guide to evaluating the evidence." Current Psychiatry Reports. December 2008. (Sept. 9, 2017) http://bit.ly/2xWWUD5
  • Turner, Eric H., etal. "Selective Publication of Antidepressant Trials and Its Influence on Apparent Efficacy." New England Journal of Medicine. Jan. 17, 2008. (Sept. 10, 2017) http://bit.ly/2xYsGzx
  • Van Hilten, Lucy Goodchild. "Why it's time to publish research "failures." Elsevier.com. May 5, 2015. (Sept. 10, 2017) http://bit.ly/2xYyLfr
  • Whoriskey, Peter. "As drug industry's influence over research grows, so does the potential for bias." Washington Post. Nov. 24, 2012. (Sept. 9, 2017)


Can You Be Addicted to Endorphins?

Can You Be Addicted to Endorphins?

HowStuffWorks finds out whether you can be addicted to endorphins and whether riding roller coasters can help or hurt.

More to Explore