10 Types of Study Bias

Sampling Bias
Thanks to a sampling bias, the Literary Digest incorrectly predicted that Alf Landon (right) would defeat Franklin D. Roosevelt (left) in the 1936 presidential election. Keystone View Company/FPG/Archive Photos/Getty Images

Researchers who've done meta-analyses of scientific research have found that early, small-scale studies — ones that end up being frequently cited in other work — often overstate their results [source: Fanelli, et al.].

That can happen because of sampling bias, in which researchers conducting small studies base their findings upon a group that isn't necessarily representative of the larger population. Universities often use students for their studies but the findings for this group don't necessarily project to the wider population.

It's a problem that's seen in both medical studies and social science research. For example, if a political science researcher who's studying attitudes about gun control does surveys in an area where most people are Second Amendment supporters, that will skew the results in a way that doesn't necessarily reflect the views of the larger U.S. population.

But sampling bias can occur in bigger studies as well. One famous example of sampling bias occurred during the 1936 U.S. presidential campaign, when Literary Digest conducted a mail survey of 2.4 million people and predicted — incorrectly — that Republican Alf Landon would handily beat incumbent Democrat Franklin Roosevelt. The problem was that the magazine used phone directories, drivers' registrations and country club memberships to find people to poll — a method that tended to reach relatively affluent voters (cars and phones were luxury items back then), rather than the poorer ones among whom Roosevelt was popular. The erroneous results hastened the end of the publication [source: Oxford Math Center].