Randomized controlled trials are the gold standard in statistics, but sometimes -- in epidemiology, for example -- ethical and practical considerations force researchers to analyze available cases. Unfortunately, such observational studies risk bias, hidden variables and, worst of all, a study group that might not reflect the population as a whole. Studying a representative sample is vital; it allows researchers to apply results to people outside of the study, like the rest of us.
A case in point: hormone replacement therapy (HRT). Beyond treating symptoms associated with menopause, it was once hailed for potentially reducing coronary heart disease (CHD) risk, thanks to a much-ballyhooed 1991 observational study [source: Stampfer and Colditz]. But later randomized controlled studies, including the large-scale Women's Health Initiative, revealed either a negative relationship, or a statistically insignificant one, between HRT and CHD [sources: Lawlor et al.; New York Times].
Why the difference? For one thing, women who use HRT tend to come from higher socioeconomic strata and receive better quality of diet and exercise – a hidden explanatory relationship for which the observational study failed to fully account [source: Lawlor et al.].