10 Signs That Study Is Bogus


The Batteries Are Low

"Whatever. That lung cancer study probably had low statistical power." iStock/Thinkstock
"Whatever. That lung cancer study probably had low statistical power." iStock/Thinkstock

The field of neuroscience is in an exciting phase with powerful new technology capable of analyzing the brain with ever-increasing exactitude. Using scans, for instance, a number of different studies looked at the relationship between mental health and abnormal brain volume. Many, in fact most of them, found a correlation.

But the overwhelming rate at which these studies kept confirming one another piqued the curiosity of researcher John Ioannidis. After analyzing the data he found that, taken together, all of these studies had an average statistical power of 8. That sounds low, and it is. But what does it mean?

The statistical power of a study refers to the size of the samples used and how large or small the results were. To massively oversimplify, if you study 10,000 smokers and 10,000 non-smokers and find that 50 percent of the smokers developed lung cancer while only 5 percent of the non-smokers did, then your study has very high power. You had a huge sample population, and the results were huge as well.

But if you study 10 smokers and 10 non-smokers and find that two of the smokers developed lung cancer and one of the non-smokers did too, then you have an extremely underpowered study. The sample size is so tiny that the difference between the two groups is meaningless [source: Yong].

To be fair, most studies accused of having low power aren't as ridiculously low as that fictional example. But in recent years, concerned researchers have been blowing the whistle on the prevalence of under-powered studies. Their message? It's time to power up!