It's Plausible, Not Provable
Very serious people in white lab coats bending pensively over test tubes — that's the image that often comes to mind when we think of a scientific study. But that's just one type of study — a laboratory experiment.
Another kind of study is called "observational." That's when researchers find a group of test subjects, ask them a lot of questions, record the answers and then "data mine" the results to see what they find.
Once upon a time, observational studies suggested that having a "Type A" personality put you at higher risk for having a heart attack. But follow-up randomized clinical trials could not repeat these results, and it's now known that the original finding was completely false. How did this happen?
Observational studies can, of course, be enormously useful and enlightening. But there's a potential for misleading results. One problem is that you can get statistically significant results by chance about 5 percent of the time. So if you ask enough questions (and sometimes these studies can include thousands of questions), the data might appear to render something important. But on subsequent review, or attempts to repeat the study, the results might not be the same [source: Miller and Young].