How Scientific Peer Review Works

Peer review has earned its enemies over the years. You might think these are scientists to whom peer review has not been kind, but that's not the case. A growing number of researchers are shining the bright light of science — controlled experimentation and careful observation — into the dark corners of a process that has been around for hundreds of years. What they're finding may surprise you.

Consider a study conducted by BMJ (British Medical Journal), one of the most respected peer-reviewed journals in medicine. BMJ Editor Fiona Godlee and two colleagues took a paper about to be published in their journal and introduced eight deliberate errors. Then they sent the paper to 420 reviewers. The median number of errors detected by the 221 respondents was two. Nobody found more than five, and 16 percent didn't find any errors at all. This seems to suggest that peer review doesn't really increase the quality of published research, or does so to only a small degree. Another BMJ study showed that a single, seasoned editor could judge the quality and significance of research just as effectively as a team of external referees.

Then there's the issue of detecting fraud. As the Hwang stem cell case clearly demonstrates, peer review isn't a fraud-detection system. Referees are much more likely to find and flag plagiarism than falsified data. That's because reviewers don't generally have access to the actual data on which a paper is based. If a scientist knowingly and deliberately sets out to falsify data, a team of reviewers may not be able to detect it. However, such data will not be able to stand up to the intense scrutiny of the larger scientific community. In fact, the ability of scientists to duplicate the results of published research is another hallmark of science and another quality-control mechanism that extends beyond peer review.

Quality is not the only issue. Some critics argue that peer review slows down advances in scientific and medical knowledge. It can take a year for an article to move through the peer-review system and become published. Some journals have introduced a fast-track approach to streamline the submission process, but it's usually reserved for very high-quality work. Papers below this standard may languish for months. That's a long time to wait, especially if the research promises to provide valuable information about a disease or other issue that affects public health and safety. And yet moving methodically and with great care is not a bad thing when you're dealing with the safety of human beings.

Finally, a few critics have suggested that peer review leads to the suppression of some scientists' results. There are two ways this suppression might play out. First, a reviewer — an established scientist in their field — might reject research that challenges their particular point of view or contradicts their own findings. Such a reviewer might be accused of maintaining the "scientific establishment" at the cost of innovative ideas. The other form of suppression involves the work of Third World researchers. Studies have shown that a vast majority of mainstream journal articles come from scientists in developed countries, with the majority coming from U.S.-based scientists. Very few Third World researchers see their work published in mainstream journals, such as Nature and Science. Even when a developing nation succeeds in publishing in a peer-reviewed journal, the journal might not be listed in the Science Citation Index, a commercial database of scholarly publications widely used by researchers.

Because of limitations such as these, many scientists are campaigning to change the peer-review process. The next section highlights some of those changes and how peer review is evolving.