About the SA Blog Network

Information Culture

Information Culture

Thoughts and analysis related to science information, data, publication and culture.
Information Culture HomeAboutContact

Peer review makes science better

The views expressed are those of the author and are not necessarily those of Scientific American.

Email   PrintPrint

Part of what makes the process of science so interesting is the part where scientists invite criticism. It’s right there in the scientific method, in the part where we have to see if other experiments confirm original results. Or, as David Ng states in his “Introduction to the Scientific Method, by way of Chewbacca,” this is where other folks get to dump on you.

Peer review is one of the formalized processes that allow scientists to criticize and dump on each others work. The big goal of peer review is to make science better, although there are lots of other smaller goals. The Action Potential blog over at the Nature Blog network just posted a case study in how this can happen.

First, a brief intro to peer review for those unfamiliar with the details: Authors submit their work to a journal, saying “publish my article!” The editors of the journal need a way to figure out if the science is any good, so they ask other scientists. The original article is sent to other experts to have a look. These reviewers are looking at two big concepts: is the science sound and is the work important enough to publish. Reviewers looking for sound science will evaluate the original authors methods, examine how they analyzed their data, and make sure that their conclusions are supported by that data. Reviewers also try to determine if the results are sufficiently original or interesting to be published in that particular journal. They then make a recommendation to the editor saying “Yes! Publish this now!” or “Maybe. Can the authors clarify some things?” or “Nope.” For a much better look at what happens during peer review, see this excellent Boing Boing post.

Most of the time, we don’t get to see this process. Reviewers are typically anonymous, and their comments are not meant for public consumption (I’ll talk about new things like open review in my next post).

The Action Potential story shows us how constructive and detailed comments from reviewers can keep scientists on their toes and at the top of their game. A manuscript was submitted and sent out for review. The reviewers weren’t overly enthusiastic about the paper. They suggested some improvements to the methodology used, including the addition of various controls. They were also concerned that this study wasn’t original enough to be published in Nature, and provided several reasons why.

Although the reviewers originally rejected the paper, the reviewers comments provided the authors with a guide to revising their results, collecting additional data, making their conclusions stronger, and analyzing the data better.

So the authors got to work. After several (probably very busy) months, the authors resubmitted the paper and it was successfully published in the May 13 issue of Nature.

This is why peer review has become the standard in scientific publications – by allowing work to be criticized thoroughly at least once before publication, we are more likely to get sound scientific results that will hold up.

Of course, it isn’t all butterflies and daisies. Peer review isn’t always helpful. Sometimes it can be downright obstructionist. And as a result, many folks are working very hard to find new methods that allow for the same kind of critical review of scientific work at some point in the process. In my next post, I’ll discuss some of the challenges with traditional peer review and some of the things being proposed to make it better.

Bonnie Swoger About the Author: Bonnie J. M. Swoger is a Science and Technology Librarian at a small public undergraduate institution in upstate New York, SUNY Geneseo. She teaches students about the science literature, helps faculty and students with library research questions and leads library assessment efforts. She has a BS in Geology from St. Lawrence University, an MS in Geology from Kent State University and an MLS from the University at Buffalo. She would love to have some free time in which to indulge in hobbies. She blogs at the Undergraduate Science Librarian. Follow on Twitter @bonnieswoger.

The views expressed are those of the author and are not necessarily those of Scientific American.

Rights & Permissions

Comments 4 Comments

Add Comment
  1. 1. julianpenrod 8:22 pm 06/14/2012

    So often, among the most crucial aspects of a situation slip by all but unnoticed. Note the sentence where they dash off that peer review asks “is the science sound and is it important enough to publish”. So that’s where “science” stands today! “Science” used to be touted as the urge to understand the world. Now, only certain things people are allowed to understand, those that “peer review’ considers “important enough”! Just providing a portal to the universe isn’t sufficient. it has to fit what a reviewer considers “worthwhile”. And, since simply providing knowledge of the world doesn’t fit this individual’s idea of “important”, or else they would allow anything that was true and provable, frankly, what does inform their “judgement”? What about the possibility that, even if an item isn’t “important’ from some reviewer’s prejudiced eye, it might potentially lead someone readingabout it to discover something truly magnificent? But the reviewers don’t think about that! Only showing off how powerful they are!

    Link to this
  2. 2. Bonnie Swoger in reply to Bonnie Swoger 9:09 pm 06/14/2012

    One of the biggest challenges for reviewers is assessing the importance or impact of the work. Some journals do more of this than others. Journals like Science or Nature want to highlight the most innovative and interesting science, while other journals publish sound science that might not advance the field quite as much. Other journals, like PLoS ONE are omitting the assessment of importance or impact altogether. These journals focus on reviewing for sound scientific methods and data analysis and allow readers to evaluate how innovative or important the research is.

    Link to this
  3. 3. Jerzy v. 3.0. 10:42 am 06/15/2012

    Peer review, in fact, poorly picks good articles, although can reject very bad ones. Studies showed that breakthrough papers with most citation science-wide in a year, were repeatedly rejected in several journals as unworthy.

    Reasons include science rivalry, ideology, but most common is simple that reviewers are not paid, so usually give paper just a glance or pass it to students.

    Happily, a rise of internet means that peer review is unnecessary – publishing is no longer limited by number of pages of a journal. I am personally looking forward to peer review being replaced by voting by larger community of scientists in some mutation of Facebook “like it”. In any case, judgement by just a few editors and 2-3 reviewer creates a bias unlike larger circle of scientists.

    Link to this
  4. 4. julianpenrod 3:31 pm 06/15/2012

    Despite what Jerzey v. 3.0. contends, any “solution” based on the internet doing away with the dictatorship of the corporately influential, the politically craven and the popular power mongers seems doomed to heart breaking failure. Consorship is wielded like a New World Order truncheon now, squashing revelations of truth, and, tragically for humanity, many champion that use because it’s done on the basis of “private property”. “You don’t have the right to say just anything you want on someone else’s property”, these quislings “justify”. The fact that they are championing withholding truth that could help others means nothing to them! They are pleased that the dodge is so powerful and potentially devastating to so many!
    When a document like the Constitution applies only to those corporately rich enough to afford it, it still represents a Fascist state.
    Face facts, what would keep them from fabricating an arboitrary set of restrictions that could prevent many “scientists” from commenting? But, then, would that even be necessary? If “scientists” aren’t concerned enough to be complaining now, maybe they are all of the same corrupt mind about what should and should not be published!

    Link to this

Add a Comment
You must sign in or register as a member to submit a comment.

More from Scientific American

Email this Article