About the SA Blog Network

Information Culture

Information Culture

Thoughts and analysis related to science information, data, publication and culture.
Information Culture HomeAboutContact

Peer-review mysteries and simple things publishers can do to help readers

The views expressed are those of the author and are not necessarily those of Scientific American.

Email   PrintPrint

Scholarly scientific publications have a pretty standard structure: Abstract, Introduction, Methods, Results, Discussion, Conclusion, References. One of the primary purposes of this structure is to make the work understandable to others. Ideally, we will understand the context from the introduction, see how the experiments were performed from the methods section, and learn how future research may aid our understanding of the topic in the discussion or conclusion. All of this makes the scientific process more transparent.

But the way in which the publication came to be is often shrouded in mystery, especially for those who haven’t tried to publish something in an academic journal.

The peer review process is inherent in the “scholarly” part of “scholarly publication.” But when you examine a standard journal article, there is nothing to indicate if the text has been through such a review process.

In class earlier this week, I showed students this article and asked them to tell me why it was considered scholarly:

After answers related to the occupation of the authors and the presence of a reference section, one student said “Because it’s peer-reviewed.”

“Really,” I said, “how do you know it has been through peer review?” The student didn’t have a response.

It’s a trick question, really, because there is nothing in the article that indicates it is peer-reviewed.

In addition to the lack of information within articles themselves, the websites of many journals make it very difficult to determine if the articles they publish have been through some kind of peer review.  You almost always have to dig deep into the journal website to find out. Sometimes the “About” page will say that “Journal of Scholarly-ness is a peer reviewed journal focused on…” But more often, readers actually have to dig into the “Author Guidelines” or “Reviewer Guidelines” sections to determine what review process happens prior to publication. This gets even more complicated if a publication includes some peer-reviewed content and some not-peer-reviewed content (e.g. book reviews, news stories, etc.).

Thankfully, some publications are making this a bit clearer by labeling articles as “Peer-Reviewed,” like this PLOS ONE article:

Even when you find a statement saying “This journal uses peer review,” that doesn’t tell you a whole lot. What were the peer reviewers looking for? How did they evaluate the manuscript? How did the reviewers praise or criticize the paper?

Peer review is difficult to study (Couzin-Frankel, 2013), and the process remains mysterious even for some who have been through it. For students and the general public, peer-review can often be confused with basic proof-reading. The secrecy of the peer-review process doesn’t help students as they learn to explore and think critically about the scientific literature.

I’m happy to see new publications like PeerJ and F1000Research making reviews available to the public, alongside the earlier versions of the manuscript. Teachers can use these publications to teach students about the peer-review and publication process. And third-party peer review services are creating peer-review rubrics to guide reviewers through the process. Happily, students can examine these rubrics, and even use them on published papers they read in class to help them critically analyze the reports.

Without wading into the debate about how the peer-review process can be improved, I suggest three really simple things publishers can do to assist readers:

  • Add a note indicating if an individual article is peer-reviewed to the HTML and PDF versions of the article
  • Include the words “peer-reviewed” in the “About” section of your journal
  • Prominently post any guidelines you provide to reviewers.

My students would certainly appreciate it.

Couzin-Frankel, J. (2013). Secretive and Subjective, Peer Review Proves Resistant to Study. Science, 341(September), 1331. doi:10.1126/science.341.6152.1331

Bonnie Swoger About the Author: Bonnie J. M. Swoger is a Science and Technology Librarian at a small public undergraduate institution in upstate New York, SUNY Geneseo. She teaches students about the science literature, helps faculty and students with library research questions and leads library assessment efforts. She has a BS in Geology from St. Lawrence University, an MS in Geology from Kent State University and an MLS from the University at Buffalo. She would love to have some free time in which to indulge in hobbies. She blogs at the Undergraduate Science Librarian. Follow on Twitter @bonnieswoger.

The views expressed are those of the author and are not necessarily those of Scientific American.

Rights & Permissions

Comments 2 Comments

Add Comment
  1. 1. Jerzy v. 3.0. 5:28 am 03/3/2014

    Is peer review like yeti? Yeti is so secretive that nobody yet studied it directly. Peer review should not be.

    It would be interesting if some big journal simply made their old reviewers’ opinions directly avialable, and allowed direct studies of peer review.

    A direct study of peer review would, first and foremost, benefit the journal itself, allowing it to pick the truly most valuable papers. For now, papers rejected several times often become most cited and respectable in their 3rd or 5th journal. This means, that all the previous journals missed the chance.

    Link to this
  2. 2. Jerzy v. 3.0. 5:35 am 03/3/2014

    BTW, pick 2 or 3 people out of the whole spectrum of opinions in scientific community, and filter the whole knowledge though it. It introduces random bias and incredibly narrows the breadth of ideas.

    Re-submitting papers and editors opinions counteract the bias in peer review somehow, but still.

    The idea worked in 19. century, where 2 or 3 people were the science. Not any more.

    Link to this

Add a Comment
You must sign in or register as a member to submit a comment.

More from Scientific American

Email this Article