ADVERTISEMENT
  About the SA Blog Network













Information Culture

Information Culture


Thoughts and analysis related to science information, data, publication and culture.
Information Culture HomeAboutContact

Post publication peer-review: Everything changes, and everything stays the same

The views expressed are those of the author and are not necessarily those of Scientific American.


Email   PrintPrint



In the early days of scientific societies (i.e. the 17th century), scientists would share their experimental results with each other at meetings, and receive feedback about their experiments in person.  (The scientific journal wasn’t invented until later.)  As the scientific community grew, it was impossible for everyone to be in the same room to hear about results, and so the amount of immediate feedback offered was limited to a few conferences or other gatherings. Recently, publishers, scientific societies and entrepreneurs have begun using the web to bring back the era of immediate feedback: so-called “post-publication peer review.”

One of the hallmarks of scholarly scientific publication is the review process.  While it isn’t perfect, peer review is the process used by almost all scholarly publications to filter out bad-science, identify weak data analysis and make suggestions for better presentation of results.

Traditional peer review was done before an article was published. In the age of print publication, this made the most sense because of the costs associated with printing.

Although most scientific journals are published online these days, peer review is still most often done prior to publication and the status of “peer reviewed” is held up to undergraduate students as an important hallmark of quality. Online publication makes post-publication peer review easier than ever – most often in addition to traditional pre-publication peer review.

Conversation

Scholarly scientific communication is a conversation. New tools are enabling more scientists to take part in a public conversation. CC-BY-SA Image courtesy of Flickr user AJ Cann.

Several new ventures are allowing scientists to comment on published papers, engage in discussions online and even publish more formal reviews.

Informally, it started with blogs. Independent bloggers would read an article from the scientific literature and want to share their thoughts on it. As we have done on this blog, the blogger would cite the paper and discuss it’s results and any limitations.  Perhaps the blogger might disagree with the original findings based on their own research. But the original author of the paper might never know that it was being discussed because there was no connection between the publishers version of the article and the blog post.

Next came user commenting.  Taking a page from blogging platforms, publishers such as PLOS and BioMed Central experimented with allowing registered users to post comments on journal articles. This was only moderately successful. Some folks suggested that scientists were unwilling to directly criticize the work of others, although they might be willing to comment on a blog post discussing the article. While article comments kept the scientific dialog contained in one place, the conversations didn’t really take off, and some scientists I spoke with were skeptical about the idea of allowing anyone to comment on scientific articles.

In recent years, third parties have started taking on the post-publication peer review mantle.

PubPeer allows users to make comments on almost any article that has a DOI, but the site is separate from publishers web pages. Scientists may be more willing to engage in frank criticism, but it is harder to connect the original paper to the comments. Importantly, PubPeer contacts authors when new comments are posted about their papers – in this way, PubPeer attempts to provide feedback directly to the authors that blogging doesn’t provide. PubPeer also attempts to preserve bling peer review by not publishing reviewer names.

PubMed Commons is the platform built into PubMed for using comments. Currently in a pilot phase, PubMed Commons invites authors of PubMed papers to join and comment on papers. Their names appear by their comments. By limiting who can comment on papers, PubMed Commons seems to want to reduce instances of non-experts commenting on papers. Articles related to politically controversial topics such as evolution, vaccines and climate change would theoretically be open to abuse if comments from everyone were allowed. Users will see these comments at the bottom of the PubMed entry for each article.

Open Review is the much hyped tool from the academic social network ResearchGate. Open Review encourages authors to publish a slightly more formal review than the simple commenting systems of PubPeer and PubMed Commons.  Reviews are tied to your research gate account and to the entry for the article on ResearchGate.

As others have pointed out, the growing number of commenting systems produces a distributed system that can make it difficult for people to know where to go to get information about an article. And I suspect that these systems will only multiple in the near future.

These new tools are providing a way for scientists to make their private conversations about recent research public, and allowing them to expand the group of people they can chat with. Although the goal is to provide feedback and commentary, the limits of space (how many people can fit in the room, who has the office next door) no longer apply.

New scientists have grown up commenting on their friends pictures, their silly comments on Facebook and their favorite YouTube videos.  Will this practice carry over into their scientific publishing? Will publishers eventually link back to these commenting systems? Will database producers (e.g. Scopus, Web of Science) incorporate user comments into their systems? It will be interesting to find the answers to these questions and see how these systems impact scholarly scientific communication.

Bonnie Swoger About the Author: Bonnie J. M. Swoger is a Science and Technology Librarian at a small public undergraduate institution in upstate New York, SUNY Geneseo. She teaches students about the science literature, helps faculty and students with library research questions and leads library assessment efforts. She has a BS in Geology from St. Lawrence University, an MS in Geology from Kent State University and an MLS from the University at Buffalo. She would love to have some free time in which to indulge in hobbies. She blogs at the Undergraduate Science Librarian. Follow on Twitter @bonnieswoger.

The views expressed are those of the author and are not necessarily those of Scientific American.





Rights & Permissions

Comments 3 Comments

Add Comment
  1. 1. Jerzy v. 3.0. 5:23 am 03/27/2014

    I wonder if there is a citation ranking or a system for online works? The nature of the current science system is that nothing but citations matter from a scientist’s career perspective. Post-publication peer review may be seen by many scientists as a waste of resources. Even if traditional letters to editor are slow and very much unlike free flow of scientific thought, they are citable.

    Link to this
  2. 2. Bonnie Swoger in reply to Bonnie Swoger 8:42 am 03/27/2014

    @Jerzy I think you got it exactly right – how will the scientific community use, cite and credit these comments? Certainly some scientists are commenting on papers because they are interested in the topic discussed, but when you need to prioritize your time, how much time do you spend on things that won’t be helpful when it comes time for tenure or promotion?

    Link to this
  3. 3. Jerzy v. 3.0. 10:08 am 03/27/2014

    @Bonnie
    I noticed rather unfortunate trend in ‘Nature’. Many online comments on papers are simply self promotion ‘My group published a paper related to that, here is the link…’.

    In general, there is a need of good standard of citation of online material. Now, online material is often cited without the author name and often too broad – for example, the whole website instead of the author name, individual post and individual blog.

    Link to this

Add a Comment
You must sign in or register as a ScientificAmerican.com member to submit a comment.

More from Scientific American

Scientific American Holiday Sale

Limited Time Only!

Get 50% off Digital Gifts

Hurry sale ends 12/31 >

X

Email this Article

X