About the SA Blog Network

The Curious Wavefunction

The Curious Wavefunction

Musings on chemistry and the history and philosophy of science
The Curious Wavefunction Home

Peer review – Pitfalls, possibilities, perils, promises *: #scio13

The views expressed are those of the author and are not necessarily those of Scientific American.

Email   PrintPrint

At this year’s ScienceOnline (un)conference, Jarrett Byrnes from the University of Massachusetts, Boston and I will be moderating a session on open science and peer review. Peer review clearly faces new and urgent challenges with the advent of online science journalism and writing that can criticize and even bypass the process. Can the traditional model of peer review survive these challenges? Can it be integrated within the new framework or would it be supplanted? How can journals best modify or implement new policies for peer review in this increasingly expanding universe of open and citizen science? These are all key questions facing not just authors, journal editors and reviewers but also the taxpaying public at large whose contributions often fund peer-reviewed research. Join us on Thursday at 4 PM in Room 7 to discuss these and other issues.

With all its merits, the traditional model of anonymous peer review clearly has flaws; reviewers under the convenient cloak of anonymity can use the system to settle scores, old boys’ clubs can conspire to prevent research from seeing the light of day, and established orthodox reviewers and editors can potentially squelch speculative, groundbreaking work. In the world of open science and science blogging, all these flaws can be – and have been – potentially addressed. The question, “How on earth did the reviewers and editors allow this paper to be published?” has appeared in blogs on more than one occasion. On the other hand, research that is consistently rejected by journals can be self-published on blogs. Since bandwidth is (almost) free, nothing can stop ideas – no matter how speculative or controversial – from seeing the light of day. In addition, responses from reviewers that seem unscrupulous or conspiratorial can also be aired out into the open.

Such easy access to publishing potentially confidential and damning material poses significant challenges to journal editors, so we would especially like to hear from them. The topic of peer review is vast and multifaceted and we can only touch upon a few topics; we are hoping that the rest of the discussion will spill over into the hallways, restaurants and bars. This being an ‘unconference’, all Jarrett and I will do is kick off the discussion with one or two topics that are our personal favorites. The floor then belongs to the audience who should feel free to hold forth on their favorite topics.

On my part I would like to focus on two aspects of peer review, both illustrated with examples that I have previously blogged about. One asks if peer review has become too conservative. The other asks how transparent the process can be made. It demonstrates one of the major problems with academic reviewing – the ability of reviewers to liberally criticize and reject legitimate research under the cover of anonymity – by way of a remarkable story in which a professor could not get legitimate criticism of existing research published in a leading journal for the better part of a year.

Jarrett is part of NEACS OpenPub a blog created by the National Center for Ecological Analysis and Synthesis which is discussing a framework for open research in the field of ecology. The blog has a productive discussion going on about standards and requirements for publishing open research in the field. Be sure to check it out; Jarrett will provide much more information during the session.

As always, this discussion can only benefit from a variety of views from the entire spectrum of people who are affected by the peer review process; certainly authors, editors and reviewers but also interested bloggers, ‘whistleblowers’ and members of the public. So we hope to see a diverse representation of people during this session and we look forward to an engaging and productive discussion. See y’all on Thursday!

* With due apologies to Doc Brown.

Ashutosh Jogalekar About the Author: Ashutosh (Ash) Jogalekar is a chemist interested in the history and philosophy of science. He considers science to be a seamless and all-encompassing part of the human experience. Follow on Twitter @curiouswavefn.

The views expressed are those of the author and are not necessarily those of Scientific American.

Rights & Permissions

Comments 8 Comments

Add Comment
  1. 1. rodestar99 7:19 pm 01/29/2013

    A bigger problem is blogs that get quoted or referenced
    in journals like scientific american that have no science
    as their basis but a casual reading makes it appear that
    they are a scientific work.
    Many of the climate stories that appear here fall in that category. I think someone here has taken note of this lately as I see a lot less of it now. Blogs are great and offer much information but they should not be passed of as scientific papers.

    Link to this
  2. 2. sault 9:53 pm 01/29/2013

    rodestar99, agreed. For example, many climate deniers get all their info from bogus blogs that are either funded / supported by fossil fuel companies or their willing dupes. Since these companies have such a huge financial incentive to stall urgent action on lowering CO2 emissions, they try every trick in the book to spread doubt about established, sound science. Since they cannot win on scientific grounds, they bypass the peer-review process and spread their misinformation on these blogs, talk radio and other friendly media outlets.

    It’s a shame that quality science reporting has mostly disappeared since our society is ever more dependent on technology and the science that drives it. Consequently, bold-faced, unscientific lies get put on the same level as established climate science because journalists are too lazy to dig into scientific & technical issues and are more interested in the he said / she said horse race that they think defines EVERY issue society has to deal with. These unscientific hucksters know they don’t have to win; they just have to spread enough doubt to get people to put issues like climate change off onto the back burner. Lazy journalists let them do it and are themselves complicit. All you have to do is follow the money to understand what’s going on.

    Link to this
  3. 3. curiouswavefunction 10:59 am 01/30/2013

    rodestar99 and sault: Yes, it’s true that blogs can be used to spread misinformation just like any other source with easy access. The beauty of it though is that in many fields the practitioners learn to separate the wheat from the chaff and identify those sources that consistently produce high-quality and accurate information. Others then start linking to these more often. I agree that it’s an imperfect system – Anthony Watts’s blog was actually voted as the best science blog once – but there’s also no dearth of robust, legitimate sources.

    Link to this
  4. 4. kedarj 6:37 pm 01/30/2013

    Isn’t all this Brahminism and Neo Brahminism that certain cultures have been accused of. Fortunately the common man is hardwired with a judicious mixture of mother’s childhood ultimatum “beware of strangers” and “caveat emptor” – buyer beware. I have seen “hoax of moon landing” on National Geographic TV. And I also see them discovering “archeological non sense” and “alien captures” I also have seen/seeing sublime and the ridiculous emanating from “scientific portals” and scientific journals – so called “peer reviewed” I call it “beer reviewed” based on my mom’s dictat. If it is the prerogative of a plumber to say something smells fishy to Hawkins – It is my prerogative to “caveat emptor” you both. And even the invitation for the conference saying “this cave is empty and safe” Who knows what kind of whatever lurks there. Happy hunting and grants for research.

    Link to this
  5. 5. kedarj 6:44 pm 01/30/2013

    I may be made a mistake on the National Gegraphic TV channel – It may have been Discovery Channel – Don’tblame me for not remembering – I have no “brand loyalty” (corollary to caveat emptor)

    Link to this
  6. 6. kedarj 6:55 pm 01/30/2013

    And I may as well add for posterity that history of science debated whether gravity was not a myth conceived to sell apples and so also the string theory to sell shoe laces.

    Link to this
  7. 7. Bryan Sanctuary 8:42 am 02/11/2013

    Timely blog for me. I am about to submit a paper on the reconciliation of the EPR paradox which shows quantum mechanics has a local realistic unpinning. This threatens a lot of research and there is so much hostility toward this that I expect a rough ride. The old boys club will want to reject it, and the reason will be, “do not like the results”.

    Any suggestions?

    Link to this
  8. 8. curiouswavefunction 11:38 am 02/11/2013

    I can only say that if you have a lot of trouble getting it accepted then you should just publish it on your blog or on arXiv.

    Link to this

Add a Comment
You must sign in or register as a member to submit a comment.

More from Scientific American

Email this Article