This article was published in Scientific American’s former blog network and reflects the views of the author, not necessarily those of Scientific American
On supporting science journalism
If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.
Following a surprisingly unscientific line of reasoning, the editors at the most renowned and prestigious of science journals have rationalized away the need to fix an ailing peer-review system. Increasing skepticism about the effectiveness and integrity of single-blind peer reviewthe process by which most academic papers submitted for publication are accepted or rejectedhas prompted empirical evaluation of the system. Standard practice is: reviewersselected for their expertise and fluency in the chosen disciplineare aware of all authors names and affiliations, while authors are kept in the dark about the identity of their reviewers (although some journals allow them to request specific referees). The growing argument against this lopsided method is that knowledge of authors identitygender, nationality, research institution, level of experience in the fieldcan (and does) bias reviewers opinions on the merit of the research. The most vocal critics of the current system are those who believe their submissions do not get fair considerationwomen, early-career scientists, people with foreign-sounding nameswhen matched up against authors who sail through the submission process on the status of their lab or the history of their career. And in an environment in which research funding, hiring, tenure, salary, and academic reputation are massively dependent on publishing record, one can easily imagine the ripple effects such a disadvantage would bring. Problematically, much of the evidence of bias is anecdotal. But a few studies show pretty convincingly that the current system is broken: in their editorial the Nature staff themselves point out that "editors have rejected previously published papers when they were resubmitted with false obscure affiliations." Further, the increased fairness of double-blind review seems a no-brainer. "The one bright light in favour of double-blind peer review," the editorial continues, "is the measured reduction in bias against authors with female first names," which has been shown in numerous studies, such as this one (subscription required for full text).These are not new ideas. In 1997, right after the United Nations recognized Sweden as the world leader with respect to equal opportunities for men and women, two biologists with access to data from the Swedish Medical Research Council discovered that the single-blind peer-review process for awarding postdoctoral fellowship was rife with sexism. (Read more about that study, and about men and women in math and science here.)And guess where the Sweden study was publishedthats right, Nature.It is a mystery, then, how Natures editors concluded that there is a "lack of evidence that double-anonymity is beneficial." And what does it say about their views on womens potential contributions to science?Instead, they take their lead from a 2008 survey commissioned by the Publishing Research Consortium (the what?) and conductedno, not by scientistsby two men at a consulting firm.The argumentput forth by this report, and reiterated by Natures editorsthat double-blind is not an improvement over single-blind review rests essentially on a whole two studies (separate links) back in 1998 that found no significant differences in the quality of peer reviews obtained with the two methods. Interestingly, these studies weren't asking anything about biasbut about the quality of the reviewers comments. What these studies did uncover, as a byproduct (and an admitted likely impediment to any potential positive results), is that double-blind peer review, in most cases, isn't double-blind after all. Reviewers can often figure out who the submitting authors are, especially those from well-known labs. Aha, the bottom line. For double-blind review to actually work, journals would have to invest precious time and resources into creating a system in which information was masked well enough to conceal author identities. For journals, this is not desirable. But it appears to be possible. In a comment on Natures blog, where they posted their editorial, B. Kamenov points out that the universal adoption of electronic submission makes it much easier than in the past to hide author identities and affiliations. Additional measures, such as prohibiting explicit self-citation, could easily be enforced.Another commenter, Erec Stebbins, tells why the difficulty in masking author identities is a bogus excuse for sticking with the old system: While this may be an argument for open review vs. double blind, it cant be for single blind vs double blind. In the latter case, if the authors are guessed, it will simply reduce to single blind. In small fields this will happen more often. In larger fields, I think this will happen much less frequently, with much less confidence, and certainly not in any field for young investigators without a prior 'text fingerprint' to mark them&The ingenuity of intelligent scientists to effectively conceal their identities when they perceive an advantage I think is being grossly underestimated!Heres a sampling of Natures editors attempts to explain why they are "resistant to adopting [double-blind review] as the default refereeing policy any time soon":" Natures policies have always moved towards greater transparency.How charmingly literal of them to presume that double-blind anonymity is less open and, therefore, less equitable than single-blind. Earlier on this heroic path of transparency, Nature itself ran a trial of open peer review, which they concluded a failure. Non-anonymous reviewers, faced with the prospect of being held accountable, are less likely to criticize; and submitting authors are more likely to hold a grudge against a reviewer who rejects their paper, welcoming a later opportunity to retaliate when the tables are turned. If academics (in either position) cannot be trusted to behave professionally or ethically, how can we even consider carrying on with an unbalanced process? To blind one side and grant the other immunity seems suspect at best." Identifying authors stimulates referees to ask appropriate questions (for example, differentiating between a muddy technical explanation and poor experimental technique).Huh?" Knowing author identities also makes it easier to compare the new manuscript with the authors' previously published work, to ensure that a true advance is being reported.And what if they have no previous publications?! This is precisely the bias people are talking about. Everyone knows the findings should stand on their own merit. They are to be judged against the state of the entire fieldany previously published work from these authors will be as easy to find as anything else.Nature has opened up their perspective to public commentary, by posting the editorial on their blog page. The overwhelming consensus among the 50 plus comments that have been streaming in throughout the week is that double-blind is the only way junior investigators and lesser-known labs will get fair treatment in the review processan issue that was hardly addressed in the Nature editorial. As one commenter, signed in as LC, put it: I can't count how many young scientists were thrown off the academic career by the ugliness of the peer review system and of the perceived (and many times proven) bias. Single blind is damaging science and its credibility.I am eager to see what credit they give to the attitude evidenced through their equally unscientific though laudable community outreach efforts. Will the editors at Nature will throw up their hands or, instead, keep their current theory open to further testing, as any science should be, in hopes of finding a better answerand justice?--Edited by Rachel Dvoskin at 02/14/2008 3:28 PM--Edited by Rachel Dvoskin at 02/14/2008 3:28 PM--Edited by Rachel Dvoskin at 02/14/2008 3:34 PM--Edited by Rachel Dvoskin at 02/14/2008 3:36 PM--Edited by Rachel Dvoskin at 02/14/2008 3:46 PM