Skip to main content

Peer Review of Peer Review; and the Franklin Institute Awards

In this episode, Scientific American editor-in-chief John Rennie discusses peer review of scientific literature, the subject of a panel he recently served on at the World Conference of Science Journalists. He also talks about some out-of-the-box thinking about burial rites. And physicist Bo Hammer talks about the awards being presented this week at the Franklin Institute in Philadelphia. Plus we'll test your knowledge about recent science in the news. Website mentioned on this episode include www.fi.edu. To vote for the Scientific American podcast in the podcast category of the Webby Awards, go to www.webbyawards.com.

Uncertain

Welcome to Science Talk, the weekly podcast of Scientific American for the seven days starting April 25th. I am Steve Mirsky. This week on the podcast, we'll talk with Scientific American Editor-in-chief, John Rennie just back from a big science journalism conference in Australia, and Bo Hammer will talk about the Franklin Institute Awards being given out this week in Philadelphia, as well as some other things. Plus we'll test your knowledge about some recent science in the news.

First up, John Rennie—he was in Melbourne last week attending the "World Conference of Science Journalists" where he was a member of a panel talking about peer review in science publishing. And he also has some comments about a truly out-of-the-box idea about human burial rites. I spoke to him in the library at Scientific American.

Steve: Hi John—how are you?


On supporting science journalism

If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.


Rennie: Hi Steve—just fine.

Steve: You were at this conference and you were on a panel discussing peer review.

Rennie: That's right!

Steve: Let's talk about what peer review is first of all.

Rennie: That was actually one of the things that we talked about on this panel. This was a panel called "Peer Review of Peer Review" because what they wanted to do was, they wanted to look at the peer review process, which is the one in which scientists—when they are looking at one and another's work and deciding which papers deserve to appear in professional journals. And this panel called "Peer Review of Peer Review" was in effect trying to size up 'how does this process work for some of the different constituencies involved. How does it look these days from the standpoint of the people who are publishing the peer reviewed journals? How does it look from the scientists who are trying to get into them? And how does it look from the standpoint of the journalists who are looking at these journal papers and then turning them into popular articles in news papers and magazines and so on? And so overall there was a sense that the peer review process has obviously served science extremely well over the years. But there have been various problems with

afraud in some cases. Some of the big cases—for exampletheHwang Woo Suk, [the] Korean researcher who managed to get fraudulent papers [published] about stem cells and cloning—that was an example of [a] way in which somebody defrauded the system.

Steve: Did we actually say though what peer review really is?

Rennie: Yeah! Peer review is—what we were debating was whether we should really refer to it as "expert review." Because peer review is the process—if you are a scientist—of having your paper first scrutinized by your fellow scientists who are experts in a particular area and deciding that your new discovery represents a worthwhile contribution, something that should appear in new journals.

Steve: Okay, so because some people might not know

thatwhen a paper is published in a scientific journal—for the most part—most of the journals or many of the journals are peer reviewed. There are some prominent journals that are not peer reviewed.

Rennie: That's right!

Steve: But most of them are.

Rennie: But the major journals, things like Nature or Science or the New England Journal of Medicine, that sort of thing. All of those are peer reviewed. But there was some discussion about the fact that may be we should start to refer to this as 'expert review'. Because quite frankly 'peer review' doesn't convey very much to most of the general public; and in fact somebody had mentioned having seen an article just the week before in which it was supposed to be a discussion of peer review, but the writer constantly referred to it as "pier", p-i-e-r as in, "take a long walk off of a short ...."

Steve: (Laughs) Okay, that's pretty interesting. Well! I guess that's the way a lot of authors feel about the process (laughs) when their works don't get accepted.

Rennie: (Laughs) Right! And that actually was another consideration; there was some question of whether or not the process by which the papers are being reviewed by other scientists—is this fair in the way that it should be? Are there ways in which there are different types of bias that infect the system, so that some kinds of papers [that] may be coming from certain researchers or certain institutions, do they have an easier time getting into major journals? Are there any questions about whether or not the, say, the funding associated for certain papers, does that make it more likely that certain results would come up? All of that's the kind of thing that can really corrupt this expert peer review system from within.

Steve: So what was the consensus view of whoever was—who were the experts on the expert panel of "Expert Review of Expert Review?"

Rennie: Well! Shockingly, I was allowed to actually be one of them. I was apparently standing up there representing journalists everywhere. But we also had Philip Campbell, who is the editor-in-chief of the journal Nature, and Warwick Anderson, who is CEO of Australia's National Health and Medical Research Council. He was more or less representing the position of the scientists who would be going into the journals. And I don't know that it was the sort of discussion that was really trying to push for consensus as much as just airing a lot of different viewpoints and various concerns for the audience of science writers that was there. Science writers need to be on guard about the ways that, just because something appears in a major journal,

itmaybe really, it should not be treated as scientific truth with a capital "T." Phil Campbell actually had an excellent point. He made the point that "peer review really only begins with the publication of a paper." New work is ultimately being scrutinized by the scientific community as long as they continue to reproduce those experiments and work on it in the future. So, in effect, when something first appears in a journal, that is only the first stage in its peer review. In the long run, it may be discarded because whatever would be truths that paper tried to present turned out to be obsolete or incorrect.

Steve: That's probably the most important point to take away from this whole conversation, and it's a point I forget—that that's the starting point for the real process of judgment by the scientific community about the validity of a finding.

Rennie: That's right!

Steve: So you had this other interesting tid-bit you came back from the conference with.

Rennie: One presentation that I thought was particularly fascinating was one that—it was called "Way to Go"—and it was presented by Roger Short, who is a geneticist by training at the University of Melbourne, although he has been working in various areas of Human Reproduction research for quite sometime. He has a lot of concerns about how there may just basically be too many people on the Earth—that we may already be at an unsustainably large number. But he was making a point that at the end of our lives, there is maybe something we could all do. A decision having to do with the disposition of our bodies that might be more beneficial to the environment. He was arguing that cremation—which is certainly a choice that many people like—is that it's really a terrible idea with respect to global warming; that if we really all want to try to help prevent global warming, that it would be a very good idea if we did not have ourselves cremated. Because when your body is reduced to ash in that way not only, of course, all the various carbon molecules in your body being converted over to carbon dioxide— which is bad in its own way, but, of course, some of that would be happening

of coursewith normal decay. But basically you're talking about the crematorium ovenisbeing heated up to a temperature of 850 degree[s] Celsius for an hour and a half, so there is an enormous amount of fuel that's being used just to help dispose of your body in that way. And typical traditional burial is—I believe—he referred to that as an abomination of desolation that cemeteries were just—they were awful places. There's nothing really productive that grows there except for grass and the bodies of people have been embalmed and locked away inside boxes, specifically with the aim of trying to slow the rate at which they would be returned to a natural state of decay. So what Professor Short was advocating was that, in fact, when we die you should have a vertical hole that is drilled and your body should just be put down in that, no casket, just maybe just surrounded by various leaves as a kind of padding and then a tree should be planted over that. The argument for this is that over a century your average tree will sequester about one metric ton of carbon dioxide. Most of us give—in [the] sort of standard of living that we have in industrialized nations, we’re typically managing to produce the equivalent of about the amount of carbon dioxide that it takes 13 trees a year to manage to take back in; and so over the course of our lifetimes we’re probably responsible for releasing into the atmosphere the equivalent of about a 1,000 trees worth of carbon dioxide. So this "Way to Go" proposal of being buried in this way, this is one way to try to give back to Mother Earth.

Steve: How was that proposal received at the conference?

Rennie: It seemed to be received favorably. It was a fairly short presentation and unfortunately I didn't get the chance to ask—and I didn't hear any other reporters ask—whether there might be any sorts of say legal impediments to someone just digging a hole and putting grandma's body in that. I suspect there probably are some. But you know I didn't hear a lot of discussion of this and if there had been I just know that it would inevitably have been some way referring to it as out-of-the-box thinking.

Steve: But unfortunately...

Rennie: But you do have to salute the ingenuity of Professor Short for having proposed something like that.

Steve: And it also would greatly increase your chances—if anybody is interested in the potential of someday being a paleontological find, you really going to want to go that way. If you want to try to be a fossil someday, you know, hundreds of thousands of years, millions of years maybe into the future—that's the way to do it. You know, certainly obviously cremation does away with everything; but being in a box—no good for fossilization. You really want to be in the Earth and we have an opportunity for your bones to be mineralized there and to just—you know, if you want to donate your body to science that's a really good way to do it too.

Rennie: Yeah! It's much more environmentally friendly than say my own plans for my body, which are to be varnished and set up at some kind of an end table.

Steve: That's funny, because that's our plans for you as well (laughs). Thank you John. For more about the World Conference of Science Journalists, go to www.scienceinmelbourne2007.org.

If you've seen the new video news feeds at the Scientific American Web site, the[y]

reare live, easy to view, and updated three times every day. Video news just a click away at sciam.com.

Now it's time to play TOTALL.......Y BOGUS. Here are four science stories; only three are true. See if you know which story is TOTALL.......Y BOGUS.

Story number 1: Tests on kitchen sponges found that the best way to disinfect them is to soak them in chlorine bleach.

Story number 2: What could be the world's oldest tree, a fossil 385 million years old, has been found in New York.

Story number 3: Researchers have created the world's smallest pipette, which can dispense a precise billionth of a trillionth of a liter of fluid.

And story number 4: Some good news, a census of mountain gorillas in Uganda has found that the rare primates are actually increasing in number.

We'll be back with the answer, but first, as we told you last week, the Franklin Institute is giving out its awards Thursday night, April 26th, recognizing achievements in Science and Technology. Astronomer Steve Squires, who appeared on this podcast on April 12th, 2006, is one of the honorees. There are also events going on Wednesday and during the day of Thursday. Bo Hammer is vice president for the Franklin Center and the Franklin Institute in Philadelphia. I had a chance to speak to him recently in Scientific American's Library. We spoke about the awards and about his interesting background as a Congressional science fellow.

Steve: Hi Dr. Hammer; how are you today?

Hammer: Great, Steve. Thanks.

Steve: Why don't you give us some history of the awards?

Hammer: So, the Franklin Institute was founded in 1824 and basically started as a school for the Mechanical Arts. Philadelphia was the center of the Industrial Revolution at that time in U.S. It was sort of the largest city in U.S. It was [an] intellectual, cultural, innovation center of the United States at that time, kind of a like Silicon Valley of its day; and at that time there weren't a lot of universities and so the Franklin Institute founders saw a role to play to educate primarily young men to essentially be draftsmen, designers, engineers— to create all the things that were needed for to feed the economy. And so one of the things that the school would do is put out calls for important inventions and inventors would send these things in models of their inventions, descriptions of these inventions. And they would be used as sort of hands-on examples for the students at that time; and gradually what happened was that a committee was formed to actually evaluate the best of these things and that evaluation process is what evolved into this awards program today. This started with, you know, industrial technology, so to speak, and over the years evolved into a program that's more focused on the basic sciences.

Steve: One of the really interesting things about the awards is the education component. The award winners do a series of lectures, and these are really aimed at the general public, school kids. Why don't you talk about that?

Hammer: Yeah! The Franklin Institute is basically a hands-on science museum, and our mission is to inspire a passion for science learning, to try to get young people and their families to come in and get turned on about science. Yet at the same time we've got this awards program that's about 180 years—more than 180 years old. So the question is, why the heck is Franklin Institute even doing [an] awards program like this? What's the connection? And it really gets down to that mission to inspire, and so what we do is we bring our laureates in. The ceremony itself is on Thursday, typically the last Thursday of April every year. That's sort of the end of a long week's worth of activities and the focus of these activities is really this educational part. On Wednesday and Thursday we have symposia at the local universities which are much more academically flavored events, but it's an opportunity for the laureates' local sponsor—the person who is responsible for championing their case to be a Franklin medalist. The sponsors are one of our local universities—University of Pennsylvania, Drexel, Temple, Villanova—and their sponsor organizes the symposium in their honor at their home institution.

I[I]t gives us an opportunity to get graduate students and post docs and faculty out there andgetto meet these great individuals and learn about their science.

Steve: You've been a Congressional fellow while being a physicist. How would you characterize the relationship between science and politics that you saw up close?

Hammer: It was interesting; I was a fellow in '93-'94, and it was a fascinating year regarding this interaction between science and politics because this was the year—'93 was the year the Superconducting Super Collider was killed by Congress and that basically happened— that legislative battle happened—in the committee that I was working for; and a lot of the reason that the science politics came to that point was because of—I won't say mistakes, but the scientific community, I would argue in some respects, didn't do the best job it could at making the case why this Superconducting Super Collider was important. They didn't make the case for the Congress to justify spending—I think it would have been 14-plus billion dollars—on this device that would basically answer very fundamental question[s] about the nature of the universe—an important question; but the case wasn't made strong[ly] enough in a time when science funding in general was very physically challenged. So it was interesting to be a part of that, to see it happen and then to see the work. That was [a] real wake-up call to the high-energy physics community. And it sort of energized them to think harder about priority setting and making their case stronger. And so what our committee did [in] the following months is basically put in place some mechanism to restore some of that high-energy physics funding in order to save the field, basically. Because the goal wasn't to kill physics; the goal for the members of Congress who wanted to kill the Super Collider was to kill this project, but not eliminate the science altogether. But it sort of put it on a different track.

Steve: But we've seen real funding problems in the last year especially.

Hammer: Yeah!

Steve: You know the Department of Energy funding of the National Lab[oratorie]s has really come under a lot of scrutiny because of the potential that Fermilab might have to close for a month basically because there's just no money there.

Hammer: Yeah! And that's—a lot of that's been due to the change in power and Congress and the budget bills. But actually the science community again got mobilized and I think was able to, in its continuing resolution, to restore some of that funding to actually boost some of those projects and keep them going.

Steve: Since you were a fellow, true physicists have been elected to House of Representatives, one on each side of the aisle; so I would assume that you are in favor of that trend and would like to see more PhD physicists in congress?

Hammer: Yeah, in politics in general. I mean I think Rush Holt and Vernon Ehlers are great members of Congress. You know they come from completely different political perspectives but they have a pretty much

ofcommon vision on the importance of funding basic research and the importance of science education and what all that means for U.S. competitiveness. And sort of at the completely other end of the political spectrum is—and I think perhaps even more important for scientists to think about—is getting involved in local politics for example school boards. I am on my local school board and that's a really great way to get involved and have a lot of impact on local education.

Steve: Do you have kids?

Hammer: Yeah! Two kids.

Steve: But you don't have to have kids to be on a local school board.

Hammer: No, absolutely not.

Steve: Right! So anybody out there who is really concerned about the curriculum choices and the quality of science education at the local level can get involved by running for the local school board.

Hammer: Yeah! And scientists have had a lot of impact nationally, either at the local school board level or the state school board level. For example, you know, in Kansas with what's been happening with evolution battles out there;

andscientists who have gotten energized by this issue have had a huge impact on reversing some of these destructive decisions.

Steve: Dr. Hammer, thanks very much

Hammer: Oh! You are welcome. It's been a pleasure being here. I appreciate the opportunity.

Steve: For more information about the Franklin Institute Awards, a schedule of the events going on Wednesday and Thursday April 25th and 26th, and a complete list of the honorees, go to www.fi.edu.

Now it's time to see which story was TOTALL.......Y BOGUS. Let's review the four stories.

Story number 1: Chlorine bleach, best way to disinfect a kitchen's sponge.

Story number 2: Oldest fossil tree found in New York.

Story number 3: World's smallest pipette created.

And Story number 4: Ugandan gorilla population increase.

Time is up.

Story number 4 is true. The most recent census of mountain gorillas in Uganda's 127 square mile Bwindi Impenetrable National Park—that's the official name—finds a 6 percent increase in the gorilla population since 2002, up from 320 to 340 individuals, as according to the Wildlife Conservation Society. Mountain gorillas live only in the impenetrable park and in an area just south of the park called the Virunga Volcanoes on the border of Uganda, Rwanda, and Congo.

Story number 3 is true. Brookhaven National Laboratory scientists have created a pipette that can deliver a precise zeptoliter—which is fun to say—of fluid—that's a billionth of a trillionth of a liter. If you want to know why you'd want a zeptoliter of anything, listen to the April 18th edition of the daily SciAm podcast 60-Second Science.

And story number 2 is true. The 385 million year old fossil tree found in New York looked like a modern palm. The recently discovered crown of the tree goes with stumps discovered over a century ago. For more, check out the April 18th article called, "Earth's Oldest Tree had Fronds, not Leaves," at our Web site, www.sciam.com.

All of which means that story number 1, about chlorine bleach being the best way to disinfect a kitchen sponge is TOTALL.......Y BOGUS. Because a study done by the agricultural research service tested bleach, lemon juice, plain old de-ionized water, a dishwasher, and a microwave oven as disinfecting agents for sponges that [are]

issoaked for two days in a solution made from ground beef and laboratory cell culture medium. Each dirty sponge had about 20 million microbes on it. Bleach, lemon juice, and de-ionized water only got between 37 and 87 percent of bacteria and between 7 and 63 percent of any yeasts and molds, but sticking the sponge in a dishwasher with a drying cycle killed 99.9998 percent of bacteria. Microwaving the sponge got the other little iota—they got it up to 99.9999%. The dishwasher and the microwave also destroyed well over 99 percent of yeasts and molds. So when you are done cooking and cleaning, cook the sponges to clean them.

Well! That's it for this edition of the weekly Scientific American podcast. You can write to us at podcast@sciam.com, check out news articles and video news at the Web site, www.sciam.com, and the daily SciAm podcast 60-Second Science is at the Web site and at iTunes. And you can vote for us in the podcast category at webbyawards.com. For Science Talk, the weekly podcast of Scientific American, I am Steve Mirsky. Thanks for clicking on us.

Web sites mentioned on this episode include www.fi.edu.

Steve Mirsky was the winner of a Twist contest in 1962, for which he received three crayons and three pieces of construction paper. It remains his most prestigious award.

More by Steve Mirsky
Peer Review of Peer Review; and the Franklin Institute Awards