ADVERTISEMENT
  About the SA Blog Network













Absolutely Maybe

Absolutely Maybe


Evidence and uncertainties about medicine and life
Absolutely Maybe Home

Science in the Abstract: Don’t Judge a Study by its Cover

The views expressed are those of the author and are not necessarily those of Scientific American.


Email   PrintPrint



Cartoon in a lab - You said to do an abstractA competition for attention lies at the heart of the scientific enterprise. And the abstract is its “blurb.”

A scientific abstract is a summary used to attract readers to an article and to get a piece of research accepted for a conference presentation. Other than the title, it’s the part of an article that is looked at by the most people.

But should we take a scientific abstract literally? Is it likely to tell us the most important things we need to know about a study?

Let’s unpick those words first, because the answers become clearer when you keep the science production process in mind.

Cartoon - This is not a studyPeople often call a scientific article a study: “Have you read that study in the Journal of Exciting Results?” But the article is not actually a study: it’s a report about a study.

A study is a complex activity. All sorts of artifacts are produced along the way – like grant applications, descriptions for ethics committees, data sets, and reports to funders or regulatory agencies.

Out of all the things observed, thought or recorded, some is chosen to go into one or more articles. There could be dollops of wishful thinking as a narrative is formed. Usually right at the end, and usually with an eye for the “juicy” bits, the abstract is written – and sent out into the battle for attention and citations.

Cover of the Journal des Scavans

It wasn’t always like this. Scientists used to communicate about their work in person, in dissertations, monographs and books, and in letters among peers. They had meetings where they read out papers. Peer review began before scientific journals did.

Journals grew out of summaries written by others, not the scientist who did the research. The first two publications regarded as the precursors of the science journal started in 1665: the Journal des Sçavans in Paris (pictured here on the left), and the Royal Society of London’s Philosophical Transactions. The first journals, full of news and book reviews, writes David Kronick, “were very much like the newspapers which provided their models.”

Then came the explosion of journals in Germany in the 1800s, particularly in medicine. Although the phrase “publish or perish” is a creature of the 20th Century, academic competition had started in the 19th. According to Kronick, it can be credited to the decentralization of German universities. The number of journals is estimated to have gone from 900 in 1800 to almost 60,000 in 1901.

The original blurbThe structure of a scientific article that we’re familiar with now, including an abstract, developed early in the 20th century. By then, the publicity “blurb” for books and movies had arrived on the scene, too. Along the way, scientists absorbed “PR” techniques in their work.

The abstract’s writing style follows the style of scientific articles, though. It’s a format called IMRAD, for Introduction, Methods, Results And Discussion. So they begin with the back story, and bring the “news” in at the end. That’s the opposite direction of the inverted pyramid writing style of journalism, which cuts to the chase right at the start.

In 1969, Ertl criticized abstracts for not providing the information readers need, proposing a tabular form for summarizing scientific work. That didn’t take off. In 1987, the move to structured abstracts – with formal sections like “objectives,” “methods,” and “results” – began in clinical research.

Early in this century, reporting standards for abstracts of clinical trials were developed – and they seem to have improved reporting. Last year, this was joined by standards for abstracts of systematic reviews. (Interest declaration: I’m one of the authors.)

What problems are we trying to resolve? And why does Ivan Oransky from Retraction Watch say it’s “journalistic malpractice” to report on a study after only reading an abstract or press release?

Cartoon of early statistical philosopher

It comes down to inaccuracy and academic spin, both of which are too common in abstracts. Inaccuracies have been shown in studies of abstracts in a chemistry journal (a quarter inconsistent with the article), medical journals (from 18%-68%) and pharmacy journals (61%).

To be fair, even though it’s a worry, every inaccuracy isn’t critical. But the spin is serious. It’s not just about language that exaggerates or massages results. It’s also about choosing the most exciting results, even when that’s misleading.

Spin in press releases and media coverage often reflects the spin in abstracts. And until more research is open access, that will be all that most people can get to read.

It’s possible that reading only abstracts might influence clinical decisions. Isabelle Boutron, a leading researcher of academic spin in clinical research, has done a randomized trial of whether abstracts with or without spin change decisions. There’s no article about that trial yet, so there’s not even an abstract to go by. Boutron reported at a conference I posted about here last year that experts who read abstracts with spin believed treatments were more beneficial than those who got the de-spun versions.

Of course, when there’s spin in the abstracts, there’s spin in the full article too. It’s less likely to be wall-to-wall, though. Most of the time, less impressive information will in fact be there in the full text. The potential conflicts of interest are more likely to be there, too.

You can get a more realistic picture. But first you have to resist jumping to conclusions before you’ve read the fine print.

~~~~

 

The cartoons are my own – this time with apologies to the great René Magritte for the clumsy homage to The Treachery of Images. (More cartoons at Statistically Funny.)

The original blurb is from the Library of Congress, via Wikimedia Commons.

Articles I relied on heavily in the section on the early history of scientific journals were by Jack Meadows and David Kronick.

If you’re wondering about the origin of not judging a book by its cover, according to the Flexners, the specific phrase, “You can never tell a book by its cover,” first appeared in 1946 in the mystery novel, Murder in the Glass Room by Edwin Rolfea writer blacklisted by McCarthy’s House Un-American Activities Committee – with co-author, Lester Fuller.


* The thoughts Hilda Bastian expresses here at Absolutely Maybe are personal, and do not necessarily reflect the views of the National Institutes of Health or the U.S. Department of Health and Human Services.

Hilda Bastian About the Author: Hilda Bastian likes thinking about bias, uncertainty and how we come to know all sorts of thing. Her day job is making clinical effectiveness research accessible. And she explores the limitless comedic potential of clinical epidemiology at her cartoon blog, Statistically Funny. Follow on Twitter @hildabast.

The views expressed are those of the author and are not necessarily those of Scientific American.





Rights & Permissions

Add Comment

Add a Comment
You must sign in or register as a ScientificAmerican.com member to submit a comment.

More from Scientific American

Scientific American Holiday Sale

Black Friday/Cyber Monday Blow-Out Sale

Enter code:
HOLIDAY 2014
at checkout

Get 20% off now! >

X

Email this Article

X