ADVERTISEMENT
  About the SA Blog Network













Absolutely Maybe

Absolutely Maybe


Evidence and uncertainties about medicine and life
Absolutely Maybe Home

A Viewer’s Guide to Mammography Evidence Ping-Pong

The views expressed are those of the author and are not necessarily those of Scientific American.


Email   PrintPrint



Cartoon of woman saying you bet I have questionsYou could get a very sore neck watching all the claims and counter-claims about mammography zing back and forth.

It’s like a lot of evidence ping-pong matches. There are teams with strongly held opinions at the table, smashing away at opposing arguments based on different interpretations of the same data.

Meanwhile, women are being advised to go to their doctors if they have questions. And their doctors may be just as swayed by extremist views and no more on top of the science than anyone else.

So here’s a viewer’s guide to help you make sense of play when the next new study appears and the batting begins.

First up: What’s with all the spin?

Spin is important in ping-pong. You would hope that matters of life and death would be settled without spin-doctoring, though. But the drama and emotion make that hard.

Genie cartoon - wishing for a world without bias

Around 70% of women aged 50-76 in the U.S. have mammograms. Ever since mammography started to take off in the U.S. in the late 1950s, Lerner points out it has had a “magic appeal.”

For others, though, mammography triggers ardent ideological opposition. Then there’s backlash and disillusion fueled by massive overselling of both the risk of breast cancer and the benefits of mammography. Perhaps inevitably, a strong counter-movement grew – and many of mammography’s critics succumb to the temptation of spin techniques, too.

All up, there’s a colossal tangle of bias to get through when you consider breast screening - cognitive biases and statistical ones, too. It’s not easy to be, and stay, objective about mammographic breast screening.

The rules of the game

Data about screening often trips us into believing we can hold death at bay more than we really can. Bringing the date of diagnosis forward lengthens the time people live with their disease – their survival rate – whether or not their lives are extended. You can read more about issues like these in my previous post, The Disease Prevention Illusion: A Tragedy in Five Parts.

Cartoon of statistician on ward rounds

Proof that there’s a way to spot disease accurately isn’t enough. Finding disease early has to make a real difference to the outcomes people would have had if they weren’t screened. That means there has to be effective treatment that leads to results when it’s started earlier. And people have to be helped over and above those who would have been diagnosed soon enough anyway.

That means you need very big randomized trials that follow women for years to get close to some answers about mammography. By the time you have the data, tests and treatments have changed. On top of the fact that no study is perfect anyway, people who want to reject results have plenty of ways to dismiss the evidence that doesn’t fit their beliefs.

To help us get closer to solid, reliable answers, then, there needs to be a very strong accumulation of data from several big trials among different sets of women. Which means there are multiple, complex, idiosyncratic data-sets to analyze.

If you look at data that doesn’t come from adequate randomized trials, or you look at one or two studies in isolation, you’re unlikely to get a reliable impression. Cherry-picking information is an easy way, though, to convince yourself or others that what you believe is true.

Looking at all the evidence isn’t simple, either. With a lot of data and many complicating factors, researchers have a lot of options for analyses and conclusions. Yet more room for cognitive and statistical biases to spin play. With mammography screening for women at average risk of breast cancer for their age, it gets rather extreme: 10-fold differences between the numbers batted around is commonplace. Let’s follow the trial trail to see why.

Get to know the important players

The set of statistical tools that can examine data from groups of similar trials is called meta-analysis. (Catch up here on 5 key things to know about meta-analysis.) Over the years, the meta-analyses of mammography trials have outnumbered the trials. The first completed screening mammography trial began in 1963 – the last in 1991. You can see a good description of the key trials in Table 3.1 here.

There is one more trial underway in England. It’s testing whether there’s benefit in starting screening a bit earlier and going a bit longer. Women will be recruited until at least 2016. There’ll be no results from a new trial in women at average risk for their age for a long time.

Cartoon of meta-analysts' duel

The key players to watch out for are the meta-analysts whose work is the basis of new reviews and the claims and counter-claims about mammography. There are 3 main groups who have been doing meta-analyses: other reviews and positions are based on using one or more of these as their source on the key question of breast cancer deaths. (You can reach reports of the meta-analyses via links at my comment on the Peter Gøtzsche Cochrane review at PubMed Commons.)

Gøtzsche’s Cochrane review is the longest-running player. It’s been updated several times – the last time in 2013. No further updates are planned. (Cochrane reviews are published by the Cochrane Collaboration, an international organization that does and promotes systematic reviews of health care.)

Then there are the reviews by the U.S. Agency for Healthcare Research and Quality (AHRQ) for the U.S. Preventive Services Task Force (USPSTF). Their first meta-analysis was done in 2002 and updated in 2009. An update is planned.

The third key player is the 2012 Independent U.K. Panel on Breast Screening. They did meta-analyses of their own – and they compared their results to the other meta-analyses. The key data they compared did not change in the 2013 Cochrane update, so the numbers are up-to-date.

Now let’s look at the score-card

The snapshot of the meta-analysis below is for the risk of dying from breast cancer. It comes from the U.K. Panel. Each horizontal line represents one of the key mammography studies, showing the confidence intervals (similar to the margin of error).

Data shown in a forest plot

Figure 3.1 Meta-analysis of relative risk (RR) of breast cancer mortality in trials after 13 years (From the report of the Independent U.K. Panel on Breast Cancer Screening)

Horizontal lines completely to the left of the solid black vertical line show a definite reduction in breast cancer deaths. If one of the study’s lines so much as touches that black vertical line, then it means no definite decrease or increase in breast cancer deaths. (One completely to the right would have meant a study showed a definite increase in breast cancer deaths.)

The diamond at the bottom summarizes all of the trials together. The diamond here is well to the left: a definite reduction in deaths of around 20%. The two lines that go right around the middle are the trials from Canada that recently reported longer term follow-up data. The new data don’t affect the results of this analysis. These trials had never shown benefit from mammography.

The three meta-analyses don’t come to a vastly different result on analyzing this question in these trials (15 to 20%). You use the relative risk reduction to calculate what groups of women who start with a different personal risk level might experience. If you have a very small risk of the disease in the first place, a 15-20% reduction will be very small, too. If your risk is higher, the chances of benefiting are greater.

But if they agree on this, how can it be that information based on the Cochrane review often talks about only 1 woman in 2,000 having a breast cancer death averted because of screening, while other results talk about 1 in less than 200?

Partly it’s to do with the Cochrane review concentrating on a particular subset of trials as more important than the larger pool: the two Canadian ones plus one. Others agree a subset is better quality, but not that the others are so bad you can disregard them.

Another major reason the numbers can vary dramatically is because it depends on which age group they’re talking about. Let’s look at the Independent U.K. Panel’s best effort to get data you can compare (Table 3 here).

In order to prevent one woman’s death from breast cancer, the number of women who would need to be invited for screening was:

  • Cochrane review: 2,000
  • USPSTF, for women aged 50 to 591,339 and for women aged 60 to 69377
  • Independent U.K. Panel, for women aged 55 to 79235

In order to prevent one woman’s death from breast cancer, the number of women who would need to be screened was estimated as:

  • Independent U.K. Panel, for women aged 55 to 79180

A further key difference between people’s approach is what data they use on the chances of harm. That includes, in particular, getting false positive mammography results and getting treated for breast cancers that were never going to become dangerous to your health (“over-diagnosis”). Here’s another area where people will come to very different results.

The chances of being harmed aren’t fixed. It depends on the choices you make about how often you get screened. If you go every year, you’re more likely to experience harm than if you only go once every 2 or 3 years. So the data researchers choose on this is also going to be lower or higher based on their choices. The U.S. is looking at screening women for longer and more frequently than the U.K. So the estimate of number who benefit will be lower and the estimate of harm higher.

The Independent U.K. Panel calculated that if women go every 3 years from 50 till 70, roughly 1 in 180 could have a breast cancer death prevented. On that calculation, for each woman who won’t die from their breast cancer, roughly another 3 will be treated needlessly. And a lot more will have biopsies for false positives. The experience of a false positive is generally distressing, and the distress could take quite a long time to reduce. It doesn’t seem to increase clinical depression or anxiety, though.

There’s so much uncertainty here – even about what our individual risks of breast cancer might be. Whether you think the odds are worthwhile, is a personal value judgement. But hidden within that phrase “value judgement” is something else. Our personal cognitive and ideological biases have an influence on which set of data and arguments convince us, as well as the value we place on the chances of benefit or harm. Wishful thinking, denial, fear, fatalism – all that and more are pulling on our thinking.

We often worry a lot about the biases of who is giving us information. In the end, though, the biggest bias we have to deal with is our own.

~~~~

I’ll be commenting on major developments with reviews and meta-analyses at PubMed Commons.

Illustrations are my own, including cartoons from Statistically Funny.


* The thoughts Hilda Bastian expresses here at Absolutely Maybe are personal, and do not necessarily reflect the views of the National Institutes of Health or the U.S. Department of Health and Human Services.

Hilda Bastian About the Author: Hilda Bastian likes thinking about bias, uncertainty and how we come to know all sorts of thing. Her day job is making clinical effectiveness research accessible. And she explores the limitless comedic potential of clinical epidemiology at her cartoon blog, Statistically Funny. Follow on Twitter @hildabast.

The views expressed are those of the author and are not necessarily those of Scientific American.





Rights & Permissions

Comments 7 Comments

Add Comment
  1. 1. emceedee 8:20 pm 05/1/2014

    I realize that there blog IS on mammography. Nonetheless, I would think that thermography should be at least mentioned: the fact is, it exists.
    I also realize that the $$$ investment (machines, training, personnel, selling the public, and likely much more) is such that thermography will not be widely used for many years, if ever. Comments, Ms Bastian?

    Link to this
  2. 2. LelThy 10:24 pm 05/1/2014

    Bastian’s own bias or value judgment gets her to view the value of mammography somewhere in the middle ground between the two extreme standpoints.

    However, what Bastian fails to mention here are many other issue affecting mammography, such as other collateral harm not mentioned here, the impact of vested interests, the poorly substantiated theory on breast cancer development, many large population based studies, etc. – all of which point in one consistent direction: mammography is hardly, if at all, beneficial to most women but it is seriously harmful to most women ( see Rolf Hefti’s The Mammogram Myth – more at http://www.supplements-and-health.com/mammograms.html ).

    Link to this
  3. 3. Hilda Bastian in reply to Hilda Bastian 7:12 am 05/2/2014

    I alluded to new imaging technologies, but there were lots of things I had to let go of, emceedee, or the post would have been very long. Tackling all of them would be another big job. May do in the future.

    Link to this
  4. 4. Hilda Bastian in reply to Hilda Bastian 7:34 am 05/2/2014

    LelThy, if you mean commercial and professional vested interests in promoting mammography, that’s certainly covered well in Lerner’s article that I linked to. I left all of the issue of interests in together without emphasizing it too much, because of space really, as there are so many kinds of vested interests. I think critics also have vested interests, whether it be what they’ve staked their reputations on, or where their ideological commitment is. Having a commercial or ideological interest in an alternative to mammography is also a vested interest.

    I disagree strongly that there’s evidence not addressed in the three major reviews/meta-analyses I discussed that outweighs the evidence in them. Large population studies are certainly very well-addressed on the question of harm.

    Link to this
  5. 5. hkraznodar 6:01 pm 05/9/2014

    @LelThy: The irony of a supplements dealer challenging the value of preventive screening is almost overwhelming. I’m not sure if I should laugh or cry.

    With advances in cancer research changing the way cancer is viewed there is a very real chance that mammograms may be greatly reduced or discarded outright. Until something better comes along I see no reason to throw the baby out with the bath water.

    Link to this
  6. 6. Amy Price 5:20 pm 07/16/2014

    This is the very best article on Mammogram controversy that I have yet to come across. What people want to know is what are the results and why, all the blame, finger pointing and posturing is just annoying. Did you do an analysis with the same data using random effects? There is an interesting and informative methods article on this by Josie Sandercock here http://metasurv.wordpress.com/2013/04/26/fixed-or-random-effects-how-about-the-full-variance-model-resolving-a-decades-old-bunfight/

    My other 2 issues are first if we cant yet establish strength of malignancy and how it will progress at an individual level, is ti not a rather dangerous assumption to predict this at a population level. I found the validation used for deaths inadequate as people can often have something else on the death certificate that was precipitated by the cancer or its treatment and also the very strange absence of quality of life measures for those whose malignancy was caught early (sure they may have died at the same time but was their life better with the cancer untouched or surgically removed in the life that remained?

    What bothers me about this is the meticulous attention to false positives which again the figures vary widely on, if you are going to look at the FP in detail why not the others?

    Also if we have all this research time and money to go into this why not build better breast cancer information and mammography sheets so women are not intimidated by a result with no knowledge of artifacts, lead time, survival, or staging of films and of the cancer, nor are they told of the types of breast cancer and the differences so they can make an informed choice in shared decision making that is in tandem with their own lifestyles and personal values.

    I would love to know what women think of clinical trials for breast cancer too and how they could be made better from a participant point of view as 3% enrollment is not exactly a vote of confidence for research:-)

    Link to this
  7. 7. Hilda Bastian in reply to Hilda Bastian 2:42 pm 07/20/2014

    Thanks so much, Amy. I’m pleased you found it helpful.

    On the random effects question: I didn’t do any data analysis – just a search for systematic reviews and analysis of those. I’ve got a little more detail on who used what model here on PubMed Commons. That wasn’t the main decider here: the relatively young age of the subset highlighted by the Cochrane review, by contrast, had a very big impact. I can’t comment on the quality of the causes of death on death certificates in these trials (because I haven’t tried to look at that question in detail).

    Definitely agree with you on the need for good information for women. Mostly, it reflects the bias of the producer – especially which data they choose as the basis for their information.

    Link to this

Add a Comment
You must sign in or register as a ScientificAmerican.com member to submit a comment.

More from Scientific American

Scientific American Dinosaurs

Get Total Access to our Digital Anthology

1,200 Articles

Order Now - Just $39! >

X

Email this Article

X