This article was published in Scientific American’s former blog network and reflects the views of the author, not necessarily those of Scientific American
Imagine a hypothetical scenario: a new drug is given to 100 people suffering from a disease. Ninety-nine of those people don’t improve but one person is totally cured. Is the treatment effective? What about if, say, 10 percent of the patients experience a 20 percent benefit? Although hypothetical, questions like these pose a challenge for researchers every day, made more difficult because patient experience is often hard to describe in binary terms. The answer is somewhat dependent on whom is being asked—the responses of the scientists running the experiment and the cured patient will clearly differ.
This subjectivity has real-world implications, as seen in the recent tragic case of Charlie Gard. Now 10 months old, Charlie was born with mitochondrial depletion syndrome, a rare genetic condition that means he is unable to hear and see or even breathe without a respirator. His parents fought in the European Court of Human Rights to fly Charlie to the U.S. for an experimental, unproved treatment. His doctors in the U.K., however, argued this would be unlikely to improve his condition, and recommendedhis life-support to be switched off. Here, the definition of what constitutes effective was driven—rightly or wrongly—by the power of human emotion.
In research, scientists use a combination of appropriate experimental design and statistics to retain objectivity as far as practically possible. Double-blind, placebo-controlled studies allow us to compare the effects of a treatment with a substance we know is pharmacologically inert; p-values provide an indication of how likely our results are to have occurred by chance, and whether they are “significant.” Well-designed studies possess the required “power” to make these assertions by involving sufficient numbers of subjects, cases or experiments. These mechanisms are implemented to ensure results are reliable, robust and ultimately protect the public downstream.
On supporting science journalism
If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.
Various regulatory bodies are then responsible for translating these research findings into clinical guidance—essentially deciding if a treatment is worth giving to patients. In the U.K. this is the job of the National Institute for Health and Care Excellence (NICE), in the U.S. the Food and Drug Administration. Even they are occasionally influenced by such subjectivity, however. In 2006 NICE approved the use of the breast cancer drug Herceptin for early-stage breast cancer following political pressure and a successful legal challenge by British woman Ann-Marie Rogers. Previous to this Herceptin was only approved for advanced breast cancers, with scant evidence supporting its use in the early stage. Economic factors are an obvious issue here—regulatory bodies sometimes decide treatments are worth it when the price comes down.
The above case was lauded as an example of “patient power.” It was, in fact, a case of feelings trumping evidence. In the U.S. “right to try” laws give terminal patients the legal right to try experimental treatments that have passed phase I clinical trials. There is a reason clinical testing is so stringent and extensive—it’s to protect the public from the adverse effects of the many, many drugs that don’t make it to phase II.
Unfortunately, despite all the protective and regulatory mechanisms in place, some treatments have disastrous consequences when they reach the market. The devastating fallout from drugs such as thalidomide contributes to a lingering public distrust of large pharmaceutical companies, a feeling that is then exploited by unscrupulous entities to drive an “us versus them” narrative, which can in turn be used to promote their own less rigorously tested brand of treatment.
These more dubious entities and their “treatments” flourish in the blurred region between what is effective, beneficial and harmless. Despite all the vast, available evidence that homeopathy is no more effective than a placebo, for example, people who believe they have benefitted from it continue to extol its use, and the alternative medicine market is valued at $34 billion. Ironically, “Big Pharma” has given birth to “Big Alterna”—two sides of the same, meaningless coin.
There are, of course, treatments produced through the more stringent channels that also don’t really do very much. Many over-the-counter cough medicines, for example, contain no active ingredient, and doctors have been telling patients not to waste their money for years. Many people will claim, however, they were of some benefit after using them. The key here is that there is yet another distinction to be made between treatments we consider “effective” and merely “beneficial.”
My own research is focused on the progressive neurodegenerative disease, amyotrophic lateral sclerosis. Biologically modifying treatment options for ALS are unfortunately limited. Understandably, many patients therefore look to unproved treatments in their search for hope, and a simple Google search reveals hundreds of companies making claims of various magnitude. At their best, such treatments offer some relief or comfort to the patients who seek them, which, it may be argued, fulfills their purpose. At their worst, they cynically exploit vulnerable people by preying on their desperation, often for large sums of money. Sadly, doctors have many such stories of patients being duped by promises of unobtainable results.
Ultimately, when deciding on which treatment to choose, most of us place our trust in a combination of the objectivity gained from science and the subjectivity of our own emotions. Just make sure your expectations are aligned accordingly.