ADVERTISEMENT
  About the SA Blog Network













The Curious Wavefunction

The Curious Wavefunction


Musings on chemistry and the history and philosophy of science
The Curious Wavefunction Home

Falsification and its discontents

The views expressed are those of the author and are not necessarily those of Scientific American.


Email   PrintPrint



Karl Popper's grounding in the age of physics colored his views regarding the way science is done. Falsification was one of the resulting casualties (Image: Wikipedia Commons)

One of the answers to Edge.org’s question “What scientific idea is ready for retirement”? is by physicist Sean Carroll. Carroll takes on an idea from the philosophy of science that’s usually considered a given: falsification. I mostly agree with Carroll’s take, although others seem to be unhappier, mainly because Carroll seems to be postulating that lack of falsification should not really make a dent in ideas like the multiverse and string theory.

I think falsification is one of those ideas which is a good guideline but which cannot be taken at face value and applied with abandon to every scientific paradigm or field. It’s also a good example of how ideas from the philosophy of science may have little to do with real science. Too much of anything is bad, especially when that anything is considered to be an inviolable truth.

It’s instructive to look at falsification’s father to understand the problems with the idea. Just like his successor Thomas Kuhn, Karl Popper was steeped in physics. He grew up during the heyday of the discipline and ran circles around the Vienna Circle whose members (mostly mathematicians, physicists and philosophers) never really accepted him as part of the group. Just like Kuhn Popper was heavily influenced by the revolutionary discoveries in physics during the 1920s and 30s and this colored his philosophy of science.

Popper and Kuhn are both favorite examples of mine for illustrating how the philosophy of science has been biased toward physics and by physicists. The origin of falsification was simple: Popper realized that no amount of data can really prove a theory, but that even a single key data point can potentially disprove it. The two scientific paradigms which were reigning then – quantum mechanics and relativity – certainly conformed to his theory. Physics as practiced then was adept at making very precise, quantitative predictions about a variety of phenomena, from the electron’s charge to the perihelion of Mercury. Falsification certainly worked very well when applied to these theories. Sensibly Popper advocated it as a tool to distinguish science from non-science (and from nonsense).

But in 2014 falsification has become a much less reliable and more complicated beast. Let’s run through a list of its limitations and failures. For one thing, Popper’s idea that no amount of data can confirm a theory is a dictum that’s simply not obeyed by the majority of the world’s scientists. In practice a large amount of data does improve confidence in a theory. Scientists usually don’t need to confirm a theory one hundred percent in order to trust and use it; in most cases a theory only needs to be good enough. Thus the purported lack of confidence in a theory just because we are not one hundred percent sure of its validity is a philosophical fear, more pondered by grim professors haunting the halls of academia than by practical scientists performing experiments in the everyday world.

Nor does Popper’s exhortation that a single incisive data point slay a theory hold any water in many scientists’ minds. Whether because of pride in their creations or because of simple caution, most scientists don’t discard a theory the moment there’s an experiment which disagrees with its main conclusions. Maybe the apparatus is flawed, or maybe you have done the statistics wrong; there’s always something that can rescue a theory from death. But most frequently, it’s a simple tweaking of the theory that can save it. For instance, the highly unexpected discovery of CP violation did not require physicists to discard the theoretical framework of particle physics. They could easily save their quantum universe by introducing some further principles that accounted for the anomalous phenomenon. Science would be in trouble if scientists started abandoning theories the moment an experiment disagreed with them. Of course there are some cases where a single experiment can actually make or break a theory but fortunately for the sanity of its practitioners, there are few such cases in science.

Another reason why falsification has turned into a nebulous entity is because much of modern, cutting-edge science is based on models rather than theories. Models are both simpler and less rigorous than theories and they apply to specific, complicated situations which cannot be resolved from first principles. There may be multiple models that can account for the same piece of data. As a molecular modeler I am fully aware of how one can tweak models to fit the data. Sometimes this is justified, at other times it’s a sneaky way to avoid admitting failure. But whatever the case, the fact is that falsification of a model almost never kills it instantly since a model by its very nature is supposed to be more or less a fictional construct. Both climate models and molecular models can be manipulated to agree with the data when the data disagrees with their previous incarnation, a fact that gives many climate skeptics heartburn. The issue here is not whether such manipulation is justified, rather it’s that falsification is really a blunt tool to judge the validity of such models. As science becomes even more complex and model-driven, this failure of falsification to discriminate between competing models will become even more widespread.

The last problem with falsification is that since it was heavily influenced by Popper’s training in physics it simply fails to apply to many activities pursued by scientists in other fields, such as chemistry. The Nobel Prize winning Roald Hoffmann has argued in his recent book how falsification is almost irrelevant to many chemists whose main activity is to synthesize molecules. What hypothesis are you falsifying, exactly, when you are making a new drug to treat cancer or a new polymer to sense toxic environmental chemicals? Now you could get very vague and general and claim that every scientific experiment is a falsification experiment since it’s implicitly based on belief in some principle of science. But as they say, a theory that explains everything explains nothing, so such a catchall definition of falsification ceases to be useful.

All this being said, there is no doubt that falsification is a generally useful guideline for doing science. Like a few other commenters I am surprised that Carroll uses his critique of falsification to justify work in areas like string theory and the multiverse, because it seems to me that those are precisely the areas where testable and falsifiable predictions are badly needed because of lack of success. Perhaps Carroll is simply saying that too much of anything including falsification is bad. With that I resoundingly agree. In fact I would go further and contend that too much of philosophy is always bad for science; as they say, the philosophy of science is too important to be left to philosophers of science.

Ashutosh Jogalekar About the Author: Ashutosh (Ash) Jogalekar is a chemist interested in the history and philosophy of science. He considers science to be a seamless and all-encompassing part of the human experience. Follow on Twitter @curiouswavefn.

The views expressed are those of the author and are not necessarily those of Scientific American.





Rights & Permissions

Comments 23 Comments

Add Comment
  1. 1. jtdwyer 5:56 pm 01/24/2014

    Without testable, falsifiable hypotheses, what distinguishes science from philosophy or even fiction?

    Link to this
  2. 2. Marc Levesque 6:23 pm 01/24/2014

    “Perhaps Carroll is simply saying that too much of anything including falsification is bad. With that I resoundingly agree. In fact I would go further and contend that too much of philosophy is always bad for science;”

    And perhaps Jogalekar is simply saying that too much of anything including philosophy is bad for science.

    Link to this
  3. 3. Marc Levesque 7:41 pm 01/24/2014

    Sorry about that, I didn’t mean it to sound insulting.

    “the philosophy of science is too important to be left to philosophers of science”

    I’m still wondering what you mean though

    Link to this
  4. 4. Chryses 8:54 pm 01/24/2014

    If Falsification is removed as a criterion for a scientific theory, it becomes very difficult to exclude theories like Astrology from being “Science”.

    Link to this
  5. 5. Johnay 11:36 pm 01/24/2014

    I think modeling can be considered as a pre-falsifiable stage.

    A model should perhaps be looked at as a system of interacting hypotheses, any one or more of which may well be false. Running the model against real data may not (heck, probably won’t) yield enough information to pinpoint which of them is false, or even identify what they are, but rather just tell you that *something* is off. Over time & with tweaking the modeller can feel their way to a model-space with less error and perhaps inform a search for the actual falsifiable hypotheses.

    In short, a model is not itself falsifiable but rather can be a tool for finding falsifiable hypotheses.

    Link to this
  6. 6. Carlyle 4:57 am 01/25/2014

    At what stage do you discard a model that repeatedly fails to be born out by empirical facts? The models predicting the effects of CO2 have repeatedly failed for many years now. I am convinced the models are wrong & that continuing to defend them is doing great harm to the credibility of science in general.

    Link to this
  7. 7. Bill_Crofut 9:33 am 01/25/2014

    Re: “I think falsification is one of those ideas which is a good guideline but which cannot be taken at face value and applied with abandon to every scientific paradigm or field. It’s also a good example of how ideas from the philosophy of science may have little to do with real science.”

    A ranking biologist may have agreed with you concerning abandon, but he certainly had no problem waxing philosophically on a selective basis:

    “Scientific creationism” is a self-contradictory, nonsense phrase precisely because it cannot be falsified. I can envision observations and experiments that would disprove any evolutionary theory I know, but I cannot imagine what potential data could lead creationists to abandon their beliefs. Unbeatable systems are dogma, not science.”

    [Prof. Stephen Jay Gould. 1981. Evolution as Fact and Theory. DISCOVER, May, p. 35]

    Gould provided no example of a falsifiable observation or experiment in that article; perhaps it was an oversight.

    Link to this
  8. 8. Spironis 11:20 am 01/25/2014

    If it does not empirically work, it is wrong. This condemns teaching, economics, quantum gravitation, SUSY, religion, and Windows 8. What is not to like?

    “If it disagrees with experiment, it is wrong. In that simple statement is the key to science. It does not make any difference how beautiful your guess is, it does not make any difference how smart you are, who made the guess, or what his name is — if it disagrees with experiment, it is wrong.” Richard Feynman. Cf: Feynman’s calculation of buffers needed in Thinking Machine’s CPU chips. Comp Sci said one more than could be afforded, Feynman said what fit was OK. Do your trust your life to an entire academic department, or Feynman who did it with continuous (rather than digital discrete) differential equations? Ask the Challenger Committee.

    Link to this
  9. 9. schatzieD 12:08 pm 01/25/2014

    FYI, everyone who is not aware: Carlyle is paid to “deny” climate change. Disregard his every single comment. He’s a shill.

    Link to this
  10. 10. Phencyclidine 3:40 pm 01/25/2014

    “For one thing, Popper’s idea that no amount of data can confirm a theory is a dictum that’s simply not obeyed by the majority of the world’s scientists”

    I don’t think that’s true and even if it is that doesn’t mean that Popper’s idea isn’t correct. I think scientists say things like “this theory has been proven” simply because it’s a lot more convenient to say than talking about how it hasn’t been falsified and because it makes it easier to communicate scientific ideas to the (nonscientist) public.

    Also, it seems to me that just because a hypothesis has a flaw (i.e. that there are data which falsify it) doesn’t mean that the hypothesis should be totally thrown away and forgotten about, especially if there are no other hypotheses available which work. It just means that the hypothesis needs improvement or that we need to work on it until the model works better. We can’t expect to propose new hypotheses every time one single datum falsifies a previously working hypothesis.

    “Nor does Popper’s exhortation that a single incisive data point slay a theory hold any water in many scientists’ minds.”

    I don’t think Popper ever said such a thing and I’ve read a few of his books. I’m pretty sure that he said it wouldn’t really make a lot of sense to completely abandon a hypothesis simply because of a single data point. So, yeah, I don’t think that many scientists would agree with such a thing, even though it may be true in the strictest sense and I don’t think Popper would have agreed with such a thing.

    “Science would be in trouble if scientists started abandoning theories the moment an experiment disagreed with them”

    I agree and I really don’t think that Popper ever suggested that a theory should be abandoned the moment that a single experiment disagreed.

    “Another reason why falsification has turned into a nebulous entity is because much of modern, cutting-edge science is based on models rather than theories.”

    And models are based on theories. I’m not really sure what you’re getting at here. Models are going to be imperfect, of course, and I don’t think anyone would suggest tossing out a theory simply because a model, which is based on theory but greatly simplified, ended up being wrong.

    “The last problem with falsification is that since it was heavily influenced by Popper’s training in physics it simply fails to apply to many activities pursued by scientists in other fields, such as chemistry.”

    I totally disagree. I’m a chemist and I think Popper’s philosophy of science is spot on. I was educated in chemistry before reading Popper, but when I read The Logic of Scientific Discovery and Conjectures and Refutations, it seemed to me that I “already knew” what Popper was saying but just never articulated it before. It also seemed really obvious to me that most chemists seemed to think about science the way Popper did.

    “What hypothesis are you falsifying, exactly, when you are making a new drug to treat cancer or a new polymer to sense toxic environmental chemicals . . .”

    This seems pretty easy to answer to me. When doing synthetic work, you’re obviously working with theory. Organic chemistry and the rules that give chemists a guideline to synthesizing new molecules are theoretical in nature (MO theory, for example). I didn’t synthesize molecules just by tossing things together in a flask and stirring. Theory played an important role because it let me know what starting materials I should use, what solvents might work and if I didn’t end up with the molecule I wanted, what other reactions might have taken place.

    If I were trying to synthesize something and out came a product which simply should not be possible to synthesize from those starting materials, then clearly falsification is relevant. To say it’s not relevant is too narrow. Really this is just a testament to how well theories in chemistry work: synthetic chemists don’t often find themselves synthesizing molecules in ways that totally violate the theories of chemistry.

    Link to this
  11. 11. Phencyclidine 3:44 pm 01/25/2014

    re: above

    “Nor does Popper’s exhortation that a single incisive data point slay a theory hold any water in many scientists’ minds.”

    I don’t think Popper ever said such a thing and I’ve read a few of his books. . . .

    In the strictest sense, Popper does say this, but that’s far too narrow an understanding of Popper. He goes to great lengths to point out that even when testing a hypothesis, one must make a number of assumptions and assume that a number of other hypotheses are correct, so a single experiment which disagrees with theory doesn’t necessarily mean the end of the theory. He also points out that from a practical point of view, it’s not sensible to toss out a working hypothesis or theory simply because of one experiment.

    In fact, I wonder if you can find a quote from one of Popper’s books or articles where he says that a single incisive data point should result in the death of a theory.

    Link to this
  12. 12. Chryses 6:53 pm 01/25/2014

    schatzieD (#9, @12:08 pm 01/25/2014)
    “… Carlyle is paid to “deny” climate change. Disregard his every single comment. He’s a shill.”
    Can you supply any evidence to support those claims? If you cannot, why do you post them?

    Link to this
  13. 13. rloldershaw 10:05 pm 01/25/2014

    Falsifiablity has always sounded a bit draconian and absolute.

    I, and others, have urged the use of the alternative term testability.

    A good scientific theory makes predictions that can be empirically tested.

    A definitive prediction is (1) feasibly tested, (2) made prior to testing, (3) quantitative, (4) NON-ADJUSTABLE, and (5) unique to the theory being tested.

    If a theory cannot make a definitive prediction, it needs further development until it can.

    RLO
    http://www3.amherst.edu/~rloldershaw

    Link to this
  14. 14. Errorstat 12:51 am 01/26/2014

    The author has quite a superficial understanding of Popper, on the order of Cliffs notes perhaps. That’s too bad, especially seeing how “freedom from testability” gets poor science off the hook. Popper demanded a repeatable phenomenon in order to corroborate a falsifying hypothesis which could then falsify a theory–scarcely a single anomaly.

    As for: “What hypothesis are you falsifying, exactly, when you are making a new drug to treat cancer..” How about, that it doesn’t work?
    http://errorstatistics.com/2012/02/01/no-pain-philosophy-skepticism-rationality-popper-and-all-that-part-2-duhems-problem-methodological-falsification/

    Link to this
  15. 15. oldfarmermac 9:51 am 01/26/2014

    Falsification is obviously the ultimate tool to be used in separating science from dogma but having said this much I think this much more needs to be said;

    First off a good bit of what is being described in this discussion is not science as such but the use of in as a tool. We don’t usually think of bookkeeper as mathematicians do we?

    Second a lot of what is obviously science as I see things is simply science aborning and not yet mature enough to stand alone and weather the test of fallibility.

    If we declare the people at the cutting edge of the knottiest problems as being outside the fold then progress in their fields may be stopped dead for various reasons relating to politics and funding and the way university labs and govt agencies are administered.

    The people at the cutting edge may need a lot of time and support to move ahead far enough to meet the falsification test.

    Link to this
  16. 16. RSchmidt 10:27 am 01/26/2014

    I understand the point of the position and I think a lot of people are taking a binary view of the scientific process, i.e. something is either “proven” or not. What people are forgetting is that science is a process of refinement. Rarely are we able to go from hypothesis to theory in one set of experiments, especially when dealing with these complex problems. It may take decades of both theoretical and practical research before one gets to a truly testable hypothesis. If we kill the idea before we get there because the steps along the way were not by themselves testable then we have become stuck in a local maximum. I think it is reasonable to “believe” in a hypothesis based on the supporting evidence but I don’t think it should be ruled out just because we haven’t figured out how to test it yet. I also think it is reasonable to say, this is the best we can do right now. In the case of string theory, it is not like we have anything else that is falsifiable.

    Again, we have to stop thinking that everything is either proven or disproven. There are shades of gray and a process of refinement. Don’t cut the journey short by demanding that every step but a complete destination unto itself.

    Link to this
  17. 17. centromere 11:26 am 01/26/2014

    @4. Chryses,

    That’s a reasonable criticism. Falsification, or testability as rloldershaw would prefer, does help separate the wheat from the chaff.

    Link to this
  18. 18. SoftLanding 3:19 pm 01/26/2014

    Good question! How do you distinguish between scientific and non-scientific theories if you drop the falsifiability criterion?

    Link to this
  19. 19. patmcgee 1:06 am 01/27/2014

    Without making any comment at all about anything else, I’d like to disagree with your example about synthetic chemists.

    I think of engineering as making things, using whatever tools they have available, including scientific theories. I think of science as developing and refining those theories. So, in your example, I would call what those people do as engineering, not science. The science part would be what they do with those things once they’ve made them.

    (Yes, I know that those are really two endpoints on a long continuum, with everyone really doing stuff somewhere in the middle. But I think each activity tends to land more on one end or the other, sometimes going from one to the other many times in a single project.)

    Engineering needs testing. Either we created what we wanted or we didn’t. Whether you call that falsifiability or not, I don’t care. it works about the same and serves about the same purpose.

    Link to this
  20. 20. David Marjanović 12:03 pm 01/27/2014

    But in 2014 falsification has become a much less reliable and more complicated beast. Let’s run through a list of its limitations and failures. For one thing, Popper’s idea that no amount of data can confirm a theory is a dictum that’s simply not obeyed by the majority of the world’s scientists. In practice a large amount of data does improve confidence in a theory.

    Well, yes: if a large amount of data is gathered and none of it falsifies the theory, that means there have been lots of opportunities to falsify it but the theory has passed all of them.

    Scientists usually don’t need to confirm a theory one hundred percent in order to trust and use it; in most cases a theory only needs to be good enough.

    I thought that was Popper’s point? It’s impossible to prove a theory, to confirm it 100 %, so at some point it’s OK to say “this is good enough for the time being”? Approportioning the strength of one’s belief to the strength of the evidence and all that?

    But most frequently, it’s a simple tweaking of the theory that can save it. For instance, the highly unexpected discovery of CP violation did not require physicists to discard the theoretical framework of particle physics. They could easily save their quantum universe by introducing some further principles that accounted for the anomalous phenomenon.

    Strictly speaking, modifying a theory creates a new theory. When CP violation was discovered, physicists quietly acknowledged that their theory was wrong, and instead they proposed a slightly different one – so slightly different that they kept the name and everything. Popper used a very strict definition of “wrong”.

    I do agree, however, that falsification is often overemphasized at the expense of parsimony. I’m a phylogeneticist. In phylogenetics, hardly anything can be really falsified; anything can evolve any which way any number of times independently. So, we try to find those phylogenetic hypotheses (trees) that require the smallest amount of assumptions to explain the data. Finding a shorter tree is a lot like falsifying all longer ones, it’s just less black-or-white.

    Indeed, there’s a lot of parsimony hidden in falsification (and even in mathematical proof). Suppose you make observations that falsify a hypothesis – can you really prove your results are real, can you prove there aren’t instead demons messing with your brain and the brains of everyone who tries to replicate your observations? Nobody entertains such ridiculously unparsimonious tosh, but I’m not aware of an actual disproof.

    The models predicting the effects of CO2 have repeatedly failed for many years now.

    Do tell. In particular, explain why the deep ocean is heating up, and why that extra heat and the extra heat in the atmosphere add up to the predicted amount.

    Gould provided no example of a falsifiable observation or experiment in that article; perhaps it was an oversight.

    One word for you: Lenski.

    FYI, everyone who is not aware: Carlyle is paid to “deny” climate change. Disregard his every single comment. He’s a shill.

    I don’t see how the second sentence follows from the others.

    I think scientists say things like “this theory has been proven” simply because it’s a lot more convenient to say than talking about how it hasn’t been falsified and because it makes it easier to communicate scientific ideas to the (nonscientist) public.

    Very few ever say “proven”, actually. The historical linguists still do it, but I can’t remember anyone outside that field saying it.

    Falsifiablity has always sounded a bit draconian and absolute.

    I, and others, have urged the use of the alternative term testability.

    Indeed this term is often used, but I wonder if it doesn’t imply that falsification and verification are equally possible – which they’re not.

    Link to this
  21. 21. Dr. Strangelove 2:29 am 01/28/2014

    “In fact I would go further and contend that too much of philosophy is always bad for science; as they say, the philosophy of science is too important to be left to philosophers of science.”

    In response I quote Feynmann “the philosophy of science is as important to scientists as ornithology is important to birds.”

    Science is what good scientists do regardless what philosophers say. There must be a physical way to determine if a theory is true or false. Otherwise it is philosophy. Experimental physicists need billion-dollar particle accelerator. Theoretical physicists only need paper and pencil. Philosophers don’t even need trash can.

    Link to this
  22. 22. Jerzy v. 3.0. 10:41 am 01/30/2014

    I think physicists like Popper are unaware how complex are biological systems, and changing only one variable is impossible.

    You can get two identical electrons or metal balls to test physical theory. But, in genetic experiments, you cannot get two groups of cancer cells, mice or patients behaving in anything confortably close to an identical way.

    At Popper times, biology was mostly about macroscopic and mechanistic things and it was easier. One perhaps can operate hearts of two patients in an almost identical way.

    Falsification has its place, but eg. pretending that cancer patients are identical would be terrible mistake to make.

    Link to this
  23. 23. sfagnew 4:52 pm 02/14/2014

    As far as I know, there is no model or theory that is completely consistent with all observations. For one thing, all observations are not necessarily accurate or precise. If a theory makes important predictions that are consistent with observations, that theory is useful. If an observation falsifies a theory in one context or at one scale or at one precision, the theory might still provide useful predictions in other contexts or at other scales. And, of course, observations are uncertain things and may be in error, so science has used and continues to use many models and theories that have been wrong and falsified in some context. Testing and observation are still the mainstays of the patchwork of theories and models that we call science, not really falsification. Scientists are much keener about verification than they are about falsification and in fact, a false negative can lead to dropping a useful hypothesis whereas a false positive leads to more testing…and more funding…

    Link to this

Add a Comment
You must sign in or register as a ScientificAmerican.com member to submit a comment.

More from Scientific American

Scientific American Back To School

Back to School Sale!

12 Digital Issues + 4 Years of Archive Access just $19.99

Order Now >

X

Email this Article

X