ADVERTISEMENT
  About the SA Blog Network













Observations

Observations


Opinion, arguments & analyses from the editors of Scientific American
Observations HomeAboutContact

5 Sigma—What’s That?

The views expressed are those of the author and are not necessarily those of Scientific American.


Email   PrintPrint



A graph of the normal distribution, showing 3 standard deviations on either side of the mean µ. A five-sigma observation corresponds to data even further from the mean. Source: Wikimedia Commons/Mwtoews

Chances are, you heard this month about the discovery of a tiny fundamental physics particle that may be the long-sought Higgs boson.  The phrase five-sigma was tossed about by scientists to describe the strength of the discovery. So, what does five-sigma mean?

In short, five-sigma corresponds to a p-value, or probability, of 3×10-7, or about 1 in 3.5 million. This is not the probability that the Higgs boson does or doesn’t exist; rather, it is the probability that if the particle does not exist, the data that CERN scientists collected in Geneva, Switzerland, would be at least as extreme as what they observed. “The reason that it’s so annoying is that people want to hear declarative statements, like ‘The probability that there’s a Higgs is 99.9 percent,’ but the real statement has an ‘if’ in there. There’s a conditional. There’s no way to remove the conditional,” says Kyle Cranmer, a physicist at New York University and member of the ATLAS team, one of the two groups that announced the new particle results in Geneva on July 4.

Scientists use p-values to test the likelihood of hypotheses. In an experiment comparing some phenomenon A to phenomenon B, researchers construct two hypotheses: that “A and B are not correlated,” which is known as the null hypothesis, and that “A and B are correlated,” which is known as the research hypothesis.

The researchers then assume the null hypothesis (because it’s the most conservative supposition, intellectually) and calculate the probability of obtaining data as extreme or more extreme than what they observed, given that there is no relationship between A and B. This calculation, which yields the p-value, can be based on any of several different statistical tests. If the p-value is low, for example 0.01, this means that there is only a small chance (one percent for p=0.01) that the data would have been observed by chance without the correlation. Usually there is a pre-established threshold in a field of study for rejecting the null hypothesis and claiming that A and B are correlated. Values of p=0.05 and p=0.01 are very common in many scientific disciplines.

High-energy physics requires even lower p-values to announce evidence or discoveries. The threshold for “evidence of a particle,” corresponds to p=0.003, and the standard for “discovery” is p=0.0000003.

The reason for such stringent standards is that several three-sigma events have later turned out to be statistical anomalies, and physicists are loath to declare discovery and later find out that the result was just a blip. One factor is the “look elsewhere effect:” when analyzing very wide energy intervals, it is likely that you will see a statistically improbable event at some particular energy level. As a concrete example, there is just under a one percent chance of flipping an ordinary coin 100 times and getting at least 66 heads. But if a thousand people flip identical coins 100 times each, it becomes likely that a few people will get at least 66 heads each; one of those events on its own should not be interpreted as evidence that the coins were somehow rigged.

So where do the sigmas come in? The Greek letter sigma is used to represent standard deviation. Standard deviation measures the distribution of data points around a mean, or average, and can be thought of as how “wide” the distribution of points or values is. A sample with a high standard deviation is more spread out—it has more variability, and a sample with a low standard deviation clusters more tightly around the mean. For example, a plot of dogs’ heights would probably have a larger standard deviation than a plot of heights of dogs from a particular breed, even if that breed had the same average height as dogs in general.

For particle physics, the sigma used is the standard deviation arising from a normal distribution of data, familiar to us as a bell curve. In a perfect bell curve, 68% of the data is within one standard deviation of the mean, 95% is within two, and so on.

In the case of the results announced last week, the process was more complicated than simply taking the results from one experiment and measuring the deviation of the data from the expected background levels; data came from many different channels, and each one had a different expected background signal.  In addition, there were uncertainties about the measurements from the detectors that had to be taken into account. Researchers used a complex formula to combine all of these variables and calculate a p-value. This value was then translated into a number of sigmas above the mean, because the number of collisions observed at the energy of the newly discovered particle was higher than the expected background.

This final point led to some confusion in the media about the p-value associated with five-sigma. In a normal distribution, data is symmetrically distributed on both sides of the mean. It is twice as likely for data to be in either the high or low tail than just the high tail, so some outlets reported that five-sigma corresponded to a p-value of 0.0000006, or 1 in 1.7 million, rather than the correct value of 0.0000003, or 1 in 3.5 million. For further discussion of this subtlety, see this Understanding Uncertainty blog post.

The excitement about the Higgs discovery led the two teams to announce their results before all the data had been analyzed. Going forward, after both teams’ analyses are complete, the groups  will combine their observations. Although the two experiments are based on similar physical principles, it is not trivial to combine their data in a meaningful way. If your wallet were filled with both U.S. dollars and Euros (or Swiss Francs if you were visiting CERN), you couldn’t simply add the numbers on the bills to find out how much money you had; you would have to perform some conversions first. The groups will use what Cranmer calls “collaborative statistical modeling” to combine the results of the two experiments (ATLAS and CMS). This approach has already been used to perform “conversions” on data sets within each team’s experiment. When complete, these analyses will convey a more accurate sense of the strength of the new evidence and determine whether the observed data is consistent with the Higgs boson physicists seek.

Evelyn Lamb About the Author: Evelyn Lamb is a postdoc at the University of Utah. She writes about mathematics and other cool stuff. Follow on Twitter @evelynjlamb.

The views expressed are those of the author and are not necessarily those of Scientific American.





Rights & Permissions

Comments 10 Comments

Add Comment
  1. 1. promytius 5:15 pm 07/17/2012

    Best explanation I’ve seen of sigma; thanks!

    Link to this
  2. 2. priddseren 6:26 pm 07/17/2012

    How about a much more realistic explanation. This is the scheme scientists come up with when they cant actually prove something exists with real solid evidence. Why would they do this, not too hard to figure out. To use CERN as the example, the massively over priced experiment needs to show some sort of benefit to the politicians that paid for it. Those same scientists need to justify their existence, convince politicians to spend more money and keep getting grants to continue operating and they know the attention span of most politicians is about that of a 6 year old kid, so they come up with a scheme to “prove” discovery because they know the politicians won’t wait the years it takes to find real evidence.
    After decades of this non-sense, scientists have basically stopped looking for reality and instead have opted for finding sigma measures or just going with what computer models tell them.

    Link to this
  3. 3. elderlybloke 7:27 pm 07/17/2012

    “probability, of 3×10-7, or about 1 in 3.5 million.”

    I think that a million is 10^6.
    These are little numbers but changing from 6 to 7 is a big change in probability.

    Dear priddseren ,
    Please keep taking your pills,I think you have forgatten yours today.

    Link to this
  4. 4. bwooster47 8:47 pm 07/17/2012

    In response to #3:
    > “probability, of 3×10-7, or about 1 in 3.5 million.”
    >> I think that a million is 10^6.

    No, the article is correct.
    Probability of 0.3 means a 1 in 3.5 chance.
    Which is why the first line, taken from the article, is correct.

    Link to this
  5. 5. jtdwyer 2:00 am 07/18/2012

    A Nature News article reporting the results,
    http://www.nature.com/news/physicists-declare-victory-in-higgs-hunt-1.10940
    stated:
    “Heuer puts the possibility of the measurements being statistical flukes at the order of one in a million – in physicists’ terms, around 5 sigma.”

    “The ways in which the new particle interacts with other particles is consistent with what was expected for a Higgs boson, although further measurements will need to be made to pin down its identity. In particular, physicists will want to determine whether the new boson has zero spin as predicted, according to Incandela.”

    “The way in which the new particle decays into other particles will also be key to verifying its precise nature. Already, the new boson seems to be decaying slightly more often into pairs of gamma rays than was predicted by theories, says Bill Murray, a physicist on ATLAS, the other experiment involved in making the discovery. But, he is quick to add, the data are still very preliminary.”

    “To try to answer some of these questions in the next year, the LHC will run for three months longer than was originally planned, according to Heuer. “It’s the beginning of a long journey,” he says.”

    The article also included an inset image http://www.nature.com/polopoly_fs/7.5244.1341407852!/image/1.10940.jpg_gen/derivatives/landscape_300/1.10940.jpg
    with the caption:
    “The ATLAS experiment has seen a new type of boson decaying into four electrons – a good indicator that it is a Higgs particle.”
    The image attribution is: ATLAS EXPERIMENT © 2012 CERN

    As I understand this explanation, the 5-sigma statistical declaration applies to the possibility that the identified particle detection was the product of signal noise; it does not apply to the possibility that the detected particle is not a Higgs particle. There is much less certainty that the particle is in fact the predicted Higgs boson.

    Link to this
  6. 6. A. Tarkovsky 2:42 am 07/18/2012

    The Mikhalkov CDSM team (of which I am a member) at Vostok Station, Antarctica, celebrates CERN CMS’s historic announcement of July 4, 2012, as do scientists around the globe. We must note here, however, that we have seen no press coverage for the equally historic work of CDSM, which we have maintained at a remarkable 100 pK for 18 months. Cold Dense Sticky Mass will sufficiently cool free bosons to isolate up-bosons and down-bosons relative to observers in the northern hemisphere, a process predicted by gravitational particle-wave theory. Also not reported is that staff at Vostok Station suffer hallucinations, resulting from the cold and isolation, out of devotion to science and waiting for these lousy particles. The good news of July 4, 2012, however, encourages us to wait even longer.

    Link to this
  7. 7. tharriss 7:48 am 07/18/2012

    That is really interesting A. Tarkovsky… do you have any links you can provide that would give more information on this work? Thanks for your dedication!

    Link to this
  8. 8. TomLWaters 1:27 pm 07/18/2012

    Overall a very good explanation. However, this struck me:

    “The reason that it’s so annoying is that people want to hear declarative statements, like ‘The probability that there’s a Higgs is 99.9 percent,’ but the real statement has an ‘if’ in there. There’s a conditional. There’s no way to remove the conditional,” says Kyle Cranmer, a physicist at New York University

    Apparently Dr. Cranmet is not familiar with Bayesian statistics, which does precisely this. I found it a little ironic, as Giulio D’Asgostini of CERN has been a proponent of the use of Bayesian statistics in elementary particle research for well over a decade.

    Link to this
  9. 9. dadster 6:17 pm 07/18/2012

    Higgs particle is comparatively an extremely heavy entity. In the normal curve, heavier the particle ,it would lie towards the right end of the graph and the less heavy particles fall at the left end of the graph .Maximum number of particles would be found in the mid region and so we say that the maximum probability is to find average weight particles,indicated by the height of the curve at the middle.The concept is that all natural occurrences ,if they are random events would have a distribution symmetrical to the middle line. and all observations would snugly fit into certain limits here , sigma limits. It may be noticed that in an ideal normal (random)distribution all readings fall within 6 sigma limits and no chance of any observations falling beyond that limit ( ie,3 sigma distance on either side of the middle) .Sigma limits indicates how far away the reading is from the average value ( or how rare the event is ).For example in a student community in a university the brightest and duds would be few and far between with most of the students being average .
    Now standing this concept on its head ,we can use this to test the validity of data collected . If there are many observations that happened to fall beyond 6 sigma limits then we can start doubting the authenticity of the data .Variations from the norm cannot be beyond 6 sigma limits if the data were unbiased and truly random.

    if the data collected all fall within five sigma limits that is a good sign regarding the genuineness of the data collected and conclusions drawn from that could stand a better chance of being true .
    In this case the x-axis is indicative of the weight of particles and the y axis indicative of the relative probability(or probability density). since the weighty Higgs boson can only lie at the right end of the spectrum of data and if the frequency of occurrence of particles having the mathematically predicted weight of higgs boson is found at a distance of 2.5 sigma distance away to the right of the mid-value ( we say it as “five sigma” taking into consideration of the limits extending to 2.5 distance to both sides of the mid value of the normal distribution )then the incidence ( or existence ) of such particles could be considered as genuine. In the predicted region if enough number of particles were detected ( height of the curve is as it should be at that distance from the normal)
    then we have to accept that such particles with that massive weight do exist. Observational evidence cannot be brushed away .
    Now this raises more questions than it answers . Q1.what about so many other heavier particles that were also detected that lie beyond 5 sigma limit ? Do they constitute another layer of reality that gives Higgs Bosons its mass ? if so how did they gain mass ? Certainly not from the less heavy Higgs Bosons. Q2. How do those particles influence ordinary particles ?
    Over and above these doubts another more serious doubt hangs around .That is as follows .In quantum science the concept of “measurement” is dicey .Measurement cannot ever be objective .The observer ( along with his intentions)is very much part of the observation. This logically leads to the conclusion that the observer can create the observation (or create the “Observed” phenomena), by his/her act of observation.That is to say that before the observation of a quantum phenomena there was no such phenomena existing. The otherwise non existant Higgs boson phenomena could have taken place because of intense, continuous and consistent observation (with the intention of observing it), if at all it has occurred . Therefore unless and until another couple of teams of scientists who intensely,consistently and continuously believe that such a particle cannot exist should , even then “sight” it , can we really believe about the authenticity of the existence of the Higgs Boson .Till then Higgs Boson should remain in the mathematical equations only.

    Link to this
  10. 10. A. Tarkovsky 3:11 am 07/19/2012

    Thank you tharriss (7:48 am 07/18/2012). We are delighted go get your inquiry now, since it coincides with the imminent return to duty of Yuri Aksakov, CDSM press officer. I am confidant Yuri will post a nice overview here with links to additional resources as his recovery and healing of amputations allow. Yuri’s main interest is in practical applications of superstring theory.

    Link to this

Add a Comment
You must sign in or register as a ScientificAmerican.com member to submit a comment.

More from Scientific American

Scientific American Special Universe

Get the latest Special Collector's edition

Secrets of the Universe: Past, Present, Future

Order Now >

X

Email this Article

X