This article was published in Scientific American’s former blog network and reflects the views of the author, not necessarily those of Scientific American
Everyone learns about the senses from when we are very young. We smell with our noses, taste with our tongues, see with our eyes, touch with our skin, and listen with our ears … right?
When scientists study our senses, they find that things are a bit more complicated. The sensory inputs that we receive from our bodies interact with each other in our brains to provide us with much more information than we would receive keeping everything in isolation. Some of this is easy to believe – like the ability to smell affecting your sense of taste when you have a bad cold. Try tasting juice, fruit candy, or gum first while holding your nose closed. Then release your nostrils and you will be able to see how the flavors change from very simple (sour or sweet) to tasting like you would normally expect.
On supporting science journalism
If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.
But what about our hearing?
It turns out that when scientists study what we hear in more detail, what we see affects what we hear and what we hear actually affects how we speak!
Imagine this. You are listening to someone repeat the same sound over and over and you open and close your eyes. What you hear won’t change depending on whether your eyes are closed or open … or will it? Scientist Harry McGurk and his research assistant John MacDonald found a difference when they put the audio from one recording to the video for another recording in 1976.
When the sound for one phoneme (smallest units of sound for a language) was played at the same time that the person on the video was making the lip movements for a different phoneme, the sound that the listener perceived actually changed. Even though the sounds being played did not change (such as repeating “ba, ba, ba, ba,” watching the lip movements for an “f” sound at the same time actually results in the listener hearing “fa, fa, fa, fa.”
This BBC video will not only explain the details, but it will provide examples you can use right from home. Try it with your eyes open and your eyes closed. See what you can hear … and see!
Take a moment to realize how quickly this change is happening to what you hear. Your brain is integrating the information from the video quickly enough that it changes the phoneme you perceive in real time.
Scientists have found that your brain can use this kind of real-time feedback with the information you hear so quickly that it can change the sounds coming out of your mouth.
By using headphones that would change the speech sounds people heard themselves making as they made them, researchers were able to trick people’s brains into believing that they were making a speech error. Participants would say a word like “bed” and through the headphones – so quickly that it seemed like they were hearing the word as they spoke it – the researchers would alter the frequency of the sound to make the participant hear “bad.”
When participants would hear themselves making the wrong sound, they would change the sound they were saying (to a lower frequency sound like “bid”). They would make this change so quickly that they would hear the word they had been trying to say in their headphones. They would try to say bed, hear bad, change what they are saying to bid, and then hear bed through the headphones.
Our brains are integrating this information so quickly that we can correct these errors before a word is even finished.
What we hear (or see, or smell) is made up of so much more information than just the input from a single sense. Think about this the next time you have trouble understanding someone when they talk on the phone or your voice sounds strange when you have a cold. You can even record your own voice reading something when you are listening to music or not listening to anything at all. Do you sound any different? See what you can find out!
Sources:
McGurk, H & Macdonald, J (1976)Hearing lips and seeing voices. Nature264, 746 - 748; doi:10.1038/264746a0
Niziolek CA (2014) When bed goes bad: how the brain can fix mistakes in speech while they happen.Front. Young Minds.2:1. doi: 10.3389/frym.2014.00001
Small, D. (2008) How does the way food looks or its smell influence taste?Scientific American