ADVERTISEMENT
  About the SA Blog Network













Observations

Observations


Opinion, arguments & analyses from the editors of Scientific American
Observations HomeAboutContact

Breakthrough Could Enable Others to Watch Your Dreams and Memories [Video]

The views expressed are those of the author and are not necessarily those of Scientific American.


Email   PrintPrint



Scientists at the University of California, Berkeley, have reconstructed the internal “movie” that plays in a person’s head. To re-create dynamic visual experiences, they used functional magnetic resonance imaging (fMRI) to measure the brain activity of volunteers (the other members of the research team) as they watched short movie clips (left panel in the video below). A computational model crunched the fMRI data to reproduce the images, as shown in the right panel.

The team, led by Shinji Nishimoto and Jack Gallant, say that the technology is decades away from enabling people to read others’ thoughts and intentions. It could become a powerful tool to communicate with people who cannot verbalize, such as stroke victim and coma patients. This visual image reconstruction study appears in the September 22 Current Biology.

Philip Yam About the Author: Philip Yam is the managing editor of ScientificAmerican.com. Follow on Twitter @philipyam.

The views expressed are those of the author and are not necessarily those of Scientific American.





Rights & Permissions

Comments 9 Comments

Add Comment
  1. 1. jtdwyer 4:45 pm 09/22/2011

    This is certainly very interesting and impressive work!

    It seems to make general sense to this pedestrian, in that the visual cortex might be expected to register something analogous to a bit map of visual signals.

    If this is somewhat correct, it raises the question: do dreams or, say, analytical thought processes, similarly activate the visual cortex? In other words, does thinking about or imagining the appearance of a bird activate the visual cortex?

    It occurs to me that recollection of stored (perhaps ‘encoded’) images might not require the use of the visual cortex…

    Link to this
  2. 2. adamsunny 5:21 pm 09/22/2011

    I wonder what it would show at the moment of someone’s death.

    Link to this
  3. 3. vespoli7 1:06 am 09/23/2011

    The most interesting part is that when the subject is watching a video of another person, the image looks much clearer than of objects, but it seems like a different face. It makes you wonder how much our impressions of the people we interact with are colored by memories or completely replaced by what we expect to observe.

    Link to this
  4. 4. zstansfi 1:21 am 09/23/2011

    @jt

    Yeah visual cortex is recruited whenever you have a visual image pop up “in your head”.

    As to this paper, I’m actually not overly impressed by the video. It looks like they’ve simply designed an algorithm which recognizes when faces are present in a particular region of the visual field. This actually isn’t all that unique, given that a region in the brain called the fusiform face area (FFA) is known to be activated by faces and similar stimuli. In this sense, it would be pretty simple to create a program that would display a face-like image in response to activation of the FFA. We’ve also known for decades that the spatial location of cells in visual cortex directly corresponds to locations in your visual field (called a retinotopic map). This would similarly make it pretty trivial to correlate the region in which “activation” occurs based upon a functional scan with the location in space that a person sees an image.

    The real problem with this technology is not just that it is currently unable to perform at the high level resolution that would be required to “record” what a person is seeing, but it also doesn’t actually “read” the brain’s code. The way fMRI is typically used, you just look at how much oxygen is being used in a given region; you can’t see the pattern of electrical activity which is thought to represent the true neural code.

    Link to this
  5. 5. jtdwyer 6:48 am 09/23/2011

    zstansfi – Thanks. A spatial correspondence between the optical signals and the assemblage of images in the visual cortex is what I was guessing.

    While the visual cortex may be involved in the perception of images during dream states, I’m skeptical that it is used in abstract thought – imagining a new car or even the face of a family member does not seem to disrupt visual processing…

    Link to this
  6. 6. Scienceangela 10:35 am 09/23/2011

    I wonder if these techniques can be used to help those who suffer from schizophrenia deal with their problems.

    Link to this
  7. 7. ellington 12:43 pm 09/23/2011

    This is so poorly explained it’s laughable. This *might* be interesting research, but presenting it in this sensationalist way, failing to mention that the clips “reconstructed from brain activity” are actually collaged YouTube videos matched up with brain activity, turns it into a joke. You’ve just lost a lot of credibility with me.

    Link to this
  8. 8. zampaz 3:36 am 09/24/2011

    Observations
    Opinion, arguments & analyses from the editors of Scientific American

    I agree with @ellington.
    This is really poor journalism coming from a SciAm editor no less. Not opinion, not argument and not analysis.
    A stupid blurb to sell copy.
    How about reporting some of the science Yam?
    Coming next?
    “Particles Found to Travel Faster than Speed of Light”
    “Neutrino results challenge a cornerstone of Einstein’s special theory of relativity, which itself forms the foundation of modern physics”
    Failing to stress that the results are unexplained and unverified and no where are the specific challenges to modern physics detailed.
    How about; “Laid off SyFy writers find new home at SciAm.”
    or
    “Scientific Method deregulated by SciAm team.”
    Looks like we better start outsourcing our media staff.

    Link to this
  9. 9. jgrosay 7:26 am 09/24/2011

    The Hammer’s movie: “Quatermass and the pit” includes an interesting use of this just newly opened possibility. Many times sci-fi antecedes actual findings. Is this some kind of deductive reasoning or travelling thru time ?

    Link to this

Add a Comment
You must sign in or register as a ScientificAmerican.com member to submit a comment.

More from Scientific American

Scientific American Special Universe

Get the latest Special Collector's edition

Secrets of the Universe: Past, Present, Future

Order Now >

X

Email this Article

X