Movie audiences who went to theaters this fall to see The Theory of Everything got a glimpse of the challenges physicist Stephen Hawking has overcome to deliver his groundbreaking insights into the nature of black holes, space and time. Tuesday the world gets a peek at how new technology will let the scientist and author continue to share his discoveries with the world as he battles the degenerative motor neuron disease that has degraded his ability to communicate over the past five decades.
Hawking has long used a voluntary twitch of his cheek muscle to compose words and sentences one character at a time that are expressed through a speech-generation device connected to his computer. Each of these small movements stops a cursor that continuously scans text on a screen facing the scientist. As Hawking has become increasingly disabled by amyotrophic lateral sclerosis (ALS), the rate at which he can communicate has in recent years fallen to a mere one word per minute.
Collaboration with researchers at chipmaker Intel has boosted Hawking’s typing speed while enabling him to perform common tasks—such as navigating the Web, e-mails and documents—10 times faster, according to the company. The key is customized software that simplifies his interactions with the technology he currently uses, including a tablet PC with a forward-facing Webcam used for Skype calls.
The Intel upgrade lets Hawking benefit from the combination of a more advanced system with better predictive capabilities built into the software, allowing him to enter fewer commands for common activities, says Intel’s Lama Nachman, principal engineer and project lead. More specifically, Intel incorporated a new word predictor from London-based app-maker SwiftKey that autocompletes words as they are typed, anticipating the next most commonly-used word in a phrase or sentence. The software also learns from individual users, enabling it to “dramatically reduce the number of clicks it takes for Stephen to build words,” Nachman says. The physicist now needs to type in only 20 percent of all characters before the system can complete his thought, she adds.
The new software complements the rest of his hardware armamentarium, which includes a black box beneath his wheelchair that contains an audio amplifier, voltage regulators and a USB hardware key that receives input from an infrared sensor on Hawking’s eyeglasses. The sensor detects changes in light as he twitches his cheek. A hardware voice synthesizer sits in another black box on the back of the chair and receives commands from the computer via a USB-based serial port.
The key to upgrading Hawking’s support infrastructure was the years Nachman and her team spent observing how the scientist used the technology available to him, understanding how he wrote documents or emails, gave lectures, searched the Web and read PDFs. “If you’re using Microsoft Word, which Stephen uses a lot, there are a few sets of functions that you want to use most often—open a new document, save, edit and so on,” Nachman says. “We added a lot of contextual menus to his system to speed up use, so he can select one with a single click, rather than having to go to the mouse, then to the menu, then to select an option.” Something that may have taken him as many as 15 different steps now takes five or fewer.
Intel designed Hawking’s upgrade specifically to work with his equipment. He didn’t want a new system, according to Nachman, which meant new approaches to eye tracking were off limits, as was an electroencephalogram (EEG) that translates brain activity into simple commands. The researchers are now working on a gesture recognition system that recognizes multiple facial expressions that they would like to introduce to Hawking, although that technology remains in the research phase, according to the company.
Part of Intel’s plan in developing Hawking’s new interface is to release the company’s open-source software freely to software developers next month. This will let other programmers adapt the code in ways that will make it a useful interface for others struggling with motor neuron diseases and quadriplegia.