This article was published in Scientific American’s former blog network and reflects the views of the author, not necessarily those of Scientific American
On supporting science journalism
If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.
Now that so-called surface computing has begun to trickle into the mainstream—some real estate agencies, hotels, retailers and other businesses are beginning to use the technology to help their employees and customers interact with information using hand gestures on a touch screen in lieu of a keyboard and mouse—makers of this technology are delivering new uses for the technology and studying ways to improve the touch screen interface.
Microsoft, in particular, is looking to raise the profile of its Surface tabletop computer, which the company introduced last year for upwards of $10,000. At this week's Association for Computing Machinery (ACM) Conference on Human Factors in Computing Systems (CHI) in Boston, Microsoft presented research indicating that the Surface interface could be made more intuitive for its users. The company this week also introduced Mobile Connect software for Surface that lets smart phones wirelessly exchange information with the tabletop computer.
When Microsoft last year asked a group of 20 participants to demonstrate gestures they would use to, for example, open a computer file or rotate an object on Surface's tabletop touch interface, many displayed ones that Surface could not recognize. Surface relies on the touch sensors in its 30-inch (72-centimeter) screen to follow a user's commands, only recognizing gestures that entail that make contact with the screen. These results mean that Microsoft may have to consider a system that can look down on a user's hand or view it from the side, rather than simply looking up.
Another gesture common among test participants was to try to delete digital items displayed on Surface's screen by placing a finger on them and sliding that finger to the edge of the table. "To our surprise, multiple participants conceived of a world beyond the edges of the table's projected screen," says study co-author Meredith Morris, a researcher in Microsoft Research's adaptive systems and interaction group.
One woman in the study referenced the movie Minority Report (in which Tom Cruise plays a character who manipulates computer files using a vertical touch screen) saying that touch screen computing appeared "sexy and exciting" on film, Morris says. The woman noted, however, that in reality the use of large touch screens such as Surface could be exhausting after a while. Morris says that this indicates that gestures used with the technology should be short and simple.
Now that Morris has compiled a list of the most popular gestures, she and her colleagues plan to launch a new study that will give participants a choice between using gestures developed by professional software engineers and the consumer favorites.
"We want to see which they prefer," she says. "We want to find out what's most popular and see what's fundamentally different between gestures."
Even though Microsoft is still working out the finer points of Surface's user interface, the company this week introduced its Mobile Connect software that allows users to communicate with the tabletop via a smart phone. To use Connect, users load a program onto their phones, and then set the phone on the tabletop with the camera facing down.
The software lets Connect communicate with the phone via the camera by flashing patterns of colors picked up by the camera's lens. Once the connection between Surface and the phone is made, users can load contacts, photos and other information stored on the phone into the Surface computer and view them on the tabletop. Likewise, users can upload new software from Surface onto their phones, such as new ring tones. AT&T last year began offering software similar to Connect on Surface computers installed at the company's retail stores in New York City, Atlanta, San Antonio and San Francisco.
Image © Microsoft Corp.