Skip to main content

A Modest Proposal: Virtual Keyboards via Leap Eyeglasses

This article was published in Scientific American’s former blog network and reflects the views of the author, not necessarily those of Scientific American


In the series "A Modest Proposal," my colleagues and I will propose inventions and projects that I think are eminently doable and would love made real.

Mobile devices are unquestionably more powerful than PCs of a generation ago. However, desktop and laptop computers still remain useful because of their peripherals, such as monitors, keyboards and mice or touchpads — their equivalents on mobile devices are far smaller and crammed together, greatly limiting their utility.

Attaching peripherals onto mobile devices typically adds bulk, thus sacrificing the virtues of having a handheld device to begin with. Ideally, one wants a peripheral that greatly increases the input and output capacities of mobile devices while taking up as little amount of space as possible.


On supporting science journalism

If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.


I've argued that electronic glasses combined with a virtual floating keyboard would accomplish just this. The glasses would display a screen and a virtual keyboard, have sensors that could detect where your fingertips are, and wirelessly communicate data to and from your mobile device.

I first proposed the idea with the Microsoft Kinect sensor, but its resolution is not really good enough to pick up the subtle finger motions needed to make a virtual keyboard work. However, San Francisco-based Leap Motion recently debuted the Leap gesture-based computer interaction system, a USB device roughly the size of a pocket knife that can recognize the differences between finger gestures, apparently tracking movements down to 1/100th of a millimeter, far more precise than the Kinect.

https://www.youtube.com/watch?feature=player_embedded&v=_d6KuiuteIA

I've also thought about the possibilities suggested by Touché by Disney Research, which can sense the way an object is held, even the human body. In principle, one can have wristbands that detect the way in which your hands and fingers are felt, which could also lead to a virtual keyboard. My problem with that idea is that you'd still want electronic glasses to see the virtual keyboard with, so why not use the Leap device?

Hopefully we'll hear serious discussion about incorporating the Leap or other next-generation Kinect-like device with electronic glasses soon.

You can email me regarding A Modest Proposal attoohardforscience@gmail.comand follow the series on Twitter at #modestproposal.

Charles Q. Choi is a frequent contributor to Scientific American. His work has also appeared in The New York Times, Science, Nature, Wired, and LiveScience, among others. In his spare time, he has traveled to all seven continents.

More by Charles Q. Choi