ADVERTISEMENT
  About the SA Blog Network













Assignment: Impossible

Assignment: Impossible


Exploring the area between the unknown and the impossible.
Assignment: Impossible Home

A Modest Proposal: Virtual Keyboards via Kinect Eyeglasses

The views expressed are those of the author and are not necessarily those of Scientific American.


Email   PrintPrint



In the series “A Modest Proposal,” my colleagues and I will propose inventions and projects that I think are eminently doable and would love made real.

I love my iPad. I scan vast amounts of information daily for work and play, and tablet computers are ideal for displaying it. But tablets and mobile devices in general have a key weakness — as much as they help people consume information, they are not very good at helping us produce it. This is why I am proposing virtual floating keyboards via Kinect-loaded eyeglasses.

A virtue and a vice of iPads is that they use the same surface for inputting and outputting data. Since the display is the interface, all you need are your fingers to manipulate information, which means figuring out how to use them comes naturally to infants, senior citizens and even cats. The problem is that when it comes to writing, the keyboard that pops up to input data competes with the visual real estate needed for the iPad to output data.

The way that conventional computers solve this problem is through peripherals such as keyboards and mice. Tablets can use wireless versions of those too, but those detract from the streamlined experience they offer. The new iOS 5 that Apple has for its latest generation of mobile devices can split the iPad’s keyboard up into two smaller halves so it takes up less visual real estate, but the problem there is that you can only use your two thumbs to input data, as opposed to all 10 fingers, limiting your input bandwidth.

The iPhone 4S has Siri, which allows users to interact with applications by voice, which to me shows that Apple is trying to come up with new interfaces to overcome the size limits that mobile devices have when it comes to keyboards. This is very promising, but speech recognition still has a ways to go before it can always accurately transcribe what you are saying, and even if it was perfect, keyboards and mice are still better at helping people cut, paste, copy and otherwise edit data — 10 fingers are better than one voice in that regard.

So what to do? Ideally you would have a virtual keyboard you could use without actually having a physical keyboard to lug around. Inventors have developed many techniques over the years that allow any surface to be turned into keyboards — a recent version can be seen here — but to me it seems a bit laborious to have to try and find a surface to project a keyboard onto. Virtual reality gloves also seem cumbersome, and are not a one-size-fits-all peripheral.

I suggest eyeglasses that display virtual keyboards. This would essentially be an augmented reality system that overlays a floating keyboard onto your field of vision. I also suggest a Kinect sensor mounted on top of these glasses.

After the sensor recognize your 10 fingertips, you can then type in the air onto the virtual keyboard to input data into your mobile device. I suggest these glasses only be used to overlay a keyboard instead of a full screen — a virtue of tablets is that they only take up as much of your field of view as you want, and you can easily put them down if you want.

Kinect sensors are currently rather clunky to fit onto eyeglasses, but I wager optimized sensors could be much smaller and integrated into the glasses. I imagine the system would also require a rather healthy amount of computing power, but perhaps that could be exported onto the tablet in question, with data transmitted wirelessly to and from the glasses.

I suspect the main bottleneck of such augmented reality glasses would be power. There’s not a lot of space in glasses to support batteries for what must be a rather power-hungry application.

You can email me regarding A Modest Proposal at toohardforscience@gmail.com and follow the series on Twitter at #modestproposal.

Charles Q. Choi About the Author: Charles Q. Choi is a frequent contributor to Scientific American. His work has also appeared in The New York Times, Science, Nature, Wired, and LiveScience, among others. In his spare time, he has traveled to all seven continents. Follow on Twitter @cqchoi.

The views expressed are those of the author and are not necessarily those of Scientific American.





Rights & Permissions

Comments 1 Comment

Add Comment
  1. 1. nilsdavis 12:15 pm 12/17/2011

    Charles – I have been thinking for quite a while that the next critical limit to overcome to really accelerate progress is how to get info into the darn computer. And until we have full augmented reality in our contact lenses (as in Vinge’s “Rainbows End”), the Vuzix Augmented Reality glasses, announced in August at the Consumer Electronics show, and as far as I know, now shipping, will have to do. They would be perfect for this. Much better than wearing a Kinect receiver on your head!

    I wrote a short blog post about these glasses btw – they are on my Xmas list this year for sure! http://nilsdavis.com/2011/08/07/warning-one-of-you-will-probably-drop-5k-on-a-pair-of-these/

    Link to this

Add a Comment
You must sign in or register as a ScientificAmerican.com member to submit a comment.

More from Scientific American

Scientific American Special Universe

Get the latest Special Collector's edition

Secrets of the Universe: Past, Present, Future

Order Now >

X

Email this Article

X