"Eye-Controlled Interaction" for your GUI coming soon

Doug Engelbart, prepare to be usurped by the Fraunhofer Institute for Industrial Engineering. Yes, the folks that brought you the MP3 and the Multimedia Dome are now working on the Eye-Controlled Interaction system -- a way to control a computer GUI just by staring at it the right way, possibly rendering the mouse a quaint invention of 20th century computing. Hyperbole aside, the mouse is probably safe for now, given that the EYCIN is mainly designed to be used by "disabled people, or professionals, such as maintenance technicians or chefs, who need both hands free to carry out their work." Of course, this isn't the first eye tracking system out there. One of the previous iterations' major hurdles is involuntary eye movements that are easily misinterpreted as mouse movements, and that staring at an on-screen button for slightly too long can be misunderstood as well. Dr. Fabian Hermann, a usability engineer on the project, told The Engineer that by using a "statistical average filter" to account for the jerky involuntary motions, the software can track eye movements much more smoothly. Further, on-screen buttons will change color twice to ensure that the selection is exactly what the user meant. Hermann also said that he forsees the day when by just looking at your stereo or a light panel, you could turn it on or off. He adds: "You could even control a kitchen display showing recipes or other information when your hands are busy or covered in ingredients."

[Via Futurismic]

[CharlieX @ Oct 3rd 2006 6:41PM]

Imagine what a pain it's going to be to quickly hide that porn when your girlfriend walks into the room.

[angelo caduto @ Nov 29th 2006 9:58AM]

think outside the porn...

[Ryan @ Jan 10th 2009 8:50PM]

I think if it moves the mouse to where you're looking I suppose there would be some inadvertant censoring because the damn mouse would get in the way.

[Team Bop @ Oct 3rd 2006 6:44PM]

"You could even control a kitchen display showing recipes or other information when your hands are busy or covered in ingredients."

and

"who need both hands free to carry out their work."

The above statements imply that one would be able to perform these tasks simultaneously. I would have thought that controlling the said display would involve some sort of effort/concentration - like moving the pointer to a button for example. Wouldn't voice activation be alot more convenient in this scenario?

[Greg Mcp @ Oct 4th 2006 4:48AM]

Since the earliest days of home computing... the late 1970's... when a technology comes along that people don't quite know what it can be used for, somehow storing or displaying Recipies is offered up as a use.

It does sound like something with a specialized use. I dont want windows getting dragged around the screen just because I look at them. Your hands are the traditional tools for manipulating things.

[foe83 @ Oct 3rd 2006 7:03PM]

what happened to that gadged that you would put on your finger and it would read your thoughts to control the mouse? Or was it just a hoax?!?

[Andrew Blossom @ Oct 3rd 2006 7:15PM]

It seems that a device might need slightly more input than just 'looking' at it to interpret your wants with regards to its functionality.

Unless everything is going to have clearly distinguishable, highly visible 'buttons' to look at. Somehow I think handheld remote controls might be the more efficient option until the machines can read my mind.

[Scabies @ Oct 3rd 2006 7:21PM]

Boom! Headshot!
...that would be weird

[Sunil @ Oct 3rd 2006 7:22PM]

Imagine, Could that be the reason that all Macintosh computers starting with the iMac to the Mac Books have webcam built in, looking at the facial expressions of the person? That would be great to see if they bring it out this January... or something similar to that.

[Zoundguy @ Oct 3rd 2006 8:11PM]

Interesting Story, I'm a student, who after getting a final project on eye tracking contacted EyeTech digital systems, and was given a month to try out their product and use... real time in my demonstration. (I received the Quick Glance 2) and let me tell you, it was AWESOME. pure coolness. anybody with serious disabilities that would like to work on a computer should try this thing out, because it WORKS. Now, it wasn't all happy and fun though either, it took a couple of hours, and the lighting has to be right, but wow, when you get the lighting right, and understand how it works, way way way too cool. (I got an A)

[Zoundguy @ Oct 3rd 2006 8:14PM]

Sunil, it is much more than "just a camera" you need a high quality camera (one that gets a very accurate image of your eye) and IR sensors which reflect on your eye. (you can't see the light but the camera can). simply put, nope no Eyetracking built into Macs for the foreseeable future... but after the foreseeable future, SCORE.

[Jay @ Oct 3rd 2006 8:23PM]

I'm a believer in the idea that humans can get used to anything. Take for instance, walking. It's really hard to do. It takes us nearly a year to force ourselves against gravity and to learn how to balance. Imagine how long it takes us to get used to typing. Or how about steering a car while smoking a cigarette and talking on the phone.

If we can get used to those things we can easily get used to something as simple as LOOKING at something and blinking!

[alex @ Oct 3rd 2006 8:54PM]

I was think of ways you could control it with your eyes. Just looking at it or blinking seem like they could be misinterpreted too easily but what about winking? You can even right or left click depending on which eye you use.

[yubastard @ Oct 3rd 2006 10:08PM]

"What's french for creepy?"

I think I wouldn't like all my device's cameras staring @ my eyes all the time trying to see if I want to turn them on or something... what if I look @ them the wrong way? will they get mad @ me?

interesting, nonetheless.

[RuthLessPirate @ Oct 3rd 2006 11:12PM]

They were working on something like this at Mississippi State University as well. They showed us their prototype and it looked very useful and not intrusive. Theirs was some lightweight glasses with mirrors and tiny IR cameras to track the pupil. It seemed to work really well.

[Virtical @ Oct 3rd 2006 11:32PM]

Personally i cant see how it can be particuarly accurate, and as Zoundguy said you have to have near perfect lighting and all that, for the disabled, maybe, for the casual user or gamer, no.
Id rather sponsor brain pattern research thanks.

[schwal @ Oct 4th 2006 12:21AM]

I saw a prototype at SIGGRAPH, it was very flaky, and doesnt work if you are wearing glasses. might have been a different company, but it's still very beta.

[sneaker @ Oct 4th 2006 12:38AM]

The work presented here is not revolutionary by anymeans. The origins of eye-tracking can be traced back to the early 1900s and with some of the pioneering work being done in the 1930s (Buswell).

Video-based eye-tracking has been around for several years now, with some of the pioneering work done in the IBM BlueEyes project (Flickner, Zhai, Koons, Morimoto and Amir).

There are several (>10) commercial companies that provide video based eye-tracking products with the leaders in Sweden (Tobii), Australia (SeeingMachines) and the US (ASL, LC Technologies).

Gaze-based interaction has been tried for a long time and there is a rich history of academic literature which discusses various approaches (in case you're curious, blink are a bad idea). If you're really curious try searching Google Scholar or drop me an email for list of references.

In the interest of full-disclosure, this is the area of my thesis work at Stanford. As part of the research at Stanford we have come up with novel ways of combining gaze and other input modalities.

For more information, I would invite any interested people to visit http://hci.stanford.edu/research/GUIDe for more information.

[VBAjedi @ Oct 4th 2006 4:33AM]

Hmmm. . . if they are having trouble knowing when you are just looking at a button and when you want to "click" it, why not use blinks like clicks? Default of 2 blinks for left-click, 3 for right-click. "Blink speed" could be set like double-click speed is currently to customize for each user.

[ariel @ Oct 4th 2006 5:09AM]

the united states air force and NAVY.
have this types of systems in there air fighterplanes
and attack helopicters

[beatwaffles @ Oct 4th 2006 1:00PM]

lol, try pushing the power button on that bad boy.

[esubvaria @ Oct 4th 2006 1:19PM]

As a person who is profoundly disabled, the eye tracking systems work only when you have no head movement. I use brainfingers.

[damian @ Oct 4th 2006 4:23PM]

I'd be quite happy using eye tracking to move/focus on things and a handheld button/spacebar being used to actually click things. Best of both worlds, of course it would be less useful for disabled people but for the rest of us a hybrid click/eye system would be perfect.

[PunkRokAdrenalin @ Jun 13th 2007 8:19AM]

I agree with damian!
How much faster could we work with multi contact interfaces on our computers? I mean say I was working on a document/photo/video/whatever and I could switch between windows just by some eye movement!?!?! and for those linux users out there, change desktops "in the blink of an eye" haha sorry had to do it!

[Ryan @ Jan 10th 2009 8:50PM]

Okay, so this may seem totally "far out" but why not combine this software with Nia or Brainfingers?? No more need for a mouse, and almost no need for a keyboard. I mean, honestly, anybody who can use Nia to play Unreal Tournament 3 and beat the level 4 bots using only a mouse for aiming and Nia for movement should already be looking into development of an eye tracking function.

I love the idea of helping disabled people, but honestly I want this for FPS purposes!!!!!!