Video: NeoVisus Gaze prototype enables handsfree computer control

If you're tired of worrying about carpal tunnel syndrome setting in and ruining your ability to surf the intarwebz, Martin Tall is cooking up an alternative to the tried-and-true mouse / keyboard control combo. The NeoVisus Gaze prototype, quite simply, enables computer users to control the action on screen with just their eyes. Granted, this setup works entirely better when viewing photos than, say, typing up a thesis -- but there's always voice recognition software for that, right? Check out the demonstration video below.

[lance @ Jun 11th 2008 8:57AM]

finally, NO FINGERPRINTS!
This is has way better response than the touch stuff lately.
I'm in. Where do I get it.

[OneLove @ Jun 11th 2008 10:37AM]

My name is martin tall and i am gaze and i like to play with track balls.

[Neeko @ Jun 11th 2008 9:01AM]

OK so now we have touch,voice and visual/gesture control of a computer. Can someone please just incorporate all of these into one package and we finally have the next gen OS we have all been waiting for.

This software would be perfect for the disabled or handicapped indiv. Also for anyone with slight mental disabilities.

I would use it to control my Media from my tv without having to buy a expensive remote. Hell i could turn on my TV startup my Tv and begin a movie all without even moving an inch...YES PLEASE I WANT IT NOW.

[Phoenixfury @ Jun 11th 2008 12:57PM]

When I saw just the topic, I immediately thought about a friend of mine that is a quadriplegic. Currently he does use a voice navigation system to get around his computer.. It's not perfect either, I've seen it do other things than what he commanded it to do.. Often it was things that involved the UI like opening programs or firing off commands. This gaze thing might be the extra mile he needs for a (less) frustration free interface. I don't see it replacing tasks like typing for him, but this would make getting around his computer much easier.

[thedesolate1 @ Jun 11th 2008 11:54PM]

yes we want the OS to be like the one in minority report... It can also tell when a hacker is going to make a virus before he does and call the cops on him..

[Mike @ Jun 11th 2008 9:02AM]

Pretty cool. That pie interface is getting pretty popular.

[d @ Jun 11th 2008 9:21AM]

blame/thank the iPod.

Yes, I did just turn this into an apple post.

Please don't rate me down because I turned this into an apple post.

But seriously, the iPod did help popularize that, and the music controls are completely identical except for the stop/menu switch. Can't argue that.

[FatDude @ Jun 11th 2008 10:20AM]

Dear d,

There are plenty of devices that have used radial dials to do something before the ipod came out. A VCR is a classic exmample. Maybe you might not have been exposed to this, but before the ipod craze, I had a VCR that had a similar layout to that of the ipod. another vcr later actually used a twisty dial for seeking!(the whole layout "reeks" of copying the ipod which at that point was still the future). I originally thought this wasn't worth saying anything towards, but when i read your bold and loud statement, the first thing I could think of was other round things, like a rotary phone, or simone says.

Do I own an iPod? yes. Do I compare every UI design out there to it, or another apple product? no.

[brickwood @ Jun 11th 2008 9:08AM]

rad!

[initialxy @ Jun 11th 2008 9:09AM]

my concern is that sometimes you want to look at somewhere else while selecting a different spot. for example move down the scroll bar of a page. did he address that? cuz i have sound off (i'm at work).

[Stiv @ Jun 11th 2008 9:14AM]

Exactly. I'm often not looking at the exact control I'm manipulating. Like when I will soon click the "Add Your Comments" button, I won't look right at it.

I think this would be great for physically challenged people, but I think I would find it tedious and slow.

[Bunson @ Jun 11th 2008 11:57AM]

I think this would be more practical for a media based computer, not a work one. In the end it's more stylish but less efficient.

Blinking naturally would cause problems
Multitasking would become impossible
Typing documents would still require a keyboard

However with a media computer this would make switching channels, browsing pictures, listening to music more convenient.

IMO

[Homeboy @ Jun 11th 2008 9:18AM]

What if you're wearing sunglasses

[Frank @ Jun 11th 2008 9:33AM]

Then you should wonder why you're wearing sunglasses indoors.

[gad get @ Jun 11th 2008 3:35PM]

Why not just take off the sunglasses for the duration of your computer session? Or does the monitor hurt your eyes?

[Eli @ Jun 11th 2008 9:18AM]

I can see some great applications for this thing.

Two of the main ones I think of are gaming, primarily FPS, imagine taking out the mouse movement, and instead using the eyes to control the crosshair and using the mouse just to shoot.

The second application is coding.
instead of moving your hand every time you need to move from one file to another, or move 25 lines UP/DOWN or to a specific location (rather then using the up/down/left/right keys on the keyboard), and I'd love it that I had some kind of a mouse button on my keyboard (under the space bar?) and then just keep on working without actually moving your hands too much.

Any tips on how to create one of these setups on my own? this could be sweet as hell...

Eli

[Pfanne @ Jun 11th 2008 9:20AM]

this is really cool, but why is this eyetracking device THIS big!?

[David MacMillan @ Jun 11th 2008 9:22AM]

I, for one, welcome our eye-controlling mind-controlling infrared overlords.

Sorry.

Mac OS X Leopard's Quick Look application is just the kind of thing that could be used with something like this (perhaps with a combination of touchscreen input) for the next-generation OS. Think about it.

What do you use a computer for? The one I'm sitting at right now is the computer at the office...I make appointments with Lotus Notes, use the Windows Explorer to browse through filesystems and open files with WordPerfect and Quattro Pro, and I occasionally go online to check Engadget, etc.

My home computer? Movies (online or otherwise), photos, Facebook, intarwebz, typing up a resume, video chat, music recording, listening to music, etc.

I think that the next generation OS should be file-oriented rather than program-oriented. In other words, the OS itself (like an uberpowerful implementation of QuickLook) can open, edit, and save a huge variety of files...whether it is .docx, .mov, .mp3, .jpg, .xlsx, or whatever. As new file formats come out, the OS company (or the open source developers, depending on whether this OS is open source or not) would release the necessary file protocols for the new format, allowing the program t stay up to date.

The point will be that it will be just as easy to open a movie file and watch it was it would be to open a movie file, start to watch it, and jump into editing mode. All of a user's files can be stored in a virtual 3D workspace, almost like the program shown in the video above. You'll have a dedicated keyboard (or touchscreen keyboard) for major text entry, but the mouse will be mostly obsolete since everything will be controlled and opened with glances and touches.

The internet browser will be the same way...built into the architecture of the OS for seamless integration. Sure, you can replace it with another browser of your choice, but the internet is the internet.

Video chat programs, games, etc would run alongside as separate programs (almost like separate OSes within the OS).

[Nishant @ Jun 11th 2008 9:47AM]

That's a pretty good idea, an OS with all the opensource standards integrated in, but the one problem with that I can think of is security.
A browser exploit would compromise your OS directly.

[David MacMillan @ Jun 11th 2008 9:59AM]

Well in a way, this could be much MORE secure. The browser would be integrated in that it was "always there" - no windows as though it was a separate program. I'm pretty sure that people use browsers more than any other program. However, it would be perfectly secure because nothing would transfer between the browser and the rest of the filesystem without the user explicitly trying to make it happen. In a way, it's safer to have a "hither shalt thou go, and no further" browser than one that stands alone as a separate program and has a lot of access to file systems, etc.

It's much harder to exploit a browser file transfer system when the security is built directly into the architecture of the OS. How do I know this? **ahem** Never mind.

[Tiogshi @ Jun 11th 2008 10:48AM]

Nishant; never heard of capability-based security?

Your browser would only have network-protocol access, read access to its own configuration folders, and write access only to a unified cache system. If it wants more, you'll have to give it that access at your own discretion and risk.

It's long time we stopped treating computer security as a single "my" key (one key for your front door, car, file cabinet, and gun chest), and started treating it as giving out keyrings for finer-grain components (one key for each).

[Californian @ Jun 11th 2008 9:23AM]

That would get really annoying really quick. If you happen to look at an ad online, you are redirected there. But for some things it would be fine.

[Neeko @ Jun 11th 2008 9:28AM]

Yeah i would def have to disable all ADS within firefox. Sorry advertisers but you blinking glowing ads force my eyes to glance your way, causing me to view your page by accident. ...DAMN YOU.

[TareX @ Jun 11th 2008 10:22AM]

The developer never even hinted that this could replace the mouse/touchscreen/keyboard. It is just a new method of input.

I see that the applications he showed off were very suitable for this method of input. Hearing comments like this is similar to saying "And how will you TYPE with that" when the first mouse came out.

[Neeko @ Jun 11th 2008 9:24AM]

Software actually seemed pretty fast. it prob wouldnt wouldnt benefit everyone every application but i can imagine a majority of ur daily computer use besides typing could benefit from this.

I can imaging this would eliminate some use for a mouse and only a keyboard would be needed for some people. Your basic mouse is just a point and click. I could use Firefox just using my eyes esp since i have all my bookmarks on my screen.

The possibilities for this application are endless.

[person157 @ Jun 11th 2008 9:26AM]

"Quits when I blink." Sounds painful.

[gad get @ Jun 11th 2008 3:48PM]

Or you look at a link/icon/menu you have no intention of going into and-- d'oh! "Hon, I swear that was accidental, the image of that girl naturally attracted my eye, I can't help it! I didn't even mean to look. I swear!"
It seems you'd have to really avoid letting your eye pause on anything you didn't intend to "click." You'd be constantly flicking your gaze around the screen and would have a hard time reading links or what have you. I cannot overstate how much that kind of a system would drive me nuts!

[drewmk2 @ Jun 11th 2008 9:44AM]

It would be so easy to get confused/not intend on "clicking" something. I see more potential in touchscreen.

[eh @ Jun 11th 2008 9:49AM]

Are there any studies to whether or not this constant additional IR in the eyes is ok and not going to cause the user to go blind prematurely or cause other optical issues?

Has lots of limitations, like others said. sunglasses, controlling something you are not looking at, minute controls ie collapsed winamp, etc.

[Motoken @ Jun 11th 2008 2:26PM]

your getting hit by IR waves everytime your outside, due to the sun, also 2nd IR waves carry much less energy, less energy then even visible light.
If your eye is not damaged by regular light then you should have no problem with infrared lights.

[David MacMillan @ Jun 11th 2008 10:07AM]

@drewmk2, @eh:

You're absolutely right...it would have a lot of limitations. The two biggest ones I can see are: you can't demonstrate anything on your computer screen, and peripheral vision becomes worthless (oh, THAT will make us better drivers). I mean, seriously.

I see this as an awesome input device, but not the primary input device. Someone needs to figure out how to integrate this with a keyboard and/or mouse/touchpad. It would be most effective in synch with a large tactile feedback multitouch touchpad...you could essentially direct where you wanted your fingers to be "inputting".

Here's one place it would POSITIVELY SHINE...combine it with a keyboard (function keys to change what you are actually inputting) and use it with Photoshop. Drawing...pictures...with...your...eyes. BEAUTIFUL!

Seriously. Move the cursor with your eyes and use a, s, d, f, g, and h (or whatever) to determine whether you are painting, erasing, selecting, deselecting, copying, or pasting. THAT would make the jobs of us graphics design people SO much easier.

[TareX @ Jun 11th 2008 10:50AM]

Picture this too:

In 5 years, you could be holding a foldable XO-2 OLPC with this eye-sensor built in. You could be reading an e-comic on both screens, and everytime your eyes hit a comic frame, things start moving in the frame (instead of the old drawings indicating an action/movement is taking place).

Of course the usual ZAAM! and BANG! will be there, but BY GOD this would be much more immersive.... (and an audio file could run too with the video, detailing the background noise and coversation taking place in this frame of the comic)

[TareX @ Jun 11th 2008 10:19AM]

I'd buy it for the music application alone. Having something like this at home... just look at the album and it plays. Magnificent.

It's also very accurate and "applicable". Before seeing the youtube video I was like "How the hell is this gonna help... mouse rules", but there are indeed a plethora of applications for something like this.

Great work man.

[Tiogshi @ Jun 11th 2008 10:53AM]

I haven't noticed it in the comments yet... please tell me someone noticed how useful this would be for those under partial-or-worse paralysis? Many of them are not lucky enough to have a limb for a mouse, and joysticks are not very graceful input mechanisms.

[Holger @ Jun 11th 2008 11:13AM]

Hm.. very good work, only thing is that it requires a special/custom interface/program to work. ie. getting it to work with windows or other standard programs would be awesome and something that disabled peple would probably love to have. I guess the "only" thing he's missing is left and right click. he could do something like closing the right eye is right click and closing left eye is left click. then combined with voice recognition for typing in text, you'd have a hands free computer

[Bunson @ Jun 11th 2008 11:50AM]

Blinking naturally would be a bitch.

[YG2012 @ Jun 11th 2008 12:06PM]

Found this:
http://annhyg.oxfordjournals.org/cgi/content/abstract/38/4/351?ck=nck
"Exposure to intense optical radiation leads to the development of infra-red (i.r ) cataracts in the workplace."
I am sure that prolonged exposure to infra-red light may not be the best replacement to prolonged use of your hands on keyboards. I'd rather suffer from carpel tunnel syndrome than blindness.

[mansbarklund @ Jun 11th 2008 12:15PM]

When it comes to assistive technology it has been going on for a quite a while already. Check Tobii out for instance they have had these kind of screens attached to whelchairs for some years already.

http://www.youtube.com/watch?v=fyNzA2ZcsPs

[mck @ Jun 11th 2008 12:16PM]

left eye blink == left click
right eye blink == right click
Is blinking both eyes the middle button or double-click?

[Motoken @ Jun 11th 2008 2:08PM]

this looks very useful and not very complicated or expensive to do, all you need is the infrared, the camera and the software.

[Motoken @ Jun 11th 2008 2:10PM]

also wanted to add, that unlike the touch screen, you don't need to buy a new monitor/tv, it looks simple enough to be set up at home by yourself.

[zezam @ Jun 11th 2008 2:15PM]

What about http://www.tobii.com ???
Isn´t this just a rippoff?

[m @ Jun 12th 2008 1:05AM]

It's a different approach in terms of interaction method. Besides that the Tobii software is very inaccessible due to pricing strategy, they have well over 100 employees. I'm just one guy. This was beta 0.1. My interface and components are likely to go open source in a near future. Stay tuned..