Wiimote re-purposed for glove-based multi-touch system

Our new hero Johnny Chung Lee has already put the Wiimote to use for various multi-touch and multi-point systems (not to mention head-tracking), but it looks like others are slowly starting to follow in his footsteps, and the folks at Cynergy Labs have now produced one of the most polished "hacks" to date. Dubbed Project Maestro, the setup is built upon Microsoft's Windows Presentation System (or WPF) and uses a pair of Minority Report-style IR gloves to give you "multi-touch at a distance." That, as you can see in the video after the break, allows for an interface not unlike Microsoft's Surface -- without the "surface," of course. Unlike Johnny Lee's projects, however, the goods behind this one aren't available to the public just yet (for free or otherwise), but hopefully that'll change before too long.

[Thanks, Henry]

[whatishalo? @ Jan 3rd 2008 1:13PM]

Wow, 8 days to cobble that together...impressive!

[Halfmad @ Jan 3rd 2008 1:14PM]

His "pinching" seems a bit hit and miss, might not be use to using it without the gimp mask?

[Yevon @ Jan 3rd 2008 1:19PM]

Why in the hell has no one ever thought of this before? This is amazing stuff. *applause*

Who wants to guess on when M$ will cash in on this and make the Microsoft Glove?
"Someday your computer will be a big ass table, um, glove."

[Jay @ Jan 3rd 2008 1:26PM]

want!

[Homeboy @ Jan 3rd 2008 1:29PM]

Apple and Nintendo need to smash their brains together and supply this type of input device to the masses before 2011. Minority report came out 6 years ago so they have had enough time to work on this.

[Ruben @ Jan 3rd 2008 2:06PM]

Just because they both make white devices doesnt mean they are going to be friends.

[Ruben @ Jan 3rd 2008 2:06PM]

Just because they both make white devices doesnt mean they are going to be friends.

[Homeboy @ Jan 3rd 2008 2:43PM]

Ruben

My Nintendo 8-bit isn't white.

[DSeaver @ Jan 3rd 2008 3:36PM]

Neither was my gameboy, snes, n64 or sweet indigo gamecube, but you dont see me complaining!

[Solivagus @ May 15th 2009 5:23AM]

apple doesn't need to team up with nintendo (it would be great), because all macbook(pro)s already have an built in camera wich can also capture infrared light,.. just need to find away to focus only on infrared and you could even do more than 4 trackings at a time,..

[Yasir @ Jan 3rd 2008 1:29PM]

*this* should be the implementation to desktops as multitouch is to the iphone

[broli @ Jan 3rd 2008 1:34PM]

The wii controller obviously shows that's it's not that responsive. I'm pretty sure that with better IR hardware you can have an insanely cheap and an even better product that beats touchscreens on every way.

[iofthestorm @ Jan 3rd 2008 6:42PM]

Actually, the Wii remote can track up to 4 IR blobs at 100hz, that's pretty fast for logic running inside a $40 remote.

[steve @ Jan 3rd 2008 1:36PM]

I like the concept and there is a lot of room for this to go, but practicality at this point is pretty bad (yeah, I know this is a quick test from them). Imagine doing this for more than a few minutes.

[ejay boy @ Jan 3rd 2008 1:40PM]

This is awesome. That wiimote can do a lot of stuff with the help of different mods and hacks.

It would be better for Apple and Nintendo to team up to make the new UI platform. Imagine Steve doing his keynote using those gloves.

[epsilon343 @ Jan 3rd 2008 1:51PM]

Color me impressed. I was kind of expecting something a little gimmicky but that's something that I'd actually like to use on a day to day basis. Just put the receiver so that you can keep your hands in a natural position and just work your computer from the couch with a small wireless keyboard.

[biggun @ Jan 3rd 2008 1:59PM]

Gotta love twits who say things like "Why didn't *they* do this before?" and "*They* should have thought of that by now".

If you see a need then fill it. If you think of a product that should be developed then develop it. Don't complain that *others* haven't done it or that *others* should have done it sooner.

Hats off to those who saw the potential for the Wiimote and are making use of it. A kick in the rear to those couch potatoes who are just complaining.

[Jonathan Bergeron @ Jan 3rd 2008 2:09PM]

Thank God. One day closer to a Minority Report type world... I can't wait.

[Elliott @ Jan 3rd 2008 2:15PM]

*type type type type type*

Enhance.

*type type type type type type*

Enhance.

[Douglas Knudsen @ Jan 3rd 2008 2:21PM]

Lot of folks working on this sort of thing at several places, levels, and from different points. for example back in July we saw this http://dougmccune.com/blog/2007/07/23/tileui-desktop-demo-video/

DK

[Matt Smith @ Jan 3rd 2008 2:38PM]

Pretty neat stuff... interesting that it took the Wiimote for this to be created... The Wiimote IR seems to have a bit of a lag, but very cool that it can sense multiple IR signals!

[shiizun @ Jan 3rd 2008 2:52PM]

The wiimote can track up to 4 IR signals. Notice the guys only uses less than 4 fingers at a time.

[Chrismix @ Jan 3rd 2008 3:00PM]

Mmm.. so is the reason that all this is possible now because of what is inside the wiimote? or has this infrared technology been out before the wiimote and only got noticed because of the wii?

I think all this stuff is amazing, and cant wait till it becomes a bit more commercialized. since im too lazy to dl all the hacks and build everything myself.

...though if they take any longer i might have to.

[Elliott @ Jan 3rd 2008 3:21PM]

Wonder if naysayers will be starting talk of IR technology causing cancer because of mods like this and the Wii even though we've been using this tech for years.

[JC @ Jan 3rd 2008 3:27PM]

"I love the PowerGlove. It's so baaaaad." - Lucas

[JasonB @ Jan 3rd 2008 3:28PM]

This is not a first for Nintendo... does no one here remember the power glove??

http://en.wikipedia.org/wiki/Power_Glove

[Fank @ Jan 3rd 2008 4:00PM]

Yeah, that obviously has everything to do with it.

[pearce_dom @ Jan 3rd 2008 3:30PM]

nice :) especially for 8 days.. i wonder whether they can make the z axis work also, it would be amazing to sculpt in zbrush with gloves instead of a pen, squeezing and stretching the mesh... long way off though :(

[SoreThumb @ Jan 3rd 2008 3:54PM]

What if I don't want to upgrade to "arm fatigue" and want to stay with my "Carpal tunnel" model?

[Fank @ Jan 3rd 2008 4:00PM]

Looks bad. Also, imagine holding your hands up like that for long periods of time. Yeah, that will work well. And why would anyone want to show pictures that way? It's not easier, it's just flashier. And that will wear out pretty fast.

[Rich @ Jan 3rd 2008 4:47PM]

This would actually be much cooler if he were actually touching the surface of the tv to manipulate the objects.

[Chrismix @ Jan 3rd 2008 6:26PM]

...we call that the DS and adding multiple touch points wouldnt b hard.

this is new...ish.

and johnny chung didnt exactly do the same thing.. though it was similar. lots of wat there excited about is the software.

[Ryan @ Jan 3rd 2008 5:14PM]

Johnny Lee Chung did this already...

http://www.cs.cmu.edu/~johnny/projects/wii/

[Kawaiipikachu @ Jan 3rd 2008 5:17PM]

When people learn it's Wii Remote not Wiimote .
It's just as bad as calling the Olympics the Pics or Picasso as Asso .

[superaktieboy @ Jan 3rd 2008 7:06PM]

i remember this sci-fi movie with precogs predicting the future, those precogs predicted that some police man would do some crime.. that police man had exactly (or nearly) that same gloves.. can't remember the name of the movie.. any one remember? anyways.. this would be also awesome if the gloves were full gloves (covering all of your fingers/thumbs) and then you basically could throw your mouse and keyboard of the window :D then just a gamepad for games 8)

greeetzz

[superaktieboy @ Jan 3rd 2008 7:08PM]

aah found it.. the movie was called Minority Report..

Guys after seeing this.. I THINK WE'RE ONTO SOMETHING!! now.. just the precogs needed, anyone?

[Jeff Lewis @ Jan 3rd 2008 7:58PM]

Easy technology. We did something like this in the 90s for head tracking systems.

The IR emitters blink say 10 times a second.

You put a webcam up (most webcams are really sensitive to IR).

You take a picture of the person 20 times a second.

Subtract the two pix. The only thing left is the IR dots.

You now have one dot for every fingertip.

Even better, use TWO webcams separated by a foot or so and now you can compute the DISTANCE to the IR dot.

Any new patent on this should check with a couple of companies including Madentec Ltd. (their Tracker device) on it.

[jibba jabba @ Jan 3rd 2008 9:06PM]

@jc
you brilliant bastard you beat me to it!! That line was the first thing that popped in my head when I read this article. cali-fornia! CALIFORNIA!! now I'll take my lunch box full of pics and try and find a dino park to cry in.