DIY head-tracker takes Wiimote hacking to dizzying new heights

DIY virtuoso Johnny Chung Lee has already more than proven himself with his Wiimote whiteboard and finger-tracking hacks (to say nothing of the famous $14 steadycam), but he now looks to have vaulted himself into a whole new league with his latest project, which uses a Wiimote for a full-on VR head-tracking system. As with his other recent hacks, the Wiimote and a sensor bar substitute swap their usual positions, with a pair of IR-equipped safety glasses stylishly getting the job done in this case. Toss in some custom-made software (available at the link below) and you've got a setup that's sure to make anyone's jaw drop. Whatever you do, be sure to hit up the video after the break (and watch 'till the end), as the image above certainly doesn't do it justice.

[Via Hack a Day]

[xxx8sxe @ Dec 21st 2007 5:55PM]

And now, get sued by Mr. Crews

[skulldriveshaft @ Dec 22nd 2007 2:30AM]

What you talking about? Minority Report again?

We're just contributing to the shortage of Wii Remotes :]

And of course Blu Soleil is getting rich because their adapter is the one most referenced for Wii Remote connectivity :]

[mcheddadi @ Dec 21st 2007 5:57PM]

this is EPIC

[Jevanzz @ Dec 21st 2007 6:25PM]

THIS IS SPARTA!

[Deoki @ Dec 23rd 2007 7:42PM]

THE CAKE IS A LIE!!!
THE CAKE IS A LIE!!!
THE CAKE I... oh, wrong thread!

[ahelk09 @ Dec 21st 2007 5:59PM]

This is ingenious.

[Rich White @ Dec 21st 2007 5:59PM]

Gotta love these hacks, still waiting for the Mac Wiimote Whiteboard App ! .. Please Johnny ! *LOL*

[VasR @ Dec 21st 2007 6:00PM]

Great job Johnny. Once again... you never cease to amaze.

[Randavance @ Dec 21st 2007 6:01PM]

Johnny Lee is my hero. I'm leading my school computers club through trying out the whiteboard idea, and this one gives me whole new ideas. He's really unleashed the potential of the Wii mote

[hitjim @ Dec 21st 2007 6:03PM]

that is amazing. what a resourceful guy.

[paul34 @ Dec 21st 2007 6:03PM]

Jesus, this is incredible

[whatishalo? @ Dec 21st 2007 6:04PM]

crazee...was waiting for those targets to come off the screen 3D...

[Randavance @ Dec 22nd 2007 1:53AM]

targets may be one thing, but think of the potential.

If you ever watched the .hack series it was about an MMORPG that the main interface was a controller and a VR goggles. Just think if this was paired with some wearable video glasses people use for ipods with the IR LEDs on the side. You could control where you were looking completely naturally and really immerse yourself in the game.

[tiuk @ Dec 21st 2007 6:06PM]

Johnny is the freaking man.

[ryan @ Dec 21st 2007 6:07PM]

That's awsome. I made this paper dragon that follows you around like that when you close one eye...but this, this takes actual skill! (not to mention it might actually be useful.)

[William @ Dec 21st 2007 6:08PM]

Wow.

[mushrooshi @ Dec 21st 2007 6:08PM]

Trippy.

[CarpeD1em500 @ Dec 21st 2007 6:09PM]

wow...just....wow

I am so excited for the next ten years of my life...

[absurdio @ Dec 22nd 2007 1:31AM]

Yeah. I'll say. For the millionth time in my life: The future is going to be effing awesome.

[Neeko @ Dec 21st 2007 6:10PM]

This could revolutionize gaming. Now if only MS could pay attention. Also could be a reason to buy a wii if i could even find one.

[MMalecky @ Dec 21st 2007 11:31PM]

Don't bother buying one!

All you need to buy is the Wiimote and the glasses with IR LEDs.
And of course, a computer with bluetooth.


For all those not aware, he is running this software on his PC, not on the Wii.
He simply is using the television as an external monitor.

[Ricardo @ Dec 21st 2007 6:10PM]

Nintendo seriously needs to hire that guy.

[pquistgard @ Dec 21st 2007 6:17PM]

absolutely...although he's released it now and it's easy to copy...still, he has the credit
...Patents anyone?

[pquistgard @ Dec 21st 2007 6:14PM]

Wow this might get me into gaming...

As I'm writing this I haven't even seen the whole video yet

That guy's a champ

[Andy @ Dec 21st 2007 6:15PM]

This is incredible.

[Reader @ Dec 21st 2007 6:17PM]

That's so cool, would love to play an FPS like that.

[ethana2 @ Jan 25th 2008 6:19PM]

From your avatar, I judge that that game would be the BEST GAME EVER.

[tony @ Dec 21st 2007 6:20PM]

immense. that's an incredible little piece of software... if only i could get my wii to stay connected to my macbook in windows. anyone able to tell me why it always disconnects once it goes out of discoverable (flashing blue lights) mode?

[rumblerjon @ Dec 21st 2007 6:34PM]

now to make those sunglasses into 3d goggles

[Wwhat @ Dec 22nd 2007 4:11PM]

If the display tracks your head then you already have superior 3D, without the headache..

[Seroth @ Dec 21st 2007 6:40PM]

I really hope Nintendo jumps on this idea. MUCH better than their plastic Wiimote holder known as the "Zapper." They'd just need to pack in some IR glasses, Wiimote stand, and an extra Wiimote with a game and it'd be awesome as hell.

[hugoliva @ Dec 21st 2007 6:41PM]

I wanna have your children!!!

[Bombaclaat @ Dec 21st 2007 6:42PM]

Call of duty 4 with head tracking! OMG!

[Neeko @ Dec 22nd 2007 9:12AM]

YES YES YES. Thats exactly what i was thinging. Bring that technology into a game like that and i will spend years playing it over and over again.

[Bombaclaat @ Dec 22nd 2007 4:49PM]

although, I probably would suck even more. lol.

[Ian @ Dec 21st 2007 6:42PM]

You're not exaggerating when you say it's jaw-dropping! The whiteboard, finger-tracking, and especially the head-tracking are beyond way-cool!

[Yoursolace @ Dec 21st 2007 6:49PM]

I am completely in awe. That was amazing, this guy is the best.

I hope someone pays him a lot of money for this stuff, this is amazing!
Someone should hire him while they can!!!

[Michael @ Dec 21st 2007 6:52PM]

Title should read "DIY head-tracker takes Wiimote hacking to dizzying new depths." Makes more sense, doesn't it?

[Case @ Dec 21st 2007 7:00PM]

NaturalPoint already makes an optical (Infrared) head tracker called TrackIR. It's in some pretty big sims and games like FSX, Armed Assault, DiRT, and a bunch of others. Even been around before the Wii ;)

[Phi Nguyen @ Dec 21st 2007 8:55PM]

Yeah, but the problem with that computer peripheral is that most games that support it assume that you are sitting in a computer chair and they don't account much at all for z-axis movement. From what I have read about TrackIR, the games that use it tack it on and just use it to change in-car game cameras and first/third person cameras similar to what a mouse/right-analog-stick would typically do in a traditional game. This actually affects the whole image rendering displayed on the screen. Awesome. Just can't imagine how tricky it would be to implement two wii remote/IR sensor bar combos to use this in a FPS shooter. Just the triangulation on both sides would be mind boggling.

[mushyho @ Dec 21st 2007 7:12PM]

but can it play DOOM....in 3D?


(yea im gonna get flamed arent i....)

[james sylvester @ Dec 21st 2007 7:15PM]

thats friggin amazing. ea or someone should give him a job.

[Hailfire753 @ Dec 21st 2007 7:34PM]

EPIC! INSANE! CRAZY! You, Mr.Lee, are AMAZING!

[mike @ Dec 21st 2007 7:58PM]

He's actually already got a pretty good job...

http://www.cs.cmu.edu/~johnny/

So, the real lesson, kids, is...
Don't be a fool,
stay in school!

- Mike
I have to agree he is the man though.. this stuff is awesome!

[mike @ Dec 21st 2007 8:02PM]

doh!...

actually he is a grad student there...
So... yes... somebody should hire him and give him lots o' cash as he is obviously a brilliant guy.

- Mike

[seansli @ Dec 21st 2007 8:26PM]

Is this how Google, Microsoft, and Apple all started? One man, one idea, the intelligence and willpower to carry it through?

[Aymon F @ Dec 21st 2007 8:34PM]

Ive never been able to say this before, but thats the sweetest thing I've ever seen in my life.

[GS @ Dec 21st 2007 8:51PM]

What's interesting about this is the use of consumer (i.e. cheap) hardware for head tracking. Unfortunately the actual application isn't anything new, as fishtank VR has been around for a long time.

[D.E. Lloyd @ Dec 21st 2007 10:08PM]

Johnny is my new Hero, too. It's all so simple, really, but so ingenious. Sometimes it just takes the right little combo of technology to make something "go off" in someone's head; I'd assume that's what happened with Johnny.

Don't forget, you don't have to just make plastic glasses for this; it'd be easy to manufacture little IR-LED clip-ons for your existing glasses, IF you wear them. A tiny battery and small squeeze-clip would be all you need, since alignment isn't a huge issue, and the LED doesn't need much juice to last a long time. Just one more little way to sell more things like this to a bigger audience.

And yeah, while this tech has certainly been available for sometime, it again just comes down to what the public is not only ready for, but also when a company like Nintendo finally does something completely different than all the other guys. It's safe to say that the Wii could do more for the game industry than anyone could possibly have thought. So far the possibilities with this sound better than anything I've played in an arcade as far as interactivity goes, and I've played some good ones.

Now, if we can come up with a way to hold a real/virtual gun in front of you and be able to move forward without using our head, we'd be set. Perhaps you could make a plastic shell for the WiiMote and Nunchuck, and have a button that you press in conjunction with the "WiiTrack" when you want to move forward, back, left or right; when your finger isn't on the button, you're able to look aound a room like in the video above; when you press the button, moving your head a little forward propels your character forward, pull it back and it's backwards, left, right, etc. One simple button combined with the headgear would work, I think. One could also use the joystick on the Nunchuck to move around, of course. It'd be nice to have the plastic shell kind of look like some of the guns we're used to seeing in FPShooters, and we'd be able to do cool things like changing our weapons by pushing a button on the WiiMote or Nunchuck. Man, that would be FUN, epsecially with a big monitor. If done properly, I could see playing something like that all the time, and I'm not that big of a gamer.

Get on it, developers! It's time, we're ready for it! Or should I say... Wii're ready for it? No, no...I shouldn't. ;)

Again, GREAT job, Johnny! Keep up the good work!

[Ghen @ Dec 21st 2007 10:56PM]

With VR glasses you can actually catch the dragon.