DIY head-tracker takes Wiimote hacking to dizzying new heights
By Donald Melanson 
posted Dec 21st 2007 5:44PM
posted Dec 21st 2007 5:44PM
DIY virtuoso Johnny Chung Lee has already more than proven himself with his Wiimote whiteboard and finger-tracking hacks (to say nothing of the famous $14 steadycam), but he now looks to have vaulted himself into a whole new league with his latest project, which uses a Wiimote for a full-on VR head-tracking system. As with his other recent hacks, the Wiimote and a sensor bar substitute swap their usual positions, with a pair of IR-equipped safety glasses stylishly getting the job done in this case. Toss in some custom-made software (available at the link below) and you've got a setup that's sure to make anyone's jaw drop. Whatever you do, be sure to hit up the video after the break (and watch 'till the end), as the image above certainly doesn't do it justice.
[Via Hack a Day]
[Via Hack a Day]
First of all, its not HAKCING wtf?
hes not hacking into anything.
what hes doing is THINKING WAAAY OUTSIDE THE DAMN BOX !!!!!!
which is friggin sweet, the games that out kids play are going to blow some minds!
second, hes speaking of minority report because in the movie Tom just uses his hand to control a huge computer screen that displays information on future Crimes.
he just moves his hands to grab new folders. pictures, videos. all he is doing is waving his hands around, and that motion performs so many tasks.
Virtually replacing a Mouse all together
Looks like Sony's stealing Johnny's brilliant idea. http://www.next-gen.biz/index.php?option=com_content&task=view&id=9342&Itemid=2. Hope Johnny gets a cut from Sony's headtracking software sales. ;)
@skulldriveshaft
I got my 20$ dongle from Walmart to work. Then I found a dongle that you get with My Mii manager, witch I think, works better then the one I bought from the Walmart store. My Mii Manager, I bought from Walmart too. Only available online store.
I could not get the desktopvr to work on vista, or on my wifes xp, and I know nothing about c# or C++ programing.
Just finished setting everything up and I am completely floored. I am amazed! Great Work Mr. Lee!
DIY virtuoso Johnny Chung Lee and this is dope... this is first time i've seen this post.
And now, get sued by Mr. Crews
What you talking about? Minority Report again?
We're just contributing to the shortage of Wii Remotes :]
And of course Blu Soleil is getting rich because their adapter is the one most referenced for Wii Remote connectivity :]
this is EPIC
THIS IS SPARTA!
THE CAKE IS A LIE!!!
THE CAKE IS A LIE!!!
THE CAKE I... oh, wrong thread!
Gotta love these hacks, still waiting for the Mac Wiimote Whiteboard App ! .. Please Johnny ! *LOL*
This is ingenious.
Great job Johnny. Once again... you never cease to amaze.
Johnny Lee is my hero. I'm leading my school computers club through trying out the whiteboard idea, and this one gives me whole new ideas. He's really unleashed the potential of the Wii mote
that is amazing. what a resourceful guy.
Jesus, this is incredible
crazee...was waiting for those targets to come off the screen 3D...
targets may be one thing, but think of the potential.
If you ever watched the .hack series it was about an MMORPG that the main interface was a controller and a VR goggles. Just think if this was paired with some wearable video glasses people use for ipods with the IR LEDs on the side. You could control where you were looking completely naturally and really immerse yourself in the game.
Johnny is the freaking man.
That's awsome. I made this paper dragon that follows you around like that when you close one eye...but this, this takes actual skill! (not to mention it might actually be useful.)
Wow.
Trippy.
wow...just....wow
I am so excited for the next ten years of my life...
Yeah. I'll say. For the millionth time in my life: The future is going to be effing awesome.
This could revolutionize gaming. Now if only MS could pay attention. Also could be a reason to buy a wii if i could even find one.
Don't bother buying one!
All you need to buy is the Wiimote and the glasses with IR LEDs.
And of course, a computer with bluetooth.
For all those not aware, he is running this software on his PC, not on the Wii.
He simply is using the television as an external monitor.
Nintendo seriously needs to hire that guy.
absolutely...although he's released it now and it's easy to copy...still, he has the credit
...Patents anyone?
Wow this might get me into gaming...
As I'm writing this I haven't even seen the whole video yet
That guy's a champ
This is incredible.
That's so cool, would love to play an FPS like that.
From your avatar, I judge that that game would be the BEST GAME EVER.
Ohhhh the possibilities! I want to see this in the next installment of call of duty!
immense. that's an incredible little piece of software... if only i could get my wii to stay connected to my macbook in windows. anyone able to tell me why it always disconnects once it goes out of discoverable (flashing blue lights) mode?
I am completely in awe. That was amazing, this guy is the best.
I hope someone pays him a lot of money for this stuff, this is amazing!
Someone should hire him while they can!!!
now to make those sunglasses into 3d goggles
If the display tracks your head then you already have superior 3D, without the headache..
I really hope Nintendo jumps on this idea. MUCH better than their plastic Wiimote holder known as the "Zapper." They'd just need to pack in some IR glasses, Wiimote stand, and an extra Wiimote with a game and it'd be awesome as hell.
I wanna have your children!!!
Call of duty 4 with head tracking! OMG!
YES YES YES. Thats exactly what i was thinging. Bring that technology into a game like that and i will spend years playing it over and over again.
although, I probably would suck even more. lol.
You're not exaggerating when you say it's jaw-dropping! The whiteboard, finger-tracking, and especially the head-tracking are beyond way-cool!
Title should read "DIY head-tracker takes Wiimote hacking to dizzying new depths." Makes more sense, doesn't it?
NaturalPoint already makes an optical (Infrared) head tracker called TrackIR. It's in some pretty big sims and games like FSX, Armed Assault, DiRT, and a bunch of others. Even been around before the Wii ;)
Yeah, but the problem with that computer peripheral is that most games that support it assume that you are sitting in a computer chair and they don't account much at all for z-axis movement. From what I have read about TrackIR, the games that use it tack it on and just use it to change in-car game cameras and first/third person cameras similar to what a mouse/right-analog-stick would typically do in a traditional game. This actually affects the whole image rendering displayed on the screen. Awesome. Just can't imagine how tricky it would be to implement two wii remote/IR sensor bar combos to use this in a FPS shooter. Just the triangulation on both sides would be mind boggling.
but can it play DOOM....in 3D?
(yea im gonna get flamed arent i....)
thats friggin amazing. ea or someone should give him a job.
EPIC! INSANE! CRAZY! You, Mr.Lee, are AMAZING!
He's actually already got a pretty good job...
http://www.cs.cmu.edu/~johnny/
So, the real lesson, kids, is...
Don't be a fool,
stay in school!
- Mike
I have to agree he is the man though.. this stuff is awesome!
doh!...
actually he is a grad student there...
So... yes... somebody should hire him and give him lots o' cash as he is obviously a brilliant guy.
- Mike
Is this how Google, Microsoft, and Apple all started? One man, one idea, the intelligence and willpower to carry it through?
Ive never been able to say this before, but thats the sweetest thing I've ever seen in my life.
What's interesting about this is the use of consumer (i.e. cheap) hardware for head tracking. Unfortunately the actual application isn't anything new, as fishtank VR has been around for a long time.
Johnny is my new Hero, too. It's all so simple, really, but so ingenious. Sometimes it just takes the right little combo of technology to make something "go off" in someone's head; I'd assume that's what happened with Johnny.
Don't forget, you don't have to just make plastic glasses for this; it'd be easy to manufacture little IR-LED clip-ons for your existing glasses, IF you wear them. A tiny battery and small squeeze-clip would be all you need, since alignment isn't a huge issue, and the LED doesn't need much juice to last a long time. Just one more little way to sell more things like this to a bigger audience.
And yeah, while this tech has certainly been available for sometime, it again just comes down to what the public is not only ready for, but also when a company like Nintendo finally does something completely different than all the other guys. It's safe to say that the Wii could do more for the game industry than anyone could possibly have thought. So far the possibilities with this sound better than anything I've played in an arcade as far as interactivity goes, and I've played some good ones.
Now, if we can come up with a way to hold a real/virtual gun in front of you and be able to move forward without using our head, we'd be set. Perhaps you could make a plastic shell for the WiiMote and Nunchuck, and have a button that you press in conjunction with the "WiiTrack" when you want to move forward, back, left or right; when your finger isn't on the button, you're able to look aound a room like in the video above; when you press the button, moving your head a little forward propels your character forward, pull it back and it's backwards, left, right, etc. One simple button combined with the headgear would work, I think. One could also use the joystick on the Nunchuck to move around, of course. It'd be nice to have the plastic shell kind of look like some of the guns we're used to seeing in FPShooters, and we'd be able to do cool things like changing our weapons by pushing a button on the WiiMote or Nunchuck. Man, that would be FUN, epsecially with a big monitor. If done properly, I could see playing something like that all the time, and I'm not that big of a gamer.
Get on it, developers! It's time, we're ready for it! Or should I say... Wii're ready for it? No, no...I shouldn't. ;)
Again, GREAT job, Johnny! Keep up the good work!
With VR glasses you can actually catch the dragon.
best comment in a long time.
This...rocks...so....hard....I....I have to go lay down !
This guy has a very bright future ahead of him :)
Johnny Lee for President... Johnny Lee and Johnny 5, a man and a robot, both bringing about life advancements.
Think about a High def version of this with remote tooling via robots, an engineer could be in one room, while a robot is in a radioactive bunker mimicking his inputs to re-seal a turned over radioactive isotope.
Darpa should hire this guy.
Not sure I've ever heared of Safety Goggles being descrbied as "Sporty"...
The goggles!! They do something!!
This is so fetch. I want a wii!
No you do not. You want two wiimotes, a bluetooth dongle, and a sensor bar powered by rechargeable batteries.
That is freaking awesome. And ninty can sell more accessories lol. I would love a game based on this. Kinda reminds me of Time Crisis(a little) where now you can actually duck and move to shoot things. Sweetness.
Give this guy his own department and a big budget at Nintendo, let him do what he wants, and he will make cool things happen. It took Ninty almost a freakin YEAR just to make a plastic "zapper" to hold the wiimote. :p fail!
I am in total shock. This could be the next big thing in gaming. I would love to have this with a computer game, console game, whatever.
I used to think that stereoscopic was the way to go for video games, but this would give the same effect with a LOT less hardware. Either would require the game to be programed for it though.
(P.S. - If you aren't aware of stereoscopics, look them up... it's been around in static, analog form for a century, it's about time they bring it to the digital age.)
WOW... and me thinking that such technology would not come until somewhere in 2012 or w/e :O im just amazed :O
Awesome. I'd love to see this implemented in 1 player campaigns.
I loved Time Crisis, and thought the pedal was a great idea. THis really takes it to the Nth level, as they used to say.
And here's how you make this more interesting for observers to watch; during a tournament wherein each player has a WiiTrack, you could also have a little camera so that the player's perspective can be shown on a larger screen, just like how they do it at some tourneys now, and how Johnny showed in the clip.
This has so much potential its crazy.
The main limitation to this right now is the TV - fact is that moving around doesn't help much if your still looking at it through a small frame. But imagine if you could complete immersion - putting the TV in a smaller space, like say one of those fancy new ones they were making that could fit on glasses. You could technically just connect the output display from the Wii or PC to them; and then connect the inputs to a keyboard + mouse - maybe find some way to hack or 'map' the Wii's controls into those so that the wiimote recognizes it and your PC. If you do this such that it recognizes your head tilt as basically axis x and y movement, you *could* technically look around any FPS game by, well, simply looking around and aiming like you would a regular weapon.
Although it would apply only for Wii games since you'd need a second controller to shoot / aim, the real question is how would this handle your regular FPS games - most of which "aim" is synonymous with "direction your facing", essentially making aiming a problem. The only games where the direction of your head differs from the direction of movement are generally racing games and simulators - for which this would be an utter boon.
If you could mod Crysis to have that kind of functionality though...and then had such a rig set up with goggles and all...oh Good lord it would be fabulous.
this is very good but when i try run the software i get this...
Microsoft Windows
Wiimote FishtankVR has stopped working
Windows is checking for a solution to the problem...
i have connected the wiimote and everything, i had the whiteboard software running fine
any ideas??
How about you goto the site and ask there? But here's a starting point: it says to use 'bluesoleil' as BT stack if it's not working, and that requires a compatible bluetooth chipset it seems, so it could be your BT dongle isn't compatible.
Johnny, have you considered using 2 or more Wiimotes for a wider field of view?
Someone I know tried this out and mentioned that the illusion is lost when you go off the edge of the screen; it seems like a combination of 2 or 3 Wiimotes set at an angle would fix this.
Why hasn't Nintendo snatch him up yet!?!?!? Come on! Anyone who can hire him will sell so much stuff!
It's not meant to run on a wii, it's using the wii-mote and it runs on your normal PC with windows, or linux if someone writes the code.
So what point would nintendo have? to suppress it?
And now there's going to be a reason to move your head sideways when watching... huh... late-night movies. ;)
Super Cool
Now if someone can rewrite that for posix/x/GL/SDL and compile it for linux, FreeBSD, OSX, OpenSolaris, and such.. Wait, anybody tried that source code with gcc using winelibs?
wow... btw here are some links (neither of which I am affilated with) which sell everything needed in this project (minus the wiimote), the first to links are from the same site because I thought it would save on shipping. The whole project can be done for about $10, I really hope to see this in games soon, it will be great. ^_^
Lighted White LED Safety Goggles - $8.95 http://www.goldmine-elec-products.com/prodinfo.asp?number=G15640
List of Infared LED's - most are $1.00
http://www.goldmine-elec-products.com/products.asp?dept=1220
And another site selling the IR LED's for $0.20 each:
http://www.allspectrum.com/store/product_info.php?products_id=972
Let's simplify the rig.
Get rid of the sensor bar and LED glasses altogether.
Instead put a IR source at the camera and put sticky reflector dots on your glasses. Or if you don't wear glasses go to a Dollar Store and get a pair of glasses which have "invisible rims". There is a piece of nylon holding the lens in place. Cut the nylon, the lens falls away and you are left with a pair of empty frames. Stick the IR reflective dots on the corners where the hinges are.
Note: I'll bet if the IR source is decently strong the reflective dots can be extremely small. Once again proving size isn't everything. :-)
Terry Thomas
President
PC Tech
Atlanta, Georgia
Patent Pending
(c) 2008 Terry Thomas
All Rights Reserved
Do Not Remove Tag Under Penalty of Law
Now replace the targets with bubbles, slowly floating in the air.
Combine this with his finger tracking hack.
When the finger touches a bubble, the bubble bursts, all in 3D ...
That would be a hack to cry for!
Even better: a replica of the game Asteroids.
Floating bubbles replace asteroids, no space ship, the player would have to burst the bubbles before they leave the screen, or before a limited time.
That would win all the baby boomers to the Wii (if Nintendo picks up the idea).
I am sure there are a lot of other possibilities combining these two hacks.
Andre.
What about a boxing game?
You're a clever man, Mr Lee.