DIY head-tracker takes Wiimote hacking to dizzying new heights
By Donald Melanson 
posted December 21st 2007 5:44PM
posted December 21st 2007 5:44PM
DIY virtuoso Johnny Chung Lee has already more than proven himself with his Wiimote whiteboard and finger-tracking hacks (to say nothing of the famous $14 steadycam), but he now looks to have vaulted himself into a whole new league with his latest project, which uses a Wiimote for a full-on VR head-tracking system. As with his other recent hacks, the Wiimote and a sensor bar substitute swap their usual positions, with a pair of IR-equipped safety glasses stylishly getting the job done in this case. Toss in some custom-made software (available at the link below) and you've got a setup that's sure to make anyone's jaw drop. Whatever you do, be sure to hit up the video after the break (and watch 'till the end), as the image above certainly doesn't do it justice.
[Via Hack a Day]
[Via Hack a Day]
He's actually already got a pretty good job...
http://www.cs.cmu.edu/~johnny/
So, the real lesson, kids, is...
Don't be a fool,
stay in school!
- Mike
I have to agree he is the man though.. this stuff is awesome!
doh!...
actually he is a grad student there...
So... yes... somebody should hire him and give him lots o' cash as he is obviously a brilliant guy.
- Mike
Is this how Google, Microsoft, and Apple all started? One man, one idea, the intelligence and willpower to carry it through?
Ive never been able to say this before, but thats the sweetest thing I've ever seen in my life.
What's interesting about this is the use of consumer (i.e. cheap) hardware for head tracking. Unfortunately the actual application isn't anything new, as fishtank VR has been around for a long time.
Johnny is my new Hero, too. It's all so simple, really, but so ingenious. Sometimes it just takes the right little combo of technology to make something "go off" in someone's head; I'd assume that's what happened with Johnny.
Don't forget, you don't have to just make plastic glasses for this; it'd be easy to manufacture little IR-LED clip-ons for your existing glasses, IF you wear them. A tiny battery and small squeeze-clip would be all you need, since alignment isn't a huge issue, and the LED doesn't need much juice to last a long time. Just one more little way to sell more things like this to a bigger audience.
And yeah, while this tech has certainly been available for sometime, it again just comes down to what the public is not only ready for, but also when a company like Nintendo finally does something completely different than all the other guys. It's safe to say that the Wii could do more for the game industry than anyone could possibly have thought. So far the possibilities with this sound better than anything I've played in an arcade as far as interactivity goes, and I've played some good ones.
Now, if we can come up with a way to hold a real/virtual gun in front of you and be able to move forward without using our head, we'd be set. Perhaps you could make a plastic shell for the WiiMote and Nunchuck, and have a button that you press in conjunction with the "WiiTrack" when you want to move forward, back, left or right; when your finger isn't on the button, you're able to look aound a room like in the video above; when you press the button, moving your head a little forward propels your character forward, pull it back and it's backwards, left, right, etc. One simple button combined with the headgear would work, I think. One could also use the joystick on the Nunchuck to move around, of course. It'd be nice to have the plastic shell kind of look like some of the guns we're used to seeing in FPShooters, and we'd be able to do cool things like changing our weapons by pushing a button on the WiiMote or Nunchuck. Man, that would be FUN, epsecially with a big monitor. If done properly, I could see playing something like that all the time, and I'm not that big of a gamer.
Get on it, developers! It's time, we're ready for it! Or should I say... Wii're ready for it? No, no...I shouldn't. ;)
Again, GREAT job, Johnny! Keep up the good work!
With VR glasses you can actually catch the dragon.
best comment in a long time.
This...rocks...so....hard....I....I have to go lay down !
This guy has a very bright future ahead of him :)
Johnny Lee for President... Johnny Lee and Johnny 5, a man and a robot, both bringing about life advancements.
Think about a High def version of this with remote tooling via robots, an engineer could be in one room, while a robot is in a radioactive bunker mimicking his inputs to re-seal a turned over radioactive isotope.
Darpa should hire this guy.
Not sure I've ever heared of Safety Goggles being descrbied as "Sporty"...
The goggles!! They do something!!
This is so fetch. I want a wii!
No you do not. You want two wiimotes, a bluetooth dongle, and a sensor bar powered by rechargeable batteries.
That is freaking awesome. And ninty can sell more accessories lol. I would love a game based on this. Kinda reminds me of Time Crisis(a little) where now you can actually duck and move to shoot things. Sweetness.
Give this guy his own department and a big budget at Nintendo, let him do what he wants, and he will make cool things happen. It took Ninty almost a freakin YEAR just to make a plastic "zapper" to hold the wiimote. :p fail!
I am in total shock. This could be the next big thing in gaming. I would love to have this with a computer game, console game, whatever.
I used to think that stereoscopic was the way to go for video games, but this would give the same effect with a LOT less hardware. Either would require the game to be programed for it though.
(P.S. - If you aren't aware of stereoscopics, look them up... it's been around in static, analog form for a century, it's about time they bring it to the digital age.)
WOW... and me thinking that such technology would not come until somewhere in 2012 or w/e :O im just amazed :O
Awesome. I'd love to see this implemented in 1 player campaigns.
I loved Time Crisis, and thought the pedal was a great idea. THis really takes it to the Nth level, as they used to say.
And here's how you make this more interesting for observers to watch; during a tournament wherein each player has a WiiTrack, you could also have a little camera so that the player's perspective can be shown on a larger screen, just like how they do it at some tourneys now, and how Johnny showed in the clip.
This has so much potential its crazy.
The main limitation to this right now is the TV - fact is that moving around doesn't help much if your still looking at it through a small frame. But imagine if you could complete immersion - putting the TV in a smaller space, like say one of those fancy new ones they were making that could fit on glasses. You could technically just connect the output display from the Wii or PC to them; and then connect the inputs to a keyboard + mouse - maybe find some way to hack or 'map' the Wii's controls into those so that the wiimote recognizes it and your PC. If you do this such that it recognizes your head tilt as basically axis x and y movement, you *could* technically look around any FPS game by, well, simply looking around and aiming like you would a regular weapon.
Although it would apply only for Wii games since you'd need a second controller to shoot / aim, the real question is how would this handle your regular FPS games - most of which "aim" is synonymous with "direction your facing", essentially making aiming a problem. The only games where the direction of your head differs from the direction of movement are generally racing games and simulators - for which this would be an utter boon.
If you could mod Crysis to have that kind of functionality though...and then had such a rig set up with goggles and all...oh Good lord it would be fabulous.
this is very good but when i try run the software i get this...
Microsoft Windows
Wiimote FishtankVR has stopped working
Windows is checking for a solution to the problem...
i have connected the wiimote and everything, i had the whiteboard software running fine
any ideas??
How about you goto the site and ask there? But here's a starting point: it says to use 'bluesoleil' as BT stack if it's not working, and that requires a compatible bluetooth chipset it seems, so it could be your BT dongle isn't compatible.
Johnny, have you considered using 2 or more Wiimotes for a wider field of view?
Someone I know tried this out and mentioned that the illusion is lost when you go off the edge of the screen; it seems like a combination of 2 or 3 Wiimotes set at an angle would fix this.
Why hasn't Nintendo snatch him up yet!?!?!? Come on! Anyone who can hire him will sell so much stuff!
It's not meant to run on a wii, it's using the wii-mote and it runs on your normal PC with windows, or linux if someone writes the code.
So what point would nintendo have? to suppress it?
And now there's going to be a reason to move your head sideways when watching... huh... late-night movies. ;)
Super Cool
Now if someone can rewrite that for posix/x/GL/SDL and compile it for linux, FreeBSD, OSX, OpenSolaris, and such.. Wait, anybody tried that source code with gcc using winelibs?
wow... btw here are some links (neither of which I am affilated with) which sell everything needed in this project (minus the wiimote), the first to links are from the same site because I thought it would save on shipping. The whole project can be done for about $10, I really hope to see this in games soon, it will be great. ^_^
Lighted White LED Safety Goggles - $8.95 http://www.goldmine-elec-products.com/prodinfo.asp?number=G15640
List of Infared LED's - most are $1.00
http://www.goldmine-elec-products.com/products.asp?dept=1220
And another site selling the IR LED's for $0.20 each:
http://www.allspectrum.com/store/product_info.php?products_id=972
Let's simplify the rig.
Get rid of the sensor bar and LED glasses altogether.
Instead put a IR source at the camera and put sticky reflector dots on your glasses. Or if you don't wear glasses go to a Dollar Store and get a pair of glasses which have "invisible rims". There is a piece of nylon holding the lens in place. Cut the nylon, the lens falls away and you are left with a pair of empty frames. Stick the IR reflective dots on the corners where the hinges are.
Note: I'll bet if the IR source is decently strong the reflective dots can be extremely small. Once again proving size isn't everything. :-)
Terry Thomas
President
PC Tech
Atlanta, Georgia
Patent Pending
(c) 2008 Terry Thomas
All Rights Reserved
Do Not Remove Tag Under Penalty of Law
Now replace the targets with bubbles, slowly floating in the air.
Combine this with his finger tracking hack.
When the finger touches a bubble, the bubble bursts, all in 3D ...
That would be a hack to cry for!
Even better: a replica of the game Asteroids.
Floating bubbles replace asteroids, no space ship, the player would have to burst the bubbles before they leave the screen, or before a limited time.
That would win all the baby boomers to the Wii (if Nintendo picks up the idea).
I am sure there are a lot of other possibilities combining these two hacks.
Andre.
What about a boxing game?
You're a clever man, Mr Lee.