Switched On: Token gestures
By Ross Rubin 
posted June 29th 2010 7:36PM
posted June 29th 2010 7:36PM
Each week Ross Rubin contributes Switched On, a column about consumer technology.
In early 2005, even after the launch of the Nintendo DS, Switched On critiqued pen computing, noting that it was too cumbersome and unnatural to become a mainstream input method. That column was validated by the launch of the iPhone, which banished the stylus to "blown it" status and popularized "finger-friendly" handset designs from all major smartphone OS developers.
Many have described the user interfaces of products such as the iPhone and Surface as ushering in the post-WIMP (windows-icon-mouse-pointer) era. Former Windows Magazine editor Mike Elgan has referred to the new paradigm as "MPG" (multitouch, physics, gestures) However, while these user interfaces feature streamlined designs and more direct manipulation, they still form a bridge with the graphical user interface. The main shift has been to more direct manipulation as the device processes more natural inputs.
The same can be said for Kinect. For a tidy sum and some untidiness, Kinect enables the kind of motion-sensing gameplay that has become the Wii's hallmark without having to strap the controller to various limbs (as with EA Active for Wii). In fact it eliminates the need to hold a controller entirely, just like the iPhone and iPad free users of mice and styluses. Beyond Soviet Russia, the input device uses you.
Since Kinect employs cameras to determine the position of various limbs, it can offer video calling and "reaction shots" as demoed in Microsoft's E3 press conference. Furthermore, as Microsoft showed off the Xbox's new entertainment dashboard, Kinect has applications far beyond games. The product's official full name -- Kinect for Xbox 360" -- leaves open the possibility that it will be offered for other platforms such as Windows. Indeed, creating different combinations of cameras and microphones that offer Kinect functionality could become an important new differentiator for PC companies.
In the iPhone and iPad, though, multitouch found a home in a new user interface designed for it. In contrast, just as the broad usefulness of touch on Windows 7 has been largely relegated to custom user interface layers such as those in HP's TouchSmart PCs or Surface demo applications, the impact of Kinect's blend of seeing and hearing the user will fall short of its potential tacked on to today's desktop operating systems. This was clear from a recent demonstration by startup PointGrab, which uses the kind of inexpensive webcam already found on many netbooks to control the UI. One could easily see the technology's application for, say, Windows Media Center control, but its reliance upon mouse emulation makes it less efficient for manipulating the core of the desktop operating system's functionality.
The manipulation and video calling capabilities of Kinect, then, may be more promising for a new breed of TVs and set-tops such as the Logitech Revue. Part of the rationale for Logitech to enter the thorny TV add-on market has been to create a Trojan horse for videoconferencing it has long offered via its webcams and more recently via its acquisition of corporate videoconferencing company LifeSize. As a platform, Revue is open enough to accommodate new input methods, but it lacks the core integration with TV media that has plagued all other TV add-on boxes. The key to broadening the gesture and voice recognition of Kinect, then, is to make it integral to a device that offers a new way of controlling old and new media.
Ross Rubin is executive director of industry analysis for consumer technology at market research and analysis firm The NPD Group. Views expressed in Switched On are his own.
Many have described the user interfaces of products such as the iPhone and Surface as ushering in the post-WIMP (windows-icon-mouse-pointer) era. Former Windows Magazine editor Mike Elgan has referred to the new paradigm as "MPG" (multitouch, physics, gestures) However, while these user interfaces feature streamlined designs and more direct manipulation, they still form a bridge with the graphical user interface. The main shift has been to more direct manipulation as the device processes more natural inputs.
The same can be said for Kinect. For a tidy sum and some untidiness, Kinect enables the kind of motion-sensing gameplay that has become the Wii's hallmark without having to strap the controller to various limbs (as with EA Active for Wii). In fact it eliminates the need to hold a controller entirely, just like the iPhone and iPad free users of mice and styluses. Beyond Soviet Russia, the input device uses you.
Since Kinect employs cameras to determine the position of various limbs, it can offer video calling and "reaction shots" as demoed in Microsoft's E3 press conference. Furthermore, as Microsoft showed off the Xbox's new entertainment dashboard, Kinect has applications far beyond games. The product's official full name -- Kinect for Xbox 360" -- leaves open the possibility that it will be offered for other platforms such as Windows. Indeed, creating different combinations of cameras and microphones that offer Kinect functionality could become an important new differentiator for PC companies.
Different combinations of cameras and mics with Kinect functionality could become an important new differentiator for PC companies. |
In the iPhone and iPad, though, multitouch found a home in a new user interface designed for it. In contrast, just as the broad usefulness of touch on Windows 7 has been largely relegated to custom user interface layers such as those in HP's TouchSmart PCs or Surface demo applications, the impact of Kinect's blend of seeing and hearing the user will fall short of its potential tacked on to today's desktop operating systems. This was clear from a recent demonstration by startup PointGrab, which uses the kind of inexpensive webcam already found on many netbooks to control the UI. One could easily see the technology's application for, say, Windows Media Center control, but its reliance upon mouse emulation makes it less efficient for manipulating the core of the desktop operating system's functionality.
The manipulation and video calling capabilities of Kinect, then, may be more promising for a new breed of TVs and set-tops such as the Logitech Revue. Part of the rationale for Logitech to enter the thorny TV add-on market has been to create a Trojan horse for videoconferencing it has long offered via its webcams and more recently via its acquisition of corporate videoconferencing company LifeSize. As a platform, Revue is open enough to accommodate new input methods, but it lacks the core integration with TV media that has plagued all other TV add-on boxes. The key to broadening the gesture and voice recognition of Kinect, then, is to make it integral to a device that offers a new way of controlling old and new media.
Ross Rubin is executive director of industry analysis for consumer technology at market research and analysis firm The NPD Group. Views expressed in Switched On are his own.
Good for Adult Entertainment.
@techlord Wouldn't the user accidentally scroll up a down rapidly?
@techlord Just tried Kinect at my Microsoft Store OMG it is so awesome it's so easy to use, its super precise way more then I had expected nothing like the wii or anything I have ever played before. I'm uploading the quick vids I took to youtube. My screen name is soapinmouth
@techlord iFace--PALM
@Soapinmouth Loved the vid!
Commented on it. :)
@foxh8er
Here's the link sorry I posted that before I even uploaded it can't edit
http://www.youtube.com/watch?v=3aEfyDSWBJk
@techlord
Here's the link to the vid got to play.kinnect at the mall
http://www.youtube.com/watch?v=3aEfyDSWBJk
@techlord
i wouldn't want to look at reaction shots of myself right after...
@Worm in the Apple You just need to hold it differently.
I will always be for feedback in my controls. Mouse & Keyboard ftw.
@BeaverusIV
I enjoy my physical stuff too, but I welcome the future with cautious, friendly terms.
@Archon I won't count out gestures, merely use someone else's device first :p Took me long enough to decide to buy a touch phone (Still haven't got it, but I'm looking at importing a Galaxy S).
@Archon I see a combination being best.
For example, on a racing game its still great to have actual race pedals and a force feedback steering wheel with shifter or paddles, but kinect could work as advanced head tracking so you can look around the cockpit and see through the turns just by moving your head.
Similar setup to what I'm running here but w/ more primitive headtracking: http://www.youtube.com/watch?v=d-rNaCiakvo (and yes, my driving sucks worse than a thai bargirl talking on the cellphone, but I had just gotten the game so was n00b)
@Ducman69 thats what ive been saying since kinnect was introduced. i think micro$oft will announce that closer to launch, and sony has completely missed the point that they were trying to make.
@Ducman69 Yeah I think Kinect can have many usefull use. I love the head tracking idea, it would be great in infiltration type of games to see enemies from a cover position.
Someone will be getting used alright... anyone who buys a Kinect. Clearly it's an intermediate step on the road to anti-life.
I think the first paragraph should be finished off with the phrase "In the US" Most other countries (especially Asian) love pen computing and even use pen like devices with the Iphone. So people of the USA just hate it not the world.
@psychoace Like the sausages!
@psychoace Pen inputs are also more accurate than fingers. Trying signing on a phone screen without a stylus. Even the Apple stores have a iPod stylus for when you need to sign after purchasing something.
@psychoace
I don't think that Americans hate styluses, it's just that unless your in certain fields, you've probably never had the chance to use one before. I personally love my Wacom dual digitizer equipped T4410 tablet, and I think it's much better then just having a mouse or touch capabilities alone, especially on a laptop. Hopefully they become more main stream in the near future.
First one to say first!
@techee44
first one to be downranked!:)
@The Truth Hurts
That was my intention.
The HP layer was demonstratively inferior, and I'm not sure why Engadget insists on dogging W7 multitouch functionality, when its really the first mainstream operating system designed from the outset with multitouch input in mind and works well:
http://www.youtube.com/watch?v=WBtEhQqS1dw
http://www.youtube.com/watch?v=ctlvb6UBkJA
http://www.youtube.com/watch?v=a5WXxLecW4Q
And it does so with icons, just as there are big icons on iOS.
Kinect and the like can simply supplement input options, especially for Media Center and gaming, just as the media center peripherals market has done with head-tracking and air-mouse remote controls.
In short, the whole "WIMP" acronym and "post-era" nonsense is really uncalled for IMHO. =)
@Ducman69 It's apple-speak, and Ross Rubin would rather be dead than give accolades to MS - just check his previous articles.
@Ducman69
Congrats on the videos, really liked them..
Could care less about the games, just want to be able to say "Xbox Pause"
This alone is worth $150 to me. I'm lazy, I know.
@PBB and apparently rich, too.
Wonder what happens if you're having a conversation with friends or just watching the news and something sounds like a Kinect command. Will it be confused?
And how can it tell who is giving the commands? will your spastic nephew playing with his tonka trucks flailing like a 'tard be flipping channels and maxing the volume? will you have to tell your gf's friends to shutup so you can register a voice command?
@juanvaldez
No, not rich just money smart. If I can't set aside $150 between now and November, something's wrong.
@PBB
Dude, windows media center can do that way ahead the Xbox (not to mention there's an option to turn on voice command in Windows Vista/7). Brought to you buy the same guy who do both the Windows and Xbox.
@cdf74dc9
I know that. My 360 spends most of its time as my primary MCE, but the MCE doesn't have voice commands. The windows interface might, but that's not the focus here.
@PBB You can install any of the templates though for various games and Media Center to use W7s speech recognition.
For WMC for example you can use Media Center Communicator. The downside is you have to memorize the audio commands, but they are pretty intuitive.
"Record channel 9" or "music library" followed by "play (insert song title or artist)".
Personally, I have a Gyration air-mouse remote control, and prefer to just click the music button and navigate normally, but perhaps I'm just not used to talking to non-sentient tools yet (except my johnson).
I'll be really interested in this if they can prove that Kinect can recognize gestures while I sit down.
@Ducman69
dunno, but from the demos, it sounds like it looks for the "Xbox" command with an inflection/emphasis on xbox. As far as the noise level in the room goes, 9/10 it will be me anyway so its not an issue. But that's an issue with ANY voice command system. I would HOPE that since it can "see" the user that it will keep track of the voice patterns since it will have to learn them anyway.
@PBB
I hope they set it up so you can use a headset to send commands to nata- err kinect. But maybe not since both are very different microphones. One can dream..
@JamesHks
Thats smart, as that way surrounding noise would be way less of an issue!
Kinect is Skynet!!!
Funny, am I the only one that noticed the "ghost-like" face in the top right window blind??? lol
@Sylon
Thats not a ghost. It kinda looks like NPH.
"In early 2005, even after the launch of the Nintendo DS, Switched On critiqued pen computing, noting that it was too cumbersome and unnatural to become a mainstream input method."
128 million DSes say it's not too cumbersome.
@Bananarama
Not to mention that the stylus rocks for writing, equations, and drawing/photoshop work.
Sausage fingers are too blunt and stubby, and don't work well or at all on some devices when its cold out and you're wearing gloves.
How would you fast foward? You ran away from the screen?
There will always be an input and output devices.
controlling a TV or Set-up box without a remote will be difficult, as people could be sitting on the side coach or chair and not in-front of the sensing device like Kinect and many people could waving or shouting or talking so how the the sensing device recognize two different moments or as speaking?
Also the same for keyboards and mice, how you type in the Air without a physical keyboard?
even for games it will be difficult to play most games without a controller, could you keep moving your body as a controller for 10-15 minutes? how about ab hour?
or even voice recognition, could you keep shouting to the TV to instruct it rather than using a remote? or in the office speak loud so your voice be transformed to characters or Words, it doesn't look practical.
so we will need the input and output devices for long and these sensing devices for specific limited applications or games.
Compact. Intuitive. Accurate.
If an input method meets all three criteria, it will be successful as a general purpose interface. If not, then it will be successful only in very specific niches, at best.
The machine must immediately respond exactly as the user wants it respond (accuracy), including in the acknowledgement of input capture; the user should be able to easily figure out how to explore/experiment with the interface when he find himself in a new environment (intuitiveness); and the user must be able to control the machine with minimal effort and without disturbing others (compactness).
The last two requirements hinder the adoption of voice as a general purpose interface. The first and the last will hinder Kinect-style 'full body motion towards a distant screen' interface use outside of games.
Am I the only one thinking 'upgradable socketed brain chip' as the obvious answer to all this?
A lot of the science has already been done on research to control robotic prosthetic devices.
Making it socketed and upgradable is just obvious.
You could have different control standards available - with the number and type of implement to be controlled defined by that.
And then some licensing as well - much as you would have to drive a bus or truck.
So any joe could get chipped and learn to change his tv channels or check his e-mail by thought.
But if you applied yourself and upped your chip you could learn to control - and get licensed for - large machinery.
Maybe the required focus would even be meditative.
So you could lay in your hammock and guide the lawn mower around the yard (with wireless video feedback directly to your ocular cavity) - and end feeling refreshed, calmed, and centered.
So what does Kinect have to do with Apple?
I wonder how long it will take for Kinect to be hacked to work with PC, much like all the Wiimote projects.. Or will MS simply release a version compatible with Windows...
Really, i would just like Kinect to make my 360 a crazy sweet media center, but the 360 will never beat a PC in terms of 'HTPC' status.. So if this tech was ported (and /supported/) properly.. HTPC game changer!