Microsoft SideSight project promises to take multi-touch beyond the screen

Microsoft is certainly no stranger to multi-touch interfaces, but it looks to really be pushing things with its new so-called SideSight research project, which promises to do away with that pesky need to actually touch the screen. To do that, Microsoft proposes to employ a whole range of proximity sensors around a device, which would be able to detect gestures up to ten centimeters away, with a quick motion towards the device representing a click, for instance, or a twisting motion letting you rotate an image. While Microsoft has actually built a prototype of sorts using an HTC Touch (seen above), it still has quite a ways to go, as the sensors on the prototype actually just connected to a PC via USB and then relayed back to the phone using Bluetooth. Eventually, however, Microsoft says that the system could allow for multi-touch to be used on very small devices, which could possibly even use printed sensors that would cover the entire casing.

[Image courtesy CNET News, thanks Peter]

[saccrant @ Oct 21st 2008 1:12PM]

Now who said microsoft's not innovative

[Kiwi616 @ Oct 21st 2008 1:21PM]

Bring on the "Minority Report" version already!

[maveric101 @ Oct 21st 2008 1:30PM]

MS does some amazing things in their R&D labs. I'd like to see more of their stuff actually make it to mass market.

[Hickeroar @ Oct 21st 2008 1:44PM]

Well, a lot of what they've done in recent years has been downright clone-ish in nature.

However, MS is really going to leave apple in the dust with this multitouch stuff. MS will be launching products supporting full screen multitouch, "multitouch" in the air, this new multitouch "around" thing, and so-on, all while apple is somehow adding multitouch to their mightymouse.

[Kamokazi @ Oct 21st 2008 1:48PM]

Exactly They have a ton of great products....it's just that their biggest one (Desktop Windows) is the biggest disappointment. Office 2007 is great, their mice/keyboards are great, Xbox is great, and bash it all you want, the Zune is a pretty decent DAP. I work in IT, and their enterprise stuff is my favorite, because it makes my job a hell of a lot easier. Very robust, integrates well together, and because it is primarily GUI driven, it is easier to pick up on new products/features than it is with a command-line interface. (Although that is available in many of their products...consoles can be much more efficient if you know how to use them...I manage most aspects of Exchange with the console). The other advantage is I can train interns/temps very quickly on how to add a user, and anything I don't train them on they can figure out more quickly, because it's easier to hunt and peck around a GUI for an option than trying to find the right command.

[majortom1981 @ Oct 21st 2008 1:49PM]

Hickeror how has there stuff been cloneish? Microsoft helped start this whole touch craze. Remember bill gates said about tablet pc's and stuff? Everybody thought it was crazy but now everybody including apple is getting on the bandwagon.

[Billy Gun @ Oct 21st 2008 1:54PM]

The best inovations from MS in my opinion is Media Center in 2002, The Tablet PC in 2002 and the WMP11 that apple copied on Itunes.

The table surface looks great.

[Hickeroar @ Oct 21st 2008 2:00PM]

major:

Tablet PCs are still holding on for dear life. Their sales are abysmal across the board. Apple is also not working on a tablet PC. There is also a grand total of less then 5 multitouch tablet PCs out there and all of them have 3rd party software for the multitouch support. Read: Windows doesn't yet support it.

Aside from that, look at the rest of their products. I wasn't even referring to their touch stuff. For the most part they're still adding features that have been in competitors products for a long time now. There's a video somewhere on youtube that has the Vista "features" being touted at one of the MS events. The screen is overlaid by OSX and they're showing the same features in OSX which have been there for years. MS was basically just copying things they knew people had wanted for a long time.

As a whole, MS no longer "innovates." SideSight is one of the nice exceptions, but probably won't ever make it to market. Multitouch was being researched and used by apple (and plenty of others) long before MS created the surface. Other than the surface, MS produces a grand total of ZERO multitouch products.

[Eric @ Oct 21st 2008 2:04PM]

I hate to say anything that might get me accused of being a fanboy, but Apple copied Windows Media Player 11 and made iTunes?

iTunes 7 had just been released when WMP11 was released, and if anything, iTunes had gotten uglier while WMP had gotten better-looking.

As far as funtionality, both play music... and iTunes plays some video formats.

[DirtyVegas @ Oct 21st 2008 1:15PM]

Ya think this will translate to that fancy computer Stark had in the Iron Man movie?

[Felipe @ Oct 21st 2008 1:29PM]

i hope they do

[c-dub @ Oct 21st 2008 3:01PM]

+1, thats shit was NIIIICE..especially the part where he had the blueprint of the suit and spun it around, and when he was able to see his hot rod engine "blown up".

[Charles @ Oct 21st 2008 1:15PM]

Every time Microsoft show these multi-touch displays they demonstrate them by rotating and resizing photographs. It seems nobody can think of a practical use for multi-touch so just show off these novelty features. I think I’ll be sticking with the mouse and keyboard for some time to come.

[Hickeroar @ Oct 21st 2008 1:48PM]

Someone needs to come out with a sheet-glass see-through multitouch keyboard. No moving parts on the keys, nothing to break. Sounds like a win to me. I think multitouch uses go far beyond UI quirks, and it has a lot of applications that have yet to be dreamed up, or at least yet to be dreamed up by the people who matter.

[Kamokazi @ Oct 21st 2008 1:55PM]

Normally I enjoy poking fun at it, but you really need to check out an iPhone/iPod touch. That's a start of the possibilities...gestures for common commands.

Next, think about using two hands to move windows and whatnot around, all while doing those gestures with them (think Minority Report). It looks goofy, but it can become much more efficient than a keyboard and mouse.

Finally...think of bigger screens that multiple people can use (and therefore need multi-touch). Surface hit on a lot of this...card games, retail kiosks, etc. But think of whiteboards in conference rooms that are a giant screen. You could have a teleconference window open you move around, be drawing, etc. on both ends that everyone can see.

And we have just started to think these things up. New applications will appear as the technology becomes more commonplace.

[northerngeek @ Oct 22nd 2008 6:29AM]

I would have to disagree, as a medical student I can see many applications, I was talking with a guy from MS at a recent expo whilst playing on a surface and we had a great brainstorm.

This tech for instance has the potential to allow your future doctor (scarily that may be me!) to easily manipulate X-rays to get some real detail seen on the small screens our hospital uses, theoretically allowing me to see problems developing sooner rather than later- and helping you!

Seriously MS software in healthcare rocks.


With resistant organisms doing the rounds in hospital this could literally be a life saver for the simple reason that I wouldn't have to touch the screen in this case- or if I do have to touch it then at least I can wipe it clean with alcohol wipes in ten seconds- rather than dealing with the dirty mouse and keyboard!!! If only I could get people to wash their hands in the toilet and we could save millions!

[Kali4 @ Oct 21st 2008 1:18PM]

Sweet!!!! Now I can make crank calls from someone else's pocket!!!

[Derk Kooi @ Oct 21st 2008 1:23PM]

I have a better idea, why don't they just make people not have to hard reset their HTC Touch every couple of months?

[John @ Oct 21st 2008 1:23PM]

sidesight? What happened to sideshow?

[Kali4 @ Oct 21st 2008 1:28PM]

It's a dual lens camera with a lazy eye?

[Ed T @ Oct 21st 2008 1:27PM]

Wow, will this be in Vista, er, Windows 7? It's way better than a file system that works or less restrictive DRM. Maybe it's part of Ballmer's cloud strategy, or is it just a brain fart from Jerry Seinfeld? Those R&D guys in Redmond sure know how to spend shareholder money...

[maveric101 @ Oct 21st 2008 1:31PM]

did you read the post? it's about an input device, not a file system...

[Phoenix @ Oct 21st 2008 1:38PM]

...he's sarcastically saying that it's better for Microsoft to be doing this than fixing Windows

[maveric101 @ Oct 21st 2008 2:48PM]

Windows Vista HAS been fixed.

[htd @ Oct 21st 2008 1:30PM]

htc touch? I am pretty happy with what it is now. I dont want no multi-touch on it. how can multi-touch be a big thing? I'd rather have some real functionality from M$.

[Sisyphus @ Oct 21st 2008 1:44PM]

DUDE.

Stop remotely bringing up porn on my phone from across the conference table!

[Lowest Ranked @ Oct 21st 2008 2:49PM]

Bring up more!

[mycintosh @ Oct 21st 2008 1:51PM]

I dunno ... does it really make sense, to put your mobile down on a surface to use it?
I prefer the tilt sensor or my one-hand pinch on the iPhone.

[Kali4 @ Oct 21st 2008 2:08PM]

Maybe not now, but who knows what apps will be developed that would make great use of this functionality. I could see this as a great compliment to mobile power point. Start your presentation (while wirelessly transmitting to the projector) and wave your hand next to the phone to change slides. Makes even more sense if the phone also has an integrated pico-projector. Games with a pico-projector and sidesight? But just because you don't see an immediate use for it doesn't mean it doesn't have a place. Remeber, there were some that didn't think pong made sense either.

[derX @ Oct 21st 2008 2:19PM]

When does it say it had to be on a flat surface? Or that it's only going to be in cell phones, for that matter? I could see the being implemented in digital picture frames or other electronics that are stationary. It's interesting, I'm excited to see where this is going.

Also, I like the accelerometer, too, but I wish there was a way to disable it on some occasions. When I'm listening to music while laying in bed, I naturally hold it so it's kinda on its side to it automatically switches to coverflow. I don't like coverflow; it's pretty and all, but it's also pretty useless, to me.

[Jeff @ Oct 21st 2008 1:57PM]

I kind of like the idea...many times I've wanted to be able to continue scrolling the screen off the actual screen of the iPhone. Having an expanded area of funtionality is always welcome. Sometimes we want to interact with our devices without covering the content with our fingers.

[THJ @ Oct 21st 2008 2:06PM]

So when I give it the double middle finger salute, will it properly install my shared printer?

[elliot.sawyer @ Oct 21st 2008 2:14PM]

If you can cause your phone to do something unintentional by merely approaching it, let alone touching it, how is that usable? Am I the only one that feels Microsoft has forgotten these devices are supposed to be phones first, and computers later?

I lost faith in Windows Mobile after I last tried to call 911 to report an accident, and received a "Command not supported" error. I'll make sure to restart my phone next time I have an emergency.

[DirtyVegas @ Oct 21st 2008 2:31PM]

Well, assuming there'll be a practical implementation of this, there'll be a lock mechanism at minimum. Just as how a touchscreen device in your pocket won't start randomly dialing your ex, this concept will have some safeguard.

[Billy Gun @ Oct 21st 2008 2:21PM]

@Eric Itunes 7 copyed part of the way The albuns apiers on WMP11, and on Itunes 8 it copyed more (separating Artist, genre, album view); but Itunes 8 inovated on the Genre view and I liked it. I guess that there are many other things they copy one another that we don't see. The fact is that MS also inovates some times.

[CD @ Oct 21st 2008 2:24PM]

Just when you though texting in cars couldn't get any more dangerous! "look ma no hands"

[Kali4 @ Oct 21st 2008 2:50PM]

HAH! Text by sign language!!

[Dylan @ Oct 21st 2008 2:37PM]

This is the Future !!

But being Microsoft, probably 30 years in the future, version 3.0 at least ;)

[encosion @ Oct 21st 2008 2:41PM]

EPIC FAIL! The photo suggests that you'd have to put your phone down to rotate an image... Not for me thanks! Whilst it's a great idea to think outside the box (or the phone in this case), just because something COULD be done, doesn't mean it's a good idea, nor will have much practical application... From what they've shown here, it's a dead end... BUT I'll give them the benefit of the doubt in that they just don't want to let the cat out of the bag with regards to their true intentions on this development...

[Lowest Ranked @ Oct 21st 2008 2:50PM]

You know, you could just rotate the phone if pictures are your main concern.

[rattyuk @ Oct 21st 2008 2:52PM]

So... You have to put the thing down to use it? hmmm. How... ergonomic.

[coffee @ Oct 21st 2008 3:07PM]

Holy crap, people, it's just a proof of concept, not a product introduction. Try to think beyond the picture a little bit. It's not simply about rotating a picture on a cell phone.

[Mike Silva @ Oct 21st 2008 4:26PM]

Humm, if you're using both hands around the phone, U'll need a third hand to hold the phone, or a portable table...
And if you are talking to tech phone support U'll say so many BEEP's that will attract flies around the phone lowering the volume until touchdown contact.

[wootman @ Oct 21st 2008 4:39PM]

Or one hand on the phone and another multi touchin it up.

[Iain @ Oct 21st 2008 4:35PM]

Now that's thinking out of the box;)

[Decoy @ Oct 21st 2008 5:08PM]

"Hey, check this out! They've made a system that tracks your fingers movement around cellphones so you have a greater multitouch interface! You won't even get the screen smudged!"

"Wicked! Apple?"

"No, Microsoft"

"Oh. Well, it sucks then. Let's make fun of how you have to put it down to use it."

[Alex @ Oct 21st 2008 7:14PM]

Unbelievable how HTC is always the chosen manufacturer for these revolutionary devices. Wish I had the money to invest in it. =(

[Bill @ Oct 21st 2008 7:15PM]

Maybe it's just me, but I'd feel a little self-conscious setting my phone on a table at Starbucks and waving my hands around it without actually touching it.

Kind of like the very first guy who walked around a supermarket talking on a bluetooth earpiece.

[AR @ Oct 21st 2008 8:37PM]

Sweet car in the picture on the phone, what is it?

[Brock Orr @ Oct 21st 2008 11:17PM]

why a cellphone? as others have pointed out, a cell phone would have to be set down onto a surface. I think this idea or others similar could be potentially incredibly useful and innovative if they were integrated into laptops and mounted displays.