Apple patents touchscreen control interface

We sure wish Apple would be a little more up front with their patents- if they just posted the darn things on their site (with explanations of how they'll be incorporated into future products, preferably) it would save us all a lot of time trawling around the USPTO site. Until that glorious day, however, we'll have to continue using our collective detective skills to feed our Apple obsession, as one of our readers did in finding a newly-released patent filing for a touchscreen control-interface. The pictures that Apple submitted look like your basic on-screen keyboard and the application is in the typical indecipherable legalese, so we're not sure what's so innovative here, but perhaps this is tech that will be incorporated into the iTablet and video iPod- should these mythical creatures actually exist.

[Thanks, Armin]

[7of7 @ Feb 17th 2006 9:11AM]

Yeah, because the iTools need more things to waste their money on.

[Pip @ Feb 17th 2006 9:26AM]

It would be nice to see what they're trying to patent here. Onscreen touch keyboards have been around since touch screen technology has been around. Early to mid 80s? Everything from ATMs to Hotel Reservation systems have used them since then.

[bowlhed @ Feb 17th 2006 9:28AM]

looks pretty standard stuff.i cant see a itablet pickin up much business.better off with an ibook

[odedcarmi @ Feb 17th 2006 9:41AM]

#1 the idea is that the touch screen can now detect multiple touches (2 I think). So basicly you could press shift + letter for caps, or alt + C to copy. This is would be a signifigant improvemnt over pressing to switch for all caps to all lowercase. I'm sure it has many other functions that I have not yet mentioned. Some one please correct me if I'm wrong.

[foxone @ Feb 17th 2006 9:41AM]

i sure hope apple gets a tablet interface...i find that i have a much more enjoyable computer experience when i use my wacom tablet as a mouse, then my fingers don't kill after using it for 50+ hours a week...

[bazald @ Feb 17th 2006 9:49AM]

Brilliant... nobody thought of mouse gestures or touch screen-based buttons before Apple...

[chris @ Feb 17th 2006 9:50AM]

looks more like this has to do with the application being able to resize and adjust itself when the virtual keyboard is triggered so that areas related to the input field do not become obscured by the keyboard. This is one of the thing i hate most about WinCE/PocketPC, however my Nokia 770, and specificaly GPE-PIM suite, already does this. There is definatly prior art here, without a doubt. Part of the patent also seems to deal with how the virtual keyboard is triggered, but I'm not clear on how that's not just logical progression.

[Karl @ Feb 17th 2006 10:00AM]

#5: PPC apps *do* adjust themselves when the SIP (on screen keyboard) is displayed; at least they do if they carry an MSFT "designed for" certification. There is nothing in the OS that requires developers to resize their app to accomodate the SIP, but it can certainly be done, many apps do it, and all apps that hope for official MSFT blessing must do it.

[LJW @ Feb 17th 2006 10:02AM]

think beyond the tablet... could apply to e-paper. or imagine a small personal device that projects a full screen onto a flat surface, and is able to interpret virtual key touches...

if apple has been awarded this patent, could be an interesting win.

[paul Barton @ Feb 17th 2006 10:10AM]

everyone is so down on tablets because they've been done so poorly in the past. Apple and 'multipoint' will re-ignite this market... The patent describes placing your hands down in a 'typing' position to initiate the virtual keyboard! - cool. For major typing sessions, one could just add a bluetooth keyboard. I reckon the touchscreen iPod, Tue March 28th - then, "see you in four days" for the big brother.

[Glancing Aft @ Feb 17th 2006 10:14AM]

I'm sure most people on here have seen it, but a friend sent me this multi-touch interaction video demo the other day, which shows off some of the tech that apple may use. Cool stuff, I especially like the google earth uses.
http://loop.worldofapple.com/archives/2006/02/08/multi-touch-interaction-video/

[mdeltito @ Feb 17th 2006 10:17AM]

Here's a question: This new video iPod that is rumored, the one with the onscreen touchwheel. How is that practical? Wouldnt there just be a ring of grease/dirt/scratches right in the viewing area? That seems to be unavoidable, no matter what coating or finish and no matter how careful the consumer is (and after Apple's response to the nano BS, i would never even think of touching the screen of a next gen iPod.) and I can't see how the could pull it off without having to use a stylus. Something i coulndt do while listening to music.

[Jon @ Feb 17th 2006 10:19AM]

Check this video out regarding this screen.

Amazing!

http://mblog.lib.umich.edu/~rdivecha/archives/2006/02/the_world_of_sm.html

[blablah @ Feb 17th 2006 10:23AM]

You fools that are slamming tablets? You'll be talking out the other side of your face when more quality implementaions emerge like, for example, that small, high-res Fujitsu unit w/the finger-sensitive touch screen. The technorati have been fawning over it ever since it was released, like, 8 months ago? I have been putting-off buying that specific unit in the hopes that Apple would release something. Maybe, finally, it'll happen.

[Kory @ Feb 17th 2006 10:23AM]

I think we're heading down the path toward a very powerful iPod. Remember, "iPod" does not have anything to do with music or video.

Remember one of their other patent applications had to do with making a scalable fully operating OS. Combine that with this touchscreen capability (whatever the heck it is) and you've got a device that can all of sudden do a LOT of stuff.

Then, combine that with wireless technology (being added by their chipmaker) and you've got wi-fi and bluetooth.

Holy crap! What a sweet device!!!

[YellowBox @ Feb 17th 2006 10:26AM]

Looks like this is a virtual keyboard geared towards newer multi-point touchscreen interfaces. It says it's based on this patent: http://appft1.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=HITOFF&u=%2Fnetahtml%2FPTO%2Fsearch-adv.html&r=1&p=1&f=G&l=50&d=PG01&S1=903%2C964.APN.&OS=APN/903,964&RS=APN/903,964
Working at an intellectual property law firm, it's pretty easy for me to see the points where this patent is unique:

"As yet another example, the keyboard initiation event may be triggered by the user putting her fingers on the touch screen (for example, a multipoint touch screen) in a 'typing' position. The detection of this user action may trigger a virtual keyboard ..."

"As an example of dynamic configuration, the user may change the position of the virtual input device display in the composite display by touching a portion of the virtual keyboard display and 'dragging' the virtual input device display to a desired portion of the composite display"

Patent applications are notoriously broad in its details, and this is my favorite:
"The scrolling may even be in more than two dimension (e.g., a virtual cube, or a virtual shape in more than three dimensions)"
5-dimensional scrolling, eh? Oooooooookay.

[Chris @ Feb 17th 2006 10:35AM]

BAD HEADLINE! Apple hasn't patented anything here. They have filed an application for a patent--quite different! The actual patent (if it comes) may look much different after it pops out of the USPTO.

[paul Barton @ Feb 17th 2006 10:37AM]

x, y, z and t (time) - that's more than 3...

[Adam @ Feb 17th 2006 10:45AM]

I think the multi-touch technology is compelling as a UI and the closest we'll get to that "Minority Report" user interface experience until we come a long way in display/projector technology.
I'd also like to see Apple just put out their first Tablet so I don't need even one Windows (Tablet PC) box in the house (which, sadly, I'm using right now.)
And I'd like to see Apple or anyone come out with a real-time UI. We were close to the adoption of real-time OS's in the 90's, but things went to glossy UIs and cheap everything. Hopefully we'll see real-time come back as user's choose computer solutions that are genuinely interactive instead of "click and we'll get back to you" interactive.

[Richard @ Feb 17th 2006 11:00AM]

Perfect, the Apple refrigerator (with voip! & cam) will be on the market within 6 months! (my guess)

[JL @ Feb 17th 2006 11:08AM]

I see this to be bigger than just for any "i" devices.
I watched the video that post #11 linked to and upon
watching the "virtual" dj part of it, was like wow!
new types of control for art/music interaction.

Would love to see the tech evolve into new interfaces for performance devices!

[john commenter @ Feb 17th 2006 11:17AM]

Apparently, working at an intellectual property firm, you don't have a frickin' clue. Working at the subway station, I can drive a train! Um, no.

The pending claims, the subject matter of *this
one*, are about taking into account the stuff already on screen, depending on what's there already, to combine that and a virtual keyboard coming up. See #6 chris.

That which you refer to, multipoint gestures, is in the parent case, not this one. That one has a pile of child cases like this one, which will farm ideas that relate to, but do not require, multipoint gestures).

Y'all should learn the joys of using the patent office PAIR (look it up) to see the fabulously boring inner workings of patents.

[Jeff Lewis @ Feb 17th 2006 11:21AM]

How is this novel? Touchscreens and onscreen keyboards have been around for a long time. Heck, I wrote one for MacOS that was one of the two leading on-screen keyboards in the disability market. (ScreenDoors by Madentec.) And we worked with Apple on it, so they knew about its existence.

[Shaun R. @ Feb 17th 2006 12:14PM]

I hope they are planning a universal, treo esq pda/smartphone/minilaptop/tablet running cheetah mobile and with quadband gsm/cdma/evdo/HSDPA/UMTS, etc...

I'm sick of palm or widows mobile devices, and symbian just isn't cutting it, yet..

just my 2cents...

[GeneMack @ Feb 17th 2006 12:39PM]

I'm a controls engineer and everything mentioned so far has been done in industrial HMIs (Human Machine Interfaces, or generally refered to as touch screens) for years now. Multiple touch, trigger on anything, etc.... Can't really see anything unique. Unique to consumer electronics maybe, but that is about it.

[ocelot67 @ Feb 17th 2006 1:39PM]

"We sure wish Apple would be a little more up front with their patents- if they just posted the darn things on their site (with explanations of how they'll be incorporated into future products, preferably) it would save us all a lot of time trawling around the USPTO site."

I, for one, would much rather be bewildered by stuff like this and be able to get the product the day it's announced. Cell phone manufacurers dangle the "new features" carrot in front of us for months before the actual product is available. Where's my Nokia E70?!?

[glacia00 @ Feb 17th 2006 2:27PM]

I have to agree with GeneMack. I work with test and diagnostic equipment every day that has some pretty slick touchscreen apps built in. While a few run a proprietary OS most run on a modified version of an off the shelf OS, either unix or windows. While it�s possible Apple has something unique it would have to be significantly better that what�s been shown here.

The USPTO has admitted that it is often unable to determine whether a patent application is unique. After all, what engineer would want to work for them? I wouldn�t have even considered it when I got out of school and after 20 years as an engineer I would bus tables before working there.

It's easy to file for a patent and almost as easy to get it issued but the entire patent process relies heavily on litigation making up for the weaknesses in the process.

[austinkennethlee @ Feb 17th 2006 2:35PM]

I sure wish Engadget would be a little more up front with their information. Explain the concept of the patent rather than just presenting it for people who don't understand the patent process. It leads to speculation and troll-like comments. You're doing a crappy (minimal) job with the reporting here.

(Valid) Patents are self-explanatory. It's the entire reason they're granted; an exclusive license in exchange for the knowledge. Dig?


The way I read the patent is that the virtual input display is part of an application, that customizes itself in both function and appearance based on what task it needs to accomplish.

That is: if you summon the almighty virtual input demon when you're running iTunes, you get a different display (say, an iPod wheel) than you would with Word (qwerty keyboard). It's an input device with a changing format and function.


Current virtual keyboards may be able to scale themselves, but can they change the button placement and functionality? Are touch-screen applications a static program or can they be tailored to fit other applications?

[glacia00 @ Feb 17th 2006 2:50PM]

I have to agree with GeneMack. I work with test and diagnostic equipment every day that has some pretty slick touchscreen apps built in. While a few run a proprietary OS most run on a modified version of an off the shelf OS, either unix or windows. While it�s possible Apple has something unique it would have to be significantly better that what�s been shown here.

The USPTO has admitted that it is often unable to determine whether a patent application is unique. After all, what engineer would want to work for them? I wouldn�t have even considered it when I got out of school and after 20 years as an engineer I would bus tables before working there.

It's easy to file for a patent and almost as easy to get it issued but the entire patent process relies heavily on litigation making up for the weaknesses in the process.

[Toshiba_Lover @ Feb 17th 2006 3:36PM]

I smell a marketing rat... The Register is running another similar "story" noting that there is a job posting "still" up at Apple's website for a handwriting recognition software engineer. Sounds like some covert mac promotion. Getting buzz going, blah. Well if it works for them, then the public is as gullible as they believe. Capitalism is the biggest mind f@#$ ever!

http://www.reghardware.co.uk/2006/02/17/apple_handwriting_recog_osx/

[Did I. Offend @ Feb 17th 2006 5:33PM]

There is no news here. This is just another patent application by Apple Computer (they have many issued patents dealing with this sort of thing). Any issued patent could be years off. Besides, DO THEY REALLY NEED ANY MORE PUBLICITY?! Apple, Apple, Apple, Apple, iPod, iPod, iPod, iPod, iPod, Apple, Apple, Apple, Apple, Apple, Apple, Apple, iPod, iPod, iPod, Apple, iPod, iPod, Apple, Apple, iPod, iPod, iPod, iPod, iPod, iPod, iPod, iPod, iPod, baaa-a-a-a-a-a-a-a-a (I digress), iPod, Apple, Apple, iPod, iPod, hurl!!! (excuse me), Apple, Apple, Apple, iPod, iPod, what's going on?! My mind is going numb...Help! Oh yeah... must have iPod, Apple, iPapple, pod pod pod, drool, ba pibble, od, od, apple, odpod, oooo, ahd, piddle ljk[';l mnmm...

[Ronald J Riley @ Feb 17th 2006 9:01PM]

We think Apple is a hypocrite. They want patents for themselves even as they work to take other inventors property without compensation and are trying to tilt the balance of America's patent system even further in favor of big companies.

Apple and the other companies pushing so called patent reform are well known as patent pirates in the inventor community. Patent pirates try to paint their victims as abusers and developed slurs like patent troll to describe those inventors who have the audacity to fight back. When the inventor fights back companies like Apple howl in rage.

PIAUSA is working to keep the patent system accessible for all. Large companies, including Apple are working to make use of the patent system a kings sport.

The real issue is that most breakthrough inventions start with an individual inventor, often in a basement or in a garage. If they are able to avoid the pirates they may well mature to become a new major company. In their quest to succeed they create jobs and tax base.

But far too often some patent pirate steals the upstart startup's property. And then they crush the startup.

Ronald J Riley, President
Professional Inventors Alliance
www.PIAUSA.org
RJR (at) PIAUSA.org
Change "at" to @
RJR Direct # [phone]

[andrew @ Feb 17th 2006 11:00PM]

apple takes a breath, more at 11.

[SCOTT BENJAMIN @ Feb 19th 2006 2:44PM]

I heard theres a new ipod coming out. EMAIL ME AT [e-mail] if you know about it. BUT I think this is a good article.

[Shaaheen @ Mar 30th 2006 1:29PM]

Main screen is scrolled up to give more space to the keyboard, under a system running OS X. Interesting, is Apple creating another Newton or just a typical tablet?

[d.nye @ May 5th 2006 4:07PM]

Hell my Newton MP2100 on screen keyboard is waaaaaaaaaaay better than that.

[digitsu @ Sep 28th 2006 5:48PM]

is these multitouch screens only limited to 2 similtaneous touches? a 3 tap would be keen to launch the dashboard.