Apple patent embeds thousands of cameras among LCD pixels

Oh Barry Fox, does a week ever go by when you don't find a great patent or two? Today the intrepid Mr. Fox manages to dig up an application by consumer-darling Apple for an LCD display embedded with thousands of microscopic image sensors that would allow users to video-conference while looking straight into the "camera." Data accumulated by the individual sensors would be stitched into actual images using special software, which will probably be bundled into future versions of iLife. Since the patent specifies almost as many sensors per screen as there are pixels, some of those sensors could have different focal lengths, with a defacto zoom lens created by switching between them. Apple goes on to suggest portable uses for the technology, such as employing the displays in cellphones and PDAs, so you can add another item to the list of features we'll be expecting from the iPhone and Newton 2.0 when they finally hit stores.

[Via New Scientist]

[harrisjamieh @ Apr 26th 2006 12:09PM]

Man this is OLD news. Get with it!

[druby @ Apr 26th 2006 12:11PM]

Boy, this is REALLY big-brother-y.

[SandMan @ Apr 26th 2006 12:11PM]

This sounds like the Television Screens in 1984. Can't wait till the government demands all tv's with this technology ... "Big Brother is Watching"

[nullsmack @ Apr 26th 2006 12:13PM]

Damn you #2, I came here to say that. :(

[Alexandre D @ Apr 26th 2006 12:14PM]

Big Brother is watching you. be afraid. you cannot hide from him, cannot know if and when he is watching you. your only escape, hide your PC screen

[ALX @ Apr 26th 2006 12:15PM]

First off Sandman they are called telescreens. I never thought that Apple would be the one to bring on the dystopian future. 2 + 2 = ?

[ron g @ Apr 26th 2006 12:18PM]

Extra! Extra! Read all about it!

ummm... This patent news is from 3 months ago. It has been reported on by Macsimum News at least that long ago anyway.

[Timothy @ Apr 26th 2006 12:26PM]

Yeah, not to be one of those "this is old" people, but here ya go

http://www.macrumors.com/pages/2006/01/20060113041940.shtml

from Jan. 13.

[Pacey @ Apr 26th 2006 12:28PM]

omg, get over yourselves. So what if it's old news. those of you who post "get with it, thise is x weeks old" are just as bad as the FIRST POST'ers out there.

I hadn't heard about it yet, so it's new to me. I dont have time to waste hitting every little mundane tech web site or to troll the web looking for newly filed patents. That's why I come here. They give me things that are up and coming and a mostly timely manner. Besides, the tech is months to years away from being in any commercial uses, so really, who are you impressing with your "I knew about it first" type posts. When you develop a cure for cancer or AIDS, then you can toot your own horn, but until then, just shut up.

Now, as to the technology, I love it. I see lots of problems that could be associated with it, but the benifits seem to outweight the negatives (for now)

[Sean @ Apr 26th 2006 12:30PM]

Could this patent be used on the next video ipod with the virtual scrollwheel? If the wheel has cameras in it, no need to physically touch the wheel, thus preventing fingerprints and smudges.

[Jeff Lewis @ Apr 26th 2006 12:34PM]

Actually, it's even older than that. I remember reading an article with an interview by Alan Kay back in the 90s where he was talking about the idea of having the transistor behind the LCD be both a control for the LCD and a sensor for light so that you could lay your LCD screen down on a book or newspaper and use it as a scanner.

Same basic concept.

[George Orwell @ Apr 26th 2006 12:34PM]

We can see you now..

[Nukes and Candy @ Apr 26th 2006 12:40PM]

#2, #3, #4, I was thinking the same thing while reading. Big Brother is definitely savoring this piece spy equipment, he is happy, he wants it in your room, living room, office, and (for other reasons) in your bathroom. Big Brother, you perb.

Alexander Wunderlich

[Gordy @ Apr 26th 2006 12:46PM]

What's up with that monitor? Is that real?

[softdrink @ Apr 26th 2006 12:48PM]

I don't think it's for use as an iSight-type application. I believe that it's going to be used to detect frustrated total internal reflection. I'm sure everyone has seen this video demonstration on multi-touch interaction, which uses that idea:
http://mrl.nyu.edu/~jhan/ftirtouch/

The idea is pretty simple: shine infrared LEDs through the sides of the glass touch surface, project an image at the frosted surface from the bottom, and watch for the infrared bright spots from underneath that are caused by touching the glass. Now, the setup for this project used a projector, camera, and a glass screen... not very portable. How do you make it portable? Put tiny infrared ccd sensors between the pixels on the LCD. The only problem is that looking for and tracking those "bright spots" takes a little bit of processing power. There is a product that works on a similar principle by Fingerworks: a flat pad that can detect and track multiple touch-points. It uses capacitance rather than infrared to do the tracking, and allows for gesture-style input, similar to what you see in that video -- only without an image. Their products do all the tracking locally in the hardware itself. Well, Fingerworks has "ceased operation as a business" (http://www.fingerworks.com) over the past year. Guess what company is rumored to have purchased Fingerworks. That's right, Apple.

Fingerworks gesture hardware + infrared sensors inside of lcd = portable multi-touch

[GadgetGav @ Apr 26th 2006 12:55PM]

softdrink, now THAT'S news..! Good points and well made. You should write for this site regularly..!

[TMart @ Apr 26th 2006 12:57PM]

Anyone else think this could have military applications as a kind of active camo? or even, say, to have your monitor perfectly blend into its surrounding by mapping images onto itself of the environment? this could be a lot more exciting than video conferencing

[jordan wheeler @ Apr 26th 2006 1:05PM]

In America, you watch televistion.
In SOVIET RUSSIA, television watches YOU!

[Chris @ Apr 26th 2006 1:06PM]

Adding to #15's post, here is a video of similiar technology being used with current video games. I hope this is the future of computer interfacing. Minority Report anyone?

http://www.youtube.com/watch?v=5O1bTNbbWk4&eurl=http%3A%2F%2Fwww%2Eggleague%2Ecom%2Fnews%2Ephp%3FNewsId%3D2738

[rex @ Apr 26th 2006 1:27PM]

So basically this is like how you see in those shows or movies where you see the person looking directly at the screen when they are talking to someone on the other side... this is awesome...now we implement these things on mobile phones and home phones and you've got stuff that you see in movies.

[TheWakeUpCall @ Apr 26th 2006 1:34PM]

I know others have said this, but for those who haven't read it, 1984 is a book which tells of a world run by a corrupt government who rule all their population like prisoners. The one way they do this is though televisions which can watch you. This is the one way they keep control of the whole land. And this is scary, because it is exactly like in the story. Just to clear that up for anyone who didnt know what the Big Brother posts were on about. (The most incredible thing about 1984 was that it was written in 1948)

[Brendan @ Apr 26th 2006 1:51PM]

"Big brother is watching you"

Sometimes I think people want technology to go backwards!
This is so you can actually look at a person while 'Videoconferencing', instead of the current tooking at a camera, with the persons face below.

You guys must have never watched a film set in the future before?? In practically every film set in the future this happens, and people can actually look at each other for a change.

This is less "big brother' than staring into a tiny camera lens.

[Mark @ Apr 26th 2006 2:01PM]

#8

we have an achivist on our hands...be warned.

http://redwing.hutman.net/~mreed/warriorshtm/archivist.htm

[Jargon @ Apr 26th 2006 2:20PM]

Video iPod with eye-activated menus, you heard it here first!

[Will @ Apr 26th 2006 2:37PM]

I agree with anyone that says "SHUT UP" to the people complaining about this, or anything else, as being old news. Some of us really appreciate Engadget and the work they do. I don't catch all those obscure news sites, and quite frankly don't plan on starting.

[anonymous coward @ Apr 26th 2006 2:48PM]

"On Screen!" says Captain Picard / Kirk / et. al.

[Steve @ Apr 26th 2006 2:56PM]

What about eye tracking. Get rid of the mouse. put right and left mouse buttons on the keyboard, and use your eyes to move the cursor.

[JIleto @ Apr 26th 2006 3:03PM]

Read an article somewhere that an LED can both be used to emit and detect light.

So with some clever circuitry, you can have an LED switch between both roles, emitting light and collecting light, several times a second. If you make the LED small enough, the size of a pixel, and cheap enough, so you can cover a surface with them, then you have something that does what Apple's patent is describing - only using LEDs as opposed to LCDs.

Aside from 1984, there is also another short story about the next stage in providing content for TVs. The story talks about TVs that also spy on their owners. So if you're bored watching canned shows, you can switch over to watch how your friend is making out with their date in the "privacy" of their own bedroom.

[TH @ Apr 26th 2006 3:04PM]

Why would this be any more "Big Brother" than any other camera? Surely you'd be controlling when it will film and where the footage will be saved/sent, just like with iSight or any webcam? BB would still need to be able to control the LCD-"camera" and be able to stream the pic from your computer/tv/iWhatev to itself.

Beware, your mouse is tracking your body movements! :-)

[Alex Moran @ Apr 26th 2006 3:15PM]

You all missed the point. The real potential of this technology is recording holographic images and video and 3-d modeling. With a thousand tiny cameras you would have a thousand angles to shoot from, you would have your own 3-d camera. With all due respect to Big Brother, if he wanted to watch you he would be watching you through your web cam right now, and not twiddling his thumbs waiting for this thing to come out.

[jcg @ Apr 26th 2006 4:20PM]

Am I the only one thats think this could make a VERY cool mirrow? It would even have the apple logo for the apple fanboys

[4x4van @ Apr 26th 2006 4:33PM]

Active Cammo! That's what I'm talking about! James Bond "Die Another Day"

[dextro @ Apr 26th 2006 4:54PM]

reading what softdrink wrote here in the comments I have to start wondering... Could this tech be used to create the "long awaited" Mac Tablet? :P

[lka1988 @ Apr 26th 2006 5:58PM]

Oh my hell, do you people not READ?

"...that would allow users to video-conference while looking straight into the "camera"."

[Andreas @ Apr 26th 2006 6:09PM]

I think this would first of all make a great "multi point" touchscreen, which goes in line with apple's other patents. As far as focal length.. I think you're all totally wrong. Since each "camera" will be pointing straight forward, and parallell to each other, together they will probably make a picture that has eternal focal length, and no perspective displacement. A dude in front of the screen, and a dude 10 meter away will probably look just as big and with a very weird focus with this technology given that they're aligned in parallell...

[VL-Tone @ Apr 26th 2006 6:33PM]

In the early days of CRT Television, cameras were working much like the TV tubes themselves.

TV cameras were using a tube with an electron beam that scanned a surface of photosensitive substance that produced an electric charge when illuminated, as opposed to phosphor in CRT TV's that produce light from an electric current.

By measuring the variations on the draw of current of the electron beam, the camera was able to produce a video signal.

So technically it may have been possible to built a CRT TV that simultaneously scans to illuminate and scans to detect light, "in between lines", working much like a camera and a display at the same time.

But the only problem with that is that to produce a suitable image, it either needs a large lens in front, or micro lens for each pixels.

[hawk @ Apr 26th 2006 7:22PM]

reasons why this is like the telescreens in 1984:

a) you won't necessarily know there are camera elements in the screen
b) it's a screen that displays and captures simultaneously, without an extra 'camera'.

That's all anyone needs to say. You can end the 'big brother' comments now.

[Jason @ Apr 26th 2006 7:23PM]

Haha, you guys beat me to it.
It's 1984 and Big Brother is watching you. Of course, they've been doing it long before these things. It's a scary world; we're building our own Hell.

[sirwnstn @ Apr 26th 2006 7:34PM]

When I watched Chris Rock get a retinal scan by looking into the laptop LCD screen in "Bad Company", I nearly fell off my chair laughing... but I guess Apple may get the last laugh here.

[Chris Robinson @ Apr 26th 2006 7:52PM]

Does a week ever go by when Apple doesn't file a great patent or two?

[Allodoxaphobic_Jay @ Apr 26th 2006 8:23PM]

Well, until we also have Thought Police squads knocking down our doors (a slightly more obvious sign of authoritarian oppression), there's always the option of putting a sheet over your TV set when you're not watching it if you're convinced that the government is using it to spy on you..

[HyperHacker @ Apr 26th 2006 9:30PM]

#29, how do you plan to get LEDs to display 65,536 different colours?

[boo @ Apr 26th 2006 9:33PM]

That technology is already built into your current LCD/CRT

You can test it at

http://www.monitorcamera.com/

Smile!

[Tim @ Apr 26th 2006 10:15PM]

No wonder I feel like my boss knows what I'm doing all the time! [like reading Engadget :)]

[OC @ Apr 26th 2006 10:54PM]

All the techie geeks and dorks making comments about 'old news' instead of intelligently discussing the 'technolgy' are idiots. Next time you wanna drive somewhere, don't.. don't use the wheel. It's 'old news'.

Embrace the technology.. don't act like a bunch of 0-day or nothing dorks about the news. Months old or not.
Pathetic internet geekdom that complain about how new news is... it's all about the technology stupid.. not the microwave quickness of you geeks knowing about it.

[Timothy @ Apr 27th 2006 12:26AM]

#23

First, there's an "r" in archivist. Secondly, no, I didn't squirrel anything away, I saw it on Macrumors when it was there, then today I...wait for it...went back and found the article! Wow, that was difficult. It's called memory. Some of us have one.

[Juha @ Apr 27th 2006 5:57AM]

This might have some interesting uses... Think of eye-controlled user interfaces. How about a blink or a double-blink on things you have on screen. How about a security app that takes a mug shot of the laptop thief before he is getting arrested. How about face recognition. How about camouflaging display as if it was a mirror? How about lipsyncing/adlibbing a cartoon character? How about using a laptop like it was a flatbed scanner...

[TObject @ Apr 27th 2006 8:18AM]

Finally! Bill Gates will see frustations and pain on users faces ;-))
Content based of user's face expressions, gender, race.
Video games become more interesting - game characters interact with user on much deeper levels.
Your monitor screams :"Hey, look at me!" ! "What r u smiling about"?

[Donald @ Apr 27th 2006 1:38PM]

Wow has anyone here ever heard of a webcam? I love this idea, no more looking like I'm looking in outerspace, I can finally look at the screen and camera at the same time! And it's probably just become affordable enough to finally implement. Can you imagine it? "The new iMac... first we asked where did the computer go, now where did the iSight go?"

[JIleto @ Apr 27th 2006 8:50PM]

#42 65536 colors? If you have a red, green and blue LED in a cluster with the brightness of each LED controlled in 256 steps, you can produce 16M colors - way more than the 65536 you're after.

Of course, there is the engineering problem of reducing all these (LEDs + supporting circuitry) to the size of a pixel.