Researchers teach computers to turn 2D images into 3D

Researchers at Carnegie Mellon University appear to have solved a problem long thought impossible, teaching computers to turn static 2D images into 3D models. It was apparently a hot area for research in the 1970s but was virtually abandoned in the 80s after attempts to devise the machine learning necessary proved too demanding for the computers of the time. The key to Carnegie Mellon's research, apart from better machines, is the ability for computers to detect visual cues (such as a car) that can be used to differentiate between vertical and horizontal surfaces -- easy for us humans, but enough to turn even the most powerful computers into an incoherent mess. Apart from turning your vacation snapshots into a whole new experience, one of the big applications for this technology is obviously robotics, where it could boost their vision systems, improve navigation, and basically endow them with one more skill necessary to keep us in line after the uprising.

[diulei @ Jun 15th 2006 11:03PM]

Seems like those calculations now would take some heavy resources too. Cool research though. Has lots of potential.

[John Doe @ Jun 15th 2006 11:06PM]

I for one welcome our robot overlords

[Alex @ Jun 15th 2006 11:32PM]

Can it really work though? That's tough to teach a computer how to do, you know..

[a @ Jun 15th 2006 11:33PM]

a new era for games.

[tommy sciano @ Jun 15th 2006 11:33PM]

i always thought about it. now its true. and i agree robots would be great overlords.

[Zeus @ Jun 15th 2006 11:36PM]

I think computer/video game companies would -love- this technology too, once it's developed some more. With ingame world environments getting more and more detailed, they'd probably like something that would enable them to automate a large part of their workload. Development time is already getting insanely long on large games.

Hmm, or even better, Google takes pics from cities and uses them to make their online DB of the city's buildings so when your in-car navi system links wirelessly to google maps it shows you virtually driving through the city, with decent representations of the buildings and landmarks too. Might make driving directions more intuitive.

[hrhih @ Jun 15th 2006 11:54PM]

I also welcome and even anticipate our robot overlords.

[distant_guy @ Jun 15th 2006 11:55PM]

How does every other article posted get turned to the "robots are going to enslave us" angle? Don't get me wrong, it's funny, and I laughed at this one because I totally didn't see it coming, but I'm beginning to think you guys are paranoid. (And, you know they'll come for they paranoid ones, first, right?)

[Peter @ Jun 16th 2006 12:17AM]

dammit, people can pay me to do this, i dont want computers doing it for me.

[gerbick @ Jun 16th 2006 12:22AM]

This seriously reminds me of the long forgotten MetaCreations tool, Canoma.

This isn't my site, but here's some information about that old program. I think I still have the floppies for this laying about. Did pretty much the same thing. From picture to 3D with auto-texture mapping.

http://www.canoma.com/

[John @ Jun 16th 2006 12:27AM]

but if the building is there to photograph in the first place why would you want to buil it in 3d again? its not endagering human creativity, its expanding and automating it.

[Andrew Carnegie @ Jun 16th 2006 1:05AM]

w00t!!
I for one welcome the new digitized University Center. Our digital entities need sustanance and TBA movies too.

[James S. @ Jun 16th 2006 1:07AM]

distant_guy...it's obvious. If you do post after post of one innovation after another, you begin to see that the robot uprising is inevitable.


Which is why it's good idea to welcome them.

[Ryan @ Jun 16th 2006 1:08AM]

Gerbick, after reading through the instruction booklet of the Canoma program you mentioned, it appears you need to actually section off portions of the object you wish to make three dimensional, for example, with rectangular prisms. This is different from the Carnegie Mellon app, which creates the 3D image without user intervention.

[Cry Havoc @ Jun 16th 2006 1:22AM]

distant_guy: Actually it's not serious. It's simply a reference to the fark.com/somethingawful forums response to a silly or intruiging situation.

"I, for one, welcome our (variable) overlords."

It's actually a pop-culture reference. Futurama if I'm not mistaken.

/Though I probably am
//Fark.com says slashies are fun!

[furtim @ Jun 16th 2006 1:30AM]

Actually, it's a reference to the Simpsons episode where NASA sends Homer into space and a bunch of ants get loose in the shuttle.

[Joe @ Jun 16th 2006 2:01AM]

It seems interesting, but the pictures make it look like they've got a really messed up perspective of the model or something. Doesn't look proportional.

[Sara B. @ Jun 16th 2006 2:29AM]

It made me happy seeing the UC on Engadget. As a CMU student, any picture that makes me exclaim "home!" is excellent.

Now, if only we had had this when building the Red Team for DARPA Grand Challenge :/

[David @ Jun 16th 2006 3:06AM]

We use some stuff kind of like this for film work. Takes some cleaning up, but saves a lot of modeling and texturing time, especially for background buildings.

As to why you'd want a 3D model of a building that already exists, you might want to put it in a different location, or have it get destroyed when the aliens arrive, etc.

[Josh @ Jun 16th 2006 4:37AM]

Quote: Actually, it's a reference to the Simpsons episode where NASA sends Homer into space and a bunch of ants get loose in the shuttle.


Wasn't it giant ant overlords?

[tosh @ Jun 16th 2006 5:54AM]

yawn, realviz image modeler, and the previously mentioned canoma app.
but foremost is the work that paul debevec did on this in the late nineties, allowing image based modelling for the environments in the now infamous "bullet time" scenes in the first matrix.

http://www.debevec.org/Research/VDTM/

http://www.realviz.com/products/im/index.php

[Chris @ Jun 16th 2006 6:27AM]

They said something along those lines on a Futurama episode too.

[Chris @ Jun 16th 2006 6:36AM]

"It seems interesting, but the pictures make it look like they've got a really messed up perspective of the model or something. Doesn't look proportional."

If you scroll down to the bottom of the page and click on the link to the URL where they've got animations, it doesn't look so messed up. Looks neat.

[vc @ Jun 16th 2006 7:37AM]

I thought that porn and video games drove the development of new technologies? I might have to stop by CMU and get them to scan in some girls from a different university. And making photorealistic 3D models for video games could become much easier now.

And Google Earth should license this technology to create 3D buildings that look like the real ones.

[Steve @ Jun 16th 2006 8:41AM]

Yay the UC on engadget. CMU has been getting more and more press these days. If you go to the actual press release you will see they tested the technology on random photos from google image to help it learn better. And as someone who has seen that building 4 times a day for the past 2 years, I find the composite nearly indistinguishable form what it really looks like, as it is already a kinda distortedly long building. It will be interesting to see how it deals with more curved things like people and trees.

[Bryan @ Jun 16th 2006 9:26AM]

Whatever happened to 3D glasses. First there were the paper ones than movie theatres carried the electrically powered ones and IMAX still does. There was VirtualBoy. We need more source material. This is good.

[jonli @ Jun 16th 2006 10:06AM]

What's with all the haters?

CMU represent!

[Hassan Luwalira @ Jun 16th 2006 10:16AM]

Perfect for porn/softcore shots.

[Kevin Forde @ Jun 16th 2006 10:50AM]

So I am doing the first of my multi daily check of engadget. *scroll* *scroll* *scroll*. hmmm that building looks familiar. hmmm. oh snap, its the UC. CMU, CS represent.

Oh, I guess the research is cool too

[Randomizer @ Jun 16th 2006 11:48AM]

I for one have already baked cookies for our robotic (and giant ant too, can't be discriminatin') overlords... hope they like chocolate chip...

[Alex C @ Jun 16th 2006 1:20PM]

This is called "Photogrammetry", the process of creating 3d models from 2d images. Some time ago a PhD at UC Berkeley acheived this through photography.

http://old.siggraph.org/publications/newsletter/v33n4/contributions/debevec.html

Additional interesting items of exploration in this abstract are the use of "image based" lighting techniques and the inclusion of High Dynamic Range(HDR) images.

[Alex C @ Jun 16th 2006 1:27PM]

Though is seems as though the CMU folks have gone "one" better and done this from a single image. Nice technique.

[Jason @ Jun 16th 2006 4:43PM]

Interesting... I wonder what the implications are with PICASA and Google's new monster data center intiative.

Could Google and our Pics be used to create a virtual 3D world?

Hmmm.

[Ben @ Jun 17th 2006 7:50AM]

As long as the computation is efficient than this could be killer in the robotics field. Next up is the DARPA Urban Challenge, I wonder if this is fast enough to play a role? (CS CMU people, any idea?)