Image-editing technique lets 2D objects get flipped, turned upside-down
Sure, Photoshop and other image editors are becoming increasingly good at transforming elements of a photo, but most of those tweaks are limited to two dimensions (resizing, rotating, repositioning, etc.). Researchers at Carnegie Mellon are pushing into the third dimension, using a technique that compares the 2D objects in a regular photo with 3D models freely available online. The result? The ability to manipulate part of photo as if it were a real, three-dimensional object, even exposing angles and sides that weren't visible in the original image.
The new technique takes advantage of the growing number of publicly available 3D models online (the researchers, for example, call out TurboSquid and 3D Warehouse, in particular). First, a user selects part of a standard image, like a chair or a banana. Then they search online repositories for a 3D model that's a close match. Once imported into the image, a three-dimensional mesh is rotated and tweaked until it closely matches the 2D element (after all, not every banana curves the same way and not every pear is the same size). At that point, the technique estimates lighting based on the rest of the photo and uses data from the 3D model to guess at parts of the object that can't be seen in the original pic.
Check out the video below to see some of what's possible, like changing the angle of WWII planes to face the viewer, closing the lid of a laptop or setting upright a knocked-over chair. While the team admits the technique isn't perfect (it still has issues with translucency), and further development will likely include automating the search process, the potential is already pretty apparent.
