Latest in Gear

Image credit: Guilin Liu et. al/NVIDIA

NVIDIA's AI fixes photos by recognizing what's missing

The results are uncannily accurate.
2047 Shares
Share
Tweet
Share
Save

Sponsored Links

Guilin Liu et. al/NVIDIA

Most image editing tools aren't terribly bright when you ask them to fix a photo. They'll borrow content from adjacent pixels (such as Adobe's recently demonstrated context-aware AI fill), but they can't determine what should have been there -- and that's no good if you're trying to restore a decades-old photo where you know what's absent. NVIDIA might have a solution. It developed a deep learning system that restores photos by determining what should be present in blank or corrupted spaces. If there's a missing eye in a portrait, for instance, it knows to insert one even if the eye area is largely obscured.

There have been previous attempts at AI-guided fills, but they've typically been limited to rectangular sections, have focused on gaps near the middle of the picture and haven't scaled well to missing photo data of different sizes. NVIDIA's "partial convolution" approach, which guarantees that the output for missing pixels doesn't rely on the input values, can work with holes of any shape, size or location. That, in turn, produces uncannily realistic results in many cases -- even if the AI doesn't know exactly what's missing, the result usually looks like it fits. Previous methods tended to produce obvious glitches.

NVIDIA trained its system by generating tens of thousands of hole variations and making the AI learn how to reconstruct photos. It tested using a different set of holes to ensure the AI genuinely understood how to restore photos on its own.

The results aren't always flawless. You may see a facial feature clearly borrowed from someone else, and it's bound to struggle if the hole is so large that there isn't enough information to create a plausible reconstruction. But what's here could still be incredibly useful. You could repair seemingly hopeless images without hours of painstaking reconstruction. The scientists also envision the AI helping to upscale images without losing sharpness. In effect, you'd only ever have to worry about touching up minor details -- the days of recreating whole segments from scratch might soon be over.

All products recommended by Engadget are selected by our editorial team, independent of our parent company. Some of our stories include affiliate links. If you buy something through one of these links, we may earn an affiliate commission.
Comment
Comments
Share
2047 Shares
Share
Tweet
Share
Save

Popular on Engadget

Engadget's Guide to Privacy

Engadget's Guide to Privacy

View
Amazon Prime perks now include mobile game add-ons

Amazon Prime perks now include mobile game add-ons

View
Yamaha updates its THR desktop guitar amps for the first time in years

Yamaha updates its THR desktop guitar amps for the first time in years

View
Facebook’s latest AI experiment helps you pick what to wear

Facebook’s latest AI experiment helps you pick what to wear

View
iFixit's iPhone 11 Pro Max teardown investigates charging rumors

iFixit's iPhone 11 Pro Max teardown investigates charging rumors

View

From around the web

Page 1Page 1ear iconeye iconFill 23text filevr