Video tech uses photos to enhance, alter shots: it's the Photoshop of video, and no one is safe

We've seen some decent video alteration in our day, but this new research project by some folks at the University of Washington has the potential to turn the entire concept on its head. Using some rather advanced algorithms to analyze video and photographs of a the same scene, the software can meld the two into something slightly better or even dramatically different. In effect, it's Photoshop for video, since it brings your Photoshop chops to bear on video effects: edit up a still shot or two of the scene, and then meld that with the video, and your edits can be seamlessly integrated into the scene, without all that nasty manual labor required by Shake or After Effects. It's not the end all be all yet, since the tech only works with static scenes so far, but the researchers are working to rectify that. While video evidence hasn't been a sure thing for years, it's always been significantly harder to fiddle with than still shots. With that barrier removed, we might be in for a whole new generation of video that lies and a reality we can be none too sure of. Oh, and really good looking indie flicks. Sample vid is after the break.

Using Photographs to Enhance Videos of a Static Scene from pro on Vimeo.

[Al2x @ Aug 16th 2008 8:30PM]

This could do some real harm rather than help...

But it's motha****** science, that's AWESOME.

[Tetra @ Aug 16th 2008 10:54PM]

This was already posted on Engadget over a year ago.
They used this link last may:
http://www.youtube.com/watch?v=1PktKqyRXIE

[Kurian @ Aug 17th 2008 9:00AM]

Ya. Now I'd be tempted to make my webcam show a 9 incher.

[Flashpoint @ Aug 16th 2008 8:32PM]

so...theoreticaly...

...i could create a videotape of someone wearing a pair of Bruno Magli shoes, even if they didn't actually own them?

[Airibis @ Aug 16th 2008 8:39PM]

You could do that now... however this software would just take out alot of steps in that process.

also the person couldn't move around with this software, being that it only works in a static scene.

but lol yeh i guess you are right, if you wanted to do that why not.

[Wise idea @ Aug 16th 2008 9:34PM]

Waaaaaaiiit a second, never mind changing shoes......that means I can have someone else instead of my girlfriend in that video?

[LarryLarryLarry @ Aug 16th 2008 10:31PM]

From what I can tell, you'd need to take a whole bunch of really good still photos along with the video. This whole thing is mildly interesting but totally impractical. Is it surprising that they can use the information from super-high resolution photos to improve a super-low resolution video of exactly the same scene?

[HyperHacker @ Aug 17th 2008 12:24PM]

Surprising? No. Incredibly difficult to do this well? Yes.

Also: DO WANT

[DarrenBaker @ Aug 18th 2008 9:51AM]

I think that reference just flew over everyone's head... You win!

[Cody @ Aug 16th 2008 8:38PM]

Just what I've been waiting for...

(insert evil laugh here)

but seriously...it looks like it wouldn't have taken this long to do this.

[Dopefish @ Aug 16th 2008 9:00PM]

Yeah I know, I wrote a similar program when I was bored last night


/sarcasm

[ethana2 @ Aug 17th 2008 5:46PM]

I SEE YOU, CARMACK.

Your fake sarcasm tags don't throw me.

[Roberto @ Aug 16th 2008 8:39PM]

Someone should tell these Huskies that Vimeo can take 720p video. On the other hand, that's some pretty awesome, commercializable tech they are inventing!!! :D

[pe3ko @ Aug 16th 2008 8:41PM]

This will be useful in the porno industry!

[Macindows @ Aug 16th 2008 8:43PM]

Now if we'd combine this technique with the new Nikon D90 DSLR, which features video recording, so that the camera would take photos at the same time while shooting the video, it could revolutionize video recording!

[Harley3k @ Aug 16th 2008 8:58PM]

I was thinking the same thing, but with an HD Video camera.
1920x1080 is basically a 2 megapixel photo.

[broli @ Aug 17th 2008 8:08AM]

^ I'm pretty sure the prices are the same as well genius. The point of this is that you can have an el cheapo cam corder wich also takes high res photos. Combine them and you're rivaling a multi 1000 $ cal corder.

[Alex @ Aug 16th 2008 8:59PM]

Snap, this is l33t stuff.

I think the camera stabilization thing was crazy good for not requiring any other input.

[Technex @ Aug 16th 2008 8:48PM]

Cool, this is clever stuff, very good!

[Mic2000 @ Aug 17th 2008 12:31PM]

Exactly, great job. The first one without a comment about how to abuse it for himself ;)

[Cal @ Aug 16th 2008 8:48PM]

That is truly amazing... But to call it PhotoShop for video is almost an insult to it, this does the work for you, in PhotoShop you have to control how you use each tool (excluding filters, to a degree) and it can take a considerable amount of hard work on your part to get results in a still which are seemingly effortless in video using this.

[lukas88 @ Aug 16th 2008 9:02PM]

I'm pretty sure a great deal of the steps to create the videos using this software were not included in the demo. For example, taking the no parking sign out involved manually removing it from one frame, a very photoshop-esque chore. This is stepping up the level of complexity significantly over photoshop, and you can expect that the amount of manual labor will be similar, if not more.

[lolec @ Aug 17th 2008 3:03AM]

Actually, the only step to remove the sign is to manually mask the area, since the system know 3d data will errase the 3d object by using information of the back ( it get it from other frames ) so, basically all you need is enough data. in this case showing the back of the sign will be enough for the system to fill the hole.

[fred @ Aug 16th 2008 9:04PM]

Great, now we will have trillions of videos that will overuse the lens flare filter.

[Ross @ Aug 16th 2008 9:16PM]

Whoo! This is something i will probably never use (or be able to afford) in my life but it looks amazing :D

[ethana2 @ Aug 17th 2008 5:48PM]

All they have to do is
1. become a household software item and
2. neglect linux
and BAM, you have thousands of pissed coders making something you can afford.

[ethana2 @ Aug 17th 2008 5:49PM]

....over the next three years

[smi1ey @ Aug 16th 2008 9:18PM]

Wow... just... wow. Truly ground-breaking technology. The video demo was absolutely fascinating!

[Ito @ Aug 16th 2008 9:39PM]

I think I just shat myself

hopefully it actually works, lol

[cwj @ Aug 16th 2008 9:31PM]

See! R. Kelly was right!!

[jabber @ Aug 17th 2008 3:47AM]

That piss was digital!

[porath @ Aug 16th 2008 9:48PM]

this wouldn't actually work for most of what you guys are saying (shoe change, porn), because it's limited to scenes with no moving objects. that said, it's still an incredible piece of technology. i can hardly even believe it. hopefully they will be able to iron out that one kink. it does say they're working on it.

[quib @ Aug 17th 2008 1:16AM]

of course it will work for porn, fake boobs doesn't move.

[ethana2 @ Aug 17th 2008 5:51PM]

*sigh*
I'm not trying to be too weird here, but the physics part of my brain has the following to say:

back and forth. how hard can it be?

[Veraxus @ Aug 16th 2008 9:44PM]

That is one of the most amazing new video editing technologies I have ever seen. I HAVE to have it. PLEASE tell me this is available for sale...

[joe @ Aug 17th 2008 1:41PM]

I've seen video cameras out there that take still pictures at a higher resolution than they take video (i.e., HD stills and SD video). Think of the software possibilities on board one of these cameras. What about the possibility of having every ~5 seconds of video trigger a high-resolution still shot? Coupled with this software, are we looking at the birth of cheap(er) HD video cameras? (ok, not true HD, but humor me here).

[chris w @ Aug 16th 2008 9:53PM]

man, this is awesome!

I can't wait to get my hands on the torrent for this program!

[ethana2 @ Aug 17th 2008 5:55PM]

I just hope it's a proper, simple .msi instead of some .exe installer.
.exe installers need to go die.
...course, even .msi allows things to get more mutant than they need to be, which is why I prefer .deb and .rpm, but I haven't seen .deb and .rpm packages for windows...

What format does OSX Fink use?

[Graham @ Aug 16th 2008 9:54PM]

Thats amazing! I work at a tv station and to think at how much that will help. Say someone leaves a mike cable across the set and we dont catch it until we move the camera, this can fix it much easier than rerecording the whole thing.

[Kaschif @ Aug 17th 2008 6:28AM]

Amazing...................... wow....

[AndrewAmazed @ Aug 16th 2008 9:57PM]

wow, that looks really nice.

now all i have to do is buy a average to crappy video cam and this program and im done??
or am i incorrect...

[El Taco @ Aug 17th 2008 12:41AM]

average crappy video camera + this + good regular camera + photoshopping skills

[Russ @ Aug 16th 2008 9:57PM]

would be interesting to take a an old movie, photographing new high res photos of the old locations and 'updating the movie'. obviously the actors would remain the old res.. just saying it'd be interesting.

[porath @ Aug 16th 2008 10:00PM]

like i said, currently the system only works on scenes with no moving objects. hopefully they get it worked out, but that's a BIG hurdle, trust me :)

[raider @ Aug 16th 2008 10:05PM]

Unbelievable, but, I kinda thought people were already doing this .. obviously I was wrong, or "space time fusion" is just a new term for what can already be done ?

[Blade Genexis @ Aug 16th 2008 10:08PM]

HOLY *****!!! O_O;

-Blade Genexis

[coehort @ Aug 16th 2008 10:33PM]

Go DAWGS! this is great stuff

[chrismansley @ Aug 16th 2008 10:44PM]

I see your 2007 technology and raise you 2008 technology!
http://www.youtube.com/watch?v=suAL6eYqKeM

[farfisa @ Aug 17th 2008 10:52AM]

That is some serious moustache technology!

[Hamidxa @ Aug 17th 2008 2:26PM]

Excellent find.

Nice job Microsoft.