We've seen some decent video alteration in our day, but this new research project by some folks at the University of Washington has the potential to turn the entire concept on its head. Using some rather advanced algorithms to analyze video and photographs of a the same scene, the software can meld the two into something slightly better or even dramatically different. In effect, it's Photoshop for video, since it brings your Photoshop chops to bear on video effects: edit up a still shot or two of the scene, and then meld that with the video, and your edits can be seamlessly integrated into the scene, without all that nasty manual labor required by Shake or After Effects. It's not the end all be all yet, since the tech only works with static scenes so far, but the researchers are working to rectify that. While video evidence hasn't been a sure thing for years, it's always been significantly harder to fiddle with than still shots. With that barrier removed, we might be in for a whole new generation of video that lies and a reality we can be none too sure of. Oh, and really good looking indie flicks. Sample vid is after the break.
posted Aug 16th 2008 8:24PM
This could do some real harm rather than help...
But it's motha****** science, that's AWESOME.
This was already posted on Engadget over a year ago.
They used this link last may:
http://www.youtube.com/watch?v=1PktKqyRXIE
Ya. Now I'd be tempted to make my webcam show a 9 incher.
so...theoreticaly...
...i could create a videotape of someone wearing a pair of Bruno Magli shoes, even if they didn't actually own them?
You could do that now... however this software would just take out alot of steps in that process.
also the person couldn't move around with this software, being that it only works in a static scene.
but lol yeh i guess you are right, if you wanted to do that why not.
Waaaaaaiiit a second, never mind changing shoes......that means I can have someone else instead of my girlfriend in that video?
From what I can tell, you'd need to take a whole bunch of really good still photos along with the video. This whole thing is mildly interesting but totally impractical. Is it surprising that they can use the information from super-high resolution photos to improve a super-low resolution video of exactly the same scene?
Surprising? No. Incredibly difficult to do this well? Yes.
Also: DO WANT
I think that reference just flew over everyone's head... You win!
Just what I've been waiting for...
(insert evil laugh here)
but seriously...it looks like it wouldn't have taken this long to do this.
Yeah I know, I wrote a similar program when I was bored last night
/sarcasm
I SEE YOU, CARMACK.
Your fake sarcasm tags don't throw me.
Someone should tell these Huskies that Vimeo can take 720p video. On the other hand, that's some pretty awesome, commercializable tech they are inventing!!! :D
This will be useful in the porno industry!
Now if we'd combine this technique with the new Nikon D90 DSLR, which features video recording, so that the camera would take photos at the same time while shooting the video, it could revolutionize video recording!
I was thinking the same thing, but with an HD Video camera.
1920x1080 is basically a 2 megapixel photo.
^ I'm pretty sure the prices are the same as well genius. The point of this is that you can have an el cheapo cam corder wich also takes high res photos. Combine them and you're rivaling a multi 1000 $ cal corder.
Snap, this is l33t stuff.
I think the camera stabilization thing was crazy good for not requiring any other input.
Cool, this is clever stuff, very good!
Exactly, great job. The first one without a comment about how to abuse it for himself ;)
That is truly amazing... But to call it PhotoShop for video is almost an insult to it, this does the work for you, in PhotoShop you have to control how you use each tool (excluding filters, to a degree) and it can take a considerable amount of hard work on your part to get results in a still which are seemingly effortless in video using this.
I'm pretty sure a great deal of the steps to create the videos using this software were not included in the demo. For example, taking the no parking sign out involved manually removing it from one frame, a very photoshop-esque chore. This is stepping up the level of complexity significantly over photoshop, and you can expect that the amount of manual labor will be similar, if not more.
Actually, the only step to remove the sign is to manually mask the area, since the system know 3d data will errase the 3d object by using information of the back ( it get it from other frames ) so, basically all you need is enough data. in this case showing the back of the sign will be enough for the system to fill the hole.
Great, now we will have trillions of videos that will overuse the lens flare filter.
Whoo! This is something i will probably never use (or be able to afford) in my life but it looks amazing :D
All they have to do is
1. become a household software item and
2. neglect linux
and BAM, you have thousands of pissed coders making something you can afford.
....over the next three years
Wow... just... wow. Truly ground-breaking technology. The video demo was absolutely fascinating!
I think I just shat myself
hopefully it actually works, lol
See! R. Kelly was right!!
That piss was digital!
this wouldn't actually work for most of what you guys are saying (shoe change, porn), because it's limited to scenes with no moving objects. that said, it's still an incredible piece of technology. i can hardly even believe it. hopefully they will be able to iron out that one kink. it does say they're working on it.
of course it will work for porn, fake boobs doesn't move.
*sigh*
I'm not trying to be too weird here, but the physics part of my brain has the following to say:
back and forth. how hard can it be?
That is one of the most amazing new video editing technologies I have ever seen. I HAVE to have it. PLEASE tell me this is available for sale...
I've seen video cameras out there that take still pictures at a higher resolution than they take video (i.e., HD stills and SD video). Think of the software possibilities on board one of these cameras. What about the possibility of having every ~5 seconds of video trigger a high-resolution still shot? Coupled with this software, are we looking at the birth of cheap(er) HD video cameras? (ok, not true HD, but humor me here).
man, this is awesome!
I can't wait to get my hands on the torrent for this program!
I just hope it's a proper, simple .msi instead of some .exe installer.
.exe installers need to go die.
...course, even .msi allows things to get more mutant than they need to be, which is why I prefer .deb and .rpm, but I haven't seen .deb and .rpm packages for windows...
What format does OSX Fink use?
Thats amazing! I work at a tv station and to think at how much that will help. Say someone leaves a mike cable across the set and we dont catch it until we move the camera, this can fix it much easier than rerecording the whole thing.
Amazing...................... wow....
wow, that looks really nice.
now all i have to do is buy a average to crappy video cam and this program and im done??
or am i incorrect...
average crappy video camera + this + good regular camera + photoshopping skills
would be interesting to take a an old movie, photographing new high res photos of the old locations and 'updating the movie'. obviously the actors would remain the old res.. just saying it'd be interesting.
like i said, currently the system only works on scenes with no moving objects. hopefully they get it worked out, but that's a BIG hurdle, trust me :)
Unbelievable, but, I kinda thought people were already doing this .. obviously I was wrong, or "space time fusion" is just a new term for what can already be done ?
HOLY *****!!! O_O;
-Blade Genexis
Go DAWGS! this is great stuff
I see your 2007 technology and raise you 2008 technology!
http://www.youtube.com/watch?v=suAL6eYqKeM
That is some serious moustache technology!
Excellent find.
Nice job Microsoft.
I bet the sort of people that hate most HDR photos will hate this effect on videos (ie: me, a bit :)
Also, they say making "efforts are being made to lift the restriction" of it only working with static scenes. To me it looks to be mainly based on the fact the most, if not all, of the scene is static, not merely a solvable restriction.
HDR has a specific use, for example to show all the detail in a scene where there are both dark and light sections like the example in the article. The problem arises when people gain access to a HDR program and overuse the technique, not even knowing why HDR is necessary in the first place.
Just like how an expensive mouse and a fancy mousepad doesn't actually make you a better gamer, fancy programs are no substitute for real composition skills and knowledge of basic photographic theory (in this case: unless you want a silhouette, never shoot with the sun behind your subject).
that mean's uw could edit a football game and claim that they won even though they lost... wow. don't know how good it'll be. maybe when it's good enough you can't really trust videos anymore since they can be edited. hopefully there is a consortium looking into creating secure raws that can used as proofs and can't be altered but can be converted leavings raws as a proof.
A. not for moving subjects
B. there is a thing called people, they tend to remember things
C. football? of all the evil that could be done with this, football?
amazing - I look forward to the future of video
THIS is the reason I spent the extra money for a quad core cpu, now as long as this is a (realatively) affordable program I'm sold.
This is the better than the current best thing since sliced bread!
They could make next-generation video cameras snap high quality photos every few seconds to allow the user to apply any of these given effects later on when editing... And this can be done with ease too if they can develop it right.
So we can make better lochness monster, ufo, and bigfoot video recordings?
my thoughts exactly
Amazing. The number of times I could have used this... The shocker for me was how it is able to create a depth map from ordinary video or photographs. The other technologies I've seen, though not married together like this.
BTW, any cameras out there take/generate HDRI images with a single snap?
fuji s3 pro and s5 pro both take HDRs in one snap.
Uh... guys, don't know if anyone else noticed, but doing the depth reconstruction that is required for mapping the photos appropriately to points in space requires a set of calibrated stereo-vision cameras, which means that in order to do this you'd need a dedicated stereo setup. Point being you won't be able to download some software to let you do this with a still camera and crappy camcorder anytime soon. Still cool, though.
If the camera is moving, it can generate stereo. If the camera is still, THEN you would need a stereo camera.
Apparently one of the guys at Adobe's "Advanced Technology Labs" is one of the main guys behind this project. Perhaps we'll see this in a future version of After Effects?
I wonder if you could FAKE an airplane crash to a building in real time...
Any ideas ??
HA HA!
WAAAAHHHHH! HAHAHAHAHAHAHAHA!
...
You're silly.
if only this was available in 1963...
http://www.youtube.com/watch?v=w-rcdBNFnGs
Wow. And I already feel like I can trust the news.
Does the Xacti still allow you to take high-res stills whilst recording? If so, I think we just found our camera...
In king arthur (movie) lead actress has big boobs on promotional poster but in movie she has small ones (perky though)
with videoshop she can have big boobs on promotional poster as well as in movie.
Awesome!
Guys, this is nothing new, its even available commercially and widely used!
Imagineer systems has been selling MOKEY for 5-6 years and it removes stuff, even moving peopleaand will paint in whats missing.
The foundry in the UK have been selling FURNACE for advanced discreet/Autodesk systems also for 4-5 years, and does all of this plus WAY MORE.
Nothing new here, not even the depth cueing, you dont need stereo cameras, it just maps the fastest moving pixels closer and the slower ones farther to the camera, no rocket science but PARALLAX.
oh, and tracking and reconstructing the 3d wall from the camera motion, been done for 6 -7 years, go check out boujou.
All this tools are used daily in post-production nowadays ;-)
Imagineer's software can do a lot of what is being recognized as "new"... but, I think this issue is that this software technology will now be public domain... and, when you look at the cost of Imagineer's and Furnace's cost... public domain software could indicate a future lower cost and a resulting more broad base useage of this ability that to date has been used by only those who can afford very expensive software.... which makes me feel a little bit quesy when I think about spending the money on Imagineer's software.
I might also note... that the developers definately do have something many have not noticed... their video stabilization method seems be able to stablize on a z-axis (depth movement toward and away from the camera is reduced or eliminated).
Oh fuck. Why do people bother? Automated systems will never do aswell as things such as after effects and shake. I use after effects heaps and this dosen't even compare.
This reminds me of the end of the movie "The Manchurian Candidate". Where they swap Denzel Washington's features with someone elses.
The second they combine this with mind reading and writing technology, we are all screwed.
Screw the video, the narrator voice is either a really flawed human or a very well done voice synth - I hope for the latter!
Scary stuff, you can easily manipulate uninformed people with these technologies.
that blows my mind!
I emailed the guys who wrote it, and the main writer Pravin emailed me back with this:
Unfortunately all the code for the project is research quality code and is currently in no shape to be released. Unfortunately we are also busy working on new research projects so there isnt any immediate plans to work on releasing this software. All the details of the algorithms used in the project have already been published in our paper. We were hoping some professional company like Adobe would create a product using our ideas. But that hasnt happened yet.
They should send it to some asian company, that'll teach adobe.
Just being able to estimate depths from a single video is amazing. Everything else it does on top of that blows my mind. I'm impressed.
NOTHING is real anymore :[
Can this add/remove russians from parts of georgia too? That's handy for the news, not that they would EVER do that of course, well maybe all the time, but not more than that! I'm sure.
Wait, so if you get divorced, you can remove your previous significant other from wedding videos to make a film that looks like you're crazy and talking to thin air with everybody cheering you on and congratulating you?
Unless you claim you were demonstrating the first iphone and headset of course.
"Alot" is NOT A WORD! Is "afew" a word? Is "abunch" a word? Is "ahundred" a word? NO! Hint: Hit the big button at the bottom of your keyboard after the "a" kids.