Adobe's Project Awesome Audio cleans up recordings with a single click

Adobe likes to show off crazy tools it has been working on in a special portion of its MAX conference each year. The not-quite-ready features are known as "sneaks," and while not all of them make it into Creative Cloud apps, some of them do. Adobe is holding its annual design gathering this week, and as you might expect, it has a new batch of "sneaks" to show off. The company wouldn't give us the full details on all of them, but it did give Engadget a preview of one: Project Awesome Audio.

Imagine how helpful it would be if you could clean up janky audio recordings in Adobe Audition with a single click. That's what Adobe research scientist Zeyu Jin has done with Project Audio Awesome. This isn't Jin's first "sneak," though. Back in 2016 he demoed a tool that would allow you to seamlessly insert speech into an existing audio clip.

Along with intern Jiaqi Su, a computer science PhD student at Princeton, Jin harnessed the power of the company's Sensei AI for what he calls "end-to-end audio enhancement." He explains audio enhancement is typically situational, where you have to use specific tools to fix specific problems. In Audition, for example, to fix an issue with noise, you have to hunt through menus to find the correct tool and then spend the time to manually adjust the parameters to get something that sounds professional.

This is doable, and many audio apps allow you fix common issues like noise, microphone artifacts and other things that don't sound good, but those edits can be time consuming. To remedy this, Jin and Su created a one-click solution that doesn't merely reduce the degradation, but it extracts the speech from the audio clip. For this reason, Project Audio Awesome can handle difficult edits like reverb, or an overly echo-y room, that might not even be possible with software.

"First, we want to make everything much simpler," Jin says. "No need to learn which tool to use in each situation. We want to reduce degradation in one go." He explains that "everything" -- in this case that's noise, artifacts and reverb -- is gone with a single click.

The feature was originally called "One-Button Audio Awesomization," but has since be named with the official "sneak" nomenclature that usually includes "Project." Basically, you open a file in Adobe Audition, click the button to let the AI do its thing, and in seconds you have a new file that sounds a million times better. In its current iteration, when you have a file open in Audition, you click a button the team built that triggers a script and taps into Sensei's AI abilities to do the heavy lifting. This isn't using anything inside of Audition -- the tech is completely new.

Jin showed me a few examples of the system in action, the first of which fixed some intense microphone degradation. Someone was trying to record a clip with a microphone, but forgot to switch to it in their computer's audio settings. The audio was captured with a laptop's built-in mic instead, so there was a ton of noise and the file wasn't usable. With a click, Jin was able to clean up the clip in seconds, and the result sounded like it was recorded with a professional microphone.

You can also fix issues with audio that was recorded in two locations. Let's say you have move to a different spot in the middle of your clip, and one spot is a lot noisier than the other. Project Awesome Audio extracts the speech from the files and makes the entire file sound like it could've been captured in the same spot. It isn't perfect -- you'll still need to make some basic volume adjustments -- but the noise, extra reverb and microphone degradation were all gone.

"Even I find it hard to do [audio enhancement] well, especially when you record with a mobile device," Jin notes. I frequently find myself in situations where I need to record and only have my phone. iOS and Android apps will help you capture, but it rarely sounds good when you're using your phone's mic like a speakerphone and multiple people are talking. With Project Audio Awesome, you might not have to worry about scenarios like that in the future.

It's easy to see how this could be a massively useful feature inside Adobe Audition, but there's no guarantee this tool will make it to the desktop app. The company is using this "sneak" to gauge reaction and gather feedback before making any concrete plans. For now, you can read more about the underlying technology and the research that went into Project Awesome Audio in this paper written by Jin, Jiaqi Su and Adam Finkelstein, a computer science professor at Princeton.