AI adds background noise to Google Street View scenes
It’s thanks to a pair of neural networks that analyze scenes and provide an appropriate soundtrack.
Google Street View is a great way to imagine you're somewhere else, walking the streets of a bustling foreign city or making your way through a quiet, tree-lined neighborhood. Now, thanks to a project from Nao Tokui, it's become even more immersive. He's put together a feature called Imaginary Soundscapes that creates background noise based on what you're looking at in Google Street View. It's a soundtrack for your virtual walks that was first identified by the Tumblr user Prosthetic Knowledge.
Imaginary Soundscapes works by using two different neural networks. One is able to identify the content within the different Street View images, while the other can pinpoint audio and separate the sounds and acoustics of the environment. Tokui has put both together for this project; the two neural networks can find open source audio and play it in the background, based on what you're looking at in Google Street View. For example, on an residential street in a busy Washington, DC, urban neighborhood, I heard an imagined soundtrack of buses, cars and people chatting as they walked by.
You can give it a try yourself at Imaginary Soundscapes' website; it only works on PC in Chrome and Firefox, though I was able to access it in Chrome on a Mac. By default the site takes you to a random location. You have a choice to input your own street address or visit another random place. The sound will play automatically.
It's not perfect, of course. Fast Co.Design points out that in the quiet, grass-filled fields of Tokyo Stadium, there are the sounds of cars zooming by. Regardless, it's a pretty cool AI. You can go more in depth into how it works at the creator's Medium post and a short paper for a workshop on Machine Learning and Design.
