Video: robotic marimba player grooves autonomously with jazz pianist

We've seen an orchestra's worth of robotic musicians, but we've yet to see one that integrates this perfectly into a piece without any human intervention. Shimon -- a robotic marimba player created by Georgia Tech's Guy Hoffman (formerly of MIT), Gil Weinberg (the director of the Georgia Tech Center for Music Technology) and Roberto Aimi of Alium Labs -- recently made its stage debut by sensing the music from a piano and reacting accordingly in order to provide complementary percussion. Unlike many alternatives, there's absolutely no delay here. Instead, it analyzes the classification of chords, estimates the human's tempo and attempts to extract features from the human's melodic phrases and styles. What you're left with a robot musician that goes beyond call-and-response and actually meshes with the Earthling's playing throughout. The full performance is posted after the break, and make sure to leave a donation as you exit through the doors on the left.

[Thanks, Guy!]

Gallery: Video: robotic marimba player grooves autonomously with jazz pianist

• 
• 
• 
• 
• 

[schmitt @ Apr 26th 2009 5:15PM]

That is amazing!

[rk @ Apr 26th 2009 5:24PM]

Very very cool, great job robot and humans!

/want one

[Curtis @ Apr 26th 2009 5:44PM]

A little off in a few places...but extremely cool. Its kinda sad that you can computerize improv now. :\

[arcticpenguins @ Apr 26th 2009 5:53PM]

A computer will never replace the finesse of a human's touch

[Jamesy @ Apr 26th 2009 8:20PM]

That's what she said!

Pwned, pwned pwned!

[Fingal @ Apr 26th 2009 5:59PM]

Who will compose our smooth jazz?!

[Gad Get @ Apr 26th 2009 6:10PM]

Wow! Seriously, wow!

It doesn't seem to be able to vary impact or volume to mesh better with the piano, though, unfortunately. Maybe they're still working on that.

[jeesusfreek @ Apr 26th 2009 6:21PM]

bet its a PC

:)

[DCheat @ Apr 26th 2009 6:25PM]

I bet that's why there's iMacs in the background...

[ultimatepwnage @ Apr 26th 2009 6:36PM]

it doesnt make a difference, its about the software...

[NoAndThen @ Apr 27th 2009 12:08AM]

@ultimatepwnage:

I think you missed the fact that the difference in their argument is, in fact, the software. Thereby making your comment a contradiction to itself.

No qualms, just found it kinda interesting.

[jeesusfreek @ Apr 27th 2009 3:36PM]

I think, DCheat, you missed my sarcasm.
It's clearly interfacing on some levels with the iMac, and was probably programmed on the aforementioned platform.

Guess I shoulda made my sarcasm more obvious. I just assumed most people would have watched the video and seen the Apple logo all fifteen times it was showed...not a Mac fanboy, but I've had two and thought they were just as lovely as the custom PC I'm writing this from.

Apparently some people don't get it unless you mark off all your remarks with html tags

[Barrett Williams @ Apr 26th 2009 7:11PM]

Sorry to be cynical, but I wonder how creative/random the program actually is.

It could just complete chords in the same key and , but later in the video some "autonomy" is noticable. Impressive even if the timing's off. Version 2.0 will read the expression on player #1's face and hand motions to determine timing.

But then that would be unfair if the competition is a human. Maybe the computer would only be allowed to look 20% of the time. Or maybe there are algorithms to determine the more organic components of music based on timbre, rhythm, etc.

Still, impressive. Maybe one day I'll pick a robot over Esbjorn Svensson Trio for jazz. Well, I hope not.

[bill cant fart is BACK! @ Apr 26th 2009 8:11PM]

In order to be a man, you need to
(1) puff out your chest
(2) say "Tax Exemption"
(3) acquire a taste for freeform jazz.

I've got the first 2 down, will this help with the last one?

[Pretol @ Apr 26th 2009 8:52PM]

The performance harmony is obviously recorded. The timing is off in many parts... Not exactly impressive on any front.

It may be cool, but it's a far cry from terminators taking over the world...

[MeowR- @ Apr 26th 2009 11:47PM]

True...

I doubt terminators will take over the world with marimbas and jazz anyhow.

[Beddoes @ Apr 27th 2009 5:03AM]

It would be amazing if they did.

[VS Dude @ Apr 26th 2009 8:55PM]

I'll be impressed when I can see this in real life:
http://www.youtube.com/watch?v=JLfBafp03qc

[Mathieu @ Apr 27th 2009 4:50AM]

Sounds horrible, no ? Tech isn't everything. But, as for every new thing, first versions are bad. Ihope this won't be right for music or arts in general... I think you guys at engadget are a little too enthusiastics with this robot... It's no improvisation, the chords are written, only the rythm is adapted live.

[Arthur Nonamiss @ Apr 29th 2009 5:23PM]

Pretty cool, but that dude looks like he's going to throw up.