tokyometropolitanuniversity

Gocen optical music recognition can read a printed score, play notes in real-time (hands-on video)

It's not often that we stumble upon classical music on the floor at SIGGRAPH, so the tune of Bach's Cantata 147 was reason enough to stop by Gocen's small table in the annual graphics trade show's Emerging Technologies hall. At first glance, the four Japanese men at the booth could have been doing anything on their MacBook Pros -- there wasn't a musical instrument in sight -- but upon closer inspection, they each appeared to be holding identical loupe-like devices, connected to each laptop via USB. Below each self-lit handheld reader were small stacks of sheet music, and it soon became clear that each of the men was very slowly moving their devices from side to side, playing a seemingly perfect rendition of "Jesu, Joy of Man's Desiring." The project, called Gocen, is described by its creators as a "handwritten notation interface for musical performance and learning music." Developed at Tokyo Metropolitan University, the device can read a printed (or even handwritten) music score in real-time using optical music recognition (OMR), which is sent through each computer to an audio mixer, and then to a set of speakers. The interface is entirely text and music-based -- musicians, if you can call them that, scan an instrument name on the page before sliding over to the notes, which can be played back at different pitches by moving the reader vertically along the line. It certainly won't replace an orchestra anytime soon -- it takes an incredible amount of care to play in a group without falling out of a sync -- but Gocen is designed more as a learning tool than a practical device for coordinated performances. Hearing exactly how each note is meant to sound makes it easier for students to master musical basics during the beginning stages of their educations, providing instant feedback for those that depend on self-teaching. You can take a closer look in our hands-on video after the break, in a real-time performance demo with the Japan-based team.%Gallery-162022%