Smule's Glee app puts a song in your heart and on your phone

Enthusiastic fans of Fox's hit teen angst 'n jazz hands show Glee refer to themselves as 'gleeks,' a portmanteau of glee and geek. It's a sure bet that the gleeks will be beside themselves when they hit the App Store today and see the new US$2.99 Glee app, which brings the show's musical numbers to life with your voice in the starring role.

Developed by Smule, the new app joins the company's suite of music makers (Ocarina, Leaf Trombone, I Am T-Pain, Magic Piano, etc.) as a social-vocal mashup that puts some serious geek power behind the simple act of singing. Add in the show's iTunes-dominating songs and what will undoubtedly be a nice promotional push from the network, and it's bound to be heading for the top of the paid charts.

Glee for iPhone is straightforward enough to use, although as is sometimes the case with memory-intensive apps in their first version, you may experience a bit of instability if you haven't restarted your phone in the past few weeks. With headphones on and device in hand, you can select a song and go into karaoke/recording mode to lay it down. The app plays the backing track, displays the lyrics and shows you the pitch line for the melody and harmony parts.

Read on for more, including an interview with Smule's founder Dr. Ge Wang.

%Gallery-90913%

As you sing, the pitch indicator lets you know when you're on the note or if you're sharp or flat; don't worry too much about that, though, as the app is correcting your pitch and optionally adding harmony as you go along. The results may sound a little bit like an Imogen Heap tribute band, but they have a certain charm. Remarkably, the app kept up quite well with singing, harmonizing and recording when I tested it on an iPhone 3G; of course, it performs even better and looks great on an iPad. You can play back the recording immediately and do another take if you need it (chances are, the first few times, you'll need it).

Once you're satisfied with your performance, you can save the recording and/or upload it to the Smule servers, where it will join similar performances from all around the world. You can create a band of like-minded gleeks who follow and favorite your tracks, and quickly email or post your heartfelt howls to your Facebook, Twitter or MySpace feeds. Clicking the 'Broadcast' button adds the performance to Smule's global sing-along; at that point, other users can add their own voices to yours, extending the virtual glee club into the cloud. Likewise, if you find a performance you want to join in with, you can lay down your vocals and add them. All this lyrical back-and-forth works briskly on Wi-Fi and holds up nicely on 3G as well.

The US$2.99 app presents would-be rockers and balladeers with a selection of three included tracks (Rehab, Somebody to Love and You Keep Me Hanging On) and several more to purchase in-app, with additional songs planned for weekly release. There's also an a capella option that harmonizes whatever you choose to sing; you can post those tracks as well. It would be nice to see some additional free tracks (not to mention a free single-song version of the app to let folks try before they buy), but considering the music licensing logistics involved, it's not surprising that the app's cost structure is what it is.

We had a chance to talk to Smule founder and chief creative officer Dr. Ge Wang this week (and at Macworld Expo -- see Mike Schramm's video interview here). We asked him about the experience of developing Glee, and the roller coaster ride of the last 18 months of Smule's App Store success.

How did the idea for the app come about? Was it Fox approaching you, or did you go to them and say "We have an idea how this could work?"

Dr. Ge Wang of Smule: Well, I think it was actually a bit of both. Kind of the way things fell into place has been a little serendipitous, a bit of both sides being interested. We at Smule had been looking to do an interesting take on both karaoke and social singing, but not in its usual way; in a way that's very special to what we do. The idea of having an app that not only can you sing to, and it transforms your voice, but also you can add a ton of voices together; that's always been floating. When the opportunity for Glee came from Fox, we really jumped on it.

It was actually a pretty fast-moving thing, from saying "Do you want to do this" to "Of course we want to do this, this is something we've been waiting to do, it's the perfect opportunity." So a lot of credit to our business development folks for getting us in the door, and lining up the team to build demos and eventually of course product design and product.

Not to give away any secrets, but from that initial conversation to the release date, how much time are we talking about?

I can't say specifically, but it was a pretty short amount of time. For the number of things that are happening in this app, which we think is going to be really delightful to our users‚ it is an absurdly short amount of time.

For me personally, this is one of those projects -- I've been on a lot of projects in software, music and music technology, but this one was just really insane. Mostly in really great ways; people just totally dug into this idea, and I think a number of us worked 110, 120-hour weeks for a couple of weeks straight. But I think people wouldn't be doing that unless we were totally stoked about the product. It was an insane amount of work that went in for an insanely short schedule. Also, though, that was balanced by the lack of sleep and the sheer enthusiasm about what we could do here. I think we ended up with a product that we're very happy with. There's probably nothing else out there, not just on mobile devices, you just can't quite do anything like this anywhere else. We're very happy about that.

You mentioned the unique aspects of the product and the platform. Was it really challenging to get this working on the original iPhone, the 3G? Were you hitting the performance limits of those devices as you went through the development process?

Absolutely. There were a lot of challenges we had to overcome in actually developing the technology. This is leveraging a lot of the state of the art in computer music research, stuff that we do at Stanford and at Princeton, but also what that the whole research field is doing. To actually put that on a mobile device is a first. I think we're pretty amazed that anything works at all. Second, to say "we really want this and this to happen," our audio engineering and programming team really went the distance on this one. They really developed technology from the ground up. We got it working on 3G, working great on 3GS and on the iPad.

Part of this was the time it took to do this. It was very much that we had to do slow research, but we had to do research, productization, product development and everything else in the same span of time.

It sounds like not only inventing the flying car, but learning to fly it in midair. Was there any piece of the technology that you were trying to invent as you went along, and you said "This just isn't going to happen, there isn't a way to do this?"

Yes and no. In the process of actually trying this technology we probably tried, for everything that went in [to the app], we probably discarded 5x the amount of other things we could have done. Feature-wise, there wasn't [anything we couldn't do]; I think we really hit upon what we wanted to achieve, but the method of getting there -- we were pretty disciplined about realizing this was something new, and if something did not work we didn't feel tied to it. We tried something else, so there was definitely a spirit of research & invention in the air.

There's no major feature that we had to say "You know, this is not going to happen." There are a number of features that, as the product went along, we said "This makes a lot more sense for the 1.1 release." We really wanted to drill in to what we wanted to present as the core experiences in this 1.0.

There is some decision-making in that. In a sense, the entire product, in the usual Smule fashion, is kind of built... there's a saying in Chinese: "Crossing a muddy stream by feeling the stones under the water." Taking a step at a time. There's actually almost a weird Zen progression to it, some kind of a discipline within the chaos. [Laughs] There's a lot of serendipity, from beginning to end -- and we're certainly not at the end, I should say from beginning to now.

In terms of using the app, what kind of experiences do you want people to come away with? On one level, it seems like you're pulling some of the user experience forward from I Am T-Pain, but there's more going on here.

We actually built this from the ground up -- in some sense, it may seem like a follow up to T-Pain, but it's a complete rearchitecture and redesign. We wiped the slate clean and said "how can we do this?" The experience we want people to have is just to have fun singing. That's our #1 goal, everything else depends on it. If you're not having a blast singing with this app then none of the more extended features would make sense.

It's core that we get that fun experience right. And part of that is really getting the conditions right to lower people's inhibitions about singing. I think everyone loves music, it's hard to find someone who just hates music. But also, most people are just not comfortable putting themselves out there, and it's always been kind of a Smule mission to bring out the creativity that's in everyone. In this app, it's trying to get people closer to that point -- part of that is creating a delightful experience that you can't help but sing into.

Then, the social component completely transforms the singing experience by connecting you with others. Those two are the core experiences: fun singing, and then the social experience‚ which for the life of me I can't ever remember seeing in a piece of software until now.

Have you shown off the app to any of the show's creators?

We've been working with Fox to show them the app... they're loving it. I think the alignment is so natural. The show has always been about the underdog in all of us, and being part of something bigger -- the glee club. It's core to our vision to do the same, we're trying to enable everyone to play music. Playing music should not be harder than just picking up your phone and calling your best friend. In this case, you're literally picking up your phone, and that transforms your voice (but keeping it what it is)... making it something otherworldly, in a good way we hope. Then allowing you to become something bigger by everyone adding their voice to it

I think the show creators are digging it because they sense that alignment. As we progressed through the iterations of the app, it became more apparent that this was a really great match.

You do sound like a fan of the show.

I do think there's an undeniable magic to the show. I should note, I had not watched the show before I started working on this project, but what I love about it... the show is not your typical musical show. People don't break into song, it's something more subtle than that. When the songs come on, they're grandiose, but it's also something more cerebral.

It's not the Disney musicals where suddenly the main character breaks into song and you see it coming. In this case you almost don't notice when they begin, you're halfway through the song before you realize "Wow, we're in a number." It's happening more in the minds of the characters, which is a lot more powerful. You can definitely count me as a fan. A happy gleek!

You've really created a branded musical instrument for the show. In the past it might have been a Partridge Family guitar, but now it's a networked iPhone app. Do you think this is the first of many TV tie-ins for Smule?

I think we're always experimenting. The truth about the mobile space and what we do is that a lot of things, we're still learning, and we don't really know until we put something in the hands of a lot of users. In that sense, we are going to see how things go, but we're going to put every bit of our muscle into making this work.

Part of why we're so happy with this app is that it's not just a branding so it's "Glee." It was built from the ground up both for Glee, and it was something that we wanted to do -- something very Smule-ian, if you will. That combination is key. This is Glee, but it transforms Glee with this mobile platform. I feel like we're building a branded Glee instrument, but also a new instrument, period, a new type of instrument where you can have a great time singing. If you like music, we think you have a good chance to enjoy this app whether you sing or not.

At the same time, the social part of it makes it a new kind of social instrument. We wonder where that goes... especially that part can only happen with these kinds of mobile devices, which are also globally connected to the Internet. Half of this app isn't in the app; it's in the back end, in the cloud.

When you started work on Smule a year and a half ago, did you have any idea that your academic work on music was going to take you to this place... where you're working on an app for a prime-time TV show and you have millions of people playing imaginary trombones and impersonating rap stars?

It feels like a lot longer in some respects and a lot shorter in others. When I started at Stanford I had no intentions of starting a company, and I've come to appreciate that I don't think anyone in their right mind should ever start a company. [Laughs] It's insane. The only reason to start a company, the only reason anyone ever should, is if there's something that you just gotta do. You want to do it so much that you can't afford not to do it.

What Smule felt like, the promise of what it could be... we had no idea what it actually was. It felt like this idea of bringing music making, not just music content, to a lot of people was so close to what I want to do as a researcher that it seemed such a natural thing for me as a researcher and an academic -- almost ironically, to actually find this bridge between academia and the commercial world, to actually do something together.

I had no idea where it would take us. Not knowing has always been part of the fun. Who would have predicted that a year later 2 million people would be playing flute-like instruments on their iPhones, and listening to millions more around the globe? Or judging each other in trombone sessions, or autotuning their voice, or playing whimsical pianos‚ and now actually singing together. It's like We Are The World, but not just the stars, it's everyone. That's the vision‚ and In one sense, I had no idea we would end up where we are. In another sense, it has felt really natural and very good in terms of the progression we've been on. I'm very thankful for that.

Speaking of Magic Piano, have you seen the cat video?

Yes, it's awesome. We tried to build a piano for everyone, young and old... we've had people tell us their 2-year old loves it, their 86-year old grandmother has always wanted to play piano and never did, now you put an iPad in front of her and she's playing. But we did not design this for cats specifically. [laughs] It seems that it is also feline-friendly, so that's satisfying. It took a little bit of magic for that to happen.

Thanks to Dr. Wang for taking the time to speak with us. Want to see the app in action? Here's Nick, one of Smule's engineers, giving it his all on "I'll Stand By You."