What on earth is going on with H.264, patents, and video encoding on the web? It seems like ever since Steve Jobs published his Thoughts on Flash the world has gone crazy.
We know what you mean! It's getting pretty silly out there. OSNews just declared that H.264 would be the death of video art and culture because professional video cameras are only licensed by AT&T for personal and non-commercial usage. Terrifying, although most of the creative people we know have continued working free of devastating laser attacks from space.
You're an ass. Really, what's the deal?
Okay, well, as you might know, H.264 is the codec used in everything from YouTube to Flip cams to HD-DSLRs to Blu-ray, and it's the standard Apple's backing over Flash for video on the web. Microsoft and Google are also backing H.264, but neither is being as militant about Flash. H.264 is thus a Big Deal -- it's very likely the future of video on the internet and beyond.
That's got people all hot and bothered, because depending on how you use H.264 you have to pay license fees to the MPEG-LA, which represents various patent-holders that came together to create the standard. We're talking some major industry heavyweights here: in addition to Apple and Microsoft, the H.264 patent roster includes Panasonic, Sony, Dolby, Thomson, and Toshiba -- in all there are 26 companies or organizations listed as holding H.264 patents. (As an aside, Apple has a single patent in the pool, while Microsoft has around 75, and Microsoft says it actually pays more in license fees than it collects in royalties.)
Oh, and AT&T also has some patents on MPEG-4 it wants license fees for, but it's not part of MPEG-LA, and those rates have to be negotiated separately.
That doesn't sound very open. Did Steve Jobs lie to me?
There's a crucial difference between "open" and "free" here. Although H.264 is an open standard, in that it was developed by a consortium of companies and anyone can make and sell an encoder or decoder, it's not free -- you've got to pay for a royalty fee to use it, and the rates are set by the MPEG-LA, which collects payments and distributes them to its members. The basic rate sheet and license terms are publicly available in summary form (PDF) on the MPEG-LA's website -- it's certainly possible for companies to strike custom deals, but for our purposes the public document offers a good baseline.
So what are the rates, and who has to pay?
An excellent question that cuts to the heart of the matter. Remember, patents cover making, using, and selling the patented technology, so the MPEG-LA actually offers two licenses: one for codec developers (who make and sell the patented H.264 technology) and one for video content and service providers (who use it to distribute H.264 encoded content). The rates vary significantly; the yearly royalties for distributing an encoder range from free to $5m, while the royalties for distributing for-pay content are subject to complex rules about distribution but also range from free to $5m. In any event, MPEG-LA has said to us that only the parties at the top and bottom of the H.264 tool chain are generally required to pay royalties; that is, the party who makes the encoder, and the party who distributes the encoded file to the end users. You can think of that as the first and last transaction, if you like -- the person who sells the encoder and the person who sells the content are the ones who have to pay.
That ultimately means products that come with an H.264 codec don't also come with a license to use the codec commercially -- in order to distribute H.264 content in a way that makes money, the distributor has to pay for a separate license. So products like Windows 7, Mac OS X, Final Cut Pro, Avid, and modern video cameras aren't licensed to distribute video for commercial use -- they all have fine print somewhere that says they're for personal and non-commercial use only. It's language that feels incredibly aggressive and broad, especially since it apparently conflicts with the MPEG-LA's general position that only the final link in the chain -- the party selling or distributing the video to the end user -- has to pay royalties for using the H.264 encoder.
Ouch. So people are mad because they have to pay to make and distribute videos for commercial use?
Using H.264 to distribute free internet video to end users doesn't cost a thing, and won't cost anything until at least 2015.
Yes, but it's not as bad as it seems. First off, we've directly asked MPEG-LA whether or not using an H.264 camera simply to shoot video for a commercial purpose requires a license, and the answer is no. We've also asked whether an end user watching H.264 videos would ever have to pay or be licensed, and the answer to that question is also no. Yes, the license terms are worded poorly, but those are the answers straight from the patent horse's mouth. Everyone can breathe again, 'kay?
On top of that, there's a gigantic exception to the license rules that should put any lingering fears to rest: using H.264 to distribute free internet video to end users doesn't cost a thing, and won't cost anything until at least 2015. After that, it's up in the air, and that's a bridge we'll have to cross when we come to it -- there's a chance the MPEG-LA could start charging a royalty for free video in five years. But for right now MPEG-LA president Larry Horn says the group doesn't want to "plug a royalty into a business model that's still unsettled." It's also important to note that the MPEG-LA can't just run off and do whatever it wants; its decisions are made by the various patent holders it represents, many of whom are also still trying to figure out the economics of video on the web themselves.
Yeah yeah. Can they sue me or what?
Even if a license fee for free internet video is required after 2015, it's still the apparent provider of the video that's on the hook for the license, not the content owner or end user -- Google would have to pay the royalty for the YouTube videos it hosts, just Apple now has to pay the fee for the movies it sells through iTunes and DirecTV has to pay for the content it broadcasts. That's a huge distinction, and it's one that all of these companies seem comfortable with -- they signed the contracts, after all. Yes, if you're a pro and you somehow find yourself selling H.264 videos directly to end users you'll have to sign a license and potentially pay up, but hey -- if you're doing that you're running an actual business and you need to go talk to a real lawyer, not a disembodied third person Q&A on the internet.
To repeat the point: as an end user, you'll never have to think about your legal liability over H.264, because there's no need for you to be licensed unless you're distributing commercial content to other end users or building an H.264 encoder. We'd venture a guess and say you're probably using a licensed camera and software and uploading to services like YouTube or Vimeo or Viddler, and that means you're totally in the clear.
Okay, so why all the fuss? This seems pretty simple to me.
Well, not really. This is the internet, where people hate paying for things, and although end users won't have to directly pay for H.264 licenses, the very idea of anyone or any company having to pay a license for a video codec has sparked off a firestorm of controversy, most of which is centered around whether or not H.264 is "free" or "open," and whether or not alternative open-source and royalty-free codecs like Ogg Theora are better for technical, emotional, or moral reasons.
That's an extremely important and valid debate, but things get messy when you start throwing around terms like "free" and "open" without an understanding of what they specifically mean, and they get even messier when you don't look at the actual license terms. And on top of all of that, things get downright heated when you mix in the fact that Apple's making the biggest push for H.264, because, well, people get incredibly irrational when it comes to hating or loving Apple.
Damn straight. But wouldn't an open-source and royalty-free codec like Ogg Theora actually be better? And cheaper?
Sure it would, and we'd absolutely love for a free alternative to win out. But it's not that simple. First, many people believe that H.264 is superior to Theora, which is based on the VP3.2 codec released by a company called On2 in 2000. Second, it's also possible that Theora infringes patents held by MPEG-LA members and other codec developers, and that puts whoever uses it at risk of a patent lawsuit -- Larry Horn has flatly said Theora infringes MPEG-LA member patents in recent months. Nobody knows the answer to that question, since it's ultimately for the courts to decide, but it injects a healthy amount of fear and uncertainty to the mix, and most businesses tend to shy away from uncertainty -- especially when it's about potentially massive legal liability.
So the real choice for most companies is to sign up with H.264 and the MPEG-LA in return for a baseline level of legal protection and broad compatibility with a codec that's been widely adopted in the market, or to go with Theora, save the money upfront and risk a patent lawsuit down the road while shipping a potentially inferior product. Depending on your point of view, that's either quite a racket the MPEG-LA's got going or it's just ruthless tech industry business as usual, but there's the fundamental situation. To say that there are a great many smart people out there with passionate viewpoints on either side of this debate would be a tremendously hilarious understatement.
Now, Google actually bought On2 in February, and there's lots of chatter that it might open-source a codec related to Theora called VP8 at the I/O conference this month, but releasing the source to something doesn't erase any potential patent liability, and VP8 would be way behind H.264 in terms of market adoption. We're eager to find out what Google has planned, but let's be straight here: it's going to take a major, major move for VP8 to have any effect on the H.264 juggernaut. Let's agree to call VP8 a big question mark for right now, shall we?
So why does Firefox support Theora and not H.264?
It all goes back to the licensing terms we talked about above -- in order to ship an H.264 decoder with Firefox, Mozilla would have to pay the MPEG-LA something around $5 million a year.
That's not cheap.
Nope. What's more, Mozilla remains actively opposed to doing anything that would violate its free-software principles, and shipping code that comes with non-free, non-redistributable license obligations definitely goes against that spirit. We can totally respect that.
Yeah. So what to do?
That's a rock and a hard place for Mozilla. For example, Firefox could leverage the H.264 decoders built in to Windows and OS X to play H.264 content without paying the fees, but that gets right back to having content on the web that isn't "free," and that's not something Mozilla seems to be interested in supporting. If H.264 continues to evolve and gain widespread support in the market, Mozilla may eventually have to make an impossible decision between idealism and compatibility, and we're not even going to try and predict how that one will turn out -- remember, we're having this H.264 conversation right now because it's in line to replace Flash video, which is arguably even less free and open.
Okay, this all sounds super complicated again.
Well, it sort of is. But it's not so complicated that the market can't figure it out -- and since it's not end users who are directly paying here, the market is actually the companies and organizations that have to risk real money and real consequences on their decisions. And that market is pretty familiar with patent pools: MPEG-2 has been licensed by the MPEG-LA for 20 years now, while HDMI, WiFi, 2G and 3G cell services, USB, CDs, DVDs, and Blu-ray discs are all standards managed by similar licensing organizations. We would note at this time none of those arrangements have led to the death of any sort of creativity.
More seriously, it's obvious the MPEG-LA and its licensees should be clearer with customers about what's up with commercial licensing and who needs to pay for what. That hidden personal use fine print feels really sneaky, and pro users need to get some clarity on what's required of them -- you don't mess around with people's livelihoods in the fine print, you know? A clarification from the MPEG-LA and some bigger notices at the front of the manuals are needed in short order.
But look, it's not often Apple, Microsoft, Google, DirecTV, Sony, Toshiba and some 810 other companies all back a standard together. Barring some insanity -- and we never bar any insanity -- we'd say H.264 has already won this year's version of the format war.
You talk a lot. So when is Engadget going to start offering HTML5 / H.264 videos instead of Flash?
2200 words later, and you still had to go there, didn't you?