How Blizzard mishandled the BlizzCon ticket situation

As you may or may not know, we here at WoW Insider are not an official Blizzard fansite. There are a few different reasons for that, but one of them is that within the Fansite Program Code of Conduct, there is a clause that states, "fansites should present content that is supportive of World of Warcraft and Blizzard Entertainment." We don't disagree with that clause -- fansites are run by fans, and they should support Blizzard. But our status as an unofficial site leaves us completely free to talk indepth about situations where Blizzard has messed up big time. And as many players already know, the BlizzCon ticket sales process that took place earlier this week is definitely one of those situations.

Blizzard is, of course, a game company. No one expects them to put on events like WWI and BlizzCon -- they do so to serve the community that's grown up around their games (and, let's be fair, market and advertise their products to the core of their fanbase). And the community loves those events, both hearing about and attending them. Which is why it was a surprise to no one (except maybe Blizzard themselves) that when the ticket sales kicked off Monday morning, it was a nightmare -- the site was hammered by fans trying desperately to buy tickets, the Failoc was a familiar sight, and within a few hours, even Blizzard.com's main site was down.

Everyone could have predicted that there'd be problems like that -- when a fanbase of 11 million tries to buy 12,000 tickets, of course you're going to have technical problems. But Blizzard's mishandling of the situation didn't happen on Monday morning -- anyone can suffer from server outages. It happened over the next two days, days full of frustration, endless page refreshing, and a lack of useful communication from Blizzard about just what was happening.

What we're going to do here is basically document exactly what happened over the last two days, and where all the mistakes were made (and there were plenty). Before we start, though, we should state that we here at WoW Insider are still Blizzard fans -- they're a great game company, and all of us here on the site are happy to pay every month to play a game that we love. But Blizzard, and its fans, should know exactly where the mistakes were made in this process, so that in the future, this doesn't happen again.

Monday

Hindsight is, as they say, 20/20, and looking back now, Blizzard should have been more prepared. The first bit of frustration around this process resulted not when the sites went down, but on Monday right at 12:01am PDT -- many players, having been told that tickets would start selling on August 11th (and not given a specific time), started refreshing the page and the store right at midnight. But they were disappointed for the first time -- the ticket sales page didn't go live until around 6:30am PDT. And even then, Blizzard didn't seem to have itself coordinated: most of their employees didn't show up until around 8am, which means that even while the store was dying, we had no communication on what was happening until two hours later.

Blizzard did make one good decision here -- from the beginning, they only sold their tickets through the website, not over the phone. By all accounts, everyone reached by phone at Blizzard (the lines were often busy, and a few people reported that the numbers had been disconnected a few times) was extremely nice about the situation, and though they couldn't sell tickets, they were extremely forthcoming about the problems. Choosing to sell tickets through a broken website may have been a bad idea, but choosing not to sell them over the phone was a good one -- with only one outlet for ticket sales, everyone was given a fairer chance.

But perhaps the biggest mistake Blizzard made came at about 10:51am PDT -- Auryk posted on the forums that no tickets had yet been sold. Unbelievably, after all the refreshing and frustration, ever since midnight the night before, not one single ticket sale had gone through. At this point, Blizzard should have shut the system down completely. They should have realized that it wasn't yet working, they should have posted that ticket sales would be postponed, and they should have gone and found a system that did work. When, after almost five hours of outages and problems, they hadn't sold a single ticket, they should have thrown in the towel right there and saved us all a lot of time.

Unfortunately, they didn't. They then pulled up the maintenance SCV for the first time, and spent the late morning and afternoon making changes to the website. We don't know what those changes were, obviously, but we do know that they slimmed down the pages (to keep bandwidth at a minimum -- pictures and some text were removed from the Blizzard store sales page), and we assume that some work was done on the credit card approval process, since that was the major blockage during the period where ticket sales weren't happening at all.

Finally, at about 1:40pm PDT, the store went back online, and ticket sales started squeaking through. Let's be clear again: even though people had been refreshing the page since midnight the night before (since Blizzard had neglected to give a clear sales start time), no tickets were actually sold until more than 13 hours later. Because of Blizzard's miscommunications and neglect in making sure their ticket service worked before putting it live, everyone who attempted to buy tickets before about 1:40pm Monday afternoon was wasting their time.

At 1:40pm ticket sales began, and the site was hammered. People received many, many Failoc screens, and many XML and CSS error screens (it is likely, though this is just speculation, that the CSS and XML pages for the store were kept on a separate server than the main page, and it was that separate server that was failing, not the main server). A few Apache error pages were seen, which means the server was being restarted or the necessary files were offline completely. Finally, around 6:30pm PDT, the store was taken offline for maintenance again, and it was returned to life for a few more sales around 9pm.

We should say, too, that during this time on Monday afternoon, Eyonix did finally start providing 30 minute updates on the forums. Of course, this is what Blizzard should have been doing the whole time (and on Tuesday, we saw almost no updates at all, which lead to even more frustration), but while they were coming out, some guidance from Blizzard on what happening was much better than hearing nothing at all.

At around 10pm PDT, Blizzard made another good decision, and decided to take the server offline for the night so that people didn't spend all evening refreshing their browsers. A few people didn't get that notice (and, we're told, did stay up all night), but we reported it here, and Blizzard reported it in a few places, including on the main WoW page and on the BlizzCon page, so they did do their best to get the word out.

Unfortunately, those notices are now deleted off of their respective pages, so there is no trace on Blizzard's official sites outside of the forums that any of this happened.

Tuesday

What they didn't tell us Monday night was the exact time ticket sales would start the next day. We were told it would be at the start of business, but even though Blizzard's offices usually open around 8am PDT, we didn't hear until 10:30am PDT that they would start selling tickets around noon PDT, which means there was another hour and a half in there at least that people were refreshing the maintenance SCV for no reason at all, simply because they hadn't been well informed by Blizzard.

Right after noon PDT, the store came back up, with the usual errors and problems. Ticket sales were happening, however (it was during this period that I was able to grab my ticket), and they kept happening for almost an hour -- at 1pm PDT, the store posted a "Sold Out" notice, and no one was able to buy tickets.

Once again we heard nothing from Blizzard. Representatives on the phone said that tickets were not sold out, and that they would go back on sale again, but there was no news posted on the forums or on the main pages at this time. At 1:30pm PDT, tickets started selling again -- some phone reps claimed that Blizzard was going to sell the tickets in waves, to try and deal with the demand. At 1:40pm PDT, the "Sold Out" notice came back up, and actually cut a lot of people trying to buy tickets off in the middle of their sales, with the message "The availability of one or more products in your cart have changed while you were shopping."

At this point, without any communication from Blizzard, the rumors began to fly. We heard that only a fraction of the tickets had sold out so far, or that the tickets were still being sold in phases. The phone staff told us that the CMs were being told to post, but that they hadn't yet. We heard nothing at all until 6pm PDT (which gives us another four hours of seemingly unnecessary refreshing and worrying), at which point Blizzard said the majority of tickets had been sold out, and that they would have one more reserve at 8pm PDT.

And as you know if you stayed up last night, that lasted all of about 20 minutes, with all of Blizzard's site at one point completely going down. Blizzard, apparently not prepared to start ticket sales on Monday morning, was apparently prepared for a sellout, as notices appeared on the forums and on the main site within seconds of the sellout (each notice of a sellout mentioned the DirecTV deal as well, which some would-be ticket buyers were offended by).

Which brings us to now -- all of the previous notices of ticket problems have been removed from the main sites, leaving only the fact that the event sold out, with links to the DirecTV service for those who didn't buy tickets. And though it's true that the credit card used to purchase the tickets must be there at time of pickup, that hasn't stopped scalpers from selling the tickets for hundreds of dollars on sites around the Internet. We don't have any idea what the percentage of people reselling the tickets is, but it's very clear that while some of Blizzard's fans were left out in the cold, others only looking to profit off of the ticket sales were able to grab quite a few.

Overview

Blizzard's mistakes during the debacle basically revolve around two issues: technical outages and the communication to us about them.

Outages were to be expected, of course, but Blizzard made mistakes that made them even worse. Apparently, since their sales system didn't even work from the beginning, they hadn't once tested sales before they went live. They were using a new Blizzard store (with the previous events, tickets were sold directly through Blizzard's website), and it clearly hadn't been optimized for large numbers of sales at the same time -- why weren't the steps they took for optimization later in the sales process taken beforehand? As many people trying to buy tickets have noted, it's extremely easy for Blizzard to outsource ticket sales through a company like Ticketmaster or even a smaller group like Brown Paper Tickets. While there are of course service charges and other expenses associated with using such a service, almost anyone, we believe, would say that those charges are inconsequential compared to the time and frustration wasted with the process as it was.

And speaking of the time wasted, let's break it down. Counting all the time spent refreshing, the whole process actually lasted about 44 hours -- from midnight Monday (when people started trying to buy tickets) to 8:15pm Tuesday night, when tickets officially sold out. Of that 44 hours, tickets were only actually on sale for just over six hours -- during a four hour period on Monday, a short time Monday night, two periods on Tuesday afternoon, and fifteen minutes on Tuesday evening. And perhaps most disturbingly, during that 44 hours, there were about eight whole hours where people were refreshing a page that couldn't actually sell them tickets (the first period on Monday before we were told that tickets hadn't been sold yet, and the period on Tuesday where we were getting the sold out notice but were unsure of whether tickets were sold out or not). And that doesn't even count the period on Monday and Tuesday mornings where people got up early, unsure of when tickets would actually go on sale. Though everyone suffers from technical outages, Blizzard increased the time we wasted trying to buy tickets with even more mistakes.

Which brings us to Blizzard's second problem: communication. We've talked about Blizzard's problems with communication before, and never are their issues with communicating to the fanbase more clear than here. At their root, Blizzard's problems with communication have a reasonable foundation: as a company, they don't want to tell anything to their players that might not end up being true, so they keep everything to themselves as much as possible -- game announcements, release dates, and so on -- until they're sure about what to say. And that was fine when Blizzard was a company that only released quality games every few years. But Blizzard is a much bigger company now, their fanbase has grown to be global (not to mention busy), and occasional communication, even every few hours, just doesn't cut it, especially during a situation like this.

The best bit of communication we got during this whole period was on Monday afternoon, when Eyonix started posting 30 minute updates about where the store was at and what was happening. Unfortunately, those updates didn't come at the times where we most needed them. When they first discovered they weren't actually selling tickets on Monday morning, Blizzard should have pulled the plug ASAP and informed us that we were refreshing a page that couldn't help us. When the "sold out" notice first appeared on Tuesday afternoon, we should have been told right away whether tickets were coming back, whether the sold out was official, or whether it was just a CSS error and that we should have kept trying.

By the end of the few days, Blizzard figured it out -- they had "sold out" notices ready to go at 8:15pm on Tuesday, and they posted it in as many places as possible. We only wish they'd been so forthcoming throughout the past two days.

So players are frustrated -- what now? If you choose to quit the game, that's your decision -- we definitely agree that as a customer service situation, this was nothing short of a debacle. We doubt it'll kill their game, but we have no idea how Blizzard might make this up to their community -- keep in mind that the people dealing with this nonsense over the past few days were also Blizzard's biggest fans, willing to travel cross country and even around the world just to participate in their event. Selling more DirecTV subscriptions doesn't really cut it as an apology.

But the fact remains that Blizzard is great at making games, just really, really bad at coordinating things with their customers. If they decide to do this again next year, there need to be obvious solutions to these two problems: if Blizzard can't keep their site up, they need to hire someone who can. And they need to let us know what's happening when -- the coy attitude of "soon" is fine when you're selling a computer game every few years, but it doesn't work when people are trying to coordinate trips and spend hundreds of dollars on tickets.

Even though we're not an official fansite, we remain fans of Blizzard. We'll have a great time at BlizzCon (and if you didn't get tickets, of course you can depend on us to bring you everything you'd want to see there and then some), and we'll keep playing their game. But just like Blizzard has learned from all of their patches implemented in WoW so far, they need also to learn from this experience, and make the necessary fixes for next time. And they need to figure out a way to remind their core fans, some of who pointlessly waited for up to eight hours trying to buy tickets, why they were so excited to spend so much money on the company in the first place.

Update: Blizzard's Mike Morhaime has issued a formal apology for all the trouble, and says they have made 3,000 more tickets available, to be sold to Blizzard members only by a lottery system. Morhaime says that he and everyone else at Blizzard share "the frustration and disappointment that many of you have expressed," and that Blizzard will "do what it takes to avoid this type of situation in the future."