State of DPS in Firelands
Patch 4.2 is winding to a close, patch 4.3 is on the PTR, and we've recently seen Ghostcrawler (lead systems developer) making some public comments about DPS balance in the Developer Watercooler -- in particular, discussing the increase of the AP buff for melee classes. All of this makes it a great time to take a look at the state of DPS among all of the specs.
Of course, we don't have nearly as much data as Blizzard has access to; however, we do have better data than ever before. In particular, the best data source we have comes from raiders who upload raid parses to World of Logs, which then makes these tens of thousands of parses available to all of us. Next, we have Raidbots, a site that automatically parses this data for us, letting us very easily sort through those massive numbers of raid parses.
One of the things that I think too many players get caught up on with this data is looking at the rankings -- and I've certainly been guilty of this myself. It's just so easy to look and say "Hey, my class is ranked 10 of the 22 specs." But of course, that's not terribly helpful -- after all, I don't care if I'm ranked 21 if I'm only 50 DPS behind number one.
Join me after the cut as we take a look at a different way of examining the data to see how all the DPS specs are performing in Firelands.
%Gallery-135598%
About the data
Before we look at the pretty pictures, we have to discuss the methodology for data collection and what the strengths and weaknesses are. I know this is long and boring, but you are not allowed to complain about anything in the comments if you haven't read all this. It will help keep you from saying something foolish.
All of the data here is coming from the median DPS of the top 100 parses. I prefer to look at the median rather than average for DPS parses because it's pretty common to have outliers up on top that really skew the average. For any given data set (25-man heroic, 10-man Baleroc normal, etc.), I'm taking the median DPS of the top 100 parses for each spec. Once I have all of those, I then calculate the median DPS of all specs. Then each spec is charted as its percentage above or below that median.
Now let's talk about the strengths and weaknesses of these methods.
Advantages of top 100 parses
When we look at the median of the top 100 parses, we know that we're seeing players who have very good gear, are incredibly skilled, have all raid buffs, and are also enjoying good RNG, to boot. It's a good representation of how much DPS a spec is capable of putting out in actual in-game content.
Looking at the top 100 parses has three big advantages in my mind:
We're comparing apples to apples. We're looking at players with similar gear, buffs, skills, and luck levels across all classes -- specifically, very high levels of each.
In some fights, certain classes have odd jobs like kiting, interacting with mechanics, etc. With the top 100, we can be pretty certain we're looking only at examples of players who were not doing anything to hamper their DPS. These are the lucky guys who didn't have to kite or pull the lever.
Because the top 100 is, in fact, the best performance players have produced for their class, we are looking at a true potential of the class in real content -- much more accurate than any spreadsheet or simulation. In general, class DPS is balanced around the potential (after all, if you're doing substantially less, you don't need a buff -- you just need better skills, gear or buffs). Don't get me wrong, average DPS matters too, but that should be more about balancing spec difficulty and scaling than DPS output.
Disadvantages of top 100 parses
The primary disadvantage of looking at the top 100 parses that we have to keep in mind is that it disproportionately favors the best DPS spec of any given class. The very best players in the very best gear are typically going to play the very best DPS spec for the fight. If there is a spec lagging behind, far fewer of the best players are playing that spec, and so its numbers fall even further behind.
It's very important to keep that in mind with this data. If a spec is 5% DPS behind, in reality, all of the top players will ditch it for most fights. Then because we have a lower tier of player skill, gear and buffs playing that spec, it looks like it's actually 15% behind.
In general, if a spec is doing substantially worse, it's because that spec does worse. But the amount worse is going to be exaggerated in this data.
Why deviation from median?
I like to sit around and try to think like a game designer from time to time. And if I were a designer, I wouldn't care a whole lot about spec rankings or the Raidbots spec score (which also gets skewed by outliers and doesn't show your relationship to the middle of the pack). As a designer, the ultimate (and unobtainable) goal is to have every spec do about the same DPS in current content. Sure, one will be stronger with these mechanics, but others will be stronger with those mechanics and it balances to everyone doing about the same.
What really matters is not what ranking order the specs are in, but how far away from that center line they lie. If a spec is doing 3% more or less DPS than the median, then I think they're looking pretty darned good, whether they're #1 or #15. To me, the question is how far away from that median each spec is, on average, in the actual content. I'm still not sure how far is too far. I definitely think that anything within 5% is pretty darned balanced. But how far away do they need to go from that median line before needing correction? 10%? More?
It's worth noting that this is a different way of looking at things than the way players usually do. By definition, half the specs will be above the median, but players tend to really want only the top handful. Most players look at numbers and if they see their class is consistently, say, 15% above median, they'll think they're doing great but not overpowered -- after all, that other class is even better in this fight or that fight. But anyone who is 15% below that median will be crying and complaining about being ignored and unable to play their class (though you'll rarely find a class doing that poorly -- there's usually at least one spec that's near or over the median).
In reality, both of those situations are equally problematic from a pure design theory perspective. So how far below the median should a spec be before it needs to be buffed? 10%? 15%? Because that's the exact same amount above median a spec should be before getting nerfed.
At any rate, enough explanation -- let's get to the numbers in part 2.
