Ask Massively: Reviewing and re-reviewing MMOs
Back in September, Star Trek Online Producer Daniel Stahl gave an interview that proposed to tell game journalists how to do their jobs. That's only fair; we're always telling developers how to do their jobs, right? Stahl told [a]listdaily,
"The whole game rating business doesn't necessarily do a great justice to MMOs. MMOs are designed to grow over time and get better with every major release. It might be better if sites like Metacritic could find a way to rate MMOs by releases instead of just the initial day one . . . There are plenty of MMOs that have made huge strides since day one and some that have even gotten worse. Until then, we will continue to offer the game for free and ask for people to try it out and decide for themselves."
Quipped Massively reader Matthew12, "If only there were MMO gaming blogs and websites that keep up to date with the MMOs and their updates... oh wait; there are."
For as long as I've worked at Massively, the site has eschewed formal reviews and ratings. It's not a top-down policy; it's just the way we've always done things and the way that's always made the most sense to us.
Ratings, for example, bring with them a whole slew of problems. How do you assign a number to an entertainment form that can take months or even years to fully grasp and complete? We're not talking about a two-hour movie or a 20-hour single-player title that really has but one path to the finish. How do you overcome our natural inclination to misconstrue what the numbers or stars or whatevers mean? Is a 5/5 an A or an A+? Is a 5/10 an average C or a failing F? How do you avoid number-creep and the desire to compare one title to another and push the rating up just a tad until everything is a 99/100?
But Stahl is right in saying that reviews themselves are a problem, even when they don't come with a handy score you can plug into Metacritic. MMOs change over time. We get angry when they don't change and improve over time. Heck, many of my favorite games were pretty wretched at launch and needed a few more years to bake before I fell in love with them. Nevermind the fact that no one person can legitimately "review" every imaginable launch feature in an MMO in the timely manner expected by readers!
So traditionally, Massively has stuck to impressions and re-impressions pieces. We stream older games; we cover older games in Rise and Shiny and The Game Archaeologist and even in Choose My Adventure on occasion; we cover our favorite long-running MMOs in dedicated game columns; and we check in on the most popular MMOs for their birthdays and anniversaries. We're even working on a new column specifically to revisit games that have fallen off our collective radar. Once you realize that so much of our news is about games that are already out in the wild, you have to admit that a site like Massively is primarily focused on updating its readership about existing games and their patches and expansions. Even if we don't do formal reviews or hand out ratings, we are already exactly what Stahl is proposing -- just not necessarily for the reasons he proposed it.
Commenter JamesKephart countered Stahl's complaint with a really good reason not to re-review old MMOs:
"I really have a problem with the concept of giving MMOs a free pass to officially release a substandard title and give them time to fix it. Granted, an MMO is more complex than your latest generic brown military shooter, but if some AAA studio released a game that was half as buggy as some MMO launches, they would get poor reviews and that would be that. For games like STO to ask for the system to be changed just for them is to say that they deserve special treatment. MMOs can change over a long time, but so can all the single-player titles. With DLC and free patches, many titles have changed notably in their lifetime. Essentially, if you can't put together a decent enough game to grab people initially, then you really don't deserve a free second review when you put a new coat of paint on your outhouse."
There's also the tl;dr version by lucidrenegade: "If you wanted great reviews, you should have shipped a product that earned them." That might not be realistic, as reader Porculasalvania argued, as "sometimes you have to launch with what you have before you run out of money," but I think James and lucid are on to something.
The truth is, the studios don't "deserve" any coverage at all. When we take a second or third or five-millionth look at an older game, we're not doing it because studios think we should or because it's only fair or because the developers earned it or because it will help make Metacritic reflect reality. We're taking a second look because our readers care about it, because our readers can benefit from the updated perspective and information.
Kudos to Stahl for wading into our comment thread; he is right that Metacritic handles digital and F2P MMOs unfairly and that the reviews industry is really screwing over our favorite genre. But the solution isn't more formal reviews and updated scores, not when it's the existence of those things that does a disservice to the depth and scale of MMORPGs in the first place. The best thing we can do is not participate in the broken system. Ignore Metacritic. If you want to know how an older MMO plays, read an updated impressions piece from an actual MMO site, preferably one that doesn't sneer at anything not released in the last year. And then, as Stahl said, maybe just try the darn game. In the world of F2P, you have very little to lose.
What should you play? Where is the MMO industry headed? How does Massively operate? Has Lord British lost his marbles? Why is there no edit button? Should "monoclegate" be hyphenated? Editor-in-Chief Bree Royce submits to your interrogations right here in Ask Massively every Thursday. Drop your questions in the comments below or ping us at ask@massively.com. Just ask!
