PAX East 2011: Massively interviews RIFT's Scott Hartsman
With the launch of RIFT still fresh, members of the Trion team packed their bags and flew to Boston to attend PAX East. Even though they didn't have a booth, their "We're not in Azeroth anymore" banner made their presence known. In addition, Community Manager Cindy "Abigale" Bowens hosted a party in Boston to celebrate the launch with players and fans. Massively had a chance to sit down and talk with Trion CCO and RIFT Executive Producer Scott Hartsman about launch day, security, plans for the future, and a whole lot more. Read on for the full interview!
Massively: First off, launch day -- what are your overall impressions about how things went?
Scott Hartsman: It was a whole lot of activity followed by a whole lot of quiet. The beta events and the head start were all very hectic in terms of lots of new changes going out, sometimes major systems being shown to users for the first time, and a lot of stuff being proven out at scale. As things would get proven out, we'd be a little nervous seeing how they worked or didn't, but when the actual launch came around, we had been through several beta events, and every one of those was intentionally orchestrated as a launch. So by the time the actual launch came around, we had 200 people just doing the job they had done seven times before. Getting launch pre-rehearsals was the thing I think that got us the most stability towards the end.
On the actual day of launch, the tenor around the office was unreal. The vast majority of the team got told, "Just play the game." There were a couple of dozen of us watching monitors and population stats and waiting for the other shoe to drop, and it never did. During the first 24 hours, there were no server crashes, which blew our minds. The engineers and execs were ecstatic, players were streaming in and playing, and it just worked.
Server queues seemed very long the first night but then went down dramatically over most servers by the second night of Head Start. How were you able to accomplish that?
Opening up lots and lots of new servers, for starters. If you think about how most open betas work, when you do an open beta, you're opening your game to the entire world. "It's free, come on in and play." That's usually when you get your biggest numbers. After that, the tourists go away, and you end up with a smaller set of people who are ready to pay to play the game. We had the opposite; we had larger numbers of people than we had in open beta. That doesn't happen. We were definitely taken a little by surprise, because we were expecting our population trends to trend like most MMOs, and they didn't. Having more servers on standby, and standing more servers up as we needed, was really helpful. Also, people's natural inclination, if they had only invested 20 minutes in their character, to go somewhere else, helped a lot. As people spread out through the world a little more, that helped a lot more too, because we could raise server caps. And then there's some of the normal play patterns. When a game is brand-new, people will play two to three times as much in a day as they normally would, but their hours-per-day of play settles over time. That's totally normal. When something's brand-new and shiny, you want to be there the entire day.
How do you walk the tightrope of having too few servers at launch, with really long queue times, to having too many servers months later, with the unfortunate prospect of having to merge?
I think MMOs have gotten beyond the point where that is necessarily seen as unfortunate. I think there's too much focus placed on the number of people online at a time in a given world, and what we should really use to track the health of games is, "How many people are playing in a given month?" If you can make a game where enough people are playing in a given month, you're succeeding. So that's the goal for success. Once you define that as your goal for success, it becomes, "How do you best entertain them?" The way our architecture works, we can allocate more hardware to one shard over here and less hardware to another shard over there as needed. But as long as there are enough people around that the game is entertaining and the events are fun, we're pretty happy. If we do end up having to move servers around, chances are what we'd do is allocate more hardware to them and make even larger populations than they have right now. And bluntly, we might even do that in a success case, as we add more content over time. Again, I think we've matured enough to where so much of that can be considered fluid and people just understand that it's going to make their gameplay a better experience.
A big launch can be both a blessing and a curse. Obviously, your popularity right now has made you a main target of hackers/botters/third-party traders. How do you protect yourself and the players from that?
We knew it was going to come on launch day. We were not expecting it to come crazy hard on launch day, to the point that, had our ops guys not had the foresight to pick up the load balancers and the particular networking hardware that they did, we would be in a very different state. We would have been denial-of-serviced off the internet, basically. The hackers came that fast.
Think about it: They've had seven years of practice hacking other games. Every time they hack one of those accounts, they save the account name and the password. Plus, you've got other sites that have had break-ins, that aren't MMOs entirely, and the hackers have no problem running lists of 10 million names against our authentication system. So we've done some basics and some clever stuff. Our spam filter was collecting messages all during beta, and we turned it on after head-start began. The in-game mail filter goes in on Monday, because they moved to mail. And the filter feature has been collecting info the entire time, but we'll start the blocks and the auto-bans this week. At the time head start began, we already had auto-kicks and auto-bans for speed-hackers and teleport-hackers. We've been doing credit card bands on Fraudsters, because hacking and fraud is not limited to inside the game. We're selling the game too, and people have no problems stealing some poor non-player's credit card and trying to buy 260 accounts for the game.
The next thing we're working on is a mechanic called coin-locking your character. If we detect that you're logging in from a fishy location, which is generally the case if you're getting hacked, your character will be coin-locked. Until you authenticate some sort of other way, you will not be able to get rid of anything on your character. You won't be able to sell it off; you won't be able to destroy it.
We're also working on two-factor authentication, via both cell phone app as well as text message. We're not going to go with a separate hardware authenticator, because these days, pretty much everybody has the ability to receive a text message or use an app. So we're going to go out with that, because it's a lot cheaper for everyone involved, and people are more likely to use it if they can download it (as opposed to something they have to order and ship and wait for). And the goal for use is that we want these things to be used.
How do you feel about the leveling speed of the playerbase, especially in light of how there were level 50s on the second day after the head start? Is there a fear of players running out of endgame content if they get there too quickly?
We actually had a pool [to predict the time to 50] in the office, and we did have a winner -- someone who was off by an hour and a half, I believe. I think there will always be people who single-mindedly just want to get to max level to be the first guy to do it. And that's OK. We intentionally did not want to make a punitive leveling experience to try to make those people go slower. We didn't want to make it no fun for everybody else. Right now it's a fraction of one percent of everybody; it's a tiny number of people who actually even made it to cap. Because if you go and you play the game, you do some tradeskills, you do some collections, you do some achievements, you do some warfronts, you do some leveling, chances are you're probably not level 50 yet.
We have a bunch of endgame content in the game. We've got the two tiers of expert dungeons; we've got the level 50 warfront. We just last week introduced the rift raids, and we already have content planned to roll out. I'm not too worried about people running out of content. I think you need to be careful to not try to tune your game for the fastest person; you need to tune your game around where the majority of people are.
You made a post to a link that shares every discovery in RIFT, and you made the FTP files available to everyone in the community. That's unprecedented! What made you decide to do this, and what role do you envision for fansite and app builders?
When we invented the existing soul system, we said, "This thing is prime for someone to make cool web apps or cool mobile apps around the idea of building loadouts." But there were so many abilities that fansites would have had to go in the game, type out all the text, look at all the numbers, and it would have been a big pain. And we could have left it there, and yes, some things would have been built. But instead, we have an engineer who's really passionate to get some of this out to make life easier for people so they can focus on cooler features, like a soul builder that has the ability to mail and share links. You don't have to worry about spending all this time scraping data; you can just worry about building cool features.
It began with the soul system, and then as a test when we did the server population list, we gave people the ability to see what the queues were, and we intentionally left a tiny little "get the XML" link at the bottom. We didn't expect many to use it, but we figured if we put it out there, chances are someone's going to build something that's going to use that XML, and sure enough, within a day, there was a "check your server status" app.
So we spent some time, took all of the info that everyone can already see in tool tips, and we said, "What else can people build? Let's find out." It's something that we're really excited about, and next up is making sure it can update itself every day.
Scott continues the interview on page two!
