Developer round table: MMO betas, page 4

Craig Alexander, Turbine VP of Product Development

What defines a successful MMO beta?

• Targeted testing (directed by our Game QA staff)

• Realistic play patterns (enabled by Character Copy from the live servers to beta servers)

• Effective feedback mechanisms (direct feedback to QA, player submitted bugs in-game, and our forums)

• Responsive development (fast and iterative – in our case weekly deployments to the Beta world)

• Player enthusiasm (building the "hype meter" through game improvements, responsive changes and direct attention... build your advocates)

Conversely, what are some hazards to avoid when testing?

• Reflect the player population (if only one type of player tests, your feedback will be skewed)

• Trust but verify (embed developers with the players to see what they aren't saying, or you risk living in an echo chamber / hearing the same voices again and again)

• Burn in the testing (some issues just take time to surface, or player understanding of the feel and balance of the system – allow for that time)

• Maintain momentum (time available for the Beta must be balanced with the effort required to maintain enthusiasm – through Beta events, updates and new invitations)

How do you think the beta process has evolved in the last year or two?

• Testing tools have gotten much more sophisticated (our ability to analyze game data, and player tools to provide feedback)

• Player expectations are higher (even in a Beta, tolerance for "broken" or incomplete content is much lower than two years ago)

• Player communities are larger and more varied (requiring us to get more deeply embedded to get clear feedback – there is never only one answer)

• Competition for time is a reality (Beta now competes with the live game, other live MMOs, and other live MMO Betas)

How do you think the beta process could be improved?

For us, much of this comes down to tools (speed and efficiency)

• Tools to summarize and segment player behavior (daily or on-demand reports)

• Tools to prepare characters for specific, targeted tests (quickly)

• Tools for players to give specific, immediate feedback (that can be filtered or rolled up quickly)

• Tools for developers to identify and analyze players in a problem state (we have character copy and "ninja" to facilitate this now)

• Effectively qualifying Beta players is a key step we could refine (based on play patterns, interest and enthusiasm)

End-to-end, detailed planning is an important step that contributes to an effective Beta.

<< Page 3 of 4
1234