SXSW 2009: 'Funologists' and unorthodox research methods
In the panel "Funologists Live & In Person: Guerrilla Game Research," a group of experts took turns explaining methods for researching user experience in games that goes in some way beyond the normal focus groups and QA cycle. The often ad-hoc methods provide valuable data about gameplay at a fraction of the budget of most data gathering exercises. And sometimes they involve becoming a frog.
Erica Firment, User Experience designer on Second Life, discussed a rather direct method for gathering info from users: hanging out in Second Life and watching -- "hiding in the audience" as an alternative to expensive technology. This is where the frog form comes in -- Firment adopted a frog avatar and sat on a roof in the new user area, watching people figure the game out and talking to new players about their difficulties with the interface. Firment produced a flow state graph, demonstrating the delicate balance between accessibility and difficulty. She also brought up other easy, inexpensive methods of gathering gameplay data: watching blogs and Flickr feeds related to the game and search terms tied to it. If there are bugs, they will show up in people's pictures and searches.
Big Fish Games' Julie Ratner described some methods of "emotional inquiry" that the company conducted for the Mystery Case Files games. In these studies, contextual information about people's feelings about the game was gathered by gauging reactions to such abstractions as color palettes and word lists. The company had informants create collages and arrange word lists to try to analyze their reactions.
Jason Schklar, who most recently worked with From Software on Ninja Blade, provided an anecdote about research that led to control alterations in Crimson Skies. The PC-experienced developers had apparently designed a game for very smooth, nuanced movements, and a trip through the testing room revealed a cacophony of clicking joysticks and crashing planes, indicating that the players were making rapid, drastic moves rather than slow, methodical ones. The controls were then altered for a non-flight-sim audience.
USC's Tracy Fullerton then went into the challenges of doing user research for student games in the Game Design Workshop. She described a "funnel of funology" -- a sudden rush of game projects, and a single researcher to gather data related to all of them. To ameliorate this problem, she suggested solutions including self-testing exercises and education related to gauging user experience.
We know that developers and publishers put a lot of money into researching user behavior in games, and that they have to do so to provide a mass-market experience. We just never considered how much creativity could go into such a seemingly dry task.
