How Spry Fox used (human) guinea pigs to evolve Free-Range Dragons overnight

If you played Spry Fox's Free-Range Dragons at PAX Prime earlier this month, you may have a vastly different impression of the game than players who returned to the team's booth during the show's final hours.

This isn't just a matter of taste, either. Spry Fox actively updated its PAX Prime demo of Free-Range Dragons throughout the expo weekend, making sweeping changes in response to player behavior and feedback. The experiment was a valuable learning experience for Spry Fox, and produced results that will likely inform the project's future direction.

Spry Fox takes player feedback very seriously. Days before PAX Prime, the studio released a major update for its roguelike Road Not Taken, introducing a new difficulty mode in response to players who felt that the initial release was too challenging.

Spry Fox's unique studio setup facilitated quick updates at PAX. During the show, Spry Fox developers demoed the current prototype version of Free-Range Dragons and responded to player questions. Afterward, team members relayed feedback to UK-based programmer Andrew Fray, who then produced a new build of the game for the next expo day while his compatriots in Seattle slept.

"PAX is an awesome place to get real-time, high-value feedback on a game," Spry Fox CEO David Edery said. "We use services like usertesting.com to playtest our games but the people who use those services tend to behave 'abnormally.' (For example, the average tester at usertesting.com carefully and deliberately reads all the text that appears in a game -- most people do the exact opposite and don't read anything, or pay little attention to the few things they do read.)"

Edery continued: "PAX is one of the best opportunities we have each year to observe lots of 'real gamers' of all ages, sexes, etc, playing our games. Additionally, because PAX goers have LOTS of things competing for their time, their behavior is much more likely to simulate the behavior of a future potential customer than a usertesting.com playtester or a face-to-face playtester because those folks have either been paid to give you their time or have volunteered to give you their time to the exclusion of all else, and that can be expected to severely bias their attitude and behavior."

In Free-Range Dragons, players battle flying creatures, steal their loot, and bank collected treasures back at a centralized home base. Spry Fox spent much of its time tweaking the game's difficulty in the week leading up to PAX, modifying a prototype build up until it premiered on the show floor. Edery kept Skype logs detailing many changes made to Free-Range Dragons during the week of PAX, giving insight into the studio's creative process.

"Easy raptors [Ed. note: minor enemies] seem completely ineffectual now Ryan," Edery told teammate Ryan Duane Williams over Skype on the Wednesday before PAX. "Maybe something in between where they used to be and where they are now?"

"Another thing I was thinking is that maybe 10 loot is just a bit too low a threshold for the 'hard' mission, if we want it to be a reasonable amount of training for the high score mode," Edery added. "You said folks were flying through it (which in general seems good, but maybe they are flying too fast. It just takes a couple pieces of loot to hit the threshold)."

"We previously decided 20 was too high, but things have changed since then," Williams replied. "Maybe try 15? We could also reduce the value of the goose loot. Really the values of all the loots need a balancing pass; the hard goose is way more than 30% harder than the easy goose."

The first day at PAX revealed several minor issues, resulting in humorous bug reports like "sometimes spider babies cosy up to you for 2+ seconds without exploding." More serious issues resulted in AI flaws, causing active enemies to remain docile until the player stole their loot. Players additionally reported that UI elements such as charge icons were hard to see amidst the on-screen chaos.

Following Saturday's showing, programmer Fray implemented a new structure for the demo:

Later that night, Edery discovered through on-site playtesting that the "goose" enemies weren't presenting enough of a challenge for players. "I think we've gone too far in nerfing the goose's health," he told coworker Williams. "I killed it in two hits. Was not satisfying. Not sure what it used to be; 10+ ? That was way too much. Cutting it in half to 5 hits seems like a reasonable next experiment."

"Two things changed at the same time," Williams replied. "We cut the goose's health from 30 to 15, and we increased the player's likely strength from 3 to 6. So I guess we quartered it; I'll just restore it to the old value in the missions branch." Edery, Williams, and artist Brent Kobayashi stayed up past midnight discussing this and other proposed changes with Fray.

The next day's build was vastly different compared to Saturday's demo.

Fray additionally tweaked the game's reward structure in between builds, aiming to produce increased satisfaction in players. "The item reward schedule is designed to give a range of emotions: instantly useful, a useless item, an item that's useful later, stuff to build up the weak aspects of the goose, etc." he explained. "Players should have to do a tiny bit of minmaxing as they go through the missions."

By Monday, Free-Range Dragons had evolved far beyond the prototype seen days prior.

Even as last-minute fixes rolled in and the Seattle team rushed to deliver new builds as crowds flooded the expo hall, Spry Fox remained close-knit and motivated. "Go get 'em, fellas," Fray told the team on the final day of PAX Prime. "You are the James Sullivans to my Mike Wazowski."

[Images: Spry Fox]