MMO MMOnkey: MMOs as conditioned learning engines (Part 2)

In a previous column we looked at some of the many ways that game designers use positive reinforcement and reward in MMOs. Positive reinforcement occurs whenever an action is accompanied by or results in something pleasant or enjoyable. When a person is positively reinforced or rewarded they are more likely to repeat the action that is associated with the reward. The importance of reinforcement for MMOs is obvious; the more the player is rewarded, the more likely she is to play the game.

A deeper understanding of how reinforcement works in MMOs can be gained by comparing an example of where it's done well with a case where it's done poorly and Everquest 2 provides just the examples we need.

EQ2 has a large number of collection quests. Norrath is littered with shinies that sparkle to attract your attention and show a "?" when you mouse over them. You click on the shiny, wait while a "gathering" timer ticks down, and are rewarded with an item like a shattered gnoll bone, some kind of grub or beetle, or a shard of stone. You can also find collectible items that are hidden around the environment or collect pages of books if you see a fluttering page on the ground instead of a shiny. Each item you pick up falls into one or two of hundreds of different collections. When you have all of the items in a collection, you hand it in to an NPC who gives you an experience reward along with either gold or loot.

EQ2 also relies on the familiar bounty hunter quests that are staples of almost every MMO. The collection and bounty hunter quests are virtually identical in structure. In a bounty hunter quest you kill X number of Y bad guys and then turn in the quest for your quest reward; in a collection quest you collect X number of Y items before you get the quest reward. The only difference is that you kill things for one type of quest and pick things up for the other. However, there's a world of difference between how the collection and bounty quests are implemented in EQ2. In terms of how they reinforce the player, the bounty quests are very well designed while the collection quests are terrible. The bounty quests encourage the player to keep playing. The collection quests do not and I suspect they are an underutilized aspect of the game as a result.

Consider all the ways bounty quests in EQ2 reinforce the player and collection quests provide either weak reinforcement or no reinforcement at all. In a bounty quest, you know at the outset what your quest reward is going to be. Every quest mob killed reinforces the player by bringing them one step closer to a known reward. In a collection quest, the game doesn't tell you what the reward is going to be until you get it. When you pick up a collection item you know you are one step closer to something but you don't know what that something is. This is a much weaker form of reinforcement.

When you kill a bounty quest mob you get a message on the screen that you've taken a step toward completing a specific quest that the message identifies by name. You then get a second message that tells you how many of the quest mobs you've killed and how many you have left to kill in order to complete the quest. Both messages are rewards for killing the mob and they both reinforce game play. Killing a quest mob encourages you to find and kill another one until the quest is finished.

How many times have felt driven to kill the mobs to finish the quest before you either quit the game or leave the area where the mobs can be found? Why do you feel that way? The reinforcement built into the game design is part of the reason.

When you pick up a collection item a window pops up that describes the item. If you already have the item, it's name is dull gray; if you need the item, the name is a brighter gray. If you don't know about this difference, or don't notice it when the item window pops up, you have no idea whether the collection item is useful or not. Clicking a button in the item window adds it to your inventory. No message, no nothing, it just adds it to your inventory.

This is a poorly designed game mechanic. If you pick up on the gray scale difference for the item name you get a weak reward for gathering the item. If you don't, you get no reward at all. When you add the item to your inventory, the game gives you no information about which collection quest the item belongs to and no information about how far you have advanced toward completing the quest. You can get this information by opening your inventory and double clicking on the collection items you have gathered. This opens a collection quest window which shows you both the collection the item belongs to and how many of the items in the collection you have gathered. It also gives you the option to add the new item to the collection which will cause it to disappear from your inventory.

Rather than go through an inventory management exercise every time they pick up a collection item, players typically get the information about how each item completes their collections when they're out in the wild and need to make room in their inventory for new loot, or when they're back in town emptying out their bags. Because this information usually doesn't come until well after the item was picked up, adding the item to the collection doesn't reinforce gathering items for collection quests. Instead, it reinforces inventory management.

How many times have you felt driven to find and gather another shiny while playing EQ2? Rarely? Never? Why not? The failure of the game to implement a good reward system for collection gathering and thereby reinforce picking up shinies is part of the reason.

Killing a quest mob entices you to kill another one; picking up a collection item entices you to look around for something more interesting to do. If you look around the game world and find something of interest, no real harm has been done from the designer's point of view. If you look around your room and find something of interest, or start thinking about some of the other games on your hard drive, you may log out. Not what the designers had in mind.

Sometimes people equate learning through reinforcement via the principles of operant conditioning with being programmed like a mindless automaton, and identify activities that are affected by this type of learning with uncontrolled addiction. This kind of viewpoint has more to say about a person's limited or confused understanding of conditioned learning than it does about the effects this type of learning has on what we do. However, the misinformed and paranoid can take heart from the poor implementation of reward based learning in EQ2's collection quests because it implies that game designers, or at least the designer's at SOE, are not shooting for some kind of comic book version of mind control by building games on the foundation of operant learning theory. If the designer's had been aware of even the most elementary principles governing how reward reinforces game playing activity, the collection quests would never had made it into the game in their current form.

On the other hand, the unduly paranoid might not want to relax just yet for the bounty quests in EQ2 are even more sophisticated in their implementation of reward-based learning than we have let on. In addition to positive reinforcement, the bounty quests also use negative reinforcement to keep players playing the game. But we'll get into more of that later.