MMO MMOnkey: The rat Skinner, or negative reinforcement in MMOs

Like the wardens in Everquest 2 or the druids in World of Warcraft, reinforcement in MMOs is not always what it appears to be. In an earlier column we wrote about how positive reinforcement, as defined by behaviorist learning theory, occurs whenever the player receives something beneficial and pointed out a number of the simple ways this kind of reinforcement is used in MMOs. A second column used the bounty and collection quests in EQ2 as examples of how positive reinforcement could be well (the bounty quests) or poorly (the collection quests) implemented.

Several Massively readers who enjoy the collection quests took us to task and pointed out some of the ways EQ2's collection quests can be rewarding in terms of gold, experience and loot. They also made the point that bounty quests, like hot elf chicks, are old, tired and overused as staples of MMO play. I mean, really, who wants to kill another 20 rats for Fatboy the quest giver? Not me, and I'm guessing many of you don't either.

Although comparing EQ2's bounty and collection quests was never about whether or not the collection quests are enjoyable or produce valuable quest rewards, the comments on the column illustrate a common misconception about reinforcement-based learning and, more importantly, provide us with a valuable path to understanding just how powerful reinforcement can be in an MMO. The misunderstanding stems from the terms the behaviorists used to describe their findings. They talked about positive reinforcement as "reward" which carries with it the implication that the player who is rewarded finds the experience pleasurable or rewarding in the sense that it provides a sense of fulfillment or happiness. Quite often this is the case in an MMO when we attain the next lever or are rewarded with a useful and sought after piece of gear. However, reward as a form of reinforcement doesn't need to be actively enjoyed by the player to be effective. It need only be perceived by the player to be beneficial.

If you keep in mind that reward in the reinforcement sense need not be enjoyable, you can see how the observation that bounty quests are often boring drives right to the heart of how powerful reinforcement can be in MMOs. EQ2, and many other well-designed MMOs, go out of their way to make sure we are told that killing that rat right here, right now was beneficial to us in advancing our bounty quest. Okay, so we understand that each dead rodent in the pile at our feet helped us, but that doesn't make the bounty quest any more enjoyable or less tedious. Some people enjoy bounty quests while others hate them. But almost everyone does them. We don't have to do bounty quests – there are usually plenty of other ways to level – but we do. If they're no fun, why do we do them?

Part of the answer, at least as far as the EQ2 bounty quests go, is that they do a superb job of positively reinforcing the player. You kill the rat and you get an automatic message telling you the wee corpse is a quest item. That's reward in the positive reinforcement sense. It doesn't matter if killing the rat didn't make you feel all warm inside or give you a tumescent sense of personal uberness, as long as you understand that the dead rat is of benefit to you, you've been rewarded and rat killing has been reinforced. You then get a second automatic message telling you how many rats you've killed and how many you have left to go before the quest is complete. That second message is also a reward (two rewards for one kill!) but it's more than that. The second message provides negative reinforcement as well as positive reinforcement.

When talking about reinforcement "positive" and "negative" don't carry the implication of "good" and "bad"; they have the sense of "addition" and "subtraction". Reinforcement of any kind makes it more likely that an activity like killing a rat will be repeated. Positive reinforcement happens when the player is given something beneficial for killing the rat; negative reinforcement happens when something unpleasant or unwanted is removed from the player for killing the rat. The second message in EQ2 positively reinforces the player by telling them that the number of rats they have killed has increased by one. It also negatively reinforces the player by telling them that the number of rats they have left to kill has decreased by one.

Look how sweet this is from the game designer's point of view. The player who is enjoying the quest is positively reinforced by being told they have been successful one more time. The player who hates the quest is negatively reinforced by being told they are one step closer to being done with the damn thing. Both types of player are likely to keep killing rats because both are being reinforced. It's an insidious, but brilliant, use of reinforcement to encourage the playing of MMOs.

There are many design mechanics like this in MMOs where positive and negative reinforcement can be seen as two sides of the same coin. An almost universal example is the experience bar that tells you how close you are to making level. If you focus on the filled part of the bar, you are positively reinforced when you notice it has increased; if you focus on the empty part of the bar, you are negatively reinforced when you notice it has decreased. So, are you a glass-half-empty or a glass-half-full kind of person? Either way, MMOs that effectively implement the basic principles of positive and negative reinforcement keep you playing the game.

Sometimes game designers use negative reinforcement by itself, but when they do, they have to be careful. Negative reinforcement encourages continued activity through the removal of something unwanted. In order for a game to use negative reinforcement alone, it must impose some type of penalty on the player. The player is then negatively reinforced for the actions that lead to the removal of the penalty. Game designers love things like death penalties and corpse runs for this reason. The player is saddled with something they don't like - being left naked and alone in a world full of slavering, hungry monsters who think you taste like chicken in the old-school games, or suffering a stat or experience-gain penalty in the more player-friendly environments of today – and then the game reinforces the player for carrying out the actions that remove the penalty. The designers have to be careful because players don't like to be penalized. We'd much rather be positively reinforced for killing rats to gain experience than be negatively reinforced for killing rats to eliminate a death penalty. Either way, however, we keep killing the rats.

That muffled sound you hear faintly in the background is B.F. Skinner laughing in his grave.