Evidence Based Gaming

Scholarly EvidenceAs some of you may be aware, I’ve returned to school to get an MBA. This particular MBA program comes with a heavy workload. I haven’t had time to play or blog in months. Not being one to quietly acquiesce, I’ve tried to find opportunities to combine education and my passion for MMORPG’s.

One such opportunity came in my Organizational Behavior (hereafter, OB) class. OB, in a sentence, is the study of the behaviors of individuals and groups within the context of an organization (rules, tasks, hierarchy). The focus of both OB and MMORPG’s is “teams completing tasks” and from this perspective, OB is very relevant to MMORPG’s (and vice versa).

Bearded game guru Raph Koster’s book, A Theory of Fun examines game “fun” from the perspective of learning theory, or perhaps neuropsychology … the brain is designed to seek opportunities to learn and it rewards us with a fun inducing burst of chemical goodness when we learn a game’s patterns.

The brain is much more than a simple learning machine: it evolved a host of functions under pressure over hundreds of thousands of years. How a brain responds to an input is a complex function of past experiences and its evolutionary heritage. As a result, human behavior is complex and not always straight-forward. OB aims to understand this nuanced behavior in the context of a group and organization.

I’m not the first person to suggest a connection between OB and game design. Maslows’s Hierarchy of Needs is one popular theory that has been cited as relevant to game design (Google it, you’ll see!). While Maslow’s Hierarchy is an important early theory, I am taught that there is little empirical evidence in support of it and that there are newer theories that better explain motivation and that do have the support of research.

As a student, I have access to handy electronic databases of research articles on OB. Many of these papers are evidence based and sometimes this evidence contradicts so-called common-sense or historically held truths (like Maslow’s Hierarchy). I believe game design could benefit from use of current research in Organizational Behavior.

There are today researchers using MMORPG to study OB. My OB Professor was kind enough to permit me to substitute one such OB/MMORPG study for one of the required “1-page” course assignments. The goal of these “1-page” assignments is to review a current OB research article and answer five questions, all within the space of a single page.

The article I chose, “Collaborate and Share: An Experimental Study of the Effects of Task and Reward Interdependencies in Online Games”, by Choi, Lee, Choi and Kim, looked at the impact of the interaction of reward and task inter-dependence on “fun”.

Some tasks are highly interdependent: a raid requires the tight choreography of a group and everyone must play a specific role, or the group fails. Some tasks are not interdependent: a pickup-group clearing trash mobs in an open area is not task interdependent. Even though the players are grouped, they are acting independently: the failure of one member of the group likely has little impact on the others. When a player in a group loots a fallen mob, that is an individual reward. If all loot were pooled and evenly split, the individual rewards become a group reward.

Choi et al., concluded that “fun” is maximized for interdependent group tasks when the rewards are group-based. In other words, group tasks with individualized rewards are less fun than they could be. The results are consistent with larger research studies outside of MMORPG’s (see Wageman, R., & Baker, G. (1997). Incentive and cooperation: the joint effects of task and reward interdependence on group performance. Journal of Organizational Behavior).

Many MMO now include loot distribution functions for low grade loot. Distribution of high end rewards, especially Epic rewards, remains problematic. Individual looting rights in a group are sometimes a source of conflict and can reduce game enjoyment. Game-external systems like DKP have evolved specifically to address this problem.

“So what,” you might say. “We don’t need OB research to tell us that Epic loot distribution is problematic. We knew this simply by watching players set up DKP systems.” Watching players establish DKP systems might lead us to conclude that we need tools to better support the division of high-end rewards amongst high-end groups. The OB research leads us to conclude that perhaps there is a better kind of reward to give in the first place: rewards that benefit each member of the group rather than just one lucky winner. That is, of course, assuming one wants to maximize fun.

I’ve included my 1-page report below. The original research by Choi et al., can be found here. For me, the key conclusions are that current empirical OB research can inform MMORPG design decisions and, conversely, that MMORPG based-studies can be a useful approach to conducting empirical OB research.

Organizational Behavior Article Report, submitted by Tuebit’s alter-ego.

Choi, B. & Lee, I. & Choi, D. & Kim, J. (2007) Collaborate and Share: An Experimental Study of the Effects of Task and Reward Interdependencies in Online Games. CyberPsychology & Behavior, Vol. 10, No. 4.

What is the article saying? Choi et al. experimentally tested the correlation of fun, flow and perceived and actual performance to the four combinations of high and low task and reward interdependence, in a Massively Multiplayer Online role playing game (“MMO”). The study found that self-assessments of fun, flow and perceived performance were highest when the degree of task interdependence was paired with a similar degree of reward interdependence.

What do I agree with? The results of Choi et al. are broadly consistent with Wageman & Baker (1997), who established that matched task and reward interdependence typically maximizes group performance and co-operation. In situations of low task interdependence, a player’s sense of distributive justice may be offended when rewards are given on a basis other than individual performance, resulting in lower organizational commitment and reduced performance.

What do I disagree with? Choi et al. hypothesize that matched high task / reward interdependence improves performance mediated by group trust and harmony. The authors do not advance evidence for their model and competing models can be proposed. Rather than matched task / reward interdependence being beneficial, perhaps a mismatch is detrimental. In the high task / low reward interdependence scenario, high contribution players are given an advantage, but not a guarantee, to gaining the reward. Players could hypothetically shirk group responsibilities, and focus on reward taking (termed “ninja-looting”), a counterproductive behaviour that may lead to a negative bias in rating performance. Choi et al. did not find an impact of joint reward / task design on actual performance (reward count). Given that the experiment was conducted in only a few sessions, perhaps there was insufficient time to master complex group strategies to improve actual performance, or perhaps no such strategies existed.

What else should the author(s) have included? The experiment was small, with only 18 participants, making it difficult to accept the article’s conclusions. Choi et al. could have explored implications for the design of MMO, providing an empirically supported model on which to base reward design decisions. For example, some players have established rules outside of the game (i.e. “Dragon Kill Points” ) to better match task and reward interdependence, indicating a perceived problem with reward distribution. Whether group or individual game activities are “most fun” is a popular debate. Choi et al. could have explored whether fun, flow and perceived performance were highest for the optimal high or low interdependence scenarios, providing guidance for designers and contributing to the market success of MMO’s.

What is my overall assessment? While I have doubts about the models and methods presented, the article illustrates two important points. First, concepts from Organizational Behaviour can be used to better inform MMO design decisions. During MMO development, there can be a substantial cost to revising game features after play-testing (or game launch). The article also demonstrates that the technology underlying MMO’s may have a role in OB research. Complex group dynamics, leadership, impact of personality on task outcomes, and other topics could be studied in the rich and highly controllable task-oriented environment of a MMO.

This entry was posted in MMO Discussion and tagged , , , , , , , , , , , , . Bookmark the permalink.

9 Responses to "Evidence Based Gaming"

Leave a reply