Gaming Based Evidence

Previously, I reviewed a research paper that investigated the relationship between task and reward interdependence and fun, flow and performance in a MMORPG, in effect, using research to improve games (see “Evidence Based Gaming”). Today I review a research paper looking at the potential value of simulations in social psychology research (in effect, using game technology to improve research).

Blascovich, J., Loomis, J., Beall, A. C., Swinth, K. R., Hoyt, C. L., & Bailenson, J. N. (2002). Immersive Virtual Environment Technology as a Methodological Tool for Social Psychology. Psychological Inquiry, 13 (2), 103–124.

What the article is saying:

Blascovich et al. explore the potential to use Immersive Virtual Environment Technology (IVET) in social psychology research. They identify social psychology as “an attempt to understand and explain how the thought, feeling, and behavior of individuals are influenced by the actual, imagined, or implied presence of others”. Immersive Virtual Environment Technology is defined by the authors as the presentation of a simulated environment causing the user to act as if the environment were real. So-called Virtual Reality helmets, and rigs like this, would qualify.

To conduct research, social psychology researchers create experimental scenarios, sometimes as simple as a story or vignette, sometimes involving physical props and actors. These experiments face three challenges.

The first challenge described is the trade-off between “experimental control” (the degree to which every factor of the experiment is precisely controlled) and “mundane realism” (the degree to which the study participant feels as though the study setup is real).

The authors say that poor experimental control produces results that are invalidated by confounding factors. For example, an actor hired for a study may not use the same intonation in delivering their role for each subject of the study, thereby influencing and invalidating the results. Poor mundane realism may cause participants to act differently than they might in a real scenario.

The authors hold that there is a trade-off between these two. An experiment might include human actors and props, increasing mundane realism but decreasing experimental control (actors cannot precisely replicate their performance). In contrast, the authors argue, computerized simulations can be very realistic (high mundane realism) and precisely controllable and repeatable (high experimental control) simultaneously.

The second and third challenges are lack of replication and issues related to sampling. The replication of experiments is an important part of the scientific process, allowing other researchers to confirm and extend research results to new samples. It is often difficult, the authors contend, to exactly specify the research methods used in a paper. For example, even hiring the same actor is not likely to recreate the intonation used in playing a role.

The authors argue that standardized IVE technology could permit social psychology researchers to precisely replicate experiments, simply by exchanging data-files. Further, the authors argue that wide replication of experiments would allow larger, more diverse and ultimately more representative samples to be used, which improves both the validity and generalize-ability of research results.

What I liked / agreed with:

In support of their claims, the authors implemented an IVET system and successfully replicated earlier research results (non-IVET) in proxemics, social facilitation / inhibition, conformity and social comparison.

The argument that the use of IVET could improve ease of replication makes great intuitive sense. The difference between the current practice and the potential in IVET is akin to the difference between describing a picture to someone and actually sending them an electronic copy of it. This potential is premised on the assumption that researchers would be able to implement universally standard IVET.

One of the most interesting discussions in the article is the potential to use IVET to conduct experiments investigating veridicality in social exchanges. Imagine you chat with someone in a virtual world. You are told that the avatars used are a reasonable facsimile of each participant’s actual appearance, and indeed, when viewing your own avatar, you see that it is a close match. However, another person viewing your avatar is shown a different representation (for example, less or more attractive or of a different gender). A single perceptual difference is introduced into this exchange that, potentially, neither participant is aware of. In this example, the experimenters have altered the veridicality of the environment, something that is hard to imagine doing without the aid of IVET.

What I disliked / disagreed with:

Blascovich et al. talked about the importance of mundane realism, but I wasn’t convinced (at least from the contents of this paper) that realism is necessary to achieve correct and generalizable results. Perhaps it has been so well studied as to be a given amongst social psychologists (which I’m not). Even if mundane realism is important, the authors didn’t convince me that IVET improves mundane realism nor did they prove that a visually surrounding technology (VR Goggles, for example) is necessary.

While the authors talk about “photo-realistic” depictions within their IVE, this might be over-selling current capabilities. Digital recreations of real objects tend to fall into the “uncanny valley”. That is to say, the closer a recreation is to reality, the more tiny differences create a perceptual dissonance. These realistic re-creations just look wrong, even creepy.

Perhaps highly realistic recreations are not necessary to create a compelling environment. The goal is not to create a projection that is indistinguishable from reality (although that would be very cool), but to create an environment that causes the subject to suspend their disbelief and act in ways that are natural. I personally experience this all the time, playing MMORPG’s with highly stylized depictions and off-the-shelf computer hardware. Maybe the technology underlying a MMORPG (software and a standard PC) is all that’s really needed to achieve both greater mundane realism and ease of replication.

What could have been taken further:

I would have liked to have seen proof of the importance of mundane realism and experiments to confirm the degree of improvement of mundane realism achieved using a variety of IVE technology (for example, MMORPG-like environments compared with VR goggles).

There have been substantial developments in technology since the article was written, such as motion capture techniques that are very portable or that don’t rely on markers or accelerometers. I’d very much enjoy reading an update on the use of IVET in research.

Conclusions:

Blascovich et al. make a compelling case for the use of IVET in lab research in social psychology. Additional research into the degree to which various IVE systems deliver actual benefits would make their position more convincing to me. Such research might also show that sufficient value can be derived from more commonly available technology, for example, standard PC based virtual environments.

This entry was posted in Chatter. Bookmark the permalink.

One Response to "Gaming Based Evidence"

Leave a reply