The video games industry is now worth almost $150 billion, rivalling other industries traditionally thought to dominate popular culture. It also continues to grow at a rapid pace, with gameplay streaming, eSports, and spinoffs continually creating new revenue sources.
Video games are now a mainstream component of the entertainment industry at large – a fact repeated so often that it has almost become cliché. There is however one aspect that separates it from a large part of the entertainment industry – testing.
While Hollywood studios have long screen-tested, interviewed, and carried out scientific reports in order to improve their films (since at least the 1920’s), the gaming industry is only just starting to catch up. This is in part due to the inherent history of the formats – the first video game wasn’t invented until 1958 (Pong, of course) – but also due to the broadness of the challenge – studies of video games need to look not only at the visuals, plotline, characters, and dialogue, but also the user experience (not to mention the broader behavioral impact on users).
This is all without mentioning that the content is dynamic and constantly changing, and often due to the player’s in-game choices. This clearly presents a challenge to game testing, but not an impossible one. Research has taken steps in defining the problem and has also begun to present solutions.
Below, we will go through some of the research that has shaped how video games testing is currently carried out, and look to new avenues of research that are being explored.
Table of Contents
Early research
One of the first research studies to look at visual attention in a 3D gaming environment was carried out by Yan and El-Nasr in 2006 [1]. By using eye tracking to record and measure the visual behavior of participants playing either a shooting game, or an adventure game, they were able to compare how visual search patterns differed.
While a preliminary study, the researchers found that participants were more likely to keep their fixation on the center of the screen when playing the shooting game, and more likely to explore the whole scene when playing the adventure game. The authors posit that such findings can “…be used to inform designers how to compose colors and placements of objects to stimulate attention and eliminate these problems”.
While other research predates this (e.g. [2, 3]), the rise of methodologies to investigate video games from an experiential perspective using objective methods has only really happened in the last decade. Prior research has been more focused on the use of video games to investigate the dynamics of perception, learning, and other psychological phenomena.
In order to actually test the experience of gameplay, researchers first must define what it is that they are testing. This has led to the game experience questionnaire (GEQ) [4], a set of questions that determine the conscious feelings of participants playing games.
The in-game experience questionnaire (iGEQ) is also widely used – a quicker version of the GEQ that shows similar levels of statistical robustness that can be implemented within gameplay without causing too much interference.
The most commonly explored features of gameplay (that are also measured by the GEQ / iGEQ) are immersion, flow, and presence [5]. Each of these has been found to be positively related to enjoyable gameplay experiences, suggesting that a great game should feature high levels of each. Below, we will go through each of these and expand on the research that has been done to explore each.
Check out: Biometric testing from home to Improve Gaming Experience
Most commonly explored features of gameplay
Immersion
Immersion is a hotly-debated concept within video game research [6], but can largely be defined by the feeling of engrossment within the game. This manifests as showing increased attention to the game, and likewise, the suppression of attention to external stimuli [7]. The more immersed an individual feels, the more they feel absorbed in the game.
Immersiveness is particularly critical for VR gameplay, one of the fastest-growing video game formats. As interest and investment grows for VR / AR devices, testing will also become increasingly important to ensure that the content is created in the most appealing and immersive way.
Researchers have used GSR (referred to as electrodermal activity within the study) and ECG recordings of heart rate to determine relationships to mental states outlined within the iGEQ, including feelings of immersion [8]. Participant responses to three major commercial games (specifically, “Prey”, “Doom 3”, and “Bioshock”) were tested with these measures. The researchers found that heart rate was negatively correlated with immersion (as well as flow), meaning that a lower heart rate was indicative of increased feelings of immersion during gameplay.
Other research has also investigated how heart rate variability (HRV) is altered by immersion within a VR environment as compared to watching a 3D movie [9]. The VR environment appeared to induce more feelings of immersion and was associated with an increase in HRV – suggesting that the participants were experiencing an increase in physiological arousal as they completed the cognitively demanding task.
Learn how to gather real-time reactions during gameplay
A study by Jennett et al (2008) [10] used a similar approach, measuring feelings of immersion with a questionnaire while also using eye tracking. The researchers found that immersion could be objectively determined through visual behavior, with a reduced number of fixations being indicative of increased immersion. This suggests that the more a participant feels immersed, the longer they stare at the game without changing their gaze.
By relating the self-reported emotions to biosensor data, it’s hoped that future studies may be able to use such data as a proxy for these feelings. This data, if proven to be reliable, could provide robust and detailed feedback about how the participants respond to the games.
Flow
Flow is described as “the optimal psychological experience that can arise from being engaged in an activity” [8, 11]. There are many factors that come into play to determine the flow state, but it essentially boils down to the combined feelings of mastery and absorption while completing a task.
One study used a flow state eliciting game component (that was also assessed with the GEQ), and measured facial EMG (fEMG) activity from the zygomaticus major, the orbicularis oculi, and the corrugator supercilii [5]. The first two of these muscles are broadly related to positive emotions, while the final is associated with negative emotions [12].
They found that the flow game component caused a significant difference in zygomaticus major and orbicularis oculi activity, as compared to a game component intended to create feelings of boredom. This suggests that flow states are experienced as relatively positive, an important factor for an enjoyable gameplay experience.
The study also used GSR activity to measure levels of physiological arousal, finding a significant increase for the flow level in relation to the boredom level.
Several groups of researchers have also found that it’s possible to discern feelings of flow while playing video games with a simple four-electrode EEG recording (and a technical analysis of the data) [11, 12]. These studies suggest that flow is an important part of an enjoyable gameplay experience, that can be both detected and measured to improve how video games are made.
Presence
Presence, while having similarities to feelings of immersion, is defined as the feeling of existing (at least momentarily) within the game. This exists on a continuum, with complete separation at one end, and strong feelings of physical existence within the game at the other end.
A study by Ravaja and others carried out in 2004 [15] used a combination of methods, including fEMG, GSR, and ECG, as well as self-reports, to examine feelings of presence while participants played Super Monkey Ball 2 (for those of you who wonder what this game is about: “The game takes place in a surrealistic world with bright colors and includes a game board hanging in the air and a cute little monkey trapped in a transparent ball”).
The results show a significant association between zygomaticus major activity (as mentioned above, the activity of which is associated with positive emotions), feelings of presence, and the successful completion of the game. This was taken as evidence that the more present a player feels within the game, the more joyful their experience will be.
The researchers also found a positive association between GSR activity and feelings of presence within the game, yet no significant association was found from the ECG recordings. The participants who experienced higher levels of presence within the game showed a smaller reduction in GSR activity following the task completion (as compared to participants who experienced low levels of presence), suggesting that their physiological state remained more constant at the end of the task.
Usability testing using biosensors
Another study compared the feelings of presence in both 1st and 3rd person views in video games, using eye tracking and fEMG [16]. The researchers found an association, as above, with feelings of presence and activity within the zygomaticus major (as well as the corrugator supercilii).
These results could in the future be developed into indicators of feelings of presence, providing an understanding of how the experience changes over time within a game.
Check out Biosensors in Game testing – GUR virtual Summer camp featuring Nam Nguyen of iMotions
Conclusion
While the above studies cover three aspects of the experience of playing video games, there are also other psychological states that are explored within the gameplay research community, such as feelings of tension, challenge, as well as negative and positive affect. Measurements of each of these dimensions can provide a richer understanding of what makes a game enjoyable (and ultimately, whether or not it will sell well).
As gaming research continues to evolve, standard methodologies will also present ever-clearer ways of investigating how the media impacts the user at an affective and experiential level. Technologies such as eye tracking, GSR, and facial expression analysis will undoubtedly be used further to help both users play video games in new ways, as well as advancing new ways of understanding video game experiences.
If you’d like to know more about eye tracking – one of the core technologies used in video game research – then download our guide below to get an understanding of everything you need to know.
Eye Tracking
The Complete Pocket Guide
- 32 pages of comprehensive eye tracking material
- Valuable eye tracking research insights (with examples)
- Learn how to take your research to the next level
References
[1] S. Yan and M. Seif El‐Nasr. (2006). Visual Attention in 3D Video Games, in Symposium on Eye Tracking and Applications 2006, San Diego.
[2] Lowery, B. R. and Knirk, F. G. (1982). Micro-computer video games and spatial visualization acquisition. J. Educational Technol. Syst. 11, 2, 155-166.
[3] Gagnon, D. (1985). Videogames and spatial skills: An exploratory study. Educational Technology Research and Development, 33, Issue 4, pp 263–275.
[4] IJsselsteijn, W. A., de Kort, Y. A. W., and Poels, K. (2008). The game experience questionnaire, Working Paper.
[5] Nacke, L. (2009). Affective ludology: scientific measurement of user experience in interactive entertainment. Blekinge Institute of Technology Doctoral Dissertation Series No. 2009:04.
[6] Michailidis, L., Balaguer-Ballester, E., and He, X. (2018). Flow and Immersion in Video Games: The Aftermath of a Conceptual Challenge. Frontiers in Psychology, 9, p. 1682, 2018.
[7] Cairns, P., Cox, A.L., & Nordin, A.I. Immersion in Digital Games: Review of Gaming Experience Research. In MC Angelides and H. Agius, Handbook of digital games, Wiley-Blackwell, 2014, 339-361.
[8] Drachen, A., Nacke, L. E., Yannakakis, G., and Pedersen, A. L. (2010). Correlation between heart rate, electrodermal activity and player experience in first-person shooter games. In Proc. Sandbox ’10, ACM, 49–54.
[9] Malińska, M., Zużewicz, K., Bugajska, J., & Grabowski, A. (2015). Heart rate variability (HRV) during virtual reality immersion. International journal of occupational safety and ergonomics : JOSE, 21(1), 47–54. https://doi.org/10.1080/10803548.2015.1017964
[10] Jennett, C., Cox, A. L., Cairns, P., Dhoparee, S., Epps, A., Tijs, T., & Walton, A. (2008). Measuring and defining the experience of immersion in games. International Journal of Human-computer Studies, 66(9), 641-661.
[11] M. Csikszentmihalyi. (1991). Flow: The psychology of optimal experience, New York: Harper Perennial.
[12] Cacioppo, J. T., Petty, R. E., Losch, M. E., & Kim, H. S. (1986). Electromyographic activity over facial muscle regions can differentiate the valence and intensity of affective reactions. Journal of Personality and Social Psychology, 50, 260-268.
[13] Plotnikov A., Stakheika N., De Gloria A., Schatten C., Bellotti F., Berta R., Fiorini C., Ansovini F. (2012). Exploiting real-time EEG analysis for assessing flow in games. Workshop: Game based learning for 21st century transferable skills, at iCalt 2012, Rome, Italy.
[14] Berta, R., Bellotti, F., De Gloria, A., Pranantha, D., & Schatten, C. (2013). Electroencephalogram and physiological signal analysis for assessing flow in games. Computational Intelligence and AI in Games, IEEE Transactions on, 5(2), 164–175.
[15] Ravaja, N., Laarni, J., Saari, T., Kallinen, K., Salminen, M., Holopainen, J., & Järvinen, A. (2004). Spatial presence and emotional responses to success in a video game: A psychophysiological study. In M. Alcañiz Raya & B. Rey Solaz (Eds.), Proceedings of the PRESENCE, (pp. 112- 116). Valencia, Spain: Editorial de la UPV.
[16] Kallinen, K., Salminen, M., Ravaja, N., Kedzior, R., and Sääksjärvi, M. (2007). Presence and emotion in computer game players during 1st person vs. 3rd person playing view: evidence from self-report, eye-tracking and facial muscle activity data, in Proceedings of the PRESENCE, 187–190.187–190.
