Enhancing Student Models in Game-based Learning with Facial Expression Recognition

Robert Sawyer

Andy Smith

Jonathan Rowe

Roger Azevedo

James Lester

Abstract:  Recent years have seen a growing recognition of the role that affect plays in learning. Because game-based learning environments elicit a wide range of student affective states, affect-enhanced student modeling for game-based learning holds considerable promise. This paper introduces an affect-enhanced student modeling framework that leverages facial expression tracking for game-based learning. The affect-enhanced student modeling framework was used to generate predictive models of student learning and student engagement for students who interacted with CRYSTAL ISLAND, a game-based learning environment for microbiology education. Findings from the study reveal that the affect-enhanced student models significantly outperform baseline predictive student models that utilize the same gameplay traces but do not use facial expression tracking. The study also found that models based on individual facial action coding units are more effective than composite emotion models. The findings suggest that introducing facial expression tracking can improve the accuracy of student models, both for predicting student learning gains and also for predicting student engagement.

Keywords:

  • Student Modeling
  • Affect
  • Game-based learning
This publication uses Eye Tracking, Eye Tracking Screen Based, Facial Expression Analysis and GSR which is fully integrated into iMotions Lab

Learn more