Integrating metacognitive judgments and eye movements using sequential pattern mining to understand processes underlying multimedia learning

Abstract: Metacomprehension is key to successful learning of complex topics when using multimedia materials. The goal of this study was to determine if eye-movement dyads could be: (1) identified by sequence mining techniques, and (2) aligned with self-reported metacognitive judgments during learning with multimedia materials that contain conceptual discrepancies designed to interfere with participants’ metacomprehension. Thirty-two undergraduate students’ metacognitive judgments were examined with RM-MANOVAs, and sequential pattern mining and differential sequence mining were conducted on their eye movements as they learned with complex multimedia materials. Additionally, we distinguished between event– (i.e., if participants looked at specific areas of the content) and duration-based (i.e., if participants looked at areas of interest [AOIs] for a medium or long amount of time) eye-movement dyads to assess if qualitative and quantitative differences existed in their eye-movement behaviors. For content with text and graph discrepancies, results indicated participants’ metacognitive judgments were lower and less accurate, and more fixation dyads were found between the text and graph. Furthermore, specific dyads of different length (i.e., long fixations on the graph to medium fixations on the text) fixations may align with lowered and inaccurate metacognitive judgments for content with text and graph discrepancies. This study begins to address how to identify behavioral indices of metacomprehension processes during multimedia learning.