Why You Should Combine Eye Tracking with Facial Expressions
Combining eye tracking and facial expressions is an underutilized yet extremely powerful combination that allows researchers to easily get much more information about a human being’s reaction to a certain stimulus. The key reasons are listed below.
Capturing visual attention and arousal
Eye tracking is a great way of measuring visual attention. While it can be used for many different applications, its primary function is measuring where people look at, either driven by their curiosity and interest or driven by provoking stimuli that draw their attention (e.g. on websites).
These findings are incredibly powerful, however, they only tell a fraction of what a respondent is experiencing at that very moment. Additionally, to capturing gaze positions, eye trackers record pupil dilation, which is often used as a measurement for arousal, rendering eye tracking an ideal tool to track arousal and excitement levels.
Reflecting emotional reactions
Facial expressions allow researchers to observe participants’ emotional states. Camera-based facial capture and automatic emotion detection have several benefits (e.g., you don’t have to attach sensors to your participants’ skin), but analyzing the outcome is not always easy.
For example, when website elements are not working as expected or long wait times make it hard to achieve the desired outcomes fast, users can get very frustrated. This emotional state of frustration will most likely manifest in facial expressions.
Facial expression recognition is great for measuring emotions and emotional valence, which is a more abstract measure of the positivity or negativity of the emotional expression.
For example, the emotional state of “joy” most likely has a solely positive valence (“Yay, weekend!”), whereas the emotional state “surprise” can comprise both positive (“Oh, a present!”) and negative (“Oh, a burglar!”) aspects.
To maximize the analytic power of facial expression analysis, researchers could combine valence (the quality of the emotion) with arousal (the strength of the emotion). Interestingly, arousal can be estimated from measuring pupil dilation, so this is just another reason to combine facial capture and eye tracking as you can extract and combine many more metrics with higher levels of validity.
1. Both methods are non-intrusive
- The combination of screen-based eye tracking and facial expression analysis allows a completely non-intrusive research setup for capturing both emotional arousal and valence. Neither eye trackers nor facial capture requires biosensors that have to be attached to the body, allowing researchers to get insights in real-world test scenarios.
2. It is easy to get started
- Setting up and analyzing eye tracking data is fairly easy since the output as generated from software such as iMotions is user-friendly and visual. Novice researchers without expert knowledge in the field can easily see where people are looking at, how long and at what time.
3. Combine arousal and valence
- One sensor alone only gives a fragmented picture. However, both biosensors combined supply data that can easily be understood and reported to third parties.