Short-Form Videos: An Exploratory Study on the Impact of Subtitles and ASMR Split-screen Format Options Using Eyegaze and Facial Expression Data

Short-form videos, popularized by platforms like TikTok, YouTube Shorts, and Instagram Reels, have revolutionized media consumption through format options and features such as split-screen visuals, subtitles, music, accelerated audio, and pause removal. This exploratory study investigates how two of those format options, subtitles and split screen, influence viewer visual attention, recall, and emotional engagement during video consumption. Using biometric tools, including eye tracking and facial expression analysis (FEA), alongside survey-based assessments, the present study examines the interplay between video format and user engagement. The experiment involved four video conditions: a baseline TED Talk, a TED Talk with subtitles, a split-screen with additional ASMR content, and a split-screen with subtitles. Results reveal that subtitles and split-screen visuals significantly impact visual attention patterns. Emotional responses, measured via FEA, showed elevated positive emotions, engagement, and attention in split-screen conditions. However, information retention, as assessed through immediate recall, was not significantly influenced by tested video features.