Consistency and changes in eye movements during static, dynamic, and audio-visual emotion processing

Abstract

Tracking eye movements as individuals view emotional facial expressions can offer insight into the mechanisms underlying emotion recognition. Few studies compare eye gaze patterns across static and dynamic expressions, and even fewer use ecologically valid audio-visual stimuli. Research is also limited on individual differences in emotion processing and how these relate to recognition performance. In this study, we address this gap by examining the eye movement patterns of adult participants ( <jats:italic toggle="yes">n</jats:italic> = 73) during emotion recognition of individuals in static photos, dynamic videos, and audio-visual videos. Using pre-defined region-of-interest analyses and Hidden Markov Models, we explored whether individuals focused on different areas of the face across conditions, whether individual eye movement patterns were consistent across conditions, and whether certain patterns were associated with better performance. The findings showed that a) individuals focused more on the eye region when recognising static expressions compared to moving expressions; b) individuals’ eye movement patterns during emotion recognition were highly consistent, albeit most likely to change between static and moving conditions; c) there was not one particular pattern associated with improved emotion recognition accuracy. Ultimately, we report that the emotion recognition of static emotional expressions differs from moving expressions, suggesting recognition is influenced by movement, highlighting important considerations for future research to include more realistic stimuli to be better able to generalise to real-world social interactions.