What’s in a face: Automatic facial coding of untrained study participants compared to standardized inventories

• Published in: PloS one Vol. 17; no. 3; p. e0263863
• Main Authors: Höfling, T. Tim A., Alpers, Georg W., Büdenbender, Björn, Föhl, Ulrich, Gerdes, Antje B. M.
• Format: Journal Article
• Language: English
• Published: United States Public Library of Science 03.03.2022; Public Library of Science (PLoS)
• Subjects: Adult; Analysis; Anger; Automated Facial Recognition - methods; Biology and Life Sciences; Coding; Computer and Information Sciences; Datasets; Emotions; Emotions - physiology; Engineering and Technology; Error analysis; Experiments; Facial Expression; Female; Females; Humans; Information technology; Inventories; Learning algorithms; Machine Learning; Male; Medicine and Health Sciences; Methods; Pictures; Psychology; Social Sciences; Validity; Young Adult; Germany
• ISSN: 1932-6203; 1932-6203
• DOI: 10.1371/journal.pone.0263863

Automatic facial coding (AFC) is a novel research tool to automatically analyze emotional facial expressions. AFC can classify emotional expressions with high accuracy in standardized picture inventories of intensively posed and prototypical expressions. However, classification of facial expressions of untrained study participants is more error prone. This discrepancy requires a direct comparison between these two sources of facial expressions. To this end, 70 untrained participants were asked to express joy, anger, surprise, sadness, disgust, and fear in a typical laboratory setting. Recorded videos were scored with a well-established AFC software (FaceReader, Noldus Information Technology). These were compared with AFC measures of standardized pictures from 70 trained actors (i.e., standardized inventories). We report the probability estimates of specific emotion categories and, in addition, Action Unit (AU) profiles for each emotion. Based on this, we used a novel machine learning approach to determine the relevant AUs for each emotion, separately for both datasets. First, misclassification was more frequent for some emotions of untrained participants. Second, AU intensities were generally lower in pictures of untrained participants compared to standardized pictures for all emotions. Third, although profiles of relevant AU overlapped substantially across the two data sets, there were also substantial differences in their AU profiles. This research provides evidence that the application of AFC is not limited to standardized facial expression inventories but can also be used to code facial expressions of untrained participants in a typical laboratory setting.