Audio-Visual Emotion Analysis Using Semi-Supervised Temporal Clustering with Constraint Propagation

• Published in: Image Analysis and Recognition pp. 3 - 11
• Main Authors: Araujo, Rodrigo, Kamel, Mohamed S.
• Format: Book Chapter
• Language: English
• Published: Cham Springer International Publishing 2014
• Series: Lecture Notes in Computer Science
• Subjects: Clustering; Facial expression; Kernel; means; Semi-supervised; Speech emotion recognition; Temporal segmentation
• ISBN: 3319117548; 9783319117546
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-319-11755-3_1

In this paper, we investigate applying semi-supervised clustering to audio-visual emotion analysis, a complex problem that is traditionally solved using supervised methods. We propose an extension to the semi-supervised aligned cluster analysis algorithm (SSACA), a temporal clustering algorithm that incorporates pairwise constraints in the form of must-link and cannot-link. We incorporate an exhaustive constraint propagation mechanism to further improve the clustering process. To validate the proposed method, we apply it to emotion analysis on a multimodal naturalistic emotion database. Results show substantial improvements compared to the original aligned clustering analysis algorithm (ACA) and to our previously proposed semi-supervised approach.