Self-Supervised Graph Representation Learning for In-The-Wild Wearable and Smartphone based Emotion Recognition

• Published in: Proceedings of the ... IEEE International Conference on Acoustics, Speech and Signal Processing (1998) pp. 1 - 5
• Main Authors: Ziogas, Ioannis, Hadjileontiadis, Leontios J., Khandoker, Ahsan H., Shehhi, Aamna Al
• Format: Conference Proceeding
• Language: English
• Published: IEEE 06.04.2025
• Subjects: Biomedical monitoring; Emotion recognition; Graph Masking; Graph Neural Networks; Representation learning; Self-supervised learning; Signal processing; Speech processing; Speech recognition; Time series analysis; Training; Wearable devices; Wearable Emotion Recognition
• ISSN: 2379-190X
• DOI: 10.1109/ICASSP49660.2025.10888648

Wearable and smartphone-based emotion recognition (WER) remains a challenging setting in affective computing, due to the notorious difficulty and bias associated with in-thewild label collection. The high inter-and intra-subject emotional variability motivates us to explore WER modeling through graph node classification in a limited resources learning scheme powered by Self-Supervised Learning (SSL) graph masking augmentation tasks. We employ a subgraph sampling approach during training, utilizing labeled and unlabeled data, along with supervised, semi-supervised, and SSL mechanisms in a multi-task inductive graph neural network architecture. Our evaluations on K-EmoPhone through leave-one-group-out cross-validation in the binary arousal and valence tasks yield average accuracy gains of 4.3% and 7.8%, compared to the full resource setting, utilizing only 20% and 25% of the labels, respectively. Our model analysis sheds light on the relation of SSL graph augmentations to emotional arousal and valence and justifies the approach of SSL-driven subgraph training for in-the-wild WER.