An array of physical sensors and an adaptive regression strategy for emotion recognition in a noisy scenario

• Published in: Sensors and actuators. A. Physical. Vol. 267; pp. 48 - 59
• Main Authors: Mosciano, Francesco, Mencattini, Arianna, Ringeval, Fabien, Schuller, Björn, Martinelli, Eugenio, Di Natale, Corrado
• Format: Journal Article
• Language: English
• Published: Lausanne Elsevier B.V 01.11.2017; Elsevier BV; Elsevier
• Subjects: Adaptive regression strategy; Computer Science; Emotion recognition; Emotions; Regression analysis; Sensor array; Sensors; Studies; Sensor array; Emotion recognition; Adaptive regression strategy
• ISSN: 0924-4247; 1873-3069; 1873-3069
• DOI: 10.1016/j.sna.2017.09.056

•Emotion Recognition exploiting a heterogeneous array of physical sensors.•Personalized emotional model by means of Valence and Arousal estimation.•On line Feature selection to prevent noise or faulty features of the sample.•Selection of the closest training templates for the regression models of Valence and Arousal.•Validation of the proposed approach with a RECOLA database also simulating different faulty scenarios. Several studies demonstrate that since emotions are spontaneously manifested through different measurable quantities (e.g. vocal and facial expressions), this makes possible a sort of automatic estimation of emotion from objective measurements. However, the reliability of such estimations is strongly influenced by the availability of the different sensor modalities used to monitor the affective status of a subject, and furthermore the extraction of objective parameters is sometime thwarted in a noisy and disturbed environment. This paper introduces a personalized emotion estimation based on a heterogeneous array of physical sensors for the measurement of vocal, facial, and physiological (electro-cardiogram and electro-dermal) activities. As a proof of concept, changes in the levels of both emotion reactiveness and pleasantness are estimated under critical operative conditions. The estimator model takes advantage from the time-varying selection of the most relevant non-spurious sensors features and the adaptation of the k-nearest neighbour paradigm to the continuous identification of the most affine model templates. The model, once trained, demonstrated to autonomously embed new sensorial input and adapt to unwanted/unpredicted sensor noise or emotion alteration. The proposed approach has been successfully tested on the RECOLA database, a multi-sensorial corpus of spontaneous emotional interactions in French.