A Deep Learning Model for Correlation Analysis between Electroencephalography Signal and Speech Stimuli

• Published in: Sensors (Basel, Switzerland) Vol. 23; no. 19; p. 8039
• Main Authors: Alessandrini, Michele, Falaschetti, Laura, Biagetti, Giorgio, Crippa, Paolo, Luzzi, Simona, Turchetti, Claudio
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 23.09.2023; MDPI
• Subjects: canonical correlation analysis (CCA); Comparative analysis; Correlation analysis; Datasets; deep correlation analysis (DCCA); Deep learning; Electroencephalography; electroencephalography (EEG); multilayer perceptron (MLP); Neural networks; Speech; speech–EEG analysis
• ISSN: 1424-8220; 1424-8220
• DOI: 10.3390/s23198039

In recent years, the use of electroencephalography (EEG) has grown as a tool for diagnostic and brain function monitoring, being a simple and non-invasive method compared with other procedures like histological sampling. Typically, in order to extract functional brain responses from EEG signals, prolonged and repeated stimuli are needed because of the artifacts generated in recordings which adversely impact the stimulus-response analysis. To mitigate the artifact effect, correlation analysis (CA) methods are applied in the literature, where the predominant approaches focus on enhancing stimulus-response correlations through the use of linear analysis methods like canonical correlation analysis (CCA). This paper introduces a novel CA framework based on a neural network with a loss function specifically designed to maximize correlation between EEG and speech stimuli. Compared with other deep learning CA approaches (DCCAs) in the literature, this framework introduces a single multilayer perceptron (MLP) network instead of two networks for each stimulus. To validate the proposed approach, a comparison with linear CCA (LCCA) and DCCA was performed, using a dataset containing the EEG traces of subjects listening to speech stimuli. The experimental results show that the proposed method improves the overall Pearson correlation by 10.56% compared with the state-of-the-art DCCA method.