A Brain Network Analysis-Based Double Way Deep Neural Network for Emotion Recognition

• Published in: IEEE transactions on neural systems and rehabilitation engineering Vol. 31; pp. 917 - 925
• Main Authors: Niu, Weixin, Ma, Chao, Sun, Xinlin, Li, Mengyu, Gao, Zhongke
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.01.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Artificial neural networks; Brain; Brain modeling; Brain network; Classification; Correlation coefficient; Correlation coefficients; deep residual neural network; EEG; Electrodes; electroencephalogram (EEG); Electroencephalography; Emotion recognition; Emotional factors; Emotions; Feature extraction; Frequencies; Motion pictures; Network analysis; Neural networks; spearman correlation coefficient; Task analysis; Temporal variations; Wavelet transforms
• ISSN: 1534-4320; 1558-0210; 1558-0210
• DOI: 10.1109/TNSRE.2023.3236434

Constructing reliable and effective models to recognize human emotional states has become an important issue in recent years. In this article, we propose a double way deep residual neural network combined with brain network analysis, which enables the classification of multiple emotional states. To begin with, we transform the emotional EEG signals into five frequency bands by wavelet transform and construct brain networks by inter-channel correlation coefficients. These brain networks are then fed into a subsequent deep neural network block which contains several modules with residual connection and enhanced by channel attention mechanism and spatial attention mechanism. In the second way of the model, we feed the emotional EEG signals directly into another deep neural network block to extract temporal features. At the end of the two ways, the features are concatenated for classification. To verify the effectiveness of our proposed model, we carried out a series of experiments to collect emotional EEG from eight subjects. The average accuracy of the proposed model on our emotional dataset is 94.57%. In addition, the evaluation results on public databases SEED and SEED-IV are 94.55% and 78.91%, respectively, demonstrating the superiority of our model in emotion recognition tasks.