Multi-domain fusion deep graph convolution neural network for EEG emotion recognition

• Published in: Neural computing & applications Vol. 34; no. 24; pp. 22241 - 22255
• Main Authors: Bi, Jinying, Wang, Fei, Yan, Xin, Ping, Jingyu, Wen, Yongzhao
• Format: Journal Article
• Language: English
• Published: London Springer London 01.12.2022; Springer Nature B.V
• Subjects: Artificial Intelligence; Artificial neural networks; Brain; Brain research; Computational Biology/Bioinformatics; Computational Science and Engineering; Computer Science; Data Mining and Knowledge Discovery; Datasets; Domains; Electroencephalography; Emotion recognition; Emotional factors; Emotions; Feature extraction; Image Processing and Computer Vision; Neural networks; Original Article; Principal components analysis; Probability and Statistics in Computer Science; Topology; Emotion recognition; Graph convolution neural network; Multi-domain fusion features; Transfer learning
• ISSN: 0941-0643; 1433-3058
• DOI: 10.1007/s00521-022-07643-1

Electroencephalogram (EEG)-based emotion recognition has become a hot research field, with the most attention given to decoding three basic types of emotional states (i.e., positive, negative, and neutral) by EEG. Traditional EEG emotion recognition is a single feature input mode that cannot cover multiple feature information. For brain function networks, the feature extraction process is very cumbersome. There will also be individual differences in the characteristics of different subjects. Therefore, the theory of graph convolution and brain function connection is introduced into this research, and the multi-domain fusion features input deep graph convolution neural network (MdGCNN) is proposed in this paper. Pearson correlation is used to determine the adjacency matrix. The Sortpooling layer is employed as a bridge between the graph convolution neural layer and the normal neural network layer and sorts the node features in a consistent order. Based on analyzing the characteristics of a single electrode, the brain topology structure features are automatically extracted. Taking MdGCNN as the basic model and considering the method of minimizing the feature distance between the source and the target domain, we propose a transfer learning (TL) emotion recognition model for cross-subject called MdGCNN-TL. Meanwhile, MdGCNN-TL is extended to traverse the target domain of a single subject in a two-to-one domain form. According to the idea of principal component analysis, the transfer model with a high recognition effect is determined with the degree of subject correlation (SC), and MdGCNN-TL is upgraded to MdGCNN-TL-SC. Experimental analysis on the SEED dataset is performed to evaluate the proposed models. Further validation of the model is implemented on the DEAP dataset. The results show that the proposed model has achieved better performance in EEG emotion recognition.