A Dual-Branch Dynamic Graph Convolution Based Adaptive TransFormer Feature Fusion Network for EEG Emotion Recognition

• Published in: IEEE transactions on affective computing Vol. 13; no. 4; pp. 2218 - 2228
• Main Authors: Sun, Mingyi, Cui, Weigang, Yu, Shuyue, Han, Hongbin, Hu, Bin, Li, Yang
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.10.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Adapters; Brain modeling; Classification; Convolution; EEG; Electroencephalography; Emotion recognition; Emotions; Feature extraction; Feature maps; graph neural network; Graph neural networks; Human-computer interface; Machine learning; Transfer learning; Transformer; Transformers
• ISSN: 1949-3045; 1949-3045
• DOI: 10.1109/TAFFC.2022.3199075

Electroencephalograph (EEG) emotion recognition plays an important role in the brain-computer interface (BCI) field. However, most of recent methods adopted shallow graph neural networks using a single temporal feature, leading to the limited emotion classification performance. Furthermore, the existing methods generally ignore the individual divergence between different subjects, resulting in poor transfer performance. To address these deficiencies, we propose a dual-branch dynamic graph convolution based adaptive transformer feature fusion network with adapter-finetuned transfer learning (DBGC-ATFFNet-AFTL) for EEG emotion recognition. Specifically, a dual-branch graph convolution network (DBGCN) is firstly designed to effectively capture the temporal and spectral characterizations of EEG simultaneously. Second, the adaptive Transformer feature fusion network (ATFFNet) is conducted by integrating the obtained feature maps with the channel-weight unit, leading to significant difference between different channels. Finally, the adapter-finetuned transfer learning method (AFTL) is applied in cross-subject emotion recognition, which proves to be parameter-efficient with few samples of the target subject. The competitive experimental results on three datasets have shown that our proposed method achieves the promising emotion classification performance compared with the state-of-the-art methods. The code of our proposed method will be available at: https://github.com/smy17/DANet .