A Noninvasive BCI System for 2D Cursor Control Using a Spectral-Temporal Long Short-Term Memory Network

• Published in: Frontiers in computational neuroscience Vol. 16; p. 799019
• Main Authors: Pan, Kang, Li, Li, Zhang, Lei, Li, Simeng, Yang, Zhuokun, Guo, Yuzhu
• Format: Journal Article
• Language: English
• Published: Switzerland Frontiers Research Foundation 23.03.2022; Frontiers Media S.A
• Subjects: brain-computer interface; Datasets; EEG; Electroencephalography; Event-related potentials; Interfaces; Long short-term memory; Mental task performance; Neuroscience; Regression analysis; spectral-temporal LSTM network; Temporal variations; two-dimensional cursor control; User training; Velocity; velocity-constrained loss; Wavelet transforms; electroencephalography; spectral-temporal LSTM network; two-dimensional cursor control; velocity-constrained loss; brain-computer interface
• ISSN: 1662-5188; 1662-5188
• DOI: 10.3389/fncom.2022.799019

Two-dimensional cursor control is an important and challenging problem in the field of electroencephalography (EEG)-based brain computer interfaces (BCIs) applications. However, most BCIs based on categorical outputs are incapable of generating accurate and smooth control trajectories. In this article, a novel EEG decoding framework based on a spectral-temporal long short-term memory (stLSTM) network is proposed to generate control signals in the horizontal and vertical directions for accurate cursor control. Precisely, the spectral information is used to decode the subject's motor imagery intention, and the error-related P300 information is used to detect a deviation in the movement trajectory. The concatenated spectral and temporal features are fed into the stLSTM network and mapped to the velocities in vertical and horizontal directions of the 2D cursor under the velocity-constrained (VC) strategy, which enables the decoding network to fit the velocity in the imaginary direction and simultaneously suppress the velocity in the non-imaginary direction. This proposed framework was validated on a public real BCI control dataset. Results show that compared with the state-of-the-art method, the RMSE of the proposed method in the non-imaginary directions on the testing sets of 2D control tasks is reduced by an average of 63.45%. Besides, the visualization of the actual trajectories distribution of the cursor also demonstrates that the decoupling of velocity is capable of yielding accurate cursor control in complex path tracking tasks and significantly improves the control accuracy.