An EEG/EMG/EOG-Based Multimodal Human-Machine Interface to Real-Time Control of a Soft Robot Hand

• Published in: Frontiers in neurorobotics Vol. 13; p. 7
• Main Authors: Zhang, Jinhua, Wang, Baozeng, Zhang, Cheng, Xiao, Yanqing, Wang, Michael Yu
• Format: Journal Article
• Language: English
• Published: Switzerland Frontiers Research Foundation 29.03.2019; Frontiers Media S.A
• Subjects: Automation; Brain; Computer applications; EEG; electroencephalogram (EEG); Electroencephalography; electromyogram (EMG); Electromyography; electrooculogram (EOG); Engineering; Eye movements; Hand; Implants; International conferences; Mental task performance; multimodal human-machine interface (mHMI); Neuroscience; Patients; Physical therapists; Physical therapy; Plasticity (neural); Quality of life; Recovery of function; Rehabilitation; Robotics; Robots; soft robot hand; Stroke; Hong Kong; China; soft robot hand; multimodal human-machine interface (mHMI); electrooculogram (EOG); electromyogram (EMG); electroencephalogram (EEG)
• ISSN: 1662-5218; 1662-5218
• DOI: 10.3389/fnbot.2019.00007

Brain-computer interface (BCI) technology shows potential for application to motor rehabilitation therapies that use neural plasticity to restore motor function and improve quality of life of stroke survivors. However, it is often difficult for BCI systems to provide the variety of control commands necessary for multi-task real-time control of soft robot naturally. In this study, a novel multimodal human-machine interface system (mHMI) is developed using combinations of electrooculography (EOG), electroencephalography (EEG), and electromyogram (EMG) to generate numerous control instructions. Moreover, we also explore subject acceptance of an affordable wearable soft robot to move basic hand actions during robot-assisted movement. Six healthy subjects separately perform left and right hand motor imagery, looking-left and looking-right eye movements, and different hand gestures in different modes to control a soft robot in a variety of actions. The results indicate that the number of mHMI control instructions is significantly greater than achievable with any individual mode. Furthermore, the mHMI can achieve an average classification accuracy of 93.83% with the average information transfer rate of 47.41 bits/min, which is entirely equivalent to a control speed of 17 actions per minute. The study is expected to construct a more user-friendly mHMI for real-time control of soft robot to help healthy or disabled persons perform basic hand movements in friendly and convenient way.