Human-robot collaborative task planning using anticipatory brain responses

• Published in: PloS one Vol. 18; no. 7; p. e0287958
• Main Authors: Ehrlich, Stefan K., Dean-Leon, Emmanuel, Tacca, Nicholas, Armleder, Simon, Dimova-Edeleva, Viktorija, Cheng, Gordon
• Format: Journal Article
• Language: English
• Published: United States Public Library of Science 11.07.2023
• Subjects: Adaptability; Adaptation; Algorithms; Biology and Life Sciences; Brain research; Cognition & reasoning; Cognitive ability; Collaboration; Computational linguistics; EEG; Electroencephalography; Engineering and Technology; Feasibility; Feedback; Human engineering; Language processing; Learning; Machine learning; Mediation; Medicine and Health Sciences; Methods; Natural language interfaces; Planning; Research and Analysis Methods; Robot arms; Robotics; Robots; Simulation; Social Sciences; Task analysis; Task planning (robotics); Technology application; Germany
• ISSN: 1932-6203; 1932-6203
• DOI: 10.1371/journal.pone.0287958

Human-robot interaction (HRI) describes scenarios in which both human and robot work as partners, sharing the same environment or complementing each other on a joint task. HRI is characterized by the need for high adaptability and flexibility of robotic systems toward their human interaction partners. One of the major challenges in HRI is task planning with dynamic subtask assignment, which is particularly challenging when subtask choices of the human are not readily accessible by the robot. In the present work, we explore the feasibility of using electroencephalogram (EEG) based neuro-cognitive measures for online robot learning of dynamic subtask assignment. To this end, we demonstrate in an experimental human subject study, featuring a joint HRI task with a UR10 robotic manipulator, the presence of EEG measures indicative of a human partner anticipating a takeover situation from human to robot or vice-versa. The present work further proposes a reinforcement learning based algorithm employing these measures as a neuronal feedback signal from the human to the robot for dynamic learning of subtask-assignment. The efficacy of this algorithm is validated in a simulation-based study. The simulation results reveal that even with relatively low decoding accuracies, successful robot learning of subtask-assignment is feasible, with around 80% choice accuracy among four subtasks within 17 minutes of collaboration. The simulation results further reveal that scalability to more subtasks is feasible and mainly accompanied with longer robot learning times. These findings demonstrate the usability of EEG-based neuro-cognitive measures to mediate the complex and largely unsolved problem of human-robot collaborative task planning.