Brain-Inspired Driver Emotion Detection for Intelligent Cockpits Based on Real Driving Data

• Published in: IEEE intelligent transportation systems magazine Vol. 16; no. 4; pp. 62 - 80
• Main Authors: Li, Wenbo, Wu, Yingzhang, Xiao, Huafei, Li, Shen, Tan, Ruichen, Deng, Zejian, Hu, Wen, Cao, Dongpu, Guo, Gang
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.07.2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Advanced driver assistance systems; Affective computing; Annotations; Automotive engineering; Brain; Brain modeling; Cockpits; Connected vehicles; Data collection; Data models; Datasets; Emotion recognition; Emotions; Intelligent vehicles; Road traffic; Roads & highways; Task analysis
• ISSN: 1939-1390; 1941-1197
• DOI: 10.1109/MITS.2023.3339758

Affective human-vehicle interaction of intelligent cockpits is a key factor affecting the acceptance, trust, and experience for intelligent connected vehicles. Driver emotion detection is the premise of realizing affective human-machine interaction. To achieve accurate and robust driver emotion detection, we propose a novel brain-inspired framework for on-road driver emotion detection using facial expressions. Then, we conduct driver emotion data collection in an on-road context. We develop a data annotation tool, annotate the collected data, and obtain the RoadEmo dataset, a dataset of facial expressions and road scenarios under the driver's emotional driving. Finally, we validate the detection accuracy of the proposed framework. The experiment results show that our proposed framework achieves excellent detection performance in the on-road driver emotion detection task and outperforms existing frameworks.