VTD: Visual and Tactile Dataset for Driver State and Behavior Detection

• Published in: IEEE robotics and automation letters Vol. 10; no. 6; pp. 5657 - 5664
• Main Authors: Wang, Jie, Cai, Mobing, Zhu, Zhongpan, Ding, Hongjun, Yi, Jiwei, Du, Aimin
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.06.2025; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Autonomous vehicles; Biomedical monitoring; Datasets; Driver behavior; driver state; Fatigue; Feature extraction; intelligent cockpit; Modal data; Monitoring; Safety; Uncertainty; Vehicles; Visual and tactile data; Visualization
• ISSN: 2377-3766; 2377-3766
• DOI: 10.1109/LRA.2025.3561563

In the domain of autonomous vehicles, the human-vehicle co-pilot system has garnered significant research attention. To address the subjective uncertainties in driver state and interaction behaviors, which are pivotal to the safety of Human-in-the-loop co-driving systems, we introduce a novel visual-tactile detection method. Utilizing a driving simulation platform, a comprehensive dataset has been developed that encompasses multi-modal data under fatigue and distraction conditions. The experimental setup integrates driving simulation with signal acquisition, yielding 600 minutes of driver state and behavior data from 15 subjects and 102 takeover experiments with 17 drivers. The dataset, synchronized across modalities, serves as a robust resource for advancing cross-modal driver behavior detection algorithms.