Towards Zero Re-Training for Long-Term Hand Gesture Recognition via Ultrasound Sensing

• Published in: IEEE journal of biomedical and health informatics Vol. 23; no. 4; pp. 1639 - 1646
• Main Authors: Yang, Xingchen, Zhou, Dalin, Zhou, Yu, Huang, Youjia, Liu, Honghai
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.07.2019; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Adult; Algorithms; bag-of-features; Calibration; Classification; classifier self-enhancing; Electromyography; Feasibility studies; Feature extraction; Gesture recognition; Gestures; Hand - diagnostic imaging; Hand - physiology; hand gesture recognition; Humans; Hypotheses; Image detection; Image Processing, Computer-Assisted - methods; Male; Muscle contraction; Muscle Contraction - physiology; Muscles; Muscular function; Myoelectricity; Pattern recognition; probe position calibration; Probes; Robustness; speeded-up robust features; Training; Ultrasonic imaging; Ultrasonography - methods; Ultrasound; Ultrasound image; User training; Young Adult
• ISSN: 2168-2194; 2168-2208; 2168-2208
• DOI: 10.1109/JBHI.2018.2867539

While myoelectric pattern recognition is a prevailing way for gesture recognition, the inherent nonstationarity of electromyography signals hinders its long-term application. This study aims to prove a hypothesis that morphological information of muscle contraction detected by ultrasound image is potentially suitable for long-term use. A set of ultrasound-based algorithms are proposed to realize robust hand gesture recognition over multiple days, with user training only at the first day. A markerless calibration algorithm is first presented to position the ultrasound probe during donning and doffing; an algorithm combining speeded-up robust features and bag-of-features model being immune to ultrasound probe shift and rotation is then introduced; a self-enhancing classification method is next adopted to update classification model automatically by incorporating useful knowledge from testing data; finally the performance of long-term hand gesture recognition with zero re-training is validated by a six-day experiment of six healthy subjects, whose outcomes strongly support the hypothesis with about 94% of gesture recognition accuracy for each testing day. This study confirms the feasibility of adoption of ultrasound sensing for long-term musculature related applications.