Applying Multistep Classification Techniques With Pre-Classification to Recognize Static and Dynamic Hand Gestures Using a Soft Sensor-Embedded Glove

• Published in: IEEE sensors journal Vol. 24; no. 19; pp. 30668 - 30679
• Main Authors: Jeon, Sujin, Park, Soyeon, Bae, Joonbum, Lim, Sunghoon
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.10.2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Classification; Data gloves; Deep learning; Fingers; Flexible printed circuits; Gesture recognition; Heuristic algorithms; Human-computer interface; Language translation; Pattern recognition; Segmentation; Sensors; Sign language; Soft sensors; Virtual reality; wearable sensors; Wrist
• ISSN: 1530-437X; 1558-1748
• DOI: 10.1109/JSEN.2024.3445128

Hand gestures have been widely used as an efficient information source for human-computer interaction (HCI) and apply to numerous fields, such as sign language translation and virtual reality (VR). Existing research on hand gesture recognition mainly considers either static gestures or dynamic gestures as gesture classes. However, simultaneously distinguishing both static and dynamic gestures is crucial for using hand gesture recognition systems in the real world because humans perform both types of gestures, not just one type in general. In this research, a multistep classification technique with a novel step called pre-classification is applied to simultaneously segment gesture patterns and classify static and dynamic gestures using a soft sensor-embedded glove, which measures the metacarpophalangeal (MCP) and proximal interphalangeal (PIP) joint angles of each finger every 15 ms. Specifically, the proposed hand gesture recognition system classifies gesture patterns as static or dynamic with pre-classification to handle both gestures with separately designed classifiers in one system for higher recognition performance. Combined with the gesture segmentation step, pre-classification shows reliable performance when a user performs various gesture types continuously. The proposed system's effectiveness is validated through experiments with ten samples from eight subjects continuously performing eight static gestures and 11 dynamic gestures based on American Sign Language (ASL). The proposed system achieved 99.19% and 99.20% classification accuracy on static and dynamic gestures, respectively, and the lowest error in gesture segmentation than existing methods. This work can be applied to various application areas where high recognition performances are required, such as VR workforce training in manufacturing and medicine.