DeepBBWAE-Net: A CNN-RNN Based Deep SuperLearner for Estimating Lower Extremity Sagittal Plane Joint Kinematics Using Shoe-Mounted IMU Sensors in Daily Living

• Published in: IEEE journal of biomedical and health informatics Vol. 26; no. 8; pp. 3906 - 3917
• Main Authors: Hossain, Md Sanzid Bin, Dranetz, Joseph, Choi, Hwan, Guo, Zhishan
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.08.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Ankle; Artificial neural networks; Biomechanical engineering; Biomechanics; Cameras; Data models; Deep learning; ensemble learning; Estimation; Human motion; Inertial platforms; Infrared analysis; Infrared cameras; Infrared tracking; Joints (anatomy); Kinematics; kinematics estimation; Knee; Legged locomotion; Machine learning; Motion capture; Neural networks; Predictive models; Recurrent neural networks; Root-mean-square errors; Sensors; Treadmills; Walking; wearable IMU sensors
• ISSN: 2168-2194; 2168-2208; 2168-2208
• DOI: 10.1109/JBHI.2022.3165383

Measurement of human body movement is an essential step in biomechanical analysis. The current standard for human motion capture systems uses infrared cameras to track reflective markers placed on a subject. While these systems can accurately track joint kinematics, the analyses are spatially limited to the lab environment. Though Inertial Measurement Units (IMUs) can eliminate these spatial limitations, those systems are impractical for use in daily living due to the need for many sensors, typically one per body segment. Due to the need for practical and accurate estimation of joint kinematics, this study implements a reduced number of IMU sensors and employs a machine learning algorithm to map sensor data to joint angles. Our developed algorithm estimates hip, knee, and ankle angles in the sagittal plane using two shoe-mounted IMU sensors in different practical walking conditions: treadmill, overground, stair, and slope conditions. Specifically, we propose five deep learning networks that use combinations of Convolutional Neural Networks (CNN) and Gated Recurrent Unit (GRU) based Recurrent Neural Networks (RNN) as base learners for our framework. Using those five baseline models, we propose a novel framework, DeepBBWAE-Net, that implements ensemble techniques such as bagging, boosting, and weighted averaging to improve kinematic predictions. DeepBBWAE-Net predicts joint kinematics for the three joint angles for each of the walking conditions with a Root Mean Square Error (RMSE) 6.93-29.0% lower than the base models individually. This is the first study that uses a reduced number of IMU sensors to estimate kinematics in multiple walking environments.