KAMTFENet: a fall detection algorithm based on keypoint attention module and temporal feature extraction

• Published in: International journal of machine learning and cybernetics Vol. 14; no. 5; pp. 1831 - 1844
• Main Authors: Li, Jiangjiao, Gao, Mengqi, Li, Bin, Zhou, Dazheng, Zhi, Yumin, Zhang, Youmei
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.05.2023; Springer Nature B.V
• Subjects: Accuracy; Algorithms; Artificial Intelligence; Complex Systems; Computational Intelligence; Control; Datasets; Engineering; Fall detection; Falls; Feature extraction; Human mechanics; Mechatronics; Methods; Modules; Neural networks; Object recognition; Original Article; Pattern Recognition; Pose estimation; Robotics; Sensors; Sliding; Systems Biology; Keypoints; Temporal features; Sliding window; Attention
• ISSN: 1868-8071; 1868-808X
• DOI: 10.1007/s13042-022-01730-4

Falls have become the second leading cause of accidental death of the elderly. The serious consequences of falls in the elders can be reduced effectively if they can be detected early. This paper proposes a fall detection method based on keypoint attention module and temporal feature extraction. Firstly, the object detection model (YOLOv3) and the pose estimation model (Multi-stage Pose Estimation Network) are used to obtain the body keypoints. Then, we design a sliding window to preprocess the keypoints. The sliding window divides the keypoints in 30 consecutive frames into a group so that the subsequent network can extract the dynamic features from the keypoints. After that, an adaptive keypoint attention module is designed to strengthen the fall-related keypoints. We improve the long-short-term memory network, and utilize it on the strengthened features to extract the dynamic temporal features. Finally, the fully connected layers are used to recognize falls and normal poses. Experimental results show that the proposed approach achieves an accuracy of 99.73% and 99.62% when tested with UR Fall Detection Dataset and Le2i Fall Detection Dataset.