Clustering and Identification of key body extremities through topological analysis of multi-sensors 3D data

• Published in: The Visual computer Vol. 38; no. 3; pp. 1097 - 1120
• Main Authors: Mohsin, Nasreen, Payandeh, Shahram
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.03.2022; Springer Nature B.V
• Subjects: Algorithms; Artificial Intelligence; Clustering; Computer Graphics; Computer Science; Datasets; Human body; Image Processing and Computer Vision; Occlusion; Original Article; Semantics; Sensors; Three dimensional models; Topology; Geodesic mapping; Topological clustering; Pose recognition; Key body part identification
• ISSN: 0178-2789; 1432-2315
• DOI: 10.1007/s00371-021-02070-0

This paper presents a framework of a marker-less human pose recognition system by identifying key body extremity parts through a network of calibrated low-cost depth sensors. The usage of depth sensors overcomes challenges related to low illuminations which usually compromises the information from the RGB cameras. Furthermore, the addition of multiple depth sensors complements the existing information with more visibility and less self-occlusion. A simple algorithm was applied which finds the connections between aligned and updated meshes produced from multiple sensors. These connections help to fuse the meshes into one large geodesic graph network. On this graph, a novel algorithm is applied to identify key body extremities such as head, hands, and feet of a human subject. A geodesic mapping is applied to the fused point cloud to produce a set of distinct topological clusters of 3D points. These clusters generate a hierarchical skeleton tree graph (Reeb graph) and produce a set of features for semantic identification of key body extremities. The combination of both the shape model and semantic classification finally leads to pose recognition. The paper presents the assessment of the proposed framework and its comparison with another available technique in a succession of experimental configurations.