Depth features to recognise dyadic interactions

• Published in: IET computer vision Vol. 12; no. 3; pp. 331 - 339
• Main Authors: Keçeli, Ali Seydi, Kaya, Aydın, Can, Ahmet Burak
• Format: Journal Article
• Language: English
• Published: The Institution of Engineering and Technology 01.04.2018; Wiley
• Subjects: activity recognition; composite kernel SVM; depth feature extraction; depth frames; depth sensors; dyadic interaction recognition; feature extraction; gesture recognition; hand-crafted features; human-computer interaction; human-to-human interactions; image representation; image sequences; interaction sequence representation; joint angles; joint distances; joint spherical coordinates; K-nearest neighbour; nonlinear input mapping method; random forest; Relieff algorithm; Research Article; support vector machine; support vector machines; SVM classifiers; support vector machines; depth feature extraction; depth frames; joint angles; depth sensors; random forest; human-to-human interactions; interaction sequence representation; joint distances; joint spherical coordinates; SVM classifiers; Relieff algorithm; human-computer interaction; gesture recognition; support vector machine; feature extraction; image representation; hand-crafted features; composite kernel SVM; nonlinear input mapping method; dyadic interaction recognition; K-nearest neighbour; image sequences; activity recognition
• ISSN: 1751-9632; 1751-9640; 1751-9640
• DOI: 10.1049/iet-cvi.2017.0204

Usage of depth sensors in activity recognition is an emerging technology in human–computer interaction. This study presents an approach to recognise human-to-human interactions by using depth information. Both hand-crafted features and deep features extracted from depth frames are studied. After selecting and ranking strong features with Relieff algorithm, depth frames are assigned to words. Then, interaction sequences are represented as histograms of words and non-linear input mapping is applied over histogram bins to minimise differences among various subjects. Random forest, K-nearest neighbour, and support vector machine (SVM) classifiers are trained using these histograms. The final model is tested on SBU and K3HI datasets and compared with the methods in the literature. In the experiments, joint distances, joint angles and spherical coordinates of the joints were the best performing features. The most successful results are obtained with the composite kernel SVM with Relieff and input mapping methods. While Relieff algorithm helps to select and rank the best features in the feature set, input mapping reduces differences among interactions of various actors.