Focus on temporal graph convolutional networks with unified attention for skeleton-based action recognition

• Published in: Applied intelligence (Dordrecht, Netherlands) Vol. 52; no. 5; pp. 5608 - 5616
• Main Authors: Gao, Bing-Kun, Dong, Le, Bi, Hong-Bo, Bi, Yun-Ze
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.03.2022
• Subjects: Artificial Intelligence; Computer Science; Machines; Manufacturing; Mechanical Engineering; Processes; Unified attention model; Skeleton-based action recognition; Graph convolutional networks; Temporal information
• ISSN: 0924-669X; 1573-7497
• DOI: 10.1007/s10489-021-02723-6

Graph convolutional networks (GCN) have received more and more attention in skeleton-based action recognition. Many existing GCN models pay more attention to spatial information and ignore temporal information, but the completion of actions must be accompanied by changes in temporal information. Besides, the channel, spatial, and temporal dimensions often contain redundant information. In this paper, we design a temporal graph convolutional network (FTGCN) module which can concentrate more temporal information and properly balance them for each action. In order to better integrate channel, spatial and temporal information, we propose a unified attention model of the channel, spatial and temporal (CSTA). A basic block containing these two novelties is called FTC-GCN. Extensive experiments on two large-scale datasets, compared with 17 methods on NTU-RGB+D and 8 methods on Kinetics-Skeleton, show that for skeleton-based human action recognition, our method achieves the best performance.