Infrared head pose estimation with multi-scales feature fusion on the IRHP database for human attention recognition

• Published in: Neurocomputing (Amsterdam) Vol. 411; pp. 510 - 520
• Main Authors: Liu, Hai, Wang, Xiang, Zhang, Wei, Zhang, Zhaoli, Li, You-Fu
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 21.10.2020
• Subjects: Attention recognition; Convolutional neural network; Feature fusion; Head pose estimation; Infrared image; Attention recognition; Feature fusion; Head pose estimation; Convolutional neural network; Infrared image
• ISSN: 0925-2312; 1872-8286
• DOI: 10.1016/j.neucom.2020.06.066

•A first-of-its-kind infrared head pose database is proposed and established.•Multi-scales feature fusion methods are explored and applied in our convolutional neural network algorithm.•The combinational feature contains high-layer semantic information and low-layer abstract features.•The robustness of the proposed model is verified by extensive experiments. Head pose estimation (HPE) has been widely applied in human attention recognition, robot vision and assistant driving. Infrared (IR) images bear unique advantages of being still effective under visible scenarios, which are resistance to illumination changing and strong penetration. However, the lack of public IR database hinders the research progress in the low illumination environment. In this paper, we establish a first-of-its-kind infrared head pose (IRHP) database and propose a novel convolutional neural network architecture IRHP-Net on the IRHP database. The IRHP database contains 145 kinds of IR head pose images of subjects, and benchmark evaluations are conducted on our database by the facial features-based standard HPE classification methods to prove the usability and effectiveness of IRHP database. To extract the adaptive features for the IR images, a novel multi-scale feature fusion descriptor is developed in the proposed IRHP-Net model. Quantitative assessments of the proposed method on the IRHP images demonstrate the significant improvements over the traditional methods. The new proposed IRHP-Net model can be utilized in human attention recognition and intelligent driving assistant system.