TIAN: A time series Imaging Association Network for human abnormal behavior detection

• Published in: Information fusion Vol. 118; p. 102906
• Main Authors: Jin, Dawei, Hu, Yiyi, Chen, Bohan, He, Guoliang, Chen, Jingyu, Shen, Zuwu
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.06.2025
• Subjects: Anomaly detection; Generative adversarial networks; Human activity recognition; Multi-sensor time series; Multi-sensor time series; Generative adversarial networks; Human activity recognition; Anomaly detection
• ISSN: 1566-2535
• DOI: 10.1016/j.inffus.2024.102906

In smart healthcare, human activity recognition (HAR) is an effective technology that provides personalized treatment plans to patients. Specifically, HAR technology is used to quickly detect abnormal physical conditions in patients, thereby improving medical efficiency and service quality. With the growth of the Internet of Things and widespread adoption of wearable devices, human abnormal behavior detection technology based on multi-sensor time series has shown excellent application value. Although recent deep learning methods have shown great potential for anomaly detection, they still need to fully utilize information on behavioral differences across subjects. Meanwhile, cross-modal representation of most time series has demonstrated excellent performance, but few studies have considered both the internal evolutionary and periodic features of time-series. Addressing this research gap, this paper proposes TIAN, or Time-series Imaging Association Network, for anomaly detection in HAR. TIAN first fuses multiple time-series cross-modal feature representation methods to encode time-series into images, thereby better capturing rich time-series feature information. Furthermore, the critical innovation of TIAN is that it distinguishes habit noise across subjects and facilitates the learning of invariant features of various behavioral patterns. Experimental results on the WISDM, HARTH, and LRAD datasets demonstrate that TIAN performs well compared with existing baselines. Meanwhile, the ablation results on the WISDM dataset show that removing each component in the TIAN system consistently degrades the performance. In addition, TIAN is more robust to input noise compared to other baseline models on the HARTH dataset. •A novel approach for encoding time series as images is introduced.•TIAN first combines time series spatiotemporal features with subject differences.•Comparing multiple baselines, TIAN achieves SOTA results.•Ablation studies prove the effectiveness of each module in the TIAN.