Towards LLM-Powered Ambient Sensor Based Multi-Person Human Activity Recognition

• Published in: Proceedings - International Conference on Parallel and Distributed Systems pp. 609 - 616
• Main Authors: Chen, Xi, Cumin, Julien, Ramparany, Fano, Vaufreydaz, Dominique
• Format: Conference Proceeding
• Language: English
• Published: IEEE 10.10.2024
• Subjects: Cognition; Complexity theory; Data models; Estimation; Face recognition; Human activity recognition; IoT; Large Language Model; Medical services; Older adults; Robustness; Security; Smart Home
• ISSN: 2690-5965
• DOI: 10.1109/ICPADS63350.2024.00085

Human Activity Recognition (HAR) is one of the central problems in fields such as healthcare, elderly care, and security at home. However, traditional ambient-sensor-based HAR approaches face challenges including data scarcity, difficulties in model generalization, and the complexity of recognizing activities in multi-person scenarios. This paper proposes a large-language-model-based framework called LAHAR which addresses HAR in multi-person scenarios. By endowing LLMs with inter-sensor relevance estimation and sensor-subject relevance estimation abilities, LAHAR can assign sensor events to the corresponding subjects. By providing action-level descriptions of sensor events and subsequently performing activity-level reasoning based on these descriptions, LAHAR is ultimately able to process data spanning several tens of hours with second-level resolution and results in an activity timeline for each subject. We validated LAHAR on the ARAS dataset. The results demonstrate that LAHAR achieves comparable accuracy to the state-of-the-art method at higher resolutions and maintains robustness in multiperson scenarios.