Combining Users’ Activity Survey and Simulators to Evaluate Human Activity Recognition Systems

• Published in: Sensors (Basel, Switzerland) Vol. 15; no. 4; pp. 8192 - 8213
• Main Authors: Azkune, Gorka, Almeida, Aitor, López-de-Ipiña, Diego, Chen, Liming
• Format: Journal Article
• Language: English
• Published: Switzerland MDPI AG 08.04.2015; MDPI
• Subjects: Activities of daily living; activity recognition; activity survey; Annotations; Cellular telephones; Datasets; Ethics; evaluation methodology; Experiments; Generators; Human error; Human motion; Human subjects; Labeling; Methods; Recognition; Sensors; Similarity; Simulators; synthetic dataset generator
• ISSN: 1424-8220; 1424-8220
• DOI: 10.3390/s150408192

Evaluating human activity recognition systems usually implies following expensive and time-consuming methodologies, where experiments with humans are run with the consequent ethical and legal issues. We propose a novel evaluation methodology to overcome the enumerated problems, which is based on surveys for users and a synthetic dataset generator tool. Surveys allow capturing how different users perform activities of daily living, while the synthetic dataset generator is used to create properly labelled activity datasets modelled with the information extracted from surveys. Important aspects, such as sensor noise, varying time lapses and user erratic behaviour, can also be simulated using the tool. The proposed methodology is shown to have very important advantages that allow researchers to carry out their work more efficiently. To evaluate the approach, a synthetic dataset generated following the proposed methodology is compared to a real dataset computing the similarity between sensor occurrence frequencies. It is concluded that the similarity between both datasets is more than significant.