A Novel Chewing Detection System Based on PPG, Audio, and Accelerometry

• Published in: IEEE journal of biomedical and health informatics Vol. 21; no. 3; pp. 607 - 618
• Main Authors: Papapanagiotou, Vasileios, Diou, Christos, Lingchuan Zhou, van den Boer, Janet, Mars, Monica, Delopoulos, Anastasios
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.05.2017; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Accelerometry - methods; Accuracy; Acoustic sensors; acoustic signal processing; Acoustics; Adult; bioinformatics; biomedical informatics; Chewing; Diet; Ear; Eating behavior; Female; Humans; Informatics; Male; Mastication - physiology; Microphones; Monitoring; optical sensors; optical signal processing; Photoplethysmography - methods; ROC Curve; Sensor systems; Sensors; Signal processing algorithms; Signal Processing, Computer-Assisted; Support Vector Machine; Support vector machines; Young Adult
• ISSN: 2168-2194; 2168-2208
• DOI: 10.1109/JBHI.2016.2625271

In the context of dietary management, accurate monitoring of eating habits is receiving increased attention. Wearable sensors, combined with the connectivity and processing of modern smartphones, can be used to robustly extract objective and real-time measurements of human behavior. In particular, for the task of chewing detection, several approaches based on an in-ear microphone can be found in the literature, while other types of sensors have also been reported, such as strain sensors. In this paper, performed in the context of the SPLENDID project, we propose to combine an in-ear microphone with a photoplethysmography (PPG) sensor placed in the ear concha, in a new high accuracy and low sampling rate prototype chewing detection system. We propose a pipeline that initially processes each sensor signal separately, and then fuses both to perform the final detection. Features are extracted from each modality, and support vector machine (SVM) classifiers are used separately to perform snacking detection. Finally, we combine the SVM scores from both signals in a late-fusion scheme, which leads to increased eating detection accuracy. We evaluate the proposed eating monitoring system on a challenging, semifree living dataset of 14 subjects, which includes more than 60 h of audio and PPG signal recordings. Results show that fusing the audio and PPG signals significantly improves the effectiveness of eating event detection, achieving accuracy up to 0.938 and class-weighted accuracy up to 0.892.