Synthetic Data for Multi-Parameter Camera-Based Physiological Sensing

Synthetic data is a powerful tool in training data hungry deep learning algorithms. However, to date, camera-based physiological sensing has not taken full advantage of these techniques. In this work, we leverage a high-fidelity synthetics pipeline for generating videos of faces with faithful blood...

Full description

Saved in:
Bibliographic Details
Published in2021 43rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) Vol. 2021; pp. 3742 - 3748
Main Authors McDuff, Daniel, Liu, Xin, Hernandez, Javier, Wood, Erroll, Baltrusaitis, Tadas
Format Conference Proceeding Journal Article
LanguageEnglish
Published United States IEEE 01.11.2021
Subjects
Algorithms
Avatars
Blood Circulation
Deep Learning
Face
Physiology
Pipelines
Respiration
Systematics
Training
Training data
Online AccessGet full text
ISSN2694-0604
DOI10.1109/EMBC46164.2021.9631031

Cover

More Information
Summary:Synthetic data is a powerful tool in training data hungry deep learning algorithms. However, to date, camera-based physiological sensing has not taken full advantage of these techniques. In this work, we leverage a high-fidelity synthetics pipeline for generating videos of faces with faithful blood flow and breathing patterns. We present systematic experiments showing how physiologically-grounded synthetic data can be used in training camera-based multi-parameter cardiopulmonary sensing. We provide empirical evidence that heart and breathing rate measurement accuracy increases with the number of synthetic avatars in the training set. Furthermore, training with avatars with darker skin types leads to better overall performance than training with avatars with lighter skin types. Finally, we discuss the opportunities that synthetics present in the domain of camera-based physiological sensing and limitations that need to be overcome.
ISSN:2694-0604
DOI:10.1109/EMBC46164.2021.9631031