Multi-modal multi-task deep neural networks for sleep disordered breathing assessment using cardiac and audio signals

• Published in: International journal of medical informatics (Shannon, Ireland) Vol. 201; p. 105932
• Main Authors: Xie, Jiali, Fonseca, Pedro, van Dijk, Johannes P., Overeem, Sebastiaan, Long, Xi
• Format: Journal Article
• Language: English
• Published: Ireland Elsevier B.V 01.09.2025
• Subjects: Adult; Audio; Deep Learning; Electrocardiogram; Electrocardiography - methods; Female; Humans; Male; Middle Aged; Multi-modal; Neural Networks, Computer; Obstructive sleep apnea; Polysomnography - methods; Signal Processing, Computer-Assisted; Sleep Apnea Syndromes - diagnosis; Sleep Apnea Syndromes - physiopathology; Sleep-wake; Deep learning; Sleep-wake; Multi-modal; Audio; Obstructive sleep apnea; Electrocardiogram
• ISSN: 1386-5056; 1872-8243; 1872-8243
• DOI: 10.1016/j.ijmedinf.2025.105932

•A deep learning model was proposed to detect sleep disordered breathing using cardiac and audio signals.•This study trained models with inter-beat interval, EDR, and MFCC features, and compared results across input combinations.•Results show combining audio and cardiac signals improves performance with IBI alone or with both IBI and EDR available.•The estimated apnea-hypopnea index showed a significant correlation with the polysomnographic gold standard. Sleep disordered breathing (SDB) is one of the most common sleep disorders and has short-term consequences for daytime functioning while being a risk factor for several conditions, such as cardiovascular disease. Polysomnography, the current diagnostic gold standard, is expensive and has limited accessibility. Therefore, cost-effective and easily accessible methods for SDB detection are needed. Both cardiac and audio signals have received attention for SDB detection as they can be obtained with unobtrusive sensors, suitable for home applications. This paper introduces a multi-modal multi-task deep learning approach for SDB assessment using a combination of cardiac and audio signals under the assumption that they can provide complementary information. We aimed to estimate the apnea-hypopnea index (AHI) and assess AHI-based SDB severity through the detection of SDB events, combined with total sleep time estimated from simultaneous sleep-wake classification. Inter-beat interval and electrocardiogram-derived respiration from the electrocardiogram, and Mel-scale frequency cepstral coefficients from concurrent audio recordings were used as inputs. We compared the performance of several models trained with different combinations of these inputs. Using cross-validation with a dataset comprising overnight recordings of 161 subjects, we achieved an F1 score of 0.588 for SDB event detection, a correlation coefficient of 0.825 for AHI estimation, and an accuracy of 57.8% for SDB severity classification (normal, mild, moderate, and severe). Results show that combining cardiac and audio signals can enhance the performance of SDB detection and highlight the potential of multi-modal data fusion for further research in this domain.