COVID-19 Mortality Prediction From Deep Learning in a Large Multistate Electronic Health Record and Laboratory Information System Data Set: Algorithm Development and Validation

• Published in: Journal of medical Internet research Vol. 23; no. 9; p. e30157
• Main Authors: Sankaranarayanan, Saranya, Balan, Jagadheshwar, Walsh, Jesse R, Wu, Yanhong, Minnich, Sara, Piazza, Amy, Osborne, Collin, Oliver, Gavin R, Lesko, Jessica, Bates, Kathy L, Khezeli, Kia, Block, Darci R, DiGuardo, Margaret, Kreuter, Justin, O’Horo, John C, Kalantari, John, Klee, Eric W, Salama, Mohamed E, Kipp, Benjamin, Morice, William G, Jenkinson, Garrett
• Format: Journal Article
• Language: English
• Published: Toronto Gunther Eysenbach MD MPH, Associate Professor 28.09.2021; JMIR Publications
• Subjects: Acids; Age; Algorithms; Blood pressure; Cognitive style; Comorbidity; Coronaviruses; COVID-19; Data collection; Datasets; Deep learning; Diabetes; Electronic health records; Fibrinogen; Information systems; Iron; Kidney diseases; Laboratories; Medical treatment; Missing data; Mortality; Multimedia; Neural networks; Original Paper; Oxygen; Pandemics; Patients; Polymerase chain reaction; Recurrent; Saturation; Serum; Severe acute respiratory syndrome; Severe acute respiratory syndrome coronavirus 2; Stratification; Time series; Variables
• ISSN: 1438-8871; 1439-4456; 1438-8871
• DOI: 10.2196/30157

Background: COVID-19 is caused by the SARS-CoV-2 virus and has strikingly heterogeneous clinical manifestations, with most individuals contracting mild disease but a substantial minority experiencing fulminant cardiopulmonary symptoms or death. The clinical covariates and the laboratory tests performed on a patient provide robust statistics to guide clinical treatment. Deep learning approaches on a data set of this nature enable patient stratification and provide methods to guide clinical treatment. Objective: Here, we report on the development and prospective validation of a state-of-the-art machine learning model to provide mortality prediction shortly after confirmation of SARS-CoV-2 infection in the Mayo Clinic patient population. Methods: We retrospectively constructed one of the largest reported and most geographically diverse laboratory information system and electronic health record of COVID-19 data sets in the published literature, which included 11,807 patients residing in 41 states of the United States of America and treated at medical sites across 5 states in 3 time zones. Traditional machine learning models were evaluated independently as well as in a stacked learner approach by using AutoGluon, and various recurrent neural network architectures were considered. The traditional machine learning models were implemented using the AutoGluon-Tabular framework, whereas the recurrent neural networks utilized the TensorFlow Keras framework. We trained these models to operate solely using routine laboratory measurements and clinical covariates available within 72 hours of a patient’s first positive COVID-19 nucleic acid test result. Results: The GRU-D recurrent neural network achieved peak cross-validation performance with 0.938 (SE 0.004) as the area under the receiver operating characteristic (AUROC) curve. This model retained strong performance by reducing the follow-up time to 12 hours (0.916 [SE 0.005] AUROC), and the leave-one-out feature importance analysis indicated that the most independently valuable features were age, Charlson comorbidity index, minimum oxygen saturation, fibrinogen level, and serum iron level. In the prospective testing cohort, this model provided an AUROC of 0.901 and a statistically significant difference in survival (P<.001, hazard ratio for those predicted to survive, 95% CI 0.043-0.106). Conclusions: Our deep learning approach using GRU-D provides an alert system to flag mortality for COVID-19–positive patients by using clinical covariates and laboratory values within a 72-hour window after the first positive nucleic acid test result.