Machine learning algorithms performed no better than regression models for prognostication in traumatic brain injury

• Published in: Journal of clinical epidemiology Vol. 122; no. 2; pp. 95 - 107
• Main Authors: Gravesteijn, Benjamin Y., Lingsma, Hester F., Nelson, David, Steyerberg, Ewout W., Andreassen, Lasse, Anke, Audny, Audibert, Gérard, Barzó, Pál, Beauvais, Romuald, Bellander, Bo-Michael, Benali, Habib, Beretta, Luigi, Blaabjerg, Morten, Brazinova, Alexandra, Brooker, Joanne, Brorsson, Camilla, Bullinger, Monika, Lozano, Guillermo Carbayo, Chieregato, Arturo, Coburn, Mark, Dawes, Helen, Della Corte, Francesco, Đilvesi, Đula, Esser, Patrick, Martin Fabricius, Erzsébet Ezer, Feigin, Kelly Foks, Valery L., Gagliardo, Pablo, Gantner, Dashiell, Gao, Guoyi, Glocker, Ben, Gravesteijn, Benjamin, Grossi, Francesca, Gruen, Russell L., Helbok, Raimund, Jacobs, Bram, Ji-yao Jiang, Mike Jarrett, Karan, Mladen, Kolias, Angelos G., Kompanje, Erwin, Koraropoulos, Evgenios, Koskinen, Lars-Owe, Laureys, Steven, Legrand, Valerie, Lightfoot, Roger, Lingsma, Hester, Castaño-León, Ana M., Manley, Geoffrey, Maréchal, Hugues, McMahon, Catherine, Menovsky, Tomas, Mulazzi, Davide, Nair, Nandesh, Nieboer, Daan, Nyirádi, József, Oresic, Matej, Ortolano, Fabrizio, Palotie, Aarno, Parizel, Paul M., Pirinen, Matti, Polinder, Suzanne, Posti, Jussi P., Puybasset, Louis, Radoi, Andreea, Rhodes, Jonathan, Richardson, Sylvia, Richter, Sophie, Ripatti, Samuli, Rocka, Saulius, Roe, Cecilie, Rosenlund, Christina, Rossaint, Rolf, Sakowitz, Oliver, Sandor, Janos, Schoechl, Herbert, Schou, Rico Frederik, Schwendenwein, Elisabeth, Sewalt, Charlie, Stamatakis, Emmanuel, Stanworth, Simon, Ao, Braden Te, Tenovuo, Olli, Theadom, Alice, Tibboel, Dick, Vajkoczy, Peter, Vallance, Shirley, van der Naalt, Joukje, van Dijck, Jeroen T.J.M., van Essen, Thomas A., van Heugten, Caroline, Van Praag, Dominique, Vyvere, Thijs Vande, Vespa, Paul M., Voormolen, Daphne, Wang, Kevin K.W., Wiegers, Eveline, Winzeck, Stefan, Wolf, Stefan, Yang, Zhihui, Younsi, Alexander, Zelinkova, Veronika
• Format: Journal Article
• Language: English
• Published: United States Elsevier Inc 01.06.2020; Elsevier Limited; Elsevier; Elsevier USA
• Subjects: Adult; Algorithms; Artificial neural networks; Brain; Brain Injuries, Traumatic - therapy; Calibration; Cohort analysis; Cohort study; Coma; Confidence intervals; Data science; Datasets; Decision Making, Computer-Assisted; Epidemiology; Female; Glasgow Coma Scale; Head injuries; Human health and pathology; Humans; Internal Medicine; Laboratories; Learning algorithms; Life Sciences; Logistic Models; Machine Learning; Male; Medical disciplines: 700; Medical prognosis; Medical technology: 620; Medisinsk teknologi: 620; Medisinske Fag: 700; Middle Aged; Models, Statistical; Mortality; Neural networks; Neurosciences & behavior; Neurosciences & comportement; Prediction; Prediction models; Prognosis; Regression analysis; Regression models; Sciences sociales & comportementales, psychologie; Social & behavioral sciences, psychology; Support vector machines; Technology: 500; Teknologi: 500; Traumatic brain injury; Variables; VDP; Data science; Prognosis; Traumatic brain injury; Cohort study; Machine learning; Prediction; NN; LR; Gradient boosting machine; Glasgow coma scale; SVM; Support vector machine; Neural network; Random Forest; Logistic regression; GBM; RF; GCS; TBI; ML
• ISSN: 0895-4356; 1878-5921; 1878-5921
• DOI: 10.1016/j.jclinepi.2020.03.005

We aimed to explore the added value of common machine learning (ML) algorithms for prediction of outcome for moderate and severe traumatic brain injury. We performed logistic regression (LR), lasso regression, and ridge regression with key baseline predictors in the IMPACT-II database (15 studies, n = 11,022). ML algorithms included support vector machines, random forests, gradient boosting machines, and artificial neural networks and were trained using the same predictors. To assess generalizability of predictions, we performed internal, internal-external, and external validation on the recent CENTER-TBI study (patients with Glasgow Coma Scale <13, n = 1,554). Both calibration (calibration slope/intercept) and discrimination (area under the curve) was quantified. In the IMPACT-II database, 3,332/11,022 (30%) died and 5,233(48%) had unfavorable outcome (Glasgow Outcome Scale less than 4). In the CENTER-TBI study, 348/1,554(29%) died and 651(54%) had unfavorable outcome. Discrimination and calibration varied widely between the studies and less so between the studied algorithms. The mean area under the curve was 0.82 for mortality and 0.77 for unfavorable outcomes in the CENTER-TBI study. ML algorithms may not outperform traditional regression approaches in a low-dimensional setting for outcome prediction after moderate or severe traumatic brain injury. Similar to regression-based prediction models, ML algorithms should be rigorously validated to ensure applicability to new populations.