A systematic review of the diagnostic accuracy of artificial intelligence-based computer programs to analyze chest x-rays for pulmonary tuberculosis

• Published in: PloS one Vol. 14; no. 9; p. e0221339
• Main Authors: Harris, Miriam, Qi, Amy, Jeagal, Luke, Torabi, Nazi, Menzies, Dick, Korobitsyn, Alexei, Pai, Madhukar, Nathavitharana, Ruvandhi R., Ahmad Khan, Faiz
• Format: Journal Article
• Language: English
• Published: United States Public Library of Science 03.09.2019; Public Library of Science (PLoS)
• Subjects: Abnormalities; Accuracy; Area Under Curve; Artificial Intelligence; Automation; Bias; Chest; Chest x-rays; Clinical trials; Computer and Information Sciences; Computer programs; Computers; Concretes; Deep learning; Design; Diagnosis; Diagnosis, Computer-Assisted - methods; Diagnosis, Computer-Assisted - standards; Diagnosis, Computer-Assisted - statistics & numerical data; Diagnostic software; Diagnostic systems; Epidemiology; Evaluation; Humans; Infectious diseases; Learning algorithms; Lung diseases; Machine learning; Medical diagnosis; Medical research; Medicine; Medicine and Health Sciences; People and Places; Physical Sciences; Predictive Value of Tests; Pulmonary tuberculosis; Quality Assurance, Health Care; Radiography; Reading; Research and Analysis Methods; Risk assessment; Sensitivity and Specificity; Software; Studies; Systematic review; Tuberculosis; Tuberculosis, Pulmonary - diagnostic imaging; X-rays; Canada; Montreal Quebec Canada; United States > US; Massachusetts
• ISSN: 1932-6203; 1932-6203
• DOI: 10.1371/journal.pone.0221339

We undertook a systematic review of the diagnostic accuracy of artificial intelligence-based software for identification of radiologic abnormalities (computer-aided detection, or CAD) compatible with pulmonary tuberculosis on chest x-rays (CXRs). We searched four databases for articles published between January 2005-February 2019. We summarized data on CAD type, study design, and diagnostic accuracy. We assessed risk of bias with QUADAS-2. We included 53 of the 4712 articles reviewed: 40 focused on CAD design methods ("Development" studies) and 13 focused on evaluation of CAD ("Clinical" studies). Meta-analyses were not performed due to methodological differences. Development studies were more likely to use CXR databases with greater potential for bias as compared to Clinical studies. Areas under the receiver operating characteristic curve (median AUC [IQR]) were significantly higher: in Development studies AUC: 0.88 [0.82-0.90]) versus Clinical studies (0.75 [0.66-0.87]; p-value 0.004); and with deep-learning (0.91 [0.88-0.99]) versus machine-learning (0.82 [0.75-0.89]; p = 0.001). We conclude that CAD programs are promising, but the majority of work thus far has been on development rather than clinical evaluation. We provide concrete suggestions on what study design elements should be improved.