Deep Learning Models to Predict Diagnostic and Billing Codes Following Visits to a Family Medicine Practice: Development and Validation Study

• Published in: JMIR AI Vol. 4; p. e64279
• Main Authors: Rajaram, Akshay, Judd, Michael, Barber, David
• Format: Journal Article
• Language: English
• Published: Canada JMIR Publications 07.03.2025
• Subjects: Clinical Informatics; Decision Support for Health Professionals; Development and Evaluation of Research Methods, Instruments and Tools; Electronic Health Records; Foundations of AI; Machine Learning; Ontologies, Classifications, and Coding; Original Paper; Tools, Programs and Algorithms; income; predictive system; electronic medical record; predictive model; predictive analytics; EMR; health record; machine learning; artificial intelligence; billing code; personal health record; family physician; primary care; family medicine; electronic notes; family doctor; electronic health record; patient record; ML; algorithm
• ISSN: 2817-1705; 2817-1705
• DOI: 10.2196/64279

Despite significant time spent on billing, family physicians routinely make errors and miss billing opportunities. In other disciplines, machine learning models have predicted Current Procedural Terminology codes with high accuracy. Our objective was to derive machine learning models capable of predicting diagnostic and billing codes from notes recorded in the electronic medical record. We conducted a retrospective algorithm development and validation study involving an academic family medicine practice. Visits between July 1, 2015, and June 30, 2020, containing a physician-authored note and an invoice in the electronic medical record were eligible for inclusion. We trained 2 deep learning models and compared their predictions to codes submitted for reimbursement. We calculated accuracy, recall, precision, F1-score, and area under the receiver operating characteristic curve. Of the 245,045 visits eligible for inclusion, 198,802 (81%) were included in model development. Accuracy was 99.8% and 99.5% for the diagnostic and billing code models, respectively. Recall was 49.4% and 70.3% for the diagnostic and billing code models, respectively. Precision was 55.3% and 76.7% for the diagnostic and billing code models, respectively. The area under the receiver operating characteristic curve was 0.983 for the diagnostic code model and 0.993 for the billing code model. We developed models capable of predicting diagnostic and billing codes from electronic notes following visits to a family medicine practice. The billing code model outperformed the diagnostic code model in terms of recall and precision, likely due to fewer codes being predicted. Work is underway to further enhance model performance and assess the generalizability of these models to other family medicine practices.