Using Generative AI to Improve the Performance and Interpretability of Rule-Based Diagnosis of Type 2 Diabetes Mellitus

• Published in: Information (Basel) Vol. 15; no. 3; p. 162
• Main Authors: Kopitar, Leon, Fister, Iztok, Stiglic, Gregor
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 01.03.2024
• Subjects: Accuracy; Analysis; association rule mining; Classification; Computational linguistics; Data mining; Datasets; Decision analysis; Decision making; Dextrose; Diabetes; Diabetes mellitus; Diabetic retinopathy; Diagnosis; diagnostics; Generative artificial intelligence; Glucose; GPT; Insulin; interpretability; Language processing; Machine learning; Natural language interfaces; Natural language processing; Optimization algorithms; Patients; Performance enhancement; Public health; Regression analysis; Similarity; Source code; Tables (data); Type 2 diabetes; India
• ISSN: 2078-2489; 2078-2489
• DOI: 10.3390/info15030162

Introduction: Type 2 diabetes mellitus is a major global health concern, but interpreting machine learning models for diagnosis remains challenging. This study investigates combining association rule mining with advanced natural language processing to improve both diagnostic accuracy and interpretability. This novel approach has not been explored before in using pretrained transformers for diabetes classification on tabular data. Methods: The study used the Pima Indians Diabetes dataset to investigate Type 2 diabetes mellitus. Python and Jupyter Notebook were employed for analysis, with the NiaARM framework for association rule mining. LightGBM and the dalex package were used for performance comparison and feature importance analysis, respectively. SHAP was used for local interpretability. OpenAI GPT version 3.5 was utilized for outcome prediction and interpretation. The source code is available on GitHub. Results: NiaARM generated 350 rules to predict diabetes. LightGBM performed better than the GPT-based model. A comparison of GPT and NiaARM rules showed disparities, prompting a similarity score analysis. LightGBM’s decision making leaned heavily on glucose, age, and BMI, as highlighted in feature importance rankings. Beeswarm plots demonstrated how feature values correlate with their influence on diagnosis outcomes. Discussion: Combining association rule mining with GPT for Type 2 diabetes mellitus classification yields limited effectiveness. Enhancements like preprocessing and hyperparameter tuning are required. Interpretation challenges and GPT’s dependency on provided rules indicate the necessity for prompt engineering and similarity score methods. Variations in feature importance rankings underscore the complexity of T2DM. Concerns regarding GPT’s reliability emphasize the importance of iterative approaches for improving prediction accuracy.