Comparative analysis of AI algorithms on real medical data for chronic pain detection

• Published in: International journal of medical informatics (Shannon, Ireland) Vol. 203; p. 106002
• Main Authors: Comito, Carmela, Forestiero, Agostino, Macrì, Davide, Metlichin, Elisabetta, Giusti, Gian Domenico, Ramacciati, Nicola
• Format: Journal Article
• Language: English
• Published: Ireland Elsevier B.V 01.11.2025
• Subjects: Algorithms; Artificial Intelligence; Bayes Theorem; Chronic pain; Chronic Pain - diagnosis; Electronic Health Records; Healthcare; Humans; Italy; Machine Learning; Medical documentation; Natural Language Processing; Nursing care; Text analysis; Italy; Healthcare; Text analysis; Nursing care; Artificial intelligence; Medical documentation; Chronic pain
• ISSN: 1386-5056; 1872-8243; 1872-8243
• DOI: 10.1016/j.ijmedinf.2025.106002

Chronic pain is a pervasive healthcare challenge with profound implications for patient well-being, clinical decision-making, and resource allocation. Traditional detection methods often rely on subjective assessments and manual documentation review, which can be time-consuming and unpredictable. Integrating Artificial Intelligence (AI) into healthcare offers a promising approach to enhance chronic pain management through automated and standardized text analysis. This study examines the use of AI in detecting chronic pain from Italian clinical notes. We leverage machine learning (ML) and natural language processing (NLP) techniques to better understand how chronic pain is documented, thereby enabling efficient, data-driven solutions in nursing and medical practice. We trained XGBoost, Gradient Boosting (GBM), and BERT-based models (BioBit, bert-base-italian-xxl) on 1,008 annotated Italian clinical notes. Input texts were encoded using TF-IDF, Word2Vec, or FastText for tree-based models and tokenized for transformers. While models were trained on full notes, evaluation was performed on fragmented text to simulate realistic usage. Bayesian optimization and stratified cross-validation over 30 trials ensured robust hyperparameter tuning and performance estimates. Our AI-based approach achieved high overall accuracy. In particular, XGBoost with TF-IDF embeddings yielded the best performance, reaching an F1-score of 0.92 ± 0.01, with precision at 94%, sensitivity at 91%, and specificity at 93%. The chronic pain notes contained fewer total words (73.91 vs. 119.86, p = 0.0021) and unique words (57.27 vs. 92.78, p = 0.0006) than non-chronic pain notes, underscoring the significance of concise, keyword-rich clinical documentation. Our findings demonstrate the effectiveness of AI in identifying chronic pain cases from fragmentary clinical notes. By focusing on concise, keyword-oriented text, this work establishes a solid baseline for domain-specific NLP approaches in healthcare. The proposed method reduces the burden of manual review, facilitates real-time decision support, and may standardize chronic pain assessment processes. Furthermore, we plan to explore new embedding techniques specifically designed for short, context-limited clinical notes, where dynamic contextual models (e.g., BERT) often encounter challenges due to insufficient extended textual context. •Problem or Issue: Chronic pain detection often depends on subjective, labor-intensive chart reviews prone to variability.•What is Already Known: AI has been studied for pain detection, yet its use on real Italian clinical and nursing notes is limited.•What this Paper Adds: We show XGBoost plus TF-IDF reliably flags chronic-pain cases from brief, keyword-rich notes.•Who would benefit from the new: Findings enable scalable, data-driven pain assessment, aiding clinicians and nurses in workflow efficiency.