Improving Radiology Report Generation Quality and Diversity through Reinforcement Learning and Text Augmentation

• Published in: Bioengineering (Basel) Vol. 11; no. 4; p. 351
• Main Authors: Parres, Daniel, Albiol, Alberto, Paredes, Roberto
• Format: Journal Article
• Language: English
• Published: Switzerland MDPI AG 01.04.2024
• Subjects: Architecture; Automation; Computational linguistics; Datasets; Deep learning; Knowledge representation; Language processing; machine learning; Natural language interfaces; Natural language processing; Neural networks; Radiology; radiology report generation; Radiology, Medical; Reinforcement; reinforcement learning; Reinforcement learning (Machine learning); Semantics; Technology application; text augmentation; vision transformer; X-rays; Spain; text augmentation; chest X-rays; deep learning; radiology report generation; medical image; machine learning; vision transformer; text generation; reinforcement learning
• ISSN: 2306-5354; 2306-5354
• DOI: 10.3390/bioengineering11040351

Deep learning is revolutionizing radiology report generation (RRG) with the adoption of vision encoder–decoder (VED) frameworks, which transform radiographs into detailed medical reports. Traditional methods, however, often generate reports of limited diversity and struggle with generalization. Our research introduces reinforcement learning and text augmentation to tackle these issues, significantly improving report quality and variability. By employing RadGraph as a reward metric and innovating in text augmentation, we surpass existing benchmarks like BLEU4, ROUGE-L, F1CheXbert, and RadGraph, setting new standards for report accuracy and diversity on MIMIC-CXR and Open-i datasets. Our VED model achieves F1-scores of 66.2 for CheXbert and 37.8 for RadGraph on the MIMIC-CXR dataset, and 54.7 and 45.6, respectively, on Open-i. These outcomes represent a significant breakthrough in the RRG field. The findings and implementation of the proposed approach, aimed at enhancing diagnostic precision and radiological interpretations in clinical settings, are publicly available on GitHub to encourage further advancements in the field.