Adversarial prompt and fine-tuning attacks threaten medical large language models

• Published in: Nature communications Vol. 16; no. 1; pp. 9011 - 10
• Main Authors: Yang, Yifan, Jin, Qiao, Huang, Furong, Lu, Zhiyong
• Format: Journal Article
• Language: English
• Published: London Nature Publishing Group UK 09.10.2025; Nature Publishing Group; Nature Portfolio
• Subjects: 631/114/1305; 692/308; Benchmarks; Computer Security; Delivery of Health Care; Disease prevention; Health care; Health services; Humanities and Social Sciences; Humans; Immunization; Language; Large Language Models; Medical imaging; multidisciplinary; Patients; Science; Science (multidisciplinary); Ultrasonic imaging; Vaccines; X-rays
• ISSN: 2041-1723; 2041-1723
• DOI: 10.1038/s41467-025-64062-1

The integration of Large Language Models (LLMs) into healthcare applications offers promising advancements in medical diagnostics, treatment recommendations, and patient care. However, the susceptibility of LLMs to adversarial attacks poses a significant threat, potentially leading to harmful outcomes in delicate medical contexts. This study investigates the vulnerability of LLMs to two types of adversarial attacks–prompt injections with malicious instructions and fine-tuning with poisoned samples–across three medical tasks: disease prevention, diagnosis, and treatment. Utilizing real-world patient data, we demonstrate that both open-source and proprietary LLMs are vulnerable to malicious manipulation across multiple tasks. We discover that while integrating poisoned data does not markedly degrade overall model performance on medical benchmarks, it can lead to noticeable shifts in fine-tuned model weights, suggesting a potential pathway for detecting and countering model attacks. This research highlights the urgent need for robust security measures and the development of defensive mechanisms to safeguard LLMs in medical applications, to ensure their safe and effective deployment in healthcare settings. Large language models hold significant potential in healthcare settings. This study exposes their vulnerability in medical applications and demonstrates the inadequacy of existing safeguards, highlighting the need for future studies to develop reliable methods for detecting and mitigating these risks.