Artificial intelligence versus neurologists: A comparative study on multiple sclerosis expertise

• Published in: Clinical neurology and neurosurgery Vol. 250; p. 108785
• Main Authors: Yaman Kula, Aslı, Durmaz Çeli̇k, Nazlı, Özben, Serkan, Yatmazoğlu Çeti̇n, Merve, Köseoğlu, Mesrure
• Format: Journal Article
• Language: English
• Published: Netherlands Elsevier B.V 01.03.2025; Elsevier Limited
• Subjects: Algorithms; Artificial Intelligence; Central nervous system; Cerebrospinal fluid; Chatbots; Clinical Competence; Clinical decision making; Diagnosis; Disease; Female; Hospitals; Humans; Internship and Residency; Magnetic resonance imaging; Male; Multiple choice; Multiple sclerosis; Multiple Sclerosis - diagnosis; Multiple Sclerosis - therapy; Neurodegenerative diseases; Neurologist; Neurologists - standards; Neurology; Neurosurgery; Neurologist; Artificial intelligence; Multiple sclerosis
• ISSN: 0303-8467; 1872-6968; 1872-6968
• DOI: 10.1016/j.clineuro.2025.108785

Multiple sclerosis (MS) is an autoimmune neurodegenerative disease affecting the central nervous system. MS diagnosis is complex, requiring magnetic resonance imaging and cerebrospinal fluid analysis due to the lack of definitive biomarkers. Although treatment advancements have reduced disability, diagnostic and therapeutic challenges remain, even among MS-specialized neurologists. Artificial Intelligence (AI) tools, which analyze large datasets, are increasingly used in healthcare, especially for diagnosis and treatment. This study aims to assess the accuracy and scope of knowledge regarding MS, focusing on diagnosis, treatment options and management strategies, as tested among neurologists and AI bots. Twenty multiple-choice questions, developed by MS-experienced neurology academics, were administered to 37 neurology specialists and 79 neurology residents in Turkey. The same questions were posed to AI platforms, including ChatGPT-4.0, GPT-4o, Gemini 1.5 Pro, Claude 3.5, Perplexity, and Perplexity Pro. Neurology specialists answered 12.05 ± 4.01 questions correctly on average, while residents scored 9.08 ± 3.41. Among residents with more than two years of training, the correct answer rate improved to 11.96 ± 3.5. Specialists active in MS clinics scored significantly higher than other neurologists (17.67 ± 1.75). AI platforms scored between 14 and 19 out of 20; with an average of 17.0 ± 1.79 with Claude 3.5 scoring highest. The findings suggest AI holds promise in supporting MS diagnosis and treatment, though challenges remain in nuanced cases. While AI complements neurologists, further studies are essential to understand its potential and limitations. Aggregated data will be shared upon written request to the corresponding author. •A 20-question MS knowledge test was conducted with neurologists and AI bots.•MS specialists and AI bots demonstrated significantly higher accuracy than other groups.•Neurology residents’ scores improved after two years of training, nearing specialists.•AI platforms achieved an 85 % average accuracy, with Claude 3.5 scoring the highest.