Using artificial intelligence (AI) to assess the prevalence of false or misleading health-related claims

• Published in: Royal Society open science Vol. 11; no. 10; pp. 240698 - 16
• Main Authors: Rose, Les, Bewley, Susan, Payne, Mandy, Colquhoun, David, Perry, Simon
• Format: Journal Article
• Language: English
• Published: England The Royal Society Publishing 01.10.2024; The Royal Society
• Subjects: Accreditation; Acupuncture; Advertising; Alternative medicine; Artificial intelligence; Automation; Chiropractic medicine; complementary and alternative medicine; complementary healthcare; complementary medicine; Compliance; Computer Science and Artificial Intelligence; Councils; Criminal investigations; Evidence; Health care; Human performance; Hypnotherapy; misleading health claims; Osteopathic medicine; Pseudoscience; Public health; Regulation; Therapy; Websites; United Kingdom > UK; complementary medicine; misleading health claims; pseudoscience; complementary and alternative medicine; artificial intelligence; complementary healthcare
• ISSN: 2054-5703; 2054-5703
• DOI: 10.1098/rsos.240698

Complementary healthcare in the United Kingdom is subject to voluntary, publicly funded regulation. Many such practices include pseudoscience. There are concerns that regulated practitioners make misleading health claims. This study used an artificial intelligence (AI) tool to measure the prevalence of such claims. Websites operated by practitioners of pseudoscientific complementary and alternative medicine, registered with the Complementary and Natural Healthcare Council, were downloaded and assessed by the AI, which determined whether a website was relevant to the investigation and, if so, identified health-related claims that it judged as false or misleading, supplying a rationale. Of 6096 registrants, 1326 met the selection criteria, of which 872 clinics had 725 relevant and operational websites. The AI assessed text from 11 771 web pages, identifying false or misleading claims in 704 (97%) of the websites. The AI’s performance was quality-assured by four human assessors, who manually reviewed 23 relevant web pages. Humans identified on average 39.5 claims likely to be judged false or misleading by advertising regulators, the AI identified 36. Humans misidentified an average of 4.8 claims, AI misidentified two. Most practitioners of pseudoscientific therapies registered with the Complementary and Natural Healthcare Council make misleading health claims online. AI could support regulator efficiency.