Comparative performance of PD‐L1 scoring by pathologists and AI algorithms

• Published in: Histopathology Vol. 87; no. 1; pp. 90 - 100
• Main Authors: Plass, Markus, Olteanu, Gheorghe‐Emilian, Dacic, Sanja, Kern, Izidor, Zacharias, Martin, Popper, Helmut, Fukuoka, Junya, Ishijima, Sosuke, Kargl, Michaela, Murauer, Christoph, Kalson, Lipika, Brcic, Luka
• Format: Journal Article
• Language: English
• Published: England Wiley Subscription Services, Inc 01.07.2025; John Wiley and Sons Inc
• Subjects: AI algorithms; Algorithms; Artificial Intelligence; B7-H1 Antigen - analysis; B7-H1 Antigen - biosynthesis; B7-H1 Antigen - metabolism; Biomarkers, Tumor - analysis; Carcinoma, Non-Small-Cell Lung - diagnosis; Carcinoma, Non-Small-Cell Lung - pathology; Decision making; Humans; Immune checkpoint inhibitors; Immunohistochemistry; Light microscopy; Lung cancer; Lung carcinoma; Lung Neoplasms - diagnosis; Lung Neoplasms - pathology; Non-small cell lung carcinoma; NSCLC; Observer Variation; Original; Pathologists; PD-L1 protein; PD‐L1 expression; Small cell lung carcinoma; Software; tumour proportion score (TPS); PD‐L1 expression; NSCLC; tumour proportion score (TPS); AI algorithms; pathologists
• ISSN: 0309-0167; 1365-2559; 1365-2559
• DOI: 10.1111/his.15432

Aim This study evaluates the comparative effectiveness of pathologists versus artificial intelligence (AI) algorithms in scoring PD‐L1 expression in non‐small cell lung carcinoma (NSCLC). Immune‐checkpoint inhibitors have revolutionized NSCLC treatment, with PD‐L1 expression, measured as the tumour proportion score (TPS), serving as a critical predictive biomarker for therapeutic response. Methods and Results In our analysis, 51 SP263‐stained NSCLC cases were scored by six pathologists using light microscopy and whole‐slide images (WSI), alongside evaluations by two commercially available software tools: uPath software (Roche) and the PD‐L1 Lung Cancer TME application (Visiopharm). The study examined intra‐ and interobserver agreement among pathologists at TPS cutoffs of 1% and 50%, revealing moderate interobserver agreement (Fleiss' kappa 0.558) for TPS <1% and almost perfect agreement (Fleiss' kappa 0.873) for TPS ≥50%. Intraobserver consistency was high, with Cohen's kappa ranging from 0.726 to 1.0. Comparisons between the AI algorithms and the median pathologist scores showed fair agreement for uPath (Fleiss' kappa 0.354) and substantial agreement for the Visiopharm application (Fleiss' kappa 0.672) at the 50% TPS cutoff. Conclusion These results indicate that while there is strong interobserver concordance among pathologists at higher TPS levels, the performance of AI algorithms is less consistent. The study underscores the need for further refinement of AI tools to match the reliability of expert human evaluation, particularly in critical clinical decision‐making contexts. Pathologists demonstrate higher consistency than analysed AI algorithms in PD‐L1 scoring at critical Tumour Proportion Score cutoffs in non‐small cell lung cancer. This emphasizes the need to enhance AI tools to better support pathologists in everyday assessments.