Near real-time intraoperative brain tumor diagnosis using stimulated Raman histology and deep neural networks

• Published in: Nature medicine Vol. 26; no. 1; pp. 52 - 58
• Main Authors: Hollon, Todd C., Pandian, Balaji, Adapa, Arjun R., Urias, Esteban, Save, Akshay V., Khalsa, Siri Sahib S., Eichberg, Daniel G., D’Amico, Randy S., Farooq, Zia U., Lewis, Spencer, Petridis, Petros D., Marie, Tamara, Shah, Ashish H., Garton, Hugh J. L., Maher, Cormac O., Heth, Jason A., McKean, Erin L., Sullivan, Stephen E., Hervey-Jumper, Shawn L., Patil, Parag G., Thompson, B. Gregory, Sagher, Oren, McKhann, Guy M., Komotar, Ricardo J., Ivan, Michael E., Snuderl, Matija, Otten, Marc L., Johnson, Timothy D., Sisti, Michael B., Bruce, Jeffrey N., Muraszko, Karin M., Trautman, Jay, Freudiger, Christian W., Canoll, Peter, Lee, Honglak, Camelo-Piragua, Sandra, Orringer, Daniel A.
• Format: Journal Article
• Language: English
• Published: New York Nature Publishing Group US 01.01.2020; Nature Publishing Group
• Subjects: 631/114/1305; 631/1647/527/1821; 631/67/2321; 692/308/575; Algorithms; Artificial intelligence; Artificial neural networks; Biomedical and Life Sciences; Biomedicine; Brain; Brain cancer; Brain Neoplasms - diagnosis; Brain Neoplasms - diagnostic imaging; Brain tumors; Cancer; Cancer Research; Clinical trials; Clinical Trials as Topic; Computer Systems; Deep Learning; Diagnosis; Diagnostic systems; Feature recognition; Health aspects; Histology; Humans; Image Processing, Computer-Assisted; Image segmentation; Infectious Diseases; Letter; Medical imaging; Metabolic Diseases; Molecular Medicine; Monitoring, Intraoperative; Neural networks; Neural Networks, Computer; Neuroimaging; Neurosciences; Pathology; Probability; Raman spectroscopy; Real time; Spectrum Analysis, Raman; Tumors; Workflow; United States
• ISSN: 1078-8956; 1546-170X; 1546-170X
• DOI: 10.1038/s41591-019-0715-9

Intraoperative diagnosis is essential for providing safe and effective care during cancer surgery 1 . The existing workflow for intraoperative diagnosis based on hematoxylin and eosin staining of processed tissue is time, resource and labor intensive 2 , 3 . Moreover, interpretation of intraoperative histologic images is dependent on a contracting, unevenly distributed, pathology workforce 4 . In the present study, we report a parallel workflow that combines stimulated Raman histology (SRH) 5 – 7 , a label-free optical imaging method and deep convolutional neural networks (CNNs) to predict diagnosis at the bedside in near real-time in an automated fashion. Specifically, our CNNs, trained on over 2.5 million SRH images, predict brain tumor diagnosis in the operating room in under 150 s, an order of magnitude faster than conventional techniques (for example, 20–30 min) 2 . In a multicenter, prospective clinical trial ( n  = 278), we demonstrated that CNN-based diagnosis of SRH images was noninferior to pathologist-based interpretation of conventional histologic images (overall accuracy, 94.6% versus 93.9%). Our CNNs learned a hierarchy of recognizable histologic feature representations to classify the major histopathologic classes of brain tumors. In addition, we implemented a semantic segmentation method to identify tumor-infiltrated diagnostic regions within SRH images. These results demonstrate how intraoperative cancer diagnosis can be streamlined, creating a complementary pathway for tissue diagnosis that is independent of a traditional pathology laboratory. A prospective, multicenter, case–control clinical trial evaluates the potential of artificial intelligence for providing accurate bedside diagnosis of patients with brain tumors.