Near Real-Time Intraoperative Brain Tumor Diagnosis Using Stimulated Raman Histology and Deep Neural Networks

INTRODUCTION Intraoperative diagnosis is essential for providing safe and effective care during cancer surgery. The existing workflow for intraoperative diagnosis based on H&E staining of processed tissue is time-, resource-, and labor-intensive. Moreover, interpretation of intraoperative histologic images is dependent on a pathology workforce that is contracting and unevenly distributed across the centers where cancer surgery is performed worldwide. METHODS We developed an automated workflow, independent of traditional H&E histology, that combines stimulated Raman histology (SRH), a rapid label-free optical imaging method, and deep convolutional neural networks (CNN) to predict diagnosis at the bedside in near real time. Specifically, our CNN, trained on over 2.5 million SRH images, predicts brain tumor diagnosis in the operating room in under 150 s, which is an order of magnitude faster than conventional techniques (eg, 20-30 min). RESULTS To validate our workflow in the clinical setting, we designed a multicenter, prospective, noninferiority clinical trial (N = 204) that compares SRH plus CNN vs traditional H&E histology. Primary endpoint was overall diagnostic accuracy. We show that CNN-based diagnosis of SRH images was noninferior to pathologist-based interpretation of conventional histologic images (overall accuracy, 94.6% vs 95.5%). Additionally, our CNN learned a hierarchy of interpretable histologic feature representations to classify the major histopathologic classes of brain tumors. We then developed and implemented a semantic segmentation method that can identify tumor infiltrated and diagnostic regions within SRH images. Mean intersection over union values was 61 ± 28.6 for ground truth diagnostic class and 86.0 ± 28.6 for tumor-infiltrated regions. CONCLUSION We have demonstrated how combining bedside optical histology with deep learning can result in near real-time intraoperative brain tumor diagnosis. Our workflow provides a means of delivering expert-level intraoperative diagnosis where neuropathology resources are scarce and improve diagnostic accuracy in resource-rich centers.