Near real-time intraoperative brain tumor diagnosis using stimulated Raman histology and deep neural networks

Abstract INTRODUCTION Intraoperative diagnosis is essential for providing safe and effective care during cancer surgery. The existing workflow for intraoperative diagnosis based on H&E staining of processed tissue is time-, resource-, and labor-intensive. Moreover, interpretation of intraoperative histologic images is dependent on a pathology workforce that is contracting and unevenly distributed across the centers where cancer surgery is performed worldwide. METHODS We developed an automated workflow, independent of traditional H&E histology, that combines stimulated Raman histology (SRH), a rapid label-free optical imaging method, and deep convolutional neural networks (CNN) to predict diagnosis at the bedside in near real time. Specifically, our CNN, trained on over 2.5 million SRH images, predicts brain tumor diagnosis in the operating room in under 150 s, which is an order of magnitude faster than conventional techniques (eg, 20-30 min). RESULTS To validate our workflow in the clinical setting, we designed a multicenter, prospective, noninferiority clinical trial (N = 204) that compares SRH plus CNN vs traditional H&E histology. Primary endpoint was overall diagnostic accuracy. We show that CNN-based diagnosis of SRH images was noninferior to pathologist-based interpretation of conventional histologic images (overall accuracy, 94.6% vs 95.5%). Additionally, our CNN learned a hierarchy of interpretable histologic feature representations to classify the major histopathologic classes of brain tumors. We then developed and implemented a semantic segmentation method that can identify tumor infiltrated and diagnostic regions within SRH images. Mean intersection over union values was 61 ± 28.6 for ground truth diagnostic class and 86.0 ± 28.6 for tumor-infiltrated regions. CONCLUSION We have demonstrated how combining bedside optical histology with deep learning can result in near real-time intraoperative brain tumor diagnosis. Our workflow provides a means of delivering expert-level intraoperative diagnosis where neuropathology resources are scarce and improve diagnostic accuracy in resource-rich centers.

Download Full-text

Development of a real-time tactile sensing system for brain tumor diagnosis

International Journal of Computer Assisted Radiology and Surgery ◽

10.1007/s11548-010-0426-7 ◽

2010 ◽

Vol 5 (4) ◽

pp. 359-367 ◽

Cited By ~ 13

Author(s):

Yoshihiro Tanaka ◽

Qingyun Yu ◽

Kazuki Doumoto ◽

Akihito Sano ◽

Yuichiro Hayashi ◽

...

Keyword(s):

Brain Tumor ◽

Real Time ◽

Tumor Diagnosis ◽

Tactile Sensing ◽

Sensing System ◽

Brain Tumor Diagnosis

Download Full-text

Assurance monitoring of learning-enabled cyber-physical systems using inductive conformal prediction based on distance learning

Artificial intelligence for engineering design analysis and manufacturing ◽

10.1017/s089006042100010x ◽

2021 ◽

Vol 35 (2) ◽

pp. 251-264

Author(s):

Dimitrios Boursinos ◽

Xenofon Koutsoukos

Keyword(s):

Neural Networks ◽

Distance Learning ◽

Real Time ◽

Speaker Recognition ◽

Deep Neural Networks ◽

Cyber Physical Systems ◽

Error Rates ◽

Traffic Sign ◽

Conformal Prediction ◽

Physical Systems

AbstractMachine learning components such as deep neural networks are used extensively in cyber-physical systems (CPS). However, such components may introduce new types of hazards that can have disastrous consequences and need to be addressed for engineering trustworthy systems. Although deep neural networks offer advanced capabilities, they must be complemented by engineering methods and practices that allow effective integration in CPS. In this paper, we proposed an approach for assurance monitoring of learning-enabled CPS based on the conformal prediction framework. In order to allow real-time assurance monitoring, the approach employs distance learning to transform high-dimensional inputs into lower size embedding representations. By leveraging conformal prediction, the approach provides well-calibrated confidence and ensures a bounded small error rate while limiting the number of inputs for which an accurate prediction cannot be made. We demonstrate the approach using three datasets of mobile robot following a wall, speaker recognition, and traffic sign recognition. The experimental results demonstrate that the error rates are well-calibrated while the number of alarms is very small. Furthermore, the method is computationally efficient and allows real-time assurance monitoring of CPS.

Download Full-text