Automated Segmentation and Connectivity Analysis for Normal Pressure Hydrocephalus

Objective and Impact Statement. We propose an automated method of predicting Normal Pressure Hydrocephalus (NPH) from CT scans. A deep convolutional network segments regions of interest from the scans. These regions are then combined with MRI information to predict NPH. To our knowledge, this is the first method which automatically predicts NPH from CT scans and incorporates diffusion tractography information for prediction. Introduction. Due to their low cost and high versatility, CT scans are often used in NPH diagnosis. No well-defined and effective protocol currently exists for analysis of CT scans for NPH. Evans’ index, an approximation of the ventricle to brain volume using one 2D image slice, has been proposed but is not robust. The proposed approach is an effective way to quantify regions of interest and offers a computational method for predicting NPH. Methods. We propose a novel method to predict NPH by combining regions of interest segmented from CT scans with connectome data to compute features which capture the impact of enlarged ventricles by excluding fiber tracts passing through these regions. The segmentation and network features are used to train a model for NPH prediction. Results. Our method outperforms the current state-of-the-art by 9 precision points and 29 recall points. Our segmentation model outperforms the current state-of-the-art in segmenting the ventricle, gray-white matter, and subarachnoid space in CT scans. Conclusion. Our experimental results demonstrate that fast and accurate volumetric segmentation of CT brain scans can help improve the NPH diagnosis process, and network properties can increase NPH prediction accuracy.

Multimodal Metabolic Imaging Reveals Pigment Reduction and Lipid Accumulation in Metastatic Melanoma

Objective and Impact Statement. Molecular signatures are needed for early diagnosis and improved treatment of metastatic melanoma. By high-resolution multimodal chemical imaging of human melanoma samples, we identify a metabolic reprogramming from pigmentation to lipid droplet (LD) accumulation in metastatic melanoma. Introduction. Metabolic plasticity promotes cancer survival and metastasis, which promises to serve as a prognostic marker and/or therapeutic target. However, identifying metabolic alterations has been challenged by difficulties in mapping localized metabolites with high spatial resolution. Methods. We developed a multimodal stimulated Raman scattering and pump-probe imaging platform. By time-domain measurement and phasor analysis, our platform allows simultaneous mapping of lipids and pigments at a subcellular level. Furthermore, we identify the sources of these metabolic signatures by tracking deuterium metabolites at a subcellular level. By validation with mass spectrometry, a specific fatty acid desaturase pathway was identified. Results. We identified metabolic reprogramming from a pigment-containing phenotype in low-grade melanoma to an LD-rich phenotype in metastatic melanoma. The LDs contain high levels of cholesteryl ester and unsaturated fatty acids. Elevated fatty acid uptake, but not de novo lipogenesis, contributes to the LD-rich phenotype. Monounsaturated sapienate, mediated by FADS2, is identified as an essential fatty acid that promotes cancer migration. Blocking such metabolic signatures effectively suppresses the migration capacity both in vitro and in vivo. Conclusion. By multimodal spectroscopic imaging and lipidomic analysis, the current study reveals lipid accumulation, mediated by fatty acid uptake, as a metabolic signature that can be harnessed for early diagnosis and improved treatment of metastatic melanoma.

Label-Free White Blood Cell Classification Using Refractive Index Tomography and Deep Learning

Objective and Impact Statement. We propose a rapid and accurate blood cell identification method exploiting deep learning and label-free refractive index (RI) tomography. Our computational approach that fully utilizes tomographic information of bone marrow (BM) white blood cell (WBC) enables us to not only classify the blood cells with deep learning but also quantitatively study their morphological and biochemical properties for hematology research. Introduction. Conventional methods for examining blood cells, such as blood smear analysis by medical professionals and fluorescence-activated cell sorting, require significant time, costs, and domain knowledge that could affect test results. While label-free imaging techniques that use a specimen’s intrinsic contrast (e.g., multiphoton and Raman microscopy) have been used to characterize blood cells, their imaging procedures and instrumentations are relatively time-consuming and complex. Methods. The RI tomograms of the BM WBCs are acquired via Mach-Zehnder interferometer-based tomographic microscope and classified by a 3D convolutional neural network. We test our deep learning classifier for the four types of bone marrow WBC collected from healthy donors (n=10): monocyte, myelocyte, B lymphocyte, and T lymphocyte. The quantitative parameters of WBC are directly obtained from the tomograms. Results. Our results show >99% accuracy for the binary classification of myeloids and lymphoids and >96% accuracy for the four-type classification of B and T lymphocytes, monocyte, and myelocytes. The feature learning capability of our approach is visualized via an unsupervised dimension reduction technique. Conclusion. We envision that the proposed cell classification framework can be easily integrated into existing blood cell investigation workflows, providing cost-effective and rapid diagnosis for hematologic malignancy.

Recent Advances in Photoacoustic Tomography

Photoacoustic tomography (PAT) that integrates the molecular contrast of optical imaging with the high spatial resolution of ultrasound imaging in deep tissue has widespread applications in basic biological science, preclinical research, and clinical trials. Recently, tremendous progress has been made in PAT regarding technical innovations, preclinical applications, and clinical translations. Here, we selectively review the recent progresses and advances in PAT, including the development of advanced PAT systems for small-animal and human imaging, newly engineered optical probes for molecular imaging, broad-spectrum PAT for label-free imaging of biological tissues, high-throughput snapshot photoacoustic topography, and integration of machine learning for image reconstruction and processing. We envision that PAT will have further technical developments and more impactful applications in biomedicine.

Optical Detection of Distal Lung Enzyme Activity in Human Inflammatory Lung Disease

Objective and Impact Statement. There is a need to develop platforms delineating inflammatory biology of the distal human lung. We describe a platform technology approach to detect in situ enzyme activity and observe drug inhibition in the distal human lung using a combination of matrix metalloproteinase (MMP) optical reporters, fibered confocal fluorescence microscopy (FCFM), and a bespoke delivery device. Introduction. The development of new therapeutic agents is hindered by the lack of in vivo in situ experimental methodologies that can rapidly evaluate the biological activity or drug-target engagement in patients. Methods. We optimised a novel highly quenched optical molecular reporter of enzyme activity (FIB One) and developed a translational pathway for in-human assessment. Results. We demonstrate the specificity for matrix metalloproteases (MMPs) 2, 9, and 13 and probe dequenching within physiological levels of MMPs and feasibility of imaging within whole lung models in preclinical settings. Subsequently, in a first-in-human exploratory experimental medicine study of patients with fibroproliferative lung disease, we demonstrate, through FCFM, the MMP activity in the alveolar space measured through FIB One fluorescence increase (with pharmacological inhibition). Conclusion. This translational in situ approach enables a new methodology to demonstrate active drug target effects of the distal lung and consequently may inform therapeutic drug development pathways.

Real-Time High-Resolution MRI Endoscopy at up to 10 Frames per Second

Objective. Atherosclerosis is a leading cause of mortality and morbidity. Optical endoscopy, ultrasound, and X-ray offer minimally invasive imaging assessments but have limited sensitivity for characterizing disease and therapeutic response. Magnetic resonance imaging (MRI) endoscopy is a newer idea employing tiny catheter-mounted detectors connected to the MRI scanner. It can see through vessel walls and provide soft-tissue sensitivity, but its slow imaging speed limits practical applications. Our goal is high-resolution MRI endoscopy with real-time imaging speeds comparable to existing modalities. Methods. Intravascular (3 mm) transmit-receive MRI endoscopes were fabricated for highly undersampled radial-projection MRI in a clinical 3-tesla MRI scanner. Iterative nonlinear reconstruction was accelerated using graphics processor units connected via a single ethernet cable to achieve true real-time endoscopy visualization at the scanner. MRI endoscopy was performed at 6-10 frames/sec and 200-300 μm resolution in human arterial specimens and porcine vessels ex vivo and in vivo and compared with fully sampled 0.3 frames/sec and three-dimensional reference scans using mutual information (MI) and structural similarity (3-SSIM) indices. Results. High-speed MRI endoscopy at 6-10 frames/sec was consistent with fully sampled MRI endoscopy and histology, with feasibility demonstrated in vivo in a large animal model. A 20-30-fold speed-up vs. 0.3 frames/sec reference scans came at a cost of ~7% in MI and ~45% in 3-SSIM, with reduced motion sensitivity. Conclusion. High-resolution MRI endoscopy can now be performed at frame rates comparable to those of X-ray and optical endoscopy and could provide an alternative to existing modalities, with MRI’s advantages of soft-tissue sensitivity and lack of ionizing radiation.

Recent Advancements in Optical Harmonic Generation Microscopy: Applications and Perspectives

Second harmonic generation (SHG) and third harmonic generation (THG) microscopies have emerged as powerful imaging modalities to examine structural properties of a wide range of biological tissues. Although SHG and THG arise from very different contrast mechanisms, the two are complimentary and can often be collected simultaneously using a modified multiphoton microscope. In this review, we discuss the needed instrumentation for these modalities as well as the underlying theoretical principles of SHG and THG in tissue and describe how these can be leveraged to extract unique structural information. We provide an overview of recent advances showing how SHG microscopy has been used to evaluate collagen alterations in the extracellular matrix and how this has been used to advance our knowledge of cancers, fibroses, and the cornea, as well as in tissue engineering applications. Specific examples using polarization-resolved approaches and machine learning algorithms are highlighted. Similarly, we review how THG has enabled developmental biology and skin cancer studies due to its sensitivity to changes in refractive index, which are ubiquitous in all cell and tissue assemblies. Lastly, we offer perspectives and outlooks on future directions of SHG and THG microscopies and present unresolved questions, especially in terms of overall miniaturization and the development of microendoscopy instrumentation.

Bioresorbable Multilayer Photonic Cavities as Temporary Implants for Tether-Free Measurements of Regional Tissue Temperatures

Objective and Impact Statement. Real-time monitoring of the temperatures of regional tissue microenvironments can serve as the diagnostic basis for treating various health conditions and diseases. Introduction. Traditional thermal sensors allow measurements at surfaces or at near-surface regions of the skin or of certain body cavities. Evaluations at depth require implanted devices connected to external readout electronics via physical interfaces that lead to risks for infection and movement constraints for the patient. Also, surgical extraction procedures after a period of need can introduce additional risks and costs. Methods. Here, we report a wireless, bioresorbable class of temperature sensor that exploits multilayer photonic cavities, for continuous optical measurements of regional, deep-tissue microenvironments over a timeframe of interest followed by complete clearance via natural body processes. Results. The designs decouple the influence of detection angle from temperature on the reflection spectra, to enable high accuracy in sensing, as supported by in vitro experiments and optical simulations. Studies with devices implanted into subcutaneous tissues of both awake, freely moving and asleep animal models illustrate the applicability of this technology for in vivo measurements. Conclusion. The results demonstrate the use of bioresorbable materials in advanced photonic structures with unique capabilities in tracking of thermal signatures of tissue microenvironments, with potential relevance to human healthcare.

Anatomical Modeling of Brain Vasculature in Two-Photon Microscopy by Generalizable Deep Learning

Objective and Impact Statement. Segmentation of blood vessels from two-photon microscopy (2PM) angiograms of brains has important applications in hemodynamic analysis and disease diagnosis. Here, we develop a generalizable deep learning technique for accurate 2PM vascular segmentation of sizable regions in mouse brains acquired from multiple 2PM setups. The technique is computationally efficient, thus ideal for large-scale neurovascular analysis. Introduction. Vascular segmentation from 2PM angiograms is an important first step in hemodynamic modeling of brain vasculature. Existing segmentation methods based on deep learning either lack the ability to generalize to data from different imaging systems or are computationally infeasible for large-scale angiograms. In this work, we overcome both these limitations by a method that is generalizable to various imaging systems and is able to segment large-scale angiograms. Methods. We employ a computationally efficient deep learning framework with a loss function that incorporates a balanced binary-cross-entropy loss and total variation regularization on the network’s output. Its effectiveness is demonstrated on experimentally acquired in vivo angiograms from mouse brains of dimensions up to 808×808×702 μm. Results. To demonstrate the superior generalizability of our framework, we train on data from only one 2PM microscope and demonstrate high-quality segmentation on data from a different microscope without any network tuning. Overall, our method demonstrates 10× faster computation in terms of voxels-segmented-per-second and 3× larger depth compared to the state-of-the-art. Conclusion. Our work provides a generalizable and computationally efficient anatomical modeling framework for brain vasculature, which consists of deep learning-based vascular segmentation followed by graphing. It paves the way for future modeling and analysis of hemodynamic response at much greater scales that were inaccessible before.

From Neurons to Cognition: Technologies for Precise Recording of Neural Activity Underlying Behavior

Understanding how brain activity encodes information and controls behavior is a long-standing question in neuroscience. This complex problem requires converging efforts from neuroscience and engineering, including technological solutions to perform high-precision and large-scale recordings of neuronal activity in vivo as well as unbiased methods to reliably measure and quantify behavior. Thanks to advances in genetics, molecular biology, engineering, and neuroscience, in recent decades, a variety of optical imaging and electrophysiological approaches for recording neuronal activity in awake animals have been developed and widely applied in the field. Moreover, sophisticated computer vision and machine learning algorithms have been developed to analyze animal behavior. In this review, we provide an overview of the current state of technology for neuronal recordings with a focus on optical and electrophysiological methods in rodents. In addition, we discuss areas that future technological development will need to cover in order to further our understanding of the neural activity underlying behavior.