scholarly journals Deep learning classifiers for near infrared spectral imaging: a tutorial

2020 ◽  
Author(s):  
Jun-Li Xu ◽  
Cecilia Riccioli ◽  
Ana Herrero-Langreo ◽  
Aoife Gowen
Keyword(s):  
Deep Learning ◽  
Visual Recognition ◽  
Near Infrared ◽  
Three Dimensional ◽  
Spectral Imaging ◽  
Learning Approaches ◽  
Imaging Data ◽  
Spectral Image ◽  
Background Removal ◽  
Wide Range

Deep learning (DL) has recently achieved considerable successes in a wide range of applications, such as speech recognition, machine translation and visual recognition. This tutorial provides guidelines and useful strategies to apply DL techniques to address pixel-wise classification of spectral images. A one-dimensional convolutional neural network (1-D CNN) is used to extract features from the spectral domain, which are subsequently used for classification. In contrast to conventional classification methods for spectral images that examine primarily the spectral context, a three-dimensional (3-D) CNN is applied to simultaneously extract spatial and spectral features to enhance classificationaccuracy. This tutorial paper explains, in a stepwise manner, how to develop 1-D CNN and 3-D CNN models to discriminate spectral imaging data in a food authenticity context. The example image data provided consists of three varieties of puffed cereals imaged in the NIR range (943–1643 nm). The tutorial is presented in the MATLAB environment and scripts and dataset used are provided. Starting from spectral image pre-processing (background removal and spectral pre-treatment), the typical steps encountered in development of CNN models are presented. The example dataset provided demonstrates that deep learning approaches can increase classification accuracy compared to conventional approaches, increasing the accuracy of the model tested on an independent image from 92.33 % using partial least squares-discriminant analysis to 99.4 % using 3-CNN model at pixel level. The paper concludes with a discussion on the challenges and suggestions in the application of DL techniques for spectral image classification.

scholarly journals Unsupervised content-preserving transformation for optical microscopy

2021 ◽  
Vol 10 (1) ◽  
Author(s):  
Xinyang Li ◽  
Guoxun Zhang ◽  
Hui Qiao ◽  
Feng Bao ◽  
Yue Deng ◽  
...  
Keyword(s):  
Deep Learning ◽  
Optical Microscopy ◽  
Training Data ◽  
Fluorescence Labeling ◽  
Imaging Data ◽  
Image Transformation ◽  
General Applicability ◽  
Data Annotation ◽  
Biomedical Image ◽  
Wide Range

AbstractThe development of deep learning and open access to a substantial collection of imaging data together provide a potential solution for computational image transformation, which is gradually changing the landscape of optical imaging and biomedical research. However, current implementations of deep learning usually operate in a supervised manner, and their reliance on laborious and error-prone data annotation procedures remains a barrier to more general applicability. Here, we propose an unsupervised image transformation to facilitate the utilization of deep learning for optical microscopy, even in some cases in which supervised models cannot be applied. Through the introduction of a saliency constraint, the unsupervised model, named Unsupervised content-preserving Transformation for Optical Microscopy (UTOM), can learn the mapping between two image domains without requiring paired training data while avoiding distortions of the image content. UTOM shows promising performance in a wide range of biomedical image transformation tasks, including in silico histological staining, fluorescence image restoration, and virtual fluorescence labeling. Quantitative evaluations reveal that UTOM achieves stable and high-fidelity image transformations across different imaging conditions and modalities. We anticipate that our framework will encourage a paradigm shift in training neural networks and enable more applications of artificial intelligence in biomedical imaging.

An efficient approach to converting three-dimensional image data into highly accurate computational models

2008 ◽  
Vol 366 (1878) ◽  
pp. 3155-3173 ◽  
Author(s):  
P.G Young ◽  
T.B.H Beresford-West ◽  
S.R.L Coward ◽  
B Notarberardino ◽  
B Walker ◽  
...  
Keyword(s):  
Mesh Generation ◽  
Computational Models ◽  
Three Dimensional ◽  
Image Data ◽  
Model Material ◽  
Imaging Data ◽  
Material Inhomogeneity ◽  
Dimensional Image ◽  
Wide Range ◽  
Impact Modelling

Image-based meshing is opening up exciting new possibilities for the application of computational continuum mechanics methods (finite-element and computational fluid dynamics) to a wide range of biomechanical and biomedical problems that were previously intractable owing to the difficulty in obtaining suitably realistic models. Innovative surface and volume mesh generation techniques have recently been developed, which convert three-dimensional imaging data, as obtained from magnetic resonance imaging, computed tomography, micro-CT and ultrasound, for example, directly into meshes suitable for use in physics-based simulations. These techniques have several key advantages, including the ability to robustly generate meshes for topologies of arbitrary complexity (such as bioscaffolds or composite micro-architectures) and with any number of constituent materials (multi-part modelling), providing meshes in which the geometric accuracy of mesh domains is only dependent on the image accuracy (image-based accuracy) and the ability for certain problems to model material inhomogeneity by assigning the properties based on image signal strength. Commonly used mesh generation techniques will be compared with the proposed enhanced volumetric marching cubes (EVoMaCs) approach and some issues specific to simulations based on three-dimensional image data will be discussed. A number of case studies will be presented to illustrate how these techniques can be used effectively across a wide range of problems from characterization of micro-scaffolds through to head impact modelling.

scholarly journals Self-Supervised Pre-Training of Transformers for Satellite Image Time Series Classification

2020 ◽  
Author(s):  
Yuan Yuan ◽  
Lei Lin
Keyword(s):  
Time Series ◽  
Deep Learning ◽  
Large Scale ◽  
Temporal Structure ◽  
Satellite Image ◽  
Fine Tuning ◽  
Small Scale ◽  
Model Parameters ◽  
Learning Approaches ◽  
Wide Range

Satellite image time series (SITS) classification is a major research topic in remote sensing and is relevant for a wide range of applications. Deep learning approaches have been commonly employed for SITS classification and have provided state-of-the-art performance. However, deep learning methods suffer from overfitting when labeled data is scarce. To address this problem, we propose a novel self-supervised pre-training scheme to initialize a Transformer-based network by utilizing large-scale unlabeled data. In detail, the model is asked to predict randomly contaminated observations given an entire time series of a pixel. The main idea of our proposal is to leverage the inherent temporal structure of satellite time series to learn general-purpose spectral-temporal representations related to land cover semantics. Once pre-training is completed, the pre-trained network can be further adapted to various SITS classification tasks by fine-tuning all the model parameters on small-scale task-related labeled data. In this way, the general knowledge and representations about SITS can be transferred to a label-scarce task, thereby improving the generalization performance of the model as well as reducing the risk of overfitting. Comprehensive experiments have been carried out on three benchmark datasets over large study areas. Experimental results demonstrate the effectiveness of the proposed method, leading to a classification accuracy increment up to 1.91% to 6.69%. <div><b>This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible.</b></div>

scholarly journals Deep learning segmentation of gadolinium-enhancing lesions in multiple sclerosis

2020 ◽  
pp. 135245852092136 ◽  
Author(s):  
Ivan Coronado ◽  
Refaat E Gabr ◽  
Ponnada A Narayana
Keyword(s):  
Multiple Sclerosis ◽  
Deep Learning ◽  
Network Performance ◽  
Three Dimensional ◽  
Proton Density ◽  
Dice Similarity Coefficient ◽  
Lesion Volume ◽  
Imaging Data ◽  
Post Contrast ◽  
Fluid Attenuated Inversion Recovery

Objective: The aim of this study is to assess the performance of deep learning convolutional neural networks (CNNs) in segmenting gadolinium-enhancing lesions using a large cohort of multiple sclerosis (MS) patients. Methods: A three-dimensional (3D) CNN model was trained for segmentation of gadolinium-enhancing lesions using multispectral magnetic resonance imaging data (MRI) from 1006 relapsing–remitting MS patients. The network performance was evaluated for three combinations of multispectral MRI used as input: (U5) fluid-attenuated inversion recovery (FLAIR), T2-weighted, proton density-weighted, and pre- and post-contrast T1-weighted images; (U2) pre- and post-contrast T1-weighted images; and (U1) only post-contrast T1-weighted images. Segmentation performance was evaluated using the Dice similarity coefficient (DSC) and lesion-wise true-positive (TPR) and false-positive (FPR) rates. Performance was also evaluated as a function of enhancing lesion volume. Results: The DSC/TPR/FPR values averaged over all the enhancing lesion sizes were 0.77/0.90/0.23 using the U5 model. These values for the largest enhancement volumes (>500 mm3) were 0.81/0.97/0.04. For U2, the average DSC/TPR/FPR values were 0.72/0.86/0.31. Comparable performance was observed with U1. For all types of input, the network performance degraded with decreased enhancement size. Conclusion: Excellent segmentation of enhancing lesions was observed for enhancement volume ⩾70 mm3. The best performance was achieved when the input included all five multispectral image sets.

scholarly journals Tridimensional Spectroscopic Techniques: Conference Summary

1995 ◽  
Vol 149 ◽  
pp. 369-381 ◽  
Author(s):  
J. Bland-Hawthorn
Keyword(s):  
Fourier Transform ◽  
Near Infrared ◽  
Three Dimensional ◽  
Spectroscopic Techniques ◽  
Conference Summary ◽  
Wide Range ◽  
Fabry Perot ◽  
Scientific Results ◽  
General Thrust

Over the last four days, we have enjoyed a wide range of talks on developments in three dimensional spectroscopic techniques. The conference organizing committee are to be congratulated for the artful manner in which instrumental presentations were interleaved with talks on the scientific results from these instruments. The general thrust of most talks was to advance the versatility of traditional instruments either through the Jacquinot (throughput) advantage or through the multiplex advantage, or both. A number of groups have attempted to utilize the full aperture of scanning Fabry-Perot and Fourier Transform interferometers. Arguably, Fabry-Perot interferometers have a wider application at present, although imaging Fourier Transform devices appear to have finally arrived, at least in the near infrared.

scholarly journals A survey on Deep Learning Based Eye Gaze Estimation Methods

2021 ◽  
Vol 3 (3) ◽  
pp. 190-207
Author(s):  
S. K. B. Sangeetha
Keyword(s):  
Computer Vision ◽  
Deep Learning ◽  
Near Infrared ◽  
Eye Gaze ◽  
Infrared Image ◽  
Estimation Methods ◽  
Learning Technology ◽  
Learning Approaches ◽  
Gaze Tracking ◽  
Inference Models

In recent years, deep-learning systems have made great progress, particularly in the disciplines of computer vision and pattern recognition. Deep-learning technology can be used to enable inference models to do real-time object detection and recognition. Using deep-learning-based designs, eye tracking systems could determine the position of eyes or pupils, regardless of whether visible-light or near-infrared image sensors were utilized. For growing electronic vehicle systems, such as driver monitoring systems and new touch screens, accurate and successful eye gaze estimates are critical. In demanding, unregulated, low-power situations, such systems must operate efficiently and at a reasonable cost. A thorough examination of the different deep learning approaches is required to take into consideration all of the limitations and opportunities of eye gaze tracking. The goal of this research is to learn more about the history of eye gaze tracking, as well as how deep learning contributed to computer vision-based tracking. Finally, this research presents a generalized system model for deep learning-driven eye gaze direction diagnostics, as well as a comparison of several approaches.

scholarly journals Self-Supervised Pre-Training of Transformers for Satellite Image Time Series Classification

2020 ◽  
Author(s):  
Yuan Yuan ◽  
Lei Lin
Keyword(s):  
Time Series ◽  
Deep Learning ◽  
Large Scale ◽  
Temporal Structure ◽  
Satellite Image ◽  
Fine Tuning ◽  
Small Scale ◽  
Model Parameters ◽  
Learning Approaches ◽  
Wide Range

<div>Satellite image time series (SITS) classification is a major research topic in remote sensing and is relevant for a wide range of applications. Deep learning approaches have been commonly employed for SITS classification and have provided state-of-the-art performance. However, deep learning methods suffer from overfitting when labeled data is scarce. To address this problem, we propose a novel self-supervised pre-training scheme to initialize a Transformer-based network by utilizing large-scale unlabeled data. In detail, the model is asked to predict randomly contaminated observations given an entire time series of a pixel. The main idea of our proposal is to leverage the inherent temporal structure of satellite time series to learn general-purpose spectral-temporal representations related to land cover semantics. Once pre-training is completed, the pre-trained network can be further adapted to various SITS classification tasks by fine-tuning all the model parameters on small-scale task-related labeled data. In this way, the general knowledge and representations about SITS can be transferred to a label-scarce task, thereby improving the generalization performance of the model as well as reducing the risk of overfitting. Comprehensive experiments have been carried out on three benchmark datasets over large study areas. Experimental results demonstrate the effectiveness of the proposed method, leading to a classification accuracy increment up to 2.38% to 5.27%. The code and the pre-trained model will be available at https://github.com/linlei1214/SITS-BERT upon publication.</div><div><b>This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible.</b></div>

scholarly journals Deep learning approaches for neural decoding across architectures and recording modalities

2020 ◽  
Author(s):  
Jesse A Livezey ◽  
Joshua I Glaser
Keyword(s):  
Deep Learning ◽  
Scientific Development ◽  
Cognitive State ◽  
Learning Approaches ◽  
Neural Decoding ◽  
New Wave ◽  
Neural Signals ◽  
Learning Tasks ◽  
Wide Range ◽  
Deep Networks

Abstract Decoding behavior, perception or cognitive state directly from neural signals is critical for brain–computer interface research and an important tool for systems neuroscience. In the last decade, deep learning has become the state-of-the-art method in many machine learning tasks ranging from speech recognition to image segmentation. The success of deep networks in other domains has led to a new wave of applications in neuroscience. In this article, we review deep learning approaches to neural decoding. We describe the architectures used for extracting useful features from neural recording modalities ranging from spikes to functional magnetic resonance imaging. Furthermore, we explore how deep learning has been leveraged to predict common outputs including movement, speech and vision, with a focus on how pretrained deep networks can be incorporated as priors for complex decoding targets like acoustic speech or images. Deep learning has been shown to be a useful tool for improving the accuracy and flexibility of neural decoding across a wide range of tasks, and we point out areas for future scientific development.

scholarly journals Beyond the visible spectrum – applying 3D multispectral full-body imaging to the VirtoScan system

2021 ◽  
Author(s):  
Sören Kottner ◽  
Martin M. Schulz ◽  
Florian Berger ◽  
Michael Thali ◽  
Dominic Gascho
Keyword(s):  
Near Infrared ◽  
Three Dimensional ◽  
Visible Spectrum ◽  
Image Data ◽  
3D Models ◽  
Light Sources ◽  
Ct Scanner ◽  
Body Imaging ◽  
Wide Range ◽  
Full Body

AbstractMultispectral photography offers a wide range of applications for forensic investigations. It is commonly used to detect latent evidence and to enhance the visibility of findings. Additionally, three-dimensional (3D) full-body documentation has become much easier and more affordable in recent years. However, the benefits of performing 3D imaging beyond the visible (VIS) spectrum are not well known, and the technique has not been widely used in forensic medical investigations. A multicamera setup was used to employ multispectral photogrammetry between 365 and 960 nm in postmortem investigations. The multicamera setup included four modified digital cameras, ultraviolet (UV) and near-infrared (NIR) light sources and supplemental lens filters. Full-body documentation was performed in conjunction with the use of a medical X-ray computed tomography (CT) scanner to automate the imaging procedure. Textured 3D models based on multispectral datasets from four example cases were reconstructed successfully. The level of detail and overall quality of the 3D reconstructions varied depending on the spectral range of the image data. Generally, the NIR datasets showed enhanced visibility of vein patterns and specific injuries, whereas the UV-induced datasets highlighted foreign substances on the skin. Three-dimensional multispectral full-body imaging enables the detection of latent evidence that is invisible to the naked eye and allows visualization, documentation and analysis of evidence beyond the VIS spectrum.

Sign in / Sign up

Export Citation Format

Share Document