scholarly journals Applications, Promises, and Pitfalls of Deep Learning for Fluorescence Image Reconstruction

2019 ◽  
Author(s):  
Chinmay Belthangady ◽  
Loic A. Royer
Keyword(s):  
Deep Learning ◽  
Image Reconstruction ◽  
Super Resolution ◽  
Training Data ◽  
Structured Illumination ◽  
Fluorescence Image ◽  
Important Addition ◽  
Review State ◽  
Image Details ◽  
Field Imaging

Deep Learning is a recent and important addition to the computational toolbox available for image reconstruction in fluorescence microscopy. We review state-of-the-art applications such as image restoration, super-resolution, and light-field imaging, and discuss how the latest Deep Learning research can be applied to other image reconstruction tasks such as structured illumination, spectral deconvolution, and sample stabilisation. Despite its successes, Deep Learning also poses significant challenges, has often misunderstood capabilities, and overlooked limits. We will address key questions, such as: What are the challenges in obtaining training data? Can we discover structures not present in the training data? And, what is the danger of inferring unsubstantiated image details?

scholarly journals Applications, Promises, and Pitfalls of Deep Learning for Fluorescence Image Reconstruction

2018 ◽  
Author(s):  
Chinmay Belthangady ◽  
Loic A. Royer
Keyword(s):  
Deep Learning ◽  
Image Reconstruction ◽  
Super Resolution ◽  
Training Data ◽  
Structured Illumination ◽  
Fluorescence Image ◽  
Important Addition ◽  
Review State ◽  
Image Details ◽  
Field Imaging

Deep Learning is a recent and important addition to the computational toolbox available for image reconstruction in fluorescence microscopy. We review state-of-the-art applications such as image restoration, super-resolution, and light-field imaging, and discuss how the latest Deep Learning research can be applied to other image reconstruction tasks such as structured illumination, spectral deconvolution, and sample stabilisation. Despite its successes, Deep Learning also poses significant challenges, has often misunderstood capabilities, and overlooked limits. We will address key questions, such as: What are the challenges in obtaining training data? Can we discover structures not present in the training data? And, what is the danger of inferring unsubstantiated image details?

scholarly journals Deep-learning based denoising and reconstruction of super-resolution structured illumination microscopy images

2020 ◽  
Author(s):  
Zafran Hussain Shah ◽  
Marcel Müller ◽  
Tung-Cheng Wang ◽  
Philip Maurice Scheidig ◽  
Axel Schneider ◽  
...  
Keyword(s):  
Deep Learning ◽  
Image Reconstruction ◽  
Super Resolution ◽  
Image Data ◽  
Excitation Power ◽  
Structured Illumination ◽  
Structured Illumination Microscopy ◽  
High Quality ◽  
Reconstruction Methods ◽  
High Modulation

AbstractSuper-resolution structured illumination microscopy (SR-SIM) provides an up to two-fold enhanced spatial resolution of fluorescently labeled samples. The reconstruction of high quality SR-SIM images critically depends on patterned illumination with high modulation contrast. Noisy raw image data, e.g. as a result of low excitation power or low exposure times, result in reconstruction artifacts. Here, we demonstrate deep-learning based SR-SIM image denoising that results in high quality reconstructed images. A residual encoding-decoding convolution neural network (RED-Net) was used to successfully denoise computationally reconstructed noisy SR-SIM images. We also demonstrate the entirely deep-learning based denoising and reconstruction of raw SIM images into high-resolution SR-SIM images. Both image reconstruction methods prove to be very robust against image reconstruction artifacts and generalize very well over various noise levels. The combination of computational reconstruction and subsequent denoising via RED-Net shows very robust performance during inference after training even if the microscope settings change.

scholarly journals Camera-based localization microscopy optimized with calibrated structured illumination

2021 ◽  
Vol 4 (1) ◽  
Author(s):  
Martin Schmidt ◽  
Adam C. Hundahl ◽  
Henrik Flyvbjerg ◽  
Rodolphe Marie ◽  
Kim I. Mortensen
Keyword(s):  
Data Analysis ◽  
Super Resolution ◽  
Wide Field ◽  
Structured Illumination ◽  
Uniform Illumination ◽  
Localization Microscopy ◽  
Localization And Tracking ◽  
Field Imaging ◽  
Wide Field Imaging ◽  
Localization Information

AbstractUntil very recently, super-resolution localization and tracking of fluorescent particles used camera-based wide-field imaging with uniform illumination. Then it was demonstrated that structured illuminations encode additional localization information in images. The first demonstration of this uses scanning and hence suffers from limited throughput. This limitation was mitigated by fusing camera-based localization with wide-field structured illumination. Current implementations, however, use effectively only half the localization information that they encode in images. Here we demonstrate how all of this information may be exploited by careful calibration of the structured illumination. Our approach achieves maximal resolution for given structured illumination, has a simple data analysis, and applies to any structured illumination in principle. We demonstrate this with an only slightly modified wide-field microscope. Our protocol should boost the emerging field of high-precision localization with structured illumination.

Super-Resolution Based on Generative Adversarial Network for HRTEM Images

2021 ◽  
pp. 2154027
Author(s):  
Fuqi Mao ◽  
Xiaohan Guan ◽  
Ruoyu Wang ◽  
Wen Yue
Keyword(s):  
Deep Learning ◽  
High Resolution ◽  
Information Structure ◽  
Rapid Development ◽  
Super Resolution ◽  
Training Data ◽  
Learning Technology ◽  
Hrtem Image ◽  
Generative Adversarial Network ◽  
Dual Regression

As an important tool to study the microstructure and properties of materials, High Resolution Transmission Electron Microscope (HRTEM) images can obtain the lattice fringe image (reflecting the crystal plane spacing information), structure image and individual atom image (which reflects the configuration of atoms or atomic groups in crystal structure). Despite the rapid development of HTTEM devices, HRTEM images still have limited achievable resolution for human visual system. With the rapid development of deep learning technology in recent years, researchers are actively exploring the Super-resolution (SR) model based on deep learning, and the model has reached the current best level in various SR benchmarks. Using SR to reconstruct high-resolution HRTEM image is helpful to the material science research. However, there is one core issue that has not been resolved: most of these super-resolution methods require the training data to exist in pairs. In actual scenarios, especially for HRTEM images, there are no corresponding HR images. To reconstruct high quality HRTEM image, a novel Super-Resolution architecture for HRTEM images is proposed in this paper. Borrowing the idea from Dual Regression Networks (DRN), we introduce an additional dual regression structure to ESRGAN, by training the model with unpaired HRTEM images and paired nature images. Results of extensive benchmark experiments demonstrate that the proposed method achieves better performance than the most resent SISR methods with both quantitative and visual results.

scholarly journals Deep-learning based denoising and reconstruction of super-resolution structured illumination microscopy images

2021 ◽  
Author(s):  
ZAFRAN HUSSAIN SHAH ◽  
Marcel Müller ◽  
TUNG-CHENG WANG ◽  
Philip Scheidig ◽  
Axel Schneider ◽  
...  
Keyword(s):  
Deep Learning ◽  
Super Resolution ◽  
Structured Illumination ◽  
Structured Illumination Microscopy ◽  
Microscopy Images
Sign in / Sign up

Export Citation Format

Share Document