scholarly journals Performance of deep learning restoration methods for the extraction of particle dynamics in noisy microscopy image sequences

2020 ◽  
Author(s):  
Paul Kefer ◽  
Fadil Iqbal ◽  
Maelle Locatelli ◽  
Josh Lawrimore ◽  
Mengdi Zhang ◽  
...  
Keyword(s):  
Deep Learning ◽  
Image Restoration ◽  
Particle Tracking ◽  
Cell Biology ◽  
Light Exposure ◽  
Image Sequences ◽  
Single Particle Tracking ◽  
Short Exposure ◽  
Tracking Accuracy ◽  
Quantitative Microscopy

ABSTRACTImage-based particle tracking is an essential tool to answer research questions in cell biology and beyond. A major challenge of particle tracking in living systems is that low light exposure is required to avoid phototoxicity and photobleaching. In addition, high-speed imaging used to fully capture particle motion dictates fast image acquisition rates. Short exposure times come at the expense of tracking accuracy. This is generally true for quantitative microscopy approaches and particularly relevant to single molecule tracking where the number of photons emitted from a single chromophore is limited. Image restoration methods based on deep learning dramatically improve the signal-to-noise ratio in low-exposure datasets. However, it is not clear whether images generated by these methods yield accurate quantitative measurements such as diffusion parameters in (single) particle tracking experiments. Here, we evaluate the performance of two popular deep learning denoising software packages for particle tracking, using synthetic datasets and movies of diffusing chromatin as biological examples. With synthetic data, both supervised and unsupervised deep learning restored particle motions with high accuracy in two-dimensional datasets, whereas artifacts were introduced by the denoisers in 3D datasets. Experimentally, we found that, while both supervised and unsupervised approaches improved the number of trackable particles and tracking accuracy, supervised learning generally outperformed the unsupervised approach, as expected. We also highlight that with extremely noisy image sequences, deep learning algorithms produce deceiving artifacts, which underscores the need to carefully evaluate the results. Finally, we address the challenge of selecting hyper-parameters to train convolutional neural networks by implementing a frugal Bayesian optimizer that rapidly explores multidimensional parameter spaces, identifying networks yielding optional particle tracking accuracy. Our study provides quantitative outcome measures of image restoration using deep learning. We anticipate broad application of the approaches presented here to critically evaluate artificial intelligence solutions for quantitative microscopy.

scholarly journals Performance of deep learning restoration methods for the extraction of particle dynamics in noisy microscopy image sequences

2021 ◽  
pp. mbc.E20-11-0689
Author(s):  
Paul Kefer ◽  
Fadil Iqbal ◽  
Maelle Locatelli ◽  
Josh Lawrimore ◽  
Mengdi Zhang ◽  
...  
Keyword(s):  
Deep Learning ◽  
Image Restoration ◽  
Particle Tracking ◽  
Light Exposure ◽  
Image Sequences ◽  
Single Particle Tracking ◽  
Short Exposure ◽  
Tracking Accuracy ◽  
Excitation Light ◽  
Noisy Images

Particle tracking in living systems requires low light exposure and short exposure times to avoid phototoxicity and photobleaching and to fully capture particle motion with high-speed imaging. Low excitation light comes at the expense of tracking accuracy. Image restoration methods based on deep learning dramatically improve the signal-to-noise ratio in low-exposure datasets, qualitatively improving the images. However, it is not clear whether images generated by these methods yield accurate quantitative measurements such as diffusion parameters in (single) particle tracking experiments. Here, we evaluate the performance of two popular deep learning denoising software packages for particle tracking, using synthetic datasets and movies of diffusing chromatin as biological examples. With synthetic data, both supervised and unsupervised deep learning restored particle motions with high accuracy in two-dimensional datasets, whereas artifacts were introduced by the denoisers in 3D datasets. Experimentally, we found that, while both supervised and unsupervised approaches improved tracking results compared to the original noisy images, supervised learning generally outperformed the unsupervised approach. We find that nicer-looking image sequences are not synonymous with more precise tracking results and highlight that deep learning algorithms can produce deceiving artifacts with extremely noisy images. Finally, we address the challenge of selecting parameters to train convolutional neural networks by implementing a frugal Bayesian optimizer that rapidly explores multidimensional parameter spaces, identifying networks yielding optimal particle tracking accuracy. Our study provides quantitative outcome measures of image restoration using deep learning. We anticipate broad application of this approach to critically evaluate artificial intelligence solutions for quantitative microscopy.

scholarly journals Improving z-tracking accuracy in the two-photon single-particle tracking microscope

2015 ◽  
Vol 107 (15) ◽  
pp. 153701 ◽  
Author(s):  
C. Liu ◽  
Y.-L. Liu ◽  
E. P. Perillo ◽  
N. Jiang ◽  
A. K. Dunn ◽  
...  
Keyword(s):  
Particle Tracking ◽  
Single Particle ◽  
Single Particle Tracking ◽  
Tracking Accuracy ◽  
Two Photon

scholarly journals Label2label: Using deep learning and dual-labelling to retrieve cellular structures in fluorescence images

2020 ◽  
Author(s):  
Lisa Sophie Kölln ◽  
Omar Salem ◽  
Jessica Valli ◽  
Carsten Gram Hansen ◽  
Gail McConnell
Keyword(s):  
Deep Learning ◽  
Image Restoration ◽  
Cell Biology ◽  
Focal Adhesions ◽  
Ground Truth ◽  
Structural Similarity ◽  
Cellular Structures ◽  
Fluorescence Signal ◽  
Protein Distribution ◽  
Ground Truth Data

AbstractSpatial localisation of proteins dictates cellular function. Hence, visualisation of precise protein distribution is essential to obtain in-depth mechanistic insights into protein roles during cellular homeostasis, dynamic cellular processes, and dysfunction during disease. Labelling and staining of cells with protein specific antibodies is therefore a central and widely used technique in cell biology. However, unspecific binding, or cytoplasmic signals originating from the antibodies, make the distinction of the fluorescence signal from cellular structures challenging. Here we report a new image restoration method for images of cellular structures, using dual-labelling and deep learning, without requiring clean ground truth data. We name this method label2label (L2L). In L2L, a convolutional neural network (CNN) is trained with noisy fluorescence image pairs of two non-identical labels that target the same protein of interest. We show that a trained network acts as a content filter of label-specific artefacts and cytosolic content in images of the actin cytoskeleton, focal adhesions and microtubules, while the contrast of structural signal, which correlates in the images of two labels, is enhanced. We use an established CNN that was previously applied for content-aware image restoration, and show that the implementation of a multi-scale structural similarity loss function increases the performance of the network as content filter for images of cellular structures.

scholarly journals Improving Z-Tracking Accuracy in Two-Photon Single-Particle Tracking Microscope

2016 ◽  
Vol 110 (3) ◽  
pp. 633a
Author(s):  
Cong Liu ◽  
Evan Perillo ◽  
Yen-Liang Liu ◽  
Ajay Rastog ◽  
Andrew Dunn ◽  
...  
Keyword(s):  
Particle Tracking ◽  
Single Particle ◽  
Single Particle Tracking ◽  
Tracking Accuracy ◽  
Two Photon

scholarly journals Robust model-based analysis of single-particle tracking experiments with Spot-On

eLife ◽  
2018 ◽  
Vol 7 ◽  
Author(s):  
Anders S Hansen ◽  
Maxime Woringer ◽  
Jonathan B Grimm ◽  
Luke D Lavis ◽  
Robert Tjian ◽  
...  
Keyword(s):  
Particle Tracking ◽  
Single Particle ◽  
Cell Biology ◽  
Motion Blur ◽  
Single Particle Tracking ◽  
Modeling Framework ◽  
Diffusion Constants ◽  
Diffusion Dynamics ◽  
Wide Range ◽  
And Diffusion

Single-particle tracking (SPT) has become an important method to bridge biochemistry and cell biology since it allows direct observation of protein binding and diffusion dynamics in live cells. However, accurately inferring information from SPT studies is challenging due to biases in both data analysis and experimental design. To address analysis bias, we introduce ‘Spot-On’, an intuitive web-interface. Spot-On implements a kinetic modeling framework that accounts for known biases, including molecules moving out-of-focus, and robustly infers diffusion constants and subpopulations from pooled single-molecule trajectories. To minimize inherent experimental biases, we implement and validate stroboscopic photo-activation SPT (spaSPT), which minimizes motion-blur bias and tracking errors. We validate Spot-On using experimentally realistic simulations and show that Spot-On outperforms other methods. We then apply Spot-On to spaSPT data from live mammalian cells spanning a wide range of nuclear dynamics and demonstrate that Spot-On consistently and robustly infers subpopulation fractions and diffusion constants.

scholarly journals Content-aware frame interpolation (CAFI): Deep Learning-based temporal super-resolution for fast bioimaging

2021 ◽  
Author(s):  
Martin Priessner ◽  
David C.A. Gaboriau ◽  
Arlo Sheridan ◽  
Tchern Lenn ◽  
Jonathan R. Chubb ◽  
...  
Keyword(s):  
Deep Learning ◽  
Light Exposure ◽  
Super Resolution ◽  
Single Particle Tracking ◽  
Video Frame ◽  
Learning Networks ◽  
Frame Interpolation ◽  
Cellular Processes ◽  
Image Pairs ◽  
Content Aware

The development of high-resolution microscopes has made it possible to investigate cellular processes in 4D (3D over time). However, observing fast cellular dynamics remains challenging as a consequence of photobleaching and phototoxicity. These issues become increasingly problematic with the depth of the volume acquired and the speed of the biological events of interest. Here, we report the implementation of two content-aware frame interpolation (CAFI) deep learning networks, Zooming SlowMo (ZS) and Depth-Aware Video Frame Interpolation (DAIN), based on combinations of recurrent neural networks, that are highly suited for accurately predicting images in between image pairs, therefore improving the temporal resolution of image series as a post-acquisition analysis step. We show that CAFI predictions are capable of understanding the motion context of biological structures to perform better than standard interpolation methods. We benchmark CAFI's performance on six different datasets, obtained from three different microscopy modalities (point-scanning confocal, spinning-disc confocal and confocal brightfield microscopy). We demonstrate its capabilities for single-particle tracking methods applied to the study of lysosome trafficking. CAFI therefore allows for reduced light exposure and phototoxicity on the sample and extends the possibility of long-term live-cell imaging. Both DAIN and ZS as well as the training and testing data are made available for use by the wider community via the ZeroCostDL4Mic platform.

scholarly journals Information-Efficient, Off-Center Sampling Results in Improved Precision in 3D Single-Particle Tracking Microscopy

Entropy ◽  
2021 ◽  
Vol 23 (5) ◽  
pp. 498
Author(s):  
Chen Zhang ◽  
Kevin Welsher
Keyword(s):  
Fisher Information ◽  
Particle Tracking ◽  
Single Particle ◽  
Biological Samples ◽  
Tracking System ◽  
Single Particle Tracking ◽  
Uniform Sampling ◽  
Single Dimension ◽  
3D Localization ◽  
Particle Localization

In this work, we present a 3D single-particle tracking system that can apply tailored sampling patterns to selectively extract photons that yield the most information for particle localization. We demonstrate that off-center sampling at locations predicted by Fisher information utilizes photons most efficiently. When performing localization in a single dimension, optimized off-center sampling patterns gave doubled precision compared to uniform sampling. A ~20% increase in precision compared to uniform sampling can be achieved when a similar off-center pattern is used in 3D localization. Here, we systematically investigated the photon efficiency of different emission patterns in a diffraction-limited system and achieved higher precision than uniform sampling. The ability to maximize information from the limited number of photons demonstrated here is critical for particle tracking applications in biological samples, where photons may be limited.

scholarly journals Deep learning-based recognition of cell structures in fluorescence microscopy sequences with respect to their morphology on cells infected with Marburg virus

2020 ◽  
Vol 6 (3) ◽  
pp. 501-504
Author(s):  
Dennis Schmidt ◽  
Andreas Rausch ◽  
Thomas Schanze
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Infected Cell ◽  
Network Architecture ◽  
Image Sequences ◽  
Marburg Virus ◽  
Microscopic Image ◽  
Cell Structures ◽  
Fast Recognition ◽  
Subviral Particles

AbstractThe Institute of Virology at the Philipps-Universität Marburg is currently researching possible drugs to combat the Marburg virus. This involves classifying cell structures based on fluoroscopic microscopic image sequences. Conventionally, membranes of cells must be marked for better analysis, which is time consuming. In this work, an approach is presented to identify cell structures in images that are marked for subviral particles. It could be shown that there is a correlation between the distribution of subviral particles in an infected cell and the position of the cell’s structures. The segmentation is performed with a "Mask-R-CNN" algorithm, presented in this work. The model (a region-based convolutional neural network) is applied to enable a robust and fast recognition of cell structures. Furthermore, the network architecture is described. The proposed method is tested on data evaluated by experts. The results show a high potential and demonstrate that the method is suitable.

Sign in / Sign up

Export Citation Format

Share Document