scholarly journals 3DeeCellTracker, a deep learning-based pipeline for segmenting and tracking cells in 3D time lapse images

eLife ◽  
2021 ◽  
Vol 10 ◽  
Author(s):  
Chentao Wen ◽  
Takuya Miura ◽  
Venkatakaushik Voleti ◽  
Kazushi Yamaguchi ◽  
Motosuke Tsutsumi ◽  
...  
Keyword(s):  
Deep Learning ◽  
Three Dimensional ◽  
Time Lapse ◽  
Training Data ◽  
Optical Systems ◽  
New Techniques ◽  
Software Pipeline ◽  
Dynamic Cell ◽  
Initial Correction ◽  
Zebrafish Heart

Despite recent improvements in microscope technologies, segmenting and tracking cells in three-dimensional time-lapse images (3D + T images) to extract their dynamic positions and activities remains a considerable bottleneck in the field. We developed a deep learning-based software pipeline, 3DeeCellTracker, by integrating multiple existing and new techniques including deep learning for tracking. With only one volume of training data, one initial correction, and a few parameter changes, 3DeeCellTracker successfully segmented and tracked ~100 cells in both semi-immobilized and ‘straightened’ freely moving worm's brain, in a naturally beating zebrafish heart, and ~1000 cells in a 3D cultured tumor spheroid. While these datasets were imaged with highly divergent optical systems, our method tracked 90–100% of the cells in most cases, which is comparable or superior to previous results. These results suggest that 3DeeCellTracker could pave the way for revealing dynamic cell activities in image datasets that have been difficult to analyze.

scholarly journals Multi-Dimensional Underwater Point Cloud Detection Based on Deep Learning

Sensors ◽  
2021 ◽  
Vol 21 (3) ◽  
pp. 884
Author(s):  
Chia-Ming Tsai ◽  
Yi-Horng Lai ◽  
Yung-Da Sun ◽  
Yu-Jen Chung ◽  
Jau-Woei Perng
Keyword(s):  
Deep Learning ◽  
Point Cloud ◽  
Three Dimensional ◽  
Point Clouds ◽  
Training Data ◽  
Network Architectures ◽  
Point Cloud Data ◽  
Data Types ◽  
Raw Data ◽  
Cloud Data

Numerous sensors can obtain images or point cloud data on land, however, the rapid attenuation of electromagnetic signals and the lack of light in water have been observed to restrict sensing functions. This study expands the utilization of two- and three-dimensional detection technologies in underwater applications to detect abandoned tires. A three-dimensional acoustic sensor, the BV5000, is used in this study to collect underwater point cloud data. Some pre-processing steps are proposed to remove noise and the seabed from raw data. Point clouds are then processed to obtain two data types: a 2D image and a 3D point cloud. Deep learning methods with different dimensions are used to train the models. In the two-dimensional method, the point cloud is transferred into a bird’s eye view image. The Faster R-CNN and YOLOv3 network architectures are used to detect tires. Meanwhile, in the three-dimensional method, the point cloud associated with a tire is cut out from the raw data and is used as training data. The PointNet and PointConv network architectures are then used for tire classification. The results show that both approaches provide good accuracy.

scholarly journals AN EFFICIENT DEEP LEARNING APPROACH FOR GROUND POINT FILTERING IN AERIAL LASER SCANNING POINT CLOUDS

2021 ◽  
Vol XLIII-B1-2021 ◽  
pp. 31-38
Author(s):  
A. Nurunnabi ◽  
F. N. Teferle ◽  
J. Li ◽  
R. C. Lindenbergh ◽  
A. Hunegnaw
Keyword(s):  
Deep Learning ◽  
Point Cloud ◽  
Laser Scanning ◽  
Three Dimensional ◽  
Ground Surface ◽  
Point Clouds ◽  
Urban Environments ◽  
Training Data ◽  
Ground Point ◽  
End To End

Abstract. Ground surface extraction is one of the classic tasks in airborne laser scanning (ALS) point cloud processing that is used for three-dimensional (3D) city modelling, infrastructure health monitoring, and disaster management. Many methods have been developed over the last three decades. Recently, Deep Learning (DL) has become the most dominant technique for 3D point cloud classification. DL methods used for classification can be categorized into end-to-end and non end-to-end approaches. One of the main challenges of using supervised DL approaches is getting a sufficient amount of training data. The main advantage of using a supervised non end-to-end approach is that it requires less training data. This paper introduces a novel local feature-based non end-to-end DL algorithm that generates a binary classifier for ground point filtering. It studies feature relevance, and investigates three models that are different combinations of features. This method is free from the limitations of point clouds’ irregular data structure and varying data density, which is the biggest challenge for using the elegant convolutional neural network. The new algorithm does not require transforming data into regular 3D voxel grids or any rasterization. The performance of the new method has been demonstrated through two ALS datasets covering urban environments. The method successfully labels ground and non-ground points in the presence of steep slopes and height discontinuity in the terrain. Experiments in this paper show that the algorithm achieves around 97% in both F1-score and model accuracy for ground point labelling.

Reducing Manual Operation Time to Obtain a Segmentation Learning Model for Volume Electron Microscopy Using Stepwise Deep Learning With Manual Correction

Microscopy ◽  
2021 ◽  
Author(s):  
Kohki Konishi ◽  
Takao Nonaka ◽  
Shunsuke Takei ◽  
Keisuke Ohta ◽  
Hideo Nishioka ◽  
...  
Keyword(s):  
Electron Microscopy ◽  
Deep Learning ◽  
Three Dimensional ◽  
Operation Time ◽  
Training Data ◽  
Manual Operation ◽  
Annotation Method ◽  
Section Electron ◽  
Model Training ◽  
Manual Correction

Abstract Three-dimensional (3D) observation of a biological sample using serial-section electron microscopy is widely used. However, organelle segmentation requires a significant amount of manual time. Therefore, several studies have been conducted to improve their efficiency. One such promising method is 3D deep learning (DL), which is highly accurate. However, the creation of training data for 3D DL still requires manual time and effort. In this study, we developed a highly efficient integrated image segmentation tool that includes stepwise DL with manual correction. The tool has four functions: efficient tracers for annotation, model training/inference for organelle segmentation using a lightweight convolutional neural network, efficient proofreading, and model refinement. We applied this tool to increase the training data step by step (stepwise annotation method) to segment the mitochondria in the cells of the cerebral cortex. We found that the stepwise annotation method reduced the manual operation time by one-third compared with that of the fully manual method, where all the training data were created manually. Moreover, we demonstrated that the F1 score, the metric of segmentation accuracy, was 0.9 by training the 3D DL model with these training data. The stepwise annotation method using this tool and the 3D DL model improved the segmentation efficiency for various organelles.

scholarly journals DeepSynth: Three-dimensional nuclear segmentation of biological images using neural networks trained with synthetic data

2019 ◽  
Vol 9 (1) ◽  
Author(s):  
Kenneth W. Dunn ◽  
Chichen Fu ◽  
David Joon Ho ◽  
Soonam Lee ◽  
Shuo Han ◽  
...  
Keyword(s):  
Neural Networks ◽  
Image Processing ◽  
Deep Learning ◽  
Three Dimensional ◽  
Synthetic Data ◽  
Training Data ◽  
Three Dimensions ◽  
Great Promise ◽  
Biological Images ◽  
Nuclear Segmentation

AbstractThe scale of biological microscopy has increased dramatically over the past ten years, with the development of new modalities supporting collection of high-resolution fluorescence image volumes spanning hundreds of microns if not millimeters. The size and complexity of these volumes is such that quantitative analysis requires automated methods of image processing to identify and characterize individual cells. For many workflows, this process starts with segmentation of nuclei that, due to their ubiquity, ease-of-labeling and relatively simple structure, make them appealing targets for automated detection of individual cells. However, in the context of large, three-dimensional image volumes, nuclei present many challenges to automated segmentation, such that conventional approaches are seldom effective and/or robust. Techniques based upon deep-learning have shown great promise, but enthusiasm for applying these techniques is tempered by the need to generate training data, an arduous task, particularly in three dimensions. Here we present results of a new technique of nuclear segmentation using neural networks trained on synthetic data. Comparisons with results obtained using commonly-used image processing packages demonstrate that DeepSynth provides the superior results associated with deep-learning techniques without the need for manual annotation.

scholarly journals DEEP LEARNING FOR SEMANTIC SEGMENTATION OF 3D POINT CLOUD

2019 ◽  
Vol XLII-2/W15 ◽  
pp. 735-742 ◽  
Author(s):  
E. S. Malinverni ◽  
R. Pierdicca ◽  
M. Paolanti ◽  
M. Martini ◽  
C. Morbidoni ◽  
...  
Keyword(s):  
Deep Learning ◽  
Cultural Heritage ◽  
Point Cloud ◽  
Cultural Landscapes ◽  
Three Dimensional ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Training Data ◽  
Historical Building ◽  
3D Point Clouds

<p><strong>Abstract.</strong> Cultural Heritage is a testimony of past human activity, and, as such, its objects exhibit great variety in their nature, size and complexity; from small artefacts and museum items to cultural landscapes, from historical building and ancient monuments to city centers and archaeological sites. Cultural Heritage around the globe suffers from wars, natural disasters and human negligence. The importance of digital documentation is well recognized and there is an increasing pressure to document our heritage both nationally and internationally. For this reason, the three-dimensional scanning and modeling of sites and artifacts of cultural heritage have remarkably increased in recent years. The semantic segmentation of point clouds is an essential step of the entire pipeline; in fact, it allows to decompose complex architectures in single elements, which are then enriched with meaningful information within Building Information Modelling software. Notwithstanding, this step is very time consuming and completely entrusted on the manual work of domain experts, far from being automatized. This work describes a method to label and cluster automatically a point cloud based on a supervised Deep Learning approach, using a state-of-the-art Neural Network called PointNet++. Despite other methods are known, we have choose PointNet++ as it reached significant results for classifying and segmenting 3D point clouds. PointNet++ has been tested and improved, by training the network with annotated point clouds coming from a real survey and to evaluate how performance changes according to the input training data. It can result of great interest for the research community dealing with the point cloud semantic segmentation, since it makes public a labelled dataset of CH elements for further tests.</p>

scholarly journals Adaptive prospective optical gating enables day-long 3D time-lapse imaging of the beating embryonic zebrafish heart

2019 ◽  
Vol 10 (1) ◽  
Author(s):  
Jonathan M. Taylor ◽  
Carl J. Nelson ◽  
Finnius A. Bruton ◽  
Aryan Kaveh ◽  
Charlotte Buckley ◽  
...  
Keyword(s):  
Time Course ◽  
Cardiac Development ◽  
Three Dimensional ◽  
Time Lapse ◽  
Beating Heart ◽  
Cellular Mechanisms ◽  
Phase Lock ◽  
Optical Gating ◽  
Time Lapse Imaging ◽  
Zebrafish Heart

AbstractThree-dimensional fluorescence time-lapse imaging of the beating heart is extremely challenging, due to the heart’s constant motion and a need to avoid pharmacological or phototoxic damage. Although real-time triggered imaging can computationally “freeze” the heart for 3D imaging, no previous algorithm has been able to maintain phase-lock across developmental timescales. We report a new algorithm capable of maintaining day-long phase-lock, permitting routine acquisition of synchronised 3D + time video time-lapse datasets of the beating zebrafish heart. This approach has enabled us for the first time to directly observe detailed developmental and cellular processes in the beating heart, revealing the dynamics of the immune response to injury and witnessing intriguing proliferative events that challenge the established literature on cardiac trabeculation. Our approach opens up exciting new opportunities for direct time-lapse imaging studies over a 24-hour time course, to understand the cellular mechanisms underlying cardiac development, repair and regeneration.

scholarly journals Segmentor: A tool for manual refinement of 3D microscopy annotations

2021 ◽  
Author(s):  
David Borland ◽  
Carolyn M. McCormick ◽  
Niyanta K. Patel ◽  
Oleh Krupa ◽  
Jessica T. Mory ◽  
...  
Keyword(s):  
Deep Learning ◽  
High Speed ◽  
Three Dimensional ◽  
Light Sheet ◽  
Training Data ◽  
Great Promise ◽  
Manual Annotation ◽  
Light Sheet Microscopy ◽  
Link Type ◽  
Segmentation Algorithms

AbstractBackgroundRecent advances in tissue clearing techniques, combined with high-speed image acquisition through light sheet microscopy, enable rapid three-dimensional (3D) imaging of biological specimens, such as whole mouse brains, in a matter of hours. Quantitative analysis of such 3D images can help us understand how changes in brain structure lead to differences in behavior or cognition, but distinguishing features of interest, such as nuclei, from background can be challenging. Recent deep learning-based nuclear segmentation algorithms show great promise for automated segmentation, but require large numbers of manually and accurately labeled nuclei as training data.ResultsWe present Segmentor, an open-source tool for reliable, efficient, and user-friendly manual annotation and refinement of objects (e.g., nuclei) within 3D light sheet microscopy images. Segmentor employs a hybrid 2D-3D approach for visualizing and segmenting objects and contains features for automatic region splitting, designed specifically for streamlining the process of 3D segmentation of nuclei. We show that editing simultaneously in 2D and 3D using Segmentor significantly decreases time spent on manual annotations without affecting accuracy.ConclusionsSegmentor is a tool for increased efficiency of manual annotation and refinement of 3D objects that can be used to train deep learning segmentation algorithms, and is available at https://www.nucleininja.org/ and https://github.com/RENCI/Segmentor.

scholarly journals DeepCryoPicker: fully automated deep neural network for single protein particle picking in cryo-EM

2020 ◽  
Vol 21 (1) ◽  
Author(s):  
Adil Al-Azzawi ◽  
Anes Ouadou ◽  
Highsmith Max ◽  
Ye Duan ◽  
John J. Tanner ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Deep Neural Network ◽  
Signal To Noise Ratio ◽  
Three Dimensional ◽  
Training Data ◽  
Human Intervention ◽  
Protein Particle ◽  
Particle Shapes

Abstract Background Cryo-electron microscopy (Cryo-EM) is widely used in the determination of the three-dimensional (3D) structures of macromolecules. Particle picking from 2D micrographs remains a challenging early step in the Cryo-EM pipeline due to the diversity of particle shapes and the extremely low signal-to-noise ratio of micrographs. Because of these issues, significant human intervention is often required to generate a high-quality set of particles for input to the downstream structure determination steps. Results Here we propose a fully automated approach (DeepCryoPicker) for single particle picking based on deep learning. It first uses automated unsupervised learning to generate particle training datasets. Then it trains a deep neural network to classify particles automatically. Results indicate that the DeepCryoPicker compares favorably with semi-automated methods such as DeepEM, DeepPicker, and RELION, with the significant advantage of not requiring human intervention. Conclusions Our framework combing supervised deep learning classification with automated un-supervised clustering for generating training data provides an effective approach to pick particles in cryo-EM images automatically and accurately.

scholarly journals Segmentor: a tool for manual refinement of 3D microscopy annotations

2021 ◽  
Vol 22 (1) ◽  
Author(s):  
David Borland ◽  
Carolyn M. McCormick ◽  
Niyanta K. Patel ◽  
Oleh Krupa ◽  
Jessica T. Mory ◽  
...  
Keyword(s):  
Deep Learning ◽  
High Speed ◽  
Three Dimensional ◽  
Light Sheet ◽  
Training Data ◽  
Great Promise ◽  
Manual Annotation ◽  
Light Sheet Microscopy ◽  
Link Type ◽  
Segmentation Algorithms

Abstract Background Recent advances in tissue clearing techniques, combined with high-speed image acquisition through light sheet microscopy, enable rapid three-dimensional (3D) imaging of biological specimens, such as whole mouse brains, in a matter of hours. Quantitative analysis of such 3D images can help us understand how changes in brain structure lead to differences in behavior or cognition, but distinguishing densely packed features of interest, such as nuclei, from background can be challenging. Recent deep learning-based nuclear segmentation algorithms show great promise for automated segmentation, but require large numbers of accurate manually labeled nuclei as training data. Results We present Segmentor, an open-source tool for reliable, efficient, and user-friendly manual annotation and refinement of objects (e.g., nuclei) within 3D light sheet microscopy images. Segmentor employs a hybrid 2D-3D approach for visualizing and segmenting objects and contains features for automatic region splitting, designed specifically for streamlining the process of 3D segmentation of nuclei. We show that editing simultaneously in 2D and 3D using Segmentor significantly decreases time spent on manual annotations without affecting accuracy as compared to editing the same set of images with only 2D capabilities. Conclusions Segmentor is a tool for increased efficiency of manual annotation and refinement of 3D objects that can be used to train deep learning segmentation algorithms, and is available at https://www.nucleininja.org/ and https://github.com/RENCI/Segmentor.

Atomic Resolution in Crystal Aperture Stem

1990 ◽  
Vol 48 (1) ◽  
pp. 236-237
Author(s):  
J T Fourie
Keyword(s):  
Optical System ◽  
Spherical Aberration ◽  
Three Dimensional ◽  
Transmission Function ◽  
Optical Systems ◽  
Bragg Angle ◽  
Electron Optical System ◽  
Intimate Contact ◽  
Diffraction Effects ◽  
Design Aspect

The attempts at improvement of electron optical systems to date, have largely been directed towards the design aspect of magnetic lenses and towards the establishment of ideal lens combinations. In the present work the emphasis has been placed on the utilization of a unique three-dimensional crystal objective aperture within a standard electron optical system with the aim to reduce the spherical aberration without introducing diffraction effects. A brief summary of this work together with a description of results obtained recently, will be given.The concept of utilizing a crystal as aperture in an electron optical system was introduced by Fourie who employed a {111} crystal foil as a collector aperture, by mounting the sample directly on top of the foil and in intimate contact with the foil. In the present work the sample was mounted on the bottom of the foil so that the crystal would function as an objective or probe forming aperture. The transmission function of such a crystal aperture depends on the thickness, t, and the orientation of the foil. The expression for calculating the transmission function was derived by Hashimoto, Howie and Whelan on the basis of the electron equivalent of the Borrmann anomalous absorption effect in crystals. In Fig. 1 the functions for a g220 diffraction vector and t = 0.53 and 1.0 μm are shown. Here n= Θ‒ΘB, where Θ is the angle between the incident ray and the (hkl) planes, and ΘB is the Bragg angle.

Sign in / Sign up

Export Citation Format

Share Document