Deep Learning-Based Instance Segmentation Method of Litchi Canopy from UAV-Acquired Images

Instance segmentation of fruit tree canopies from images acquired by unmanned aerial vehicles (UAVs) is of significance for the precise management of orchards. Although deep learning methods have been widely used in the fields of feature extraction and classification, there are still phenomena of complex data and strong dependence on software performances. This paper proposes a deep learning-based instance segmentation method of litchi trees, which has a simple structure and lower requirements for data form. Considering that deep learning models require a large amount of training data, a labor-friendly semi-auto method for image annotation is introduced. The introduction of this method allows for a significant improvement in the efficiency of data pre-processing. Facing the high requirement of a deep learning method for computing resources, a partition-based method is presented for the segmentation of high-resolution digital orthophoto maps (DOMs). Citrus data is added to the training set to alleviate the lack of diversity of the original litchi dataset. The average precision (AP) is selected to evaluate the metric of the proposed model. The results show that with the help of training with the litchi-citrus datasets, the best AP on the test set reaches 96.25%.

Download Full-text

OpSeF: Open source Python framework for collaborative instance segmentation of bioimages

10.1101/2020.04.29.068023 ◽

2020 ◽

Cited By ~ 2

Author(s):

Tobias M. Rasse ◽

Réka Hollandi ◽

Péter Horváth

Keyword(s):

Machine Learning ◽

Deep Learning ◽

Complex Analysis ◽

Ease Of Use ◽

Problem Definition ◽

Training Data ◽

Post Processing ◽

Gpu Clusters ◽

User Tasks ◽

Instance Segmentation

AbstractVarious pre-trained deep learning models for the segmentation of bioimages have been made available as ‘developer-to-end-user’ solutions. They usually require neither knowledge of machine learning nor coding skills, are optimized for ease of use, and deployability on laptops. However, testing these tools individually is tedious and success is uncertain.Here, we present the ‘Op’en ‘Se’gmentation ‘F’ramework (OpSeF), a Python framework for deep learning-based instance segmentation. OpSeF aims at facilitating the collaboration of biomedical users with experienced image analysts. It builds on the analysts’ knowledge in Python, machine learning, and workflow design to solve complex analysis tasks at any scale in a reproducible, well-documented way. OpSeF defines standard inputs and outputs, thereby facilitating modular workflow design and interoperability with other software. Users play an important role in problem definition, quality control, and manual refinement of results. All analyst tasks are optimized for deployment on Linux workstations or GPU clusters, all user tasks may be performed on any laptop in ImageJ.OpSeF semi-automates preprocessing, convolutional neural network (CNN)-based segmentation in 2D or 3D, and post-processing. It facilitates benchmarking of multiple models in parallel. OpSeF streamlines the optimization of parameters for pre- and post-processing such, that an available model may frequently be used without retraining. Even if sufficiently good results are not achievable with this approach, intermediate results can inform the analysts in the selection of the most promising CNN-architecture in which the biomedical user might invest the effort of manually labeling training data.We provide Jupyter notebooks that document sample workflows based on various image collections. Analysts may find these notebooks useful to illustrate common segmentation challenges, as they prepare the advanced user for gradually taking over some of their tasks and completing their projects independently. The notebooks may also be used to explore the analysis options available within OpSeF in an interactive way and to document and share final workflows.Currently, three mechanistically distinct CNN-based segmentation methods, the U-Net implementation used in Cellprofiler 3.0, StarDist, and Cellpose have been integrated within OpSeF. The addition of new networks requires little, the addition of new models requires no coding skills. Thus, OpSeF might soon become both an interactive model repository, in which pre-trained models might be shared, evaluated, and reused with ease.

Download Full-text

Application of deep learning methods in biological networks

Briefings in Bioinformatics ◽

10.1093/bib/bbaa043 ◽

2020 ◽

Cited By ~ 1

Author(s):

Shuting Jin ◽

Xiangxiang Zeng ◽

Feng Xia ◽

Wei Huang ◽

Xiangrong Liu

Keyword(s):

Deep Learning ◽

Biological Networks ◽

Learning Algorithm ◽

Biological Data ◽

Training Data ◽

Learning Ability ◽

Network Data ◽

Complex Data ◽

Layer By Layer ◽

Information Layer

Abstract The increase in biological data and the formation of various biomolecule interaction databases enable us to obtain diverse biological networks. These biological networks provide a wealth of raw materials for further understanding of biological systems, the discovery of complex diseases and the search for therapeutic drugs. However, the increase in data also increases the difficulty of biological networks analysis. Therefore, algorithms that can handle large, heterogeneous and complex data are needed to better analyze the data of these network structures and mine their useful information. Deep learning is a branch of machine learning that extracts more abstract features from a larger set of training data. Through the establishment of an artificial neural network with a network hierarchy structure, deep learning can extract and screen the input information layer by layer and has representation learning ability. The improved deep learning algorithm can be used to process complex and heterogeneous graph data structures and is increasingly being applied to the mining of network data information. In this paper, we first introduce the used network data deep learning models. After words, we summarize the application of deep learning on biological networks. Finally, we discuss the future development prospects of this field.

Download Full-text

HistoFlow: Label-Efficient and Interactive Deep Learning Cell Analysis

10.1101/2020.07.15.204891 ◽

2020 ◽

Author(s):

Tim Henning ◽

Benjamin Bergner ◽

Christoph Lippert

Keyword(s):

Machine Learning ◽

Deep Learning ◽

User Interface ◽

Network Architecture ◽

Training Data ◽

Cell Segmentation ◽

Annotation Tool ◽

Cell Analysis ◽

Instance Segmentation

Instance segmentation is a common task in quantitative cell analysis. While there are many approaches doing this using machine learning, typically, the training process requires a large amount of manually annotated data. We present HistoFlow, a software for annotation-efficient training of deep learning models for cell segmentation and analysis with an interactive user interface.It provides an assisted annotation tool to quickly draw and correct cell boundaries and use biomarkers as weak annotations. It also enables the user to create artificial training data to lower the labeling effort. We employ a universal U-Net neural network architecture that allows accurate instance segmentation and the classification of phenotypes in only a single pass of the network. Transfer learning is available through the user interface to adapt trained models to new tissue types.We demonstrate HistoFlow for fluorescence breast cancer images. The models trained using only artificial data perform comparably to those trained with time-consuming manual annotations. They outperform traditional cell segmentation algorithms and match state-of-the-art machine learning approaches. A user test shows that cells can be annotated six times faster than without the assistance of our annotation tool. Extending a segmentation model for classification of epithelial cells can be done using only 50 to 1500 annotations.Our results show that, unlike previous assumptions, it is possible to interactively train a deep learning model in a matter of minutes without many manual annotations.

Download Full-text

Superpixel segmentations for thin sections: evaluation of methods to enable the generation of machine learning training data sets

10.31223/x55s65 ◽

2021 ◽

Author(s):

Jiaxin Yu ◽

Florian Wellmann ◽

Simon Virgo ◽

Marven von Domarus ◽

Mingze Jiang ◽

...

Keyword(s):

Machine Learning ◽

Deep Learning ◽

Thin Section ◽

Image Annotation ◽

Performance Testing ◽

Training Image ◽

Training Data ◽

Sandstone Sample ◽

Thin Sections ◽

Superpixel Segmentation

Training data is the backbone of developing either Machine Learning (ML) models or specific deep learning algorithms. The paucity of well-labeled training image data has significantly impeded the applications of ML-based approaches, especially the development of novel Deep Learning (DL) methods like Convolutional Neural Networks (CNNs) in mineral thin section images identification. However, image annotation, especially pixel-wise annotation is always a costly process. Manually creating dense semantic labels for rock thin section images has been long considered as an unprecedented challenge in view of the ubiquitous variety and complexity of minerals in thin sections. To speed up the annotation, we propose a human-computer collaborative pipeline in which superpixel segmentation is used as a boundary extractor to avoid hand delineation of instances boundaries. The pipeline consists of two steps: superpixel segmentation using MultiSLIC, and superpixel labeling through a specific-designed tool. We use a cutting-edge methodology Virtual Petroscopy (ViP) for automatic image acquisition. Bentheimer sandstone sample is used to conduct performance testing of the pipeline. Three standard error metrics are used to evaluate the performance of MultiSLIC. The result indicates that MultiSLIC is able to extract compact superpixels with satisfying boundary adherence given multiple input images. According to our test results, large and complex thin section images with pixel-wisely accurate labels can be annotated with the labeling tool more efficiently than in a conventional, purely manual work, and generate data of high quality.

Download Full-text

CVFlow: Open Workflow for Computer Vision

10.31219/osf.io/t2f4w ◽

2019 ◽

Author(s):

Xiaozhe Yao

Keyword(s):

Computer Vision ◽

Deep Learning ◽

Object Detection ◽

Image Annotation ◽

Learning System ◽

Commercial Products ◽

Learning In Computer Vision ◽

Instance Segmentation ◽

Image Caption

In recent years, the multimedia community has witnessed an emerging usage of deep learning in computer vision, such as image caption\cite{Vinyals_2015_CVPR}, object detection, instance segmentation and etc, and many companies are adopting deep learning into their commercial products. Developing and Deploying a fast and reliable deep learning system is complex and involves labour-intensive works, like image annotation\cite{Ratner_2017}, data and models versioning and etc. To facilitate such a process, we proposed CVFlow, which is a suite of versatile computer vision libraries and toolkits for handling computer vision tasks, including intelligent annotation toolkit, deep learning package manager, models sharing platform and computer vision serving program.

Download Full-text

Instance Segmentation Method of User Interface Component of Games

Applied Sciences ◽

10.3390/app10186502 ◽

2020 ◽

Vol 10 (18) ◽

pp. 6502

Author(s):

Shinjin Kang ◽

Jong-in Choi

Keyword(s):

Deep Learning ◽

Semantic Information ◽

Data Augmentation ◽

Image Data ◽

Learning Networks ◽

Segmentation Method ◽

Learning Network ◽

Additional Input ◽

Deep Learning Network ◽

Instance Segmentation

On the game screen, the UI interface provides key information for game play. A vision deep learning network exploits pure pixel information in the screen. Apart from this, if we separately extract the information provided by the UI interface and use it as an additional input value, we can enhance the learning efficiency of deep learning networks. To this end, by effectively segmenting UI interface components such as buttons, image icons, and gauge bars on the game screen, we should be able to separately analyze only the relevant images. In this paper, we propose a methodology that segments UI components in a game by using synthetic game images created on a game engine. We developed a tool that approximately detected the UI areas of an image in games on the game screen and generated a large amount of synthetic labeling data through this. By training this data on a Pix2Pix, we applied UI segmentation. The network trained in this way can segment the UI areas of the target game regardless of the position of the corresponding UI components. Our methodology can help analyze the game screen without applying data augmentation to the game screen. It can also help vision researchers who should extract semantic information from game image data.

Download Full-text

Semi-Supervised Deep Learning for Lunar Crater Detection Using CE-2 DOM

Remote Sensing ◽

10.3390/rs13142819 ◽

2021 ◽

Vol 13 (14) ◽

pp. 2819

Author(s):

Sudong Zang ◽

Lingli Mu ◽

Lina Xian ◽

Wei Zhang

Keyword(s):

Deep Learning ◽

Landing Site ◽

Training Data ◽

The Moon ◽

Detection Model ◽

Processing Times ◽

Crater Detection ◽

High Resolution Imagery ◽

Model Training ◽

Digital Orthophoto

Lunar craters are very important for estimating the geological age of the Moon, studying the evolution of the Moon, and for landing site selection. Due to a lack of labeled samples, processing times due to high-resolution imagery, the small number of suitable detection models, and the influence of solar illumination, Crater Detection Algorithms (CDAs) based on Digital Orthophoto Maps (DOMs) have not yet been well-developed. In this paper, a large number of training data are labeled manually in the Highland and Maria regions, using the Chang’E-2 (CE-2) DOM; however, the labeled data cannot cover all kinds of crater types. To solve the problem of small crater detection, a new crater detection model (Crater R-CNN) is proposed, which can effectively extract the spatial and semantic information of craters from DOM data. As incomplete labeled samples are not conducive for model training, the Two-Teachers Self-training with Noise (TTSN) method is used to train the Crater R-CNN model, thus constructing a new model—called Crater R-CNN with TTSN—which can achieve state-of-the-art performance. To evaluate the accuracy of the model, three other detection models (Mask R-CNN, no-Mask R-CNN, and Crater R-CNN) based on semi-supervised deep learning were used to detect craters in the Highland and Maria regions. The results indicate that Crater R-CNN with TTSN achieved the highest precision (of 91.4% and 88.5%, respectively) in the Highland and Maria regions, even obtaining the highest recall and F1 score. Compared with Mask R-CNN, no-Mask R-CNN, and Crater R-CNN, Crater R-CNN with TTSN had strong robustness and better generalization ability for crater detection within 1 km in different terrains, making it possible to detect small craters with high accuracy when using DOM data.

Download Full-text

Towards Accurate State of Charge Estimation for Lithium-ion Batteries using Self-supervised Transformer Model: A Deep Learning Approach

10.21203/rs.3.rs-687515/v1 ◽

2021 ◽

Author(s):

M Hannan ◽

Dickson How ◽

M. S. Hossain Lipu ◽

M Mansor ◽

Pin Ker ◽

...

Keyword(s):

Deep Learning ◽

Lithium Ion ◽

Absolute Error ◽

State Of Charge ◽

Training Data ◽

Ambient Temperatures ◽

Soc Estimation ◽

Proposed Model ◽

Li Ion ◽

Transformer Model

Abstract Accurate state of charge (SOC) estimation of lithium-ion (Li-ion) batteries is crucial in prolonging cell lifespan and ensuring its safe operation for electric vehicle applications. In this article, we propose the deep learning-based transformer model trained with self-supervised learning (SSL) for end-to-end SOC estimation without the requirements of feature engineering or adaptive filtering. We demonstrate that with the SSL framework, the proposed deep learning-enabled transformer model achieves the lowest root-mean-square-error (RMSE) of 1.2% and a mean-absolute-error (MAE) of 0.7% on the test dataset at various ambient temperatures. With SSL, the proposed model can be trained with as few as 5 epochs using only 20% of the total training data and still achieves less than 1.9% RMSE on the test data. Finally, we also demonstrate that the learning weights during the SSL training can be transferred to a new Li-ion cell with different chemistry and still achieve on-par performance compared to the models trained from scratch on the new cell.

Download Full-text

Cellpose: a generalist algorithm for cellular segmentation

10.1101/2020.02.02.931238 ◽

2020 ◽

Cited By ~ 21

Author(s):

Carsen Stringer ◽

Tim Wang ◽

Michalis Michaelos ◽

Marius Pachitariu

Keyword(s):

Deep Learning ◽

Training Data ◽

Data Repository ◽

Biological Applications ◽

Segmentation Method ◽

Wide Range ◽

Great Progress ◽

Microscopy Images ◽

2D Model

Many biological applications require the segmentation of cell bodies, membranes and nuclei from microscopy images. Deep learning has enabled great progress on this problem, but current methods are specialized for images that have large training datasets. Here we introduce a generalist, deep learning-based segmentation method called Cellpose, which can precisely segment cells from a wide range of image types and does not require model retraining or parameter adjustments. We trained Cellpose on a new dataset of highly-varied images of cells, containing over 70,000 segmented objects. We also demonstrate a 3D extension of Cellpose which reuses the 2D model and does not require 3D-labelled data. To support community contributions to the training data, we developed software for manual labelling and for curation of the automated results, with optional direct upload to our data repository. Periodically retraining the model on the community-contributed data will ensure that Cellpose improves constantly.

Download Full-text

Learning from Synthetic Dataset for Crop Seed Instance Segmentation

10.1101/866921 ◽

2019 ◽

Author(s):

Yosuke Toda ◽

Fumio Okura ◽

Jun Ito ◽

Satoshi Okada ◽

Toshinori Kinoshita ◽

...

Keyword(s):

Neural Network ◽

Deep Learning ◽

Large Scale ◽

Synthetic Data ◽

Seed Morphology ◽

Training Data ◽

Training Dataset ◽

Labor Costs ◽

Data Annotation ◽

Instance Segmentation

Incorporating deep learning in the image analysis pipeline has opened the possibility of introducing precision phenotyping in the field of agriculture. However, to train the neural network, a sufficient amount of training data must be prepared, which requires a time-consuming manual data annotation process that often becomes the limiting step. Here, we show that an instance segmentation neural network (Mask R-CNN) aimed to phenotype the barley seed morphology of various cultivars, can be sufficiently trained purely by a synthetically generated dataset. Our attempt is based on the concept of domain randomization, where a large amount of image is generated by randomly orienting the seed object to a virtual canvas. After training with such a dataset, performance based on recall and the average Precision of the real-world test dataset achieved 96% and 95%, respectively. Applying our pipeline enables extraction of morphological parameters at a large scale, enabling precise characterization of the natural variation of barley from a multivariate perspective. Importantly, we show that our approach is effective not only for barley seeds but also for various crops including rice, lettuce, oat, and wheat, and thus supporting the fact that the performance benefits of this technique is generic. We propose that constructing and utilizing such synthetic data can be a powerful method to alleviate human labor costs needed to prepare the training dataset for deep learning in the agricultural domain.

Download Full-text