Automatic Deep Learning Semantic Segmentation of Ultrasound Thyroid Cineclips using Recurrent Fully Convolutional Networks

Deep learning architectures have received much attention in recent years demonstrating state-of-the-art performance in several segmentation, classification and other computer vision tasks. Most of these deep networks are based on either convolutional or fully convolutional architectures. In this paper, we propose a novel object-based deep-learning framework for semantic segmentation in very high-resolution satellite data. In particular, we exploit object-based priors integrated into a fully convolutional neural network by incorporating an anisotropic diffusion data preprocessing step and an additional loss term during the training process. Under this constrained framework, the goal is to enforce pixels that belong to the same object to be classified at the same semantic category. We compared thoroughly the novel object-based framework with the currently dominating convolutional and fully convolutional deep networks. In particular, numerous experiments were conducted on the publicly available ISPRS WGII/4 benchmark datasets, namely Vaihingen and Potsdam, for validation and inter-comparison based on a variety of metrics. Quantitatively, experimental results indicate that, overall, the proposed object-based framework slightly outperformed the current state-of-the-art fully convolutional networks by more than 1% in terms of overall accuracy, while intersection over union results are improved for all semantic categories. Qualitatively, man-made classes with more strict geometry such as buildings were the ones that benefit most from our method, especially along object boundaries, highlighting the great potential of the developed approach.

Download Full-text

Deep Residual Autoencoder with Multiscaling for Semantic Segmentation of Land-Use Images

Remote Sensing ◽

10.3390/rs11182142 ◽

2019 ◽

Vol 11 (18) ◽

pp. 2142 ◽

Cited By ~ 5

Author(s):

Lianfa Li

Keyword(s):

Land Use ◽

Deep Learning ◽

Semantic Segmentation ◽

Remotely Sensed ◽

Convolutional Network ◽

Convolutional Networks ◽

Residual Learning ◽

Fully Convolutional Networks ◽

Remotely Sensed Images ◽

Real World Datasets

Semantic segmentation is a fundamental means of extracting information from remotely sensed images at the pixel level. Deep learning has enabled considerable improvements in efficiency and accuracy of semantic segmentation of general images. Typical models range from benchmarks such as fully convolutional networks, U-Net, Micro-Net, and dilated residual networks to the more recently developed DeepLab 3+. However, many of these models were originally developed for segmentation of general or medical images and videos, and are not directly relevant to remotely sensed images. The studies of deep learning for semantic segmentation of remotely sensed images are limited. This paper presents a novel flexible autoencoder-based architecture of deep learning that makes extensive use of residual learning and multiscaling for robust semantic segmentation of remotely sensed land-use images. In this architecture, a deep residual autoencoder is generalized to a fully convolutional network in which residual connections are implemented within and between all encoding and decoding layers. Compared with the concatenated shortcuts in U-Net, these residual connections reduce the number of trainable parameters and improve the learning efficiency by enabling extensive backpropagation of errors. In addition, resizing or atrous spatial pyramid pooling (ASPP) can be leveraged to capture multiscale information from the input images to enhance the robustness to scale variations. The residual learning and multiscaling strategies improve the trained model’s generalizability, as demonstrated in the semantic segmentation of land-use types in two real-world datasets of remotely sensed images. Compared with U-Net, the proposed method improves the Jaccard index (JI) or the mean intersection over union (MIoU) by 4-11% in the training phase and by 3-9% in the validation and testing phases. With its flexible deep learning architecture, the proposed approach can be easily applied for and transferred to semantic segmentation of land-use variables and other surface variables of remotely sensed images.

Download Full-text

Depth Density Achieves a Better Result for Semantic Segmentation with the Kinect System

Sensors ◽

10.3390/s20030812 ◽

2020 ◽

Vol 20 (3) ◽

pp. 812 ◽

Cited By ~ 1

Author(s):

Hanbing Deng ◽

Tongyu Xu ◽

Yuncheng Zhou ◽

Teng Miao

Keyword(s):

Neural Networks ◽

Image Segmentation ◽

Deep Learning ◽

Semantic Segmentation ◽

Depth Image ◽

Depth Information ◽

Convolutional Networks ◽

Automatic Feature Extraction ◽

Fully Convolutional Networks ◽

Simmental Cattle

Image segmentation is one of the most important methods for animal phenome research. Since the advent of deep learning, many researchers have looked at multilayer convolutional neural networks to solve the problems of image segmentation. A network simplifies the task of image segmentation with automatic feature extraction. Many networks struggle to output accurate details when dealing with pixel-level segmentation. In this paper, we propose a new concept: Depth density. Based on a depth image, produced by a Kinect system, we design a new function to calculate the depth density value of each pixel and bring this value back to the result of semantic segmentation for improving the accuracy. In the experiment, we choose Simmental cattle as the target of image segmentation and fully convolutional networks (FCN) as the verification networks. We proved that depth density can improve four metrics of semantic segmentation (pixel accuracy, mean accuracy, mean intersection over union, and frequency weight intersection over union) by 2.9%, 0.3%, 11.4%, and 5.02%, respectively. The result shows that depth information produced by Kinect can improve the accuracy of the semantic segmentation of FCN. This provides a new way of analyzing the phenotype information of animals.

Download Full-text

Deep Convolutional Neural Network for Large-Scale Date Palm Tree Mapping from UAV-Based Images

Remote Sensing ◽

10.3390/rs13142787 ◽

2021 ◽

Vol 13 (14) ◽

pp. 2787

Author(s):

Mohamed Barakat A. Gibril ◽

Helmi Zulhaidi Mohd Shafri ◽

Abdallah Shanableh ◽

Rami Al-Ruzouq ◽

Aimrun Wayayok ◽

...

Keyword(s):

Neural Network ◽

Deep Learning ◽

Convolutional Neural Network ◽

Large Scale ◽

Date Palm ◽

Semantic Segmentation ◽

Palm Tree ◽

Convolutional Networks ◽

Fully Convolutional Networks ◽

Palm Trees

Large-scale mapping of date palm trees is vital for their consistent monitoring and sustainable management, considering their substantial commercial, environmental, and cultural value. This study presents an automatic approach for the large-scale mapping of date palm trees from very-high-spatial-resolution (VHSR) unmanned aerial vehicle (UAV) datasets, based on a deep learning approach. A U-Shape convolutional neural network (U-Net), based on a deep residual learning framework, was developed for the semantic segmentation of date palm trees. A comprehensive set of labeled data was established to enable the training and evaluation of the proposed segmentation model and increase its generalization capability. The performance of the proposed approach was compared with those of various state-of-the-art fully convolutional networks (FCNs) with different encoder architectures, including U-Net (based on VGG-16 backbone), pyramid scene parsing network, and two variants of DeepLab V3+. Experimental results showed that the proposed model outperformed other FCNs in the validation and testing datasets. The generalizability evaluation of the proposed approach on a comprehensive and complex testing dataset exhibited higher classification accuracy and showed that date palm trees could be automatically mapped from VHSR UAV images with an F-score, mean intersection over union, precision, and recall of 91%, 85%, 0.91, and 0.92, respectively. The proposed approach provides an efficient deep learning architecture for the automatic mapping of date palm trees from VHSR UAV-based images.

Download Full-text

A Hybrid Semantic Segmentation Based on Level-Set Evolution Driven by Fully Convolutional Networks

IEEE Access ◽

10.1109/access.2021.3066515 ◽

2021 ◽

Vol 9 ◽

pp. 42556-42567

Author(s):

Meng Wang ◽

Yi Ma ◽

Fan Li ◽

Zhengbing Guo

Keyword(s):

Level Set ◽

Semantic Segmentation ◽

Convolutional Networks ◽

Fully Convolutional Networks ◽

Level Set Evolution

Download Full-text

Domain Adaptation for Semantic Segmentation of Historical Panchromatic Orthomosaics in Central Africa

ISPRS International Journal of Geo-Information ◽

10.3390/ijgi10080523 ◽

2021 ◽

Vol 10 (8) ◽

pp. 523

Author(s):

Nicholus Mboga ◽

Stefano D’Aronco ◽

Tais Grippa ◽

Charlotte Pelletier ◽

Stefanos Georganos ◽

...

Keyword(s):

Land Cover ◽

Domain Adaptation ◽

Central Africa ◽

Semantic Segmentation ◽

Target Domain ◽

Convolutional Networks ◽

Fully Convolutional Networks ◽

The Cost ◽

Performance Gains

Multitemporal environmental and urban studies are essential to guide policy making to ultimately improve human wellbeing in the Global South. Land-cover products derived from historical aerial orthomosaics acquired decades ago can provide important evidence to inform long-term studies. To reduce the manual labelling effort by human experts and to scale to large, meaningful regions, we investigate in this study how domain adaptation techniques and deep learning can help to efficiently map land cover in Central Africa. We propose and evaluate a methodology that is based on unsupervised adaptation to reduce the cost of generating reference data for several cities and across different dates. We present the first application of domain adaptation based on fully convolutional networks for semantic segmentation of a dataset of historical panchromatic orthomosaics for land-cover generation for two focus cities Goma-Gisenyi and Bukavu. Our experimental evaluation shows that the domain adaptation methods can reach an overall accuracy between 60% and 70% for different regions. If we add a small amount of labelled data from the target domain, too, further performance gains can be achieved.

Download Full-text

Dual Path Attention Net for Remote Sensing Semantic Image Segmentation

ISPRS International Journal of Geo-Information ◽

10.3390/ijgi9100571 ◽

2020 ◽

Vol 9 (10) ◽

pp. 571

Author(s):

Jinglun Li ◽

Jiapeng Xiu ◽

Zhengqiu Yang ◽

Chen Liu

Keyword(s):

Remote Sensing ◽

Spatial Information ◽

Semantic Segmentation ◽

Remote Sensing Images ◽

Convolutional Networks ◽

Fully Convolutional Networks ◽

Augmentation Strategies ◽

Rich Information ◽

The Mean ◽

The Rich

Semantic segmentation plays an important role in being able to understand the content of remote sensing images. In recent years, deep learning methods based on Fully Convolutional Networks (FCNs) have proved to be effective for the sematic segmentation of remote sensing images. However, the rich information and complex content makes the training of networks for segmentation challenging, and the datasets are necessarily constrained. In this paper, we propose a Convolutional Neural Network (CNN) model called Dual Path Attention Network (DPA-Net) that has a simple modular structure and can be added to any segmentation model to enhance its ability to learn features. Two types of attention module are appended to the segmentation model, one focusing on spatial information the other focusing upon the channel. Then, the outputs of these two attention modules are fused to further improve the network’s ability to extract features, thus contributing to more precise segmentation results. Finally, data pre-processing and augmentation strategies are used to compensate for the small number of datasets and uneven distribution. The proposed network was tested on the Gaofen Image Dataset (GID). The results show that the network outperformed U-Net, PSP-Net, and DeepLab V3+ in terms of the mean IoU by 0.84%, 2.54%, and 1.32%, respectively.

Download Full-text

Detecting Objects from Space: An Evaluation of Deep-Learning Modern Approaches

Electronics ◽

10.3390/electronics9040583 ◽

2020 ◽

Vol 9 (4) ◽

pp. 583 ◽

Cited By ~ 6

Author(s):

Khang Nguyen ◽

Nhut T. Huynh ◽

Phat C. Nguyen ◽

Khanh-Duy Nguyen ◽

Nguyen D. Vo ◽

...

Keyword(s):

Deep Learning ◽

Object Detection ◽

Unmanned Aircraft ◽

Aerial Images ◽

Great Success ◽

Single Shot ◽

Convolutional Networks ◽

Image Pyramids ◽

Fully Convolutional Networks ◽

Wide Range

Unmanned aircraft systems or drones enable us to record or capture many scenes from the bird’s-eye view and they have been fast deployed to a wide range of practical domains, i.e., agriculture, aerial photography, fast delivery and surveillance. Object detection task is one of the core steps in understanding videos collected from the drones. However, this task is very challenging due to the unconstrained viewpoints and low resolution of captured videos. While deep-learning modern object detectors have recently achieved great success in general benchmarks, i.e., PASCAL-VOC and MS-COCO, the robustness of these detectors on aerial images captured by drones is not well studied. In this paper, we present an evaluation of state-of-the-art deep-learning detectors including Faster R-CNN (Faster Regional CNN), RFCN (Region-based Fully Convolutional Networks), SNIPER (Scale Normalization for Image Pyramids with Efficient Resampling), Single-Shot Detector (SSD), YOLO (You Only Look Once), RetinaNet, and CenterNet for the object detection in videos captured by drones. We conduct experiments on VisDrone2019 dataset which contains 96 videos with 39,988 annotated frames and provide insights into efficient object detectors for aerial images.

Download Full-text