Image-Based Iron Slag Segmentation via Graph Convolutional Networks

Slagging-off (i.e., slag removal) is an important preprocessing operation of steel-making to improve the purity of iron. Current manual-operated slag removal schemes are inefficient and labor-intensive. Automatic slagging-off is desirable but challenging as the reliable recognition of iron and slag is difficult. This work focuses on realizing an efficient and accurate recognition algorithm of iron and slag, which is conducive to realize automatic slagging-off operation. Motivated by the recent success of deep learning techniques in smart manufacturing, we introduce deep learning methods to this field for the first time. The monotonous gray value of industry images, poor image quality, and nonrigid feature of iron and slag challenge the existing fully convolutional networks (FCNs). To this end, we propose a novel spatial and feature graph convolutional network (SFGCN) module. SFGCN module can be easily inserted in FCNs to improve the reasoning ability of global contextual information, which is helpful to enhance the segmentation accuracy of small objects and isolated areas. To verify the validity of the SFGCN module, we create an industrial dataset and conduct extensive experiments. Finally, the results show that our SFGCN module brings a consistent performance boost for a wide range of FCNs. Moreover, by adopting a lightweight network as backbone, our method achieves real-time iron and slag segmentation. In the future work, we will dedicate our efforts to the weakly supervised learning for quick annotation of big data stream to improve the generalization ability of current models.

Download Full-text

Detecting Objects from Space: An Evaluation of Deep-Learning Modern Approaches

Electronics ◽

10.3390/electronics9040583 ◽

2020 ◽

Vol 9 (4) ◽

pp. 583 ◽

Cited By ~ 6

Author(s):

Khang Nguyen ◽

Nhut T. Huynh ◽

Phat C. Nguyen ◽

Khanh-Duy Nguyen ◽

Nguyen D. Vo ◽

...

Keyword(s):

Deep Learning ◽

Object Detection ◽

Unmanned Aircraft ◽

Aerial Images ◽

Great Success ◽

Single Shot ◽

Convolutional Networks ◽

Image Pyramids ◽

Fully Convolutional Networks ◽

Wide Range

Unmanned aircraft systems or drones enable us to record or capture many scenes from the bird’s-eye view and they have been fast deployed to a wide range of practical domains, i.e., agriculture, aerial photography, fast delivery and surveillance. Object detection task is one of the core steps in understanding videos collected from the drones. However, this task is very challenging due to the unconstrained viewpoints and low resolution of captured videos. While deep-learning modern object detectors have recently achieved great success in general benchmarks, i.e., PASCAL-VOC and MS-COCO, the robustness of these detectors on aerial images captured by drones is not well studied. In this paper, we present an evaluation of state-of-the-art deep-learning detectors including Faster R-CNN (Faster Regional CNN), RFCN (Region-based Fully Convolutional Networks), SNIPER (Scale Normalization for Image Pyramids with Efficient Resampling), Single-Shot Detector (SSD), YOLO (You Only Look Once), RetinaNet, and CenterNet for the object detection in videos captured by drones. We conduct experiments on VisDrone2019 dataset which contains 96 videos with 39,988 annotated frames and provide insights into efficient object detectors for aerial images.

Download Full-text

Deep Residual Autoencoder with Multiscaling for Semantic Segmentation of Land-Use Images

Remote Sensing ◽

10.3390/rs11182142 ◽

2019 ◽

Vol 11 (18) ◽

pp. 2142 ◽

Cited By ~ 5

Author(s):

Lianfa Li

Keyword(s):

Land Use ◽

Deep Learning ◽

Semantic Segmentation ◽

Remotely Sensed ◽

Convolutional Network ◽

Convolutional Networks ◽

Residual Learning ◽

Fully Convolutional Networks ◽

Remotely Sensed Images ◽

Real World Datasets

Semantic segmentation is a fundamental means of extracting information from remotely sensed images at the pixel level. Deep learning has enabled considerable improvements in efficiency and accuracy of semantic segmentation of general images. Typical models range from benchmarks such as fully convolutional networks, U-Net, Micro-Net, and dilated residual networks to the more recently developed DeepLab 3+. However, many of these models were originally developed for segmentation of general or medical images and videos, and are not directly relevant to remotely sensed images. The studies of deep learning for semantic segmentation of remotely sensed images are limited. This paper presents a novel flexible autoencoder-based architecture of deep learning that makes extensive use of residual learning and multiscaling for robust semantic segmentation of remotely sensed land-use images. In this architecture, a deep residual autoencoder is generalized to a fully convolutional network in which residual connections are implemented within and between all encoding and decoding layers. Compared with the concatenated shortcuts in U-Net, these residual connections reduce the number of trainable parameters and improve the learning efficiency by enabling extensive backpropagation of errors. In addition, resizing or atrous spatial pyramid pooling (ASPP) can be leveraged to capture multiscale information from the input images to enhance the robustness to scale variations. The residual learning and multiscaling strategies improve the trained model’s generalizability, as demonstrated in the semantic segmentation of land-use types in two real-world datasets of remotely sensed images. Compared with U-Net, the proposed method improves the Jaccard index (JI) or the mean intersection over union (MIoU) by 4-11% in the training phase and by 3-9% in the validation and testing phases. With its flexible deep learning architecture, the proposed approach can be easily applied for and transferred to semantic segmentation of land-use variables and other surface variables of remotely sensed images.

Download Full-text

Delineation of Agricultural Field Boundaries from Sentinel-2 Images Using a Novel Super-Resolution Contour Detector Based on Fully Convolutional Networks

Remote Sensing ◽

10.3390/rs12010059 ◽

2019 ◽

Vol 12 (1) ◽

pp. 59 ◽

Cited By ~ 2

Author(s):

Khairiya Mudrik Masoud ◽

Claudio Persello ◽

Valentyn A. Tolpekin

Keyword(s):

Contextual Information ◽

Super Resolution ◽

Contour Detection ◽

Agricultural Field ◽

Convolutional Network ◽

Convolutional Networks ◽

Fully Convolutional Networks ◽

Learning Technique ◽

Field Boundaries ◽

Sentinel 2

Boundaries of agricultural fields are important features necessary for defining the location, shape, and spatial extent of agricultural units. They are commonly used to summarize production statistics at the field level. In this study, we investigate the delineation of agricultural field boundaries (AFB) from Sentinel-2 satellite images acquired over the Flevoland province, the Netherlands, using a deep learning technique based on fully convolutional networks (FCNs). We designed a multiple dilation fully convolutional network (MD-FCN) for AFB detection from Sentinel-2 images at 10 m resolution. Furthermore, we developed a novel super-resolution semantic contour detection network (named SRC-Net) using a transposed convolutional layer in the FCN architecture to enhance the spatial resolution of the AFB output from 10 m to 5 m resolution. The SRC-Net also improves the AFB maps at 5 m resolution by exploiting the spatial-contextual information in the label space. The results of the proposed SRC-Net outperform alternative upsampling techniques and are only slightly inferior to the results of the MD-FCN for AFB detection from RapidEye images acquired at 5 m resolution.

Download Full-text

Automatic Deep Learning Semantic Segmentation of Ultrasound Thyroid Cineclips using Recurrent Fully Convolutional Networks

IEEE Access ◽

10.1109/access.2020.3045906 ◽

2020 ◽

pp. 1-1

Author(s):

Jeremy M. Webb ◽

Duane D. Meixner ◽

Shaheeda A. Adusei ◽

Eric C. Polley ◽

Mostafa Fatemi ◽

...

Keyword(s):

Deep Learning ◽

Semantic Segmentation ◽

Convolutional Networks ◽

Fully Convolutional Networks

Download Full-text

Automated Video Behavior Recognition of Pigs Using Two-Stream Convolutional Networks

Sensors ◽

10.3390/s20041085 ◽

2020 ◽

Vol 20 (4) ◽

pp. 1085

Author(s):

Kaifeng Zhang ◽

Dan Li ◽

Jiayun Huang ◽

Yifei Chen

Keyword(s):

Feature Extraction ◽

Deep Learning ◽

Optical Flow ◽

Network Models ◽

Well Being ◽

Motion Information ◽

Behavior Recognition ◽

Convolutional Network ◽

Convolutional Networks ◽

Effective Manner

The detection of pig behavior helps detect abnormal conditions such as diseases and dangerous movements in a timely and effective manner, which plays an important role in ensuring the health and well-being of pigs. Monitoring pig behavior by staff is time consuming, subjective, and impractical. Therefore, there is an urgent need to implement methods for identifying pig behavior automatically. In recent years, deep learning has been gradually applied to the study of pig behavior recognition. Existing studies judge the behavior of the pig only based on the posture of the pig in a still image frame, without considering the motion information of the behavior. However, optical flow can well reflect the motion information. Thus, this study took image frames and optical flow from videos as two-stream input objects to fully extract the temporal and spatial behavioral characteristics. Two-stream convolutional network models based on deep learning were proposed, including inflated 3D convnet (I3D) and temporal segment networks (TSN) whose feature extraction network is Residual Network (ResNet) or the Inception architecture (e.g., Inception with Batch Normalization (BN-Inception), InceptionV3, InceptionV4, or InceptionResNetV2) to achieve pig behavior recognition. A standard pig video behavior dataset that included 1000 videos of feeding, lying, walking, scratching and mounting from five kinds of different behavioral actions of pigs under natural conditions was created. The dataset was used to train and test the proposed models, and a series of comparative experiments were conducted. The experimental results showed that the TSN model whose feature extraction network was ResNet101 was able to recognize pig feeding, lying, walking, scratching, and mounting behaviors with a higher average of 98.99%, and the average recognition time of each video was 0.3163 s. The TSN model (ResNet101) is superior to the other models in solving the task of pig behavior recognition.

Download Full-text

A Stacked Fully Convolutional Networks with Feature Alignment Framework for Multi-Label Land-cover Segmentation

Remote Sensing ◽

10.3390/rs11091051 ◽

2019 ◽

Vol 11 (9) ◽

pp. 1051 ◽

Cited By ~ 7

Author(s):

Guangming Wu ◽

Yimin Guo ◽

Xiaoya Song ◽

Zhiling Guo ◽

Haoran Zhang ◽

...

Keyword(s):

Deep Learning ◽

Land Cover ◽

Superior Performance ◽

Learning Methods ◽

Convolutional Networks ◽

Fully Convolutional Networks ◽

Optimal Feature ◽

Imaging Conditions ◽

Very High ◽

Feature Alignment

Applying deep-learning methods, especially fully convolutional networks (FCNs), has become a popular option for land-cover classification or segmentation in remote sensing. Compared with traditional solutions, these approaches have shown promising generalization capabilities and precision levels in various datasets of different scales, resolutions, and imaging conditions. To achieve superior performance, a lot of research has focused on constructing more complex or deeper networks. However, using an ensemble of different fully convolutional models to achieve better generalization and to prevent overfitting has long been ignored. In this research, we design four stacked fully convolutional networks (SFCNs), and a feature alignment framework for multi-label land-cover segmentation. The proposed feature alignment framework introduces an alignment loss of features extracted from basic models to balance their similarity and variety. Experiments on a very high resolution(VHR) image dataset with six categories of land-covers indicates that the proposed SFCNs can gain better performance when compared to existing deep learning methods. In the 2nd variant of SFCN, the optimal feature alignment gains increments of 4.2% (0.772 vs. 0.741), 6.8% (0.629 vs. 0.589), and 5.5% (0.727 vs. 0.689) for its f1-score, jaccard index, and kappa coefficient, respectively.

Download Full-text

A Novel Object-Based Deep Learning Framework for Semantic Segmentation of Very High-Resolution Remote Sensing Data: Comparison with Convolutional and Fully Convolutional Networks

Remote Sensing ◽

10.3390/rs11060684 ◽

2019 ◽

Vol 11 (6) ◽

pp. 684 ◽

Cited By ~ 17

Author(s):

Maria Papadomanolaki ◽

Maria Vakalopoulou ◽

Konstantinos Karantzalos

Keyword(s):

Deep Learning ◽

State Of The Art ◽

Semantic Segmentation ◽

Novel Object ◽

Convolutional Networks ◽

Learning Framework ◽

Fully Convolutional Networks ◽

Object Based ◽

Deep Networks ◽

Very High

Deep learning architectures have received much attention in recent years demonstrating state-of-the-art performance in several segmentation, classification and other computer vision tasks. Most of these deep networks are based on either convolutional or fully convolutional architectures. In this paper, we propose a novel object-based deep-learning framework for semantic segmentation in very high-resolution satellite data. In particular, we exploit object-based priors integrated into a fully convolutional neural network by incorporating an anisotropic diffusion data preprocessing step and an additional loss term during the training process. Under this constrained framework, the goal is to enforce pixels that belong to the same object to be classified at the same semantic category. We compared thoroughly the novel object-based framework with the currently dominating convolutional and fully convolutional deep networks. In particular, numerous experiments were conducted on the publicly available ISPRS WGII/4 benchmark datasets, namely Vaihingen and Potsdam, for validation and inter-comparison based on a variety of metrics. Quantitatively, experimental results indicate that, overall, the proposed object-based framework slightly outperformed the current state-of-the-art fully convolutional networks by more than 1% in terms of overall accuracy, while intersection over union results are improved for all semantic categories. Qualitatively, man-made classes with more strict geometry such as buildings were the ones that benefit most from our method, especially along object boundaries, highlighting the great potential of the developed approach.

Download Full-text

DEEP LEARNING FOR CODED TARGET DETECTION

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xliv-2-w1-2021-125-2021 ◽

2021 ◽

Vol XLIV-2/W1-2021 ◽

pp. 125-130

Author(s):

V. V. Kniaz ◽

L. Grodzitskiy ◽

V. A. Knyaz

Keyword(s):

Deep Learning ◽

Target Detection ◽

Camera Calibration ◽

Light Conditions ◽

Low Light ◽

Convolutional Network ◽

Decoding Algorithms ◽

Wide Range ◽

Optical Markers ◽

Detection And Recognition

Abstract. Coded targets are physical optical markers that can be easily identified in an image. Their detection is a critical step in the process of camera calibration. A wide range of coded targets was developed to date. The targets differ in their decoding algorithms. The main limitation of the existing methods is low robustness to new backgrounds and illumination conditions. Modern deep learning recognition-based algorithms demonstrate exciting progress in object detection performance in low-light conditions or new environments. This paper is focused on the development of a new deep convolutional network for automatic detection and recognition of the coded targets and sub-pixel estimation of their centers.

Download Full-text

Extraction and Recognition Method of Basketball Players’ Dynamic Human Actions Based on Deep Learning

Mobile Information Systems ◽

10.1155/2021/4437146 ◽

2021 ◽

Vol 2021 ◽

pp. 1-6

Author(s):

Qiulin Wang ◽

Baole Tao ◽

Fulei Han ◽

Wenting Wei

Keyword(s):

Deep Learning ◽

Action Recognition ◽

Recognition Accuracy ◽

Human Action Recognition ◽

Human Action ◽

Recognition Algorithm ◽

Basketball Players ◽

Human Actions ◽

Basketball Game ◽

Wide Range

The extraction and recognition of human actions has always been a research hotspot in the field of state recognition. It has a wide range of application prospects in many fields. In sports, it can reduce the occurrence of accidental injuries and improve the training level of basketball players. How to extract effective features from the dynamic body movements of basketball players is of great significance. In order to improve the fairness of the basketball game, realize the accurate recognition of the athletes’ movements, and simultaneously improve the level of the athletes and regulate the movements of the athletes during training, this article uses deep learning to extract and recognize the movements of the basketball players. This paper implements human action recognition algorithm based on deep learning. This method automatically extracts image features through convolution kernels, which greatly improves the efficiency compared with traditional manual feature extraction methods. This method uses the deep convolutional neural network VGG model on the TensorFlow platform to extract and recognize human actions. On the Matlab platform, the KTH and Weizmann datasets are preprocessed to obtain the input image set. Then, the preprocessed dataset is used to train the model to obtain the optimal network model and corresponding data by testing the two datasets. Finally, the two datasets are analyzed in detail, and the specific cause of each action confusion is given. Simultaneously, the recognition accuracy and average recognition accuracy rates of each action category are calculated. The experimental results show that the human action recognition algorithm based on deep learning obtains a higher recognition accuracy rate.

Download Full-text

Structured crowdsourcing enables convolutional segmentation of histology images

Bioinformatics ◽

10.1093/bioinformatics/btz083 ◽

2019 ◽

Vol 35 (18) ◽

pp. 3461-3467 ◽

Cited By ~ 12

Author(s):

Mohamed Amgad ◽

Habiba Elfandy ◽

Hagar Hussein ◽

Lamees A Atteya ◽

Mai A T Elsebaie ◽

...

Keyword(s):

Breast Cancer ◽

Deep Learning ◽

Classification Accuracy ◽

Supplementary Information ◽

Supplementary Data ◽

Digital Slide ◽

Convolutional Networks ◽

Fully Convolutional Networks ◽

Annotation Data ◽

Whole Slide Images

Abstract Motivation While deep-learning algorithms have demonstrated outstanding performance in semantic image segmentation tasks, large annotation datasets are needed to create accurate models. Annotation of histology images is challenging due to the effort and experience required to carefully delineate tissue structures, and difficulties related to sharing and markup of whole-slide images. Results We recruited 25 participants, ranging in experience from senior pathologists to medical students, to delineate tissue regions in 151 breast cancer slides using the Digital Slide Archive. Inter-participant discordance was systematically evaluated, revealing low discordance for tumor and stroma, and higher discordance for more subjectively defined or rare tissue classes. Feedback provided by senior participants enabled the generation and curation of 20 000+ annotated tissue regions. Fully convolutional networks trained using these annotations were highly accurate (mean AUC=0.945), and the scale of annotation data provided notable improvements in image classification accuracy. Availability and Implementation Dataset is freely available at: https://goo.gl/cNM4EL. Supplementary information Supplementary data are available at Bioinformatics online.

Download Full-text