Identification of maceral groups in Chinese bituminous coals based on semantic segmentation models

This work proposes and validates a simple but effective approach to train dense semantic segmentation models from sparsely labeled data. Data and labeling collection is most costly task of semantic segmentation. Our approach needs only a few pixels per image reducing the human interaction required.

Download Full-text

A Comparative Study of Semantic Segmentation Models for Identification of Grape with Different Varieties

Agriculture ◽

10.3390/agriculture11100997 ◽

2021 ◽

Vol 11 (10) ◽

pp. 997

Author(s):

Yun Peng ◽

Aichen Wang ◽

Jizhan Liu ◽

Muhammad Faheem

Keyword(s):

Image Enhancement ◽

Precision Agriculture ◽

State Of The Art ◽

Semantic Segmentation ◽

Histogram Equalization ◽

Performance Difference ◽

Convolutional Network ◽

Enhancement Method ◽

The Impact ◽

Segmentation Models

Accurate fruit segmentation in images is the prerequisite and key step for precision agriculture. In this article, aiming at the segmentation of grape cluster with different varieties, 3 state-of-the-art semantic segmentation networks, i.e., Fully Convolutional Network (FCN), U-Net, and DeepLabv3+ applied on six different datasets were studied. We investigated: (1) the segmentation performance difference of the 3 studied networks; (2) The impact of different input representations on segmentation performance; (3) The effect of image enhancement method to improve the poor illumination of images and further improve the segmentation performance; (4) The impact of the distance between grape clusters and camera on segmentation performance. The experiment results show that compared with FCN and U-Net the DeepLabv3+ combined with transfer learning is more suitable for the task with an intersection over union (IoU) of 84.26%. Five different input representations, namely RGB, HSV, L*a*b, HHH, and YCrCb obtained different IoU, ranging from 81.5% to 88.44%. Among them, the L*a*b got the highest IoU. Besides, the adopted Histogram Equalization (HE) image enhancement method could improve the model’s robustness against poor illumination conditions. Through the HE preprocessing, the IoU of the enhanced dataset increased by 3.88%, from 84.26% to 88.14%. The distance between the target and camera also affects the segmentation performance, no matter in which dataset, the closer the distance, the better the segmentation performance was. In a word, the conclusion of this research provides some meaningful suggestions for the study of grape or other fruit segmentation.

Download Full-text

DAM: Hierarchical Adaptive Feature Selection Using Convolution Encoder Decoder Network for Strawberry Segmentation

Frontiers in Plant Science ◽

10.3389/fpls.2021.591333 ◽

2021 ◽

Vol 12 ◽

Author(s):

Talha Ilyas ◽

Muhammad Umraiz ◽

Abbas Khan ◽

Hyongsuk Kim

Keyword(s):

Feature Selection ◽

Feature Fusion ◽

Semantic Segmentation ◽

Dynamic Feature ◽

Feature Maps ◽

Adaptive Feature Selection ◽

Convolutional Encoder ◽

Time Segmentation ◽

Flow Of Information ◽

Segmentation Models

Autonomous harvesters can be used for the timely cultivation of high-value crops such as strawberries, where the robots have the capability to identify ripe and unripe crops. However, the real-time segmentation of strawberries in an unbridled farming environment is a challenging task due to fruit occlusion by multiple trusses, stems, and leaves. In this work, we propose a possible solution by constructing a dynamic feature selection mechanism for convolutional neural networks (CNN). The proposed building block namely a dense attention module (DAM) controls the flow of information between the convolutional encoder and decoder. DAM enables hierarchical adaptive feature fusion by exploiting both inter-channel and intra-channel relationships and can be easily integrated into any existing CNN to obtain category-specific feature maps. We validate our attention module through extensive ablation experiments. In addition, a dataset is collected from different strawberry farms and divided into four classes corresponding to different maturity levels of fruits and one is devoted to background. Quantitative analysis of the proposed method showed a 4.1% and 2.32% increase in mean intersection over union, over existing state-of-the-art semantic segmentation models and other attention modules respectively, while simultaneously retaining a processing speed of 53 frames per second.

Download Full-text

Automated Processing of Remote Sensing Imagery Using Deep Semantic Segmentation: A Building Footprint Extraction Case

ISPRS International Journal of Geo-Information ◽

10.3390/ijgi9080486 ◽

2020 ◽

Vol 9 (8) ◽

pp. 486 ◽

Cited By ~ 2

Author(s):

Aleksandar Milosavljević

Keyword(s):

Remote Sensing ◽

Semantic Segmentation ◽

Aerial Image ◽

Empirical Examination ◽

Remote Sensing Imagery ◽

Automated Processing ◽

Building Footprint ◽

Segmentation Task ◽

Segmentation Models ◽

Selection Of

The proliferation of high-resolution remote sensing sensors and platforms imposes the need for effective analyses and automated processing of high volumes of aerial imagery. The recent advance of artificial intelligence (AI) in the form of deep learning (DL) and convolutional neural networks (CNN) showed remarkable results in several image-related tasks, and naturally, gain the focus of the remote sensing community. In this paper, we focus on specifying the processing pipeline that relies on existing state-of-the-art DL segmentation models to automate building footprint extraction. The proposed pipeline is organized in three stages: image preparation, model implementation and training, and predictions fusion. For the first and third stages, we introduced several techniques that leverage remote sensing imagery specifics, while for the selection of the segmentation model, we relied on empirical examination. In the paper, we presented and discussed several experiments that we conducted on Inria Aerial Image Labeling Dataset. Our findings confirmed that automatic processing of remote sensing imagery using DL semantic segmentation is both possible and can provide applicable results. The proposed pipeline can be potentially transferred to any other remote sensing imagery segmentation task if the corresponding dataset is available.

Download Full-text

Unsupervised Domain Adaptation in Semantic Segmentation: A Review

Technologies ◽

10.3390/technologies8020035 ◽

2020 ◽

Vol 8 (2) ◽

pp. 35

Author(s):

Marco Toldo ◽

Andrea Maracani ◽

Umberto Michieli ◽

Pietro Zanuttigh

Keyword(s):

Ad Hoc ◽

Domain Adaptation ◽

Semantic Segmentation ◽

Autonomous Driving ◽

Input Image ◽

Comprehensive Overview ◽

Open Problems ◽

Unsupervised Domain Adaptation ◽

Driving Scenario ◽

Segmentation Models

The aim of this paper is to give an overview of the recent advancements in the Unsupervised Domain Adaptation (UDA) of deep networks for semantic segmentation. This task is attracting a wide interest since semantic segmentation models require a huge amount of labeled data and the lack of data fitting specific requirements is the main limitation in the deployment of these techniques. This field has been recently explored and has rapidly grown with a large number of ad-hoc approaches. This motivates us to build a comprehensive overview of the proposed methodologies and to provide a clear categorization. In this paper, we start by introducing the problem, its formulation and the various scenarios that can be considered. Then, we introduce the different levels at which adaptation strategies may be applied: namely, at the input (image) level, at the internal features representation and at the output level. Furthermore, we present a detailed overview of the literature in the field, dividing previous methods based on the following (non mutually exclusive) categories: adversarial learning, generative-based, analysis of the classifier discrepancies, self-teaching, entropy minimization, curriculum learning and multi-task learning. Novel research directions are also briefly introduced to give a hint of interesting open problems in the field. Finally, a comparison of the performance of the various methods in the widely used autonomous driving scenario is presented.

Download Full-text

Hybrid Deep Learning Models with Sparse Enhancement Technique for Detection of Newly Grown Tree Leaves

Sensors ◽

10.3390/s21062077 ◽

2021 ◽

Vol 21 (6) ◽

pp. 2077 ◽

Cited By ~ 1

Author(s):

Shih-Yu Chen ◽

Chinsu Lin ◽

Guan-Jie Li ◽

Yu-Chun Hsu ◽

Keng-Hao Liu

Keyword(s):

Image Segmentation ◽

Deep Learning ◽

Life Cycle ◽

Climate Changes ◽

Imbalanced Data ◽

Semantic Segmentation ◽

Effect Of Temperature ◽

Tree Leaves ◽

The Difference ◽

Segmentation Models

The life cycle of leaves, from sprout to senescence, is the phenomenon of regular changes such as budding, branching, leaf spreading, flowering, fruiting, leaf fall, and dormancy due to seasonal climate changes. It is the effect of temperature and moisture in the life cycle on physiological changes, so the detection of newly grown leaves (NGL) is helpful for the estimation of tree growth and even climate change. This study focused on the detection of NGL based on deep learning convolutional neural network (CNN) models with sparse enhancement (SE). As the NGL areas found in forest images have similar sparse characteristics, we used a sparse image to enhance the signal of the NGL. The difference between the NGL and the background could be further improved. We then proposed hybrid CNN models that combined U-net and SegNet features to perform image segmentation. As the NGL in the image were relatively small and tiny targets, in terms of data characteristics, they also belonged to the problem of imbalanced data. Therefore, this paper further proposed 3-Layer SegNet, 3-Layer U-SegNet, 2-Layer U-SegNet, and 2-Layer Conv-U-SegNet architectures to reduce the pooling degree of traditional semantic segmentation models, and used a loss function to increase the weight of the NGL. According to the experimental results, our proposed algorithms were indeed helpful for the image segmentation of NGL and could achieve better kappa results by 0.743.

Download Full-text

WTS: A Weakly towards Strongly Supervised Learning Framework for Remote Sensing Land Cover Classification Using Segmentation Models

Remote Sensing ◽

10.3390/rs13030394 ◽

2021 ◽

Vol 13 (3) ◽

pp. 394

Author(s):

Wei Zhang ◽

Ping Tang ◽

Thomas Corpetti ◽

Lijun Zhao

Keyword(s):

Remote Sensing ◽

Land Cover ◽

Supervised Learning ◽

Region Growing ◽

Semantic Segmentation ◽

Land Cover Classification ◽

Training Set ◽

Seeded Region Growing ◽

Class Labels ◽

Segmentation Models

Land cover classification is one of the most fundamental tasks in the field of remote sensing. In recent years, fully supervised fully convolutional network (FCN)-based semantic segmentation models have achieved state-of-the-art performance in the semantic segmentation task. However, creating pixel-level annotations is prohibitively expensive and laborious, especially when dealing with remote sensing images. Weakly supervised learning methods from weakly labeled annotations can overcome this difficulty to some extent and achieve impressive segmentation results, but results are limited in accuracy. Inspired by point supervision and the traditional segmentation method of seeded region growing (SRG) algorithm, a weakly towards strongly (WTS) supervised learning framework is proposed in this study for remote sensing land cover classification to handle the absence of well-labeled and abundant pixel-level annotations when using segmentation models. In this framework, only several points with true class labels are required as the training set, which are much less expensive to acquire compared with pixel-level annotations through field survey or visual interpretation using high-resolution images. Firstly, they are used to train a Support Vector Machine (SVM) classifier. Once fully trained, the SVM is used to generate the initial seeded pixel-level training set, in which only the pixels with high confidence are assigned with class labels whereas others are unlabeled. They are used to weakly train the segmentation model. Then, the seeded region growing module and fully connected Conditional Random Fields (CRFs) are used to iteratively update the seeded pixel-level training set for progressively increasing pixel-level supervision of the segmentation model. Sentinel-2 remote sensing images are used to validate the proposed framework, and SVM is selected for comparison. In addition, FROM-GLC10 global land cover map is used as training reference to directly train the segmentation model. Experimental results show that the proposed framework outperforms other methods and can be highly recommended for land cover classification tasks when the pixel-level labeled datasets are insufficient by using segmentation models.

Download Full-text

Semantic Segmentation of Cabbage in the South Korea Highlands with Images by Unmanned Aerial Vehicles

Applied Sciences ◽

10.3390/app11104493 ◽

2021 ◽

Vol 11 (10) ◽

pp. 4493

Author(s):

Yongwon Jo ◽

Soobin Lee ◽

Youngjae Lee ◽

Hyungu Kahng ◽

Seonghun Park ◽

...

Keyword(s):

Deep Learning ◽

South Korea ◽

Unmanned Aerial Vehicles ◽

Rapid Development ◽

Significant Proportion ◽

Semantic Segmentation ◽

Training Data ◽

Field Surveys ◽

Aerial Vehicles ◽

Segmentation Models

Identifying agricultural fields that grow cabbage in the highlands of South Korea is critical for accurate crop yield estimation. Only grown for a limited time during the summer, highland cabbage accounts for a significant proportion of South Korea’s annual cabbage production. Thus, it has a profound effect on the formation of cabbage prices. Traditionally, labor-extensive and time-consuming field surveys are manually carried out to derive agricultural field maps of the highlands. Recently, high-resolution overhead images of the highlands have become readily available with the rapid development of unmanned aerial vehicles (UAV) and remote sensing technology. In addition, deep learning-based semantic segmentation models have quickly advanced by recent improvements in algorithms and computational resources. In this study, we propose a semantic segmentation framework based on state-of-the-art deep learning techniques to automate the process of identifying cabbage cultivation fields. We operated UAVs and collected 2010 multispectral images under different spatiotemporal conditions to measure how well semantic segmentation models generalize. Next, we manually labeled these images at a pixel-level to obtain ground truth labels for training. Our results demonstrate that our framework performs well in detecting cabbage fields not only in areas included in the training data but also in unseen areas not included in the training data. Moreover, we analyzed the effects of infrared wavelengths on the performance of identifying cabbage fields. Based on the results of our framework, we expect agricultural officials to reduce time and manpower when identifying information about highlands cabbage fields by replacing field surveys.

Download Full-text