Semantic Segmentation and Contextual Information Based Image Scene Interpretation: A Review

2020 ◽  
Author(s):  
Ajay Koul ◽  
Apeksha Koul
Keyword(s):  
Contextual Information ◽  
Semantic Segmentation ◽  
Scene Interpretation

scholarly journals Knowledge and Geo-Object Based Graph Convolutional Network for Remote Sensing Semantic Segmentation

Sensors ◽  
2021 ◽  
Vol 21 (11) ◽  
pp. 3848
Author(s):  
Wei Cui ◽  
Meng Yao ◽  
Yuanjie Hao ◽  
Ziwei Wang ◽  
Xin He ◽  
...  
Keyword(s):  
Remote Sensing ◽  
Prior Knowledge ◽  
Contextual Information ◽  
Information Aggregation ◽  
Semantic Segmentation ◽  
Spatial Correlations ◽  
Convolutional Network ◽  
Object Based ◽  
Graph Neural Networks ◽  
Salt And Pepper

Pixel-based semantic segmentation models fail to effectively express geographic objects and their topological relationships. Therefore, in semantic segmentation of remote sensing images, these models fail to avoid salt-and-pepper effects and cannot achieve high accuracy either. To solve these problems, object-based models such as graph neural networks (GNNs) are considered. However, traditional GNNs directly use similarity or spatial correlations between nodes to aggregate nodes’ information, which rely too much on the contextual information of the sample. The contextual information of the sample is often distorted, which results in a reduction in the node classification accuracy. To solve this problem, a knowledge and geo-object-based graph convolutional network (KGGCN) is proposed. The KGGCN uses superpixel blocks as nodes of the graph network and combines prior knowledge with spatial correlations during information aggregation. By incorporating the prior knowledge obtained from all samples of the study area, the receptive field of the node is extended from its sample context to the study area. Thus, the distortion of the sample context is overcome effectively. Experiments demonstrate that our model is improved by 3.7% compared with the baseline model named Cluster GCN and 4.1% compared with U-Net.

scholarly journals EFFICIENT SEMANTIC SEGMENTATION OF MAN-MADE SCENES USING FULLY-CONNECTED CONDITIONAL RANDOM FIELD

2016 ◽  
Vol XLI-B3 ◽  
pp. 633-640
Author(s):  
Weihao Li ◽  
Michael Ying Yang
Keyword(s):  
Random Field ◽  
State Of The Art ◽  
Conditional Random Field ◽  
Contextual Information ◽  
Mean Field ◽  
Semantic Segmentation ◽  
Gaussian Kernels ◽  
Previous State ◽  
Fully Connected ◽  
Better Than

In this paper we explore semantic segmentation of man-made scenes using fully connected conditional random field (CRF). Images of man-made scenes display strong contextual dependencies in the spatial structures. Fully connected CRFs can model long-range connections within the image of man-made scenes and make use of contextual information of scene structures. The pairwise edge potentials of fully connected CRF models are defined by a linear combination of Gaussian kernels. Using filter-based mean field algorithm, the inference is very efficient. Our experimental results demonstrate that fully connected CRF performs better than previous state-of-the-art approaches on both eTRIMS dataset and LabelMeFacade dataset.

scholarly journals A Hierarchical Feature Extraction Network for Fast Scene Segmentation

Sensors ◽  
2021 ◽  
Vol 21 (22) ◽  
pp. 7730
Author(s):  
◽  
Keyword(s):  
Feature Extraction ◽  
Spatial Information ◽  
Contextual Information ◽  
Semantic Segmentation ◽  
Superior Performance ◽  
Scene Segmentation ◽  
Research Topics ◽  
Time Performance ◽  
Segmentation Accuracy ◽  
Active Research

Semantic segmentation is one of the most active research topics in computer vision with the goal to assign dense semantic labels for all pixels in a given image. In this paper, we introduce HFEN (Hierarchical Feature Extraction Network), a lightweight network to reach a balance between inference speed and segmentation accuracy. Our architecture is based on an encoder-decoder framework. The input images are down-sampled through an efficient encoder to extract multi-layer features. Then the extracted features are fused via a decoder, where the global contextual information and spatial information are aggregated for final segmentations with real-time performance. Extensive experiments have been conducted on two standard benchmarks, Cityscapes and Camvid, where our network achieved superior performance on NVIDIA 2080Ti.

scholarly journals Attentively Learning Edge Distributions for Semantic Segmentation of Remote Sensing Imagery

2021 ◽  
Vol 14 (1) ◽  
pp. 102
Author(s):  
Xin Li ◽  
Tao Li ◽  
Ziqi Chen ◽  
Kaiwen Zhang ◽  
Runliang Xia
Keyword(s):  
Remote Sensing ◽  
Contextual Information ◽  
Semantic Segmentation ◽  
Matrix Analysis ◽  
Natural Image ◽  
Feature Maps ◽  
Remote Sensing Imagery ◽  
Edge Distribution ◽  
Non Local ◽  
Ablation Study

Semantic segmentation has been a fundamental task in interpreting remote sensing imagery (RSI) for various downstream applications. Due to the high intra-class variants and inter-class similarities, inflexibly transferring natural image-specific networks to RSI is inadvisable. To enhance the distinguishability of learnt representations, attention modules were developed and applied to RSI, resulting in satisfactory improvements. However, these designs capture contextual information by equally handling all the pixels regardless of whether they around edges. Therefore, blurry boundaries are generated, rising high uncertainties in classifying vast adjacent pixels. Hereby, we propose an edge distribution attention module (EDA) to highlight the edge distributions of leant feature maps in a self-attentive fashion. In this module, we first formulate and model column-wise and row-wise edge attention maps based on covariance matrix analysis. Furthermore, a hybrid attention module (HAM) that emphasizes the edge distributions and position-wise dependencies is devised combing with non-local block. Consequently, a conceptually end-to-end neural network, termed as EDENet, is proposed to integrate HAM hierarchically for the detailed strengthening of multi-level representations. EDENet implicitly learns representative and discriminative features, providing available and reasonable cues for dense prediction. The experimental results evaluated on ISPRS Vaihingen, Potsdam and DeepGlobe datasets show the efficacy and superiority to the state-of-the-art methods on overall accuracy (OA) and mean intersection over union (mIoU). In addition, the ablation study further validates the effects of EDA.

scholarly journals Feature fusion network based on strip pooling

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Gaihua Wang ◽  
Qianyu Zhai
Keyword(s):  
Feature Fusion ◽  
Contextual Information ◽  
Region Of Interest ◽  
Semantic Segmentation ◽  
Attention Mechanism ◽  
Affinity Matrix ◽  
The Public ◽  
Key Factor ◽  
Public Datasets ◽  
The Relationship

AbstractContextual information is a key factor affecting semantic segmentation. Recently, many methods have tried to use the self-attention mechanism to capture more contextual information. However, these methods with self-attention mechanism need a huge computation. In order to solve this problem, a novel self-attention network, called FFANet, is designed to efficiently capture contextual information, which reduces the amount of calculation through strip pooling and linear layers. It proposes the feature fusion (FF) module to calculate the affinity matrix. The affinity matrix can capture the relationship between pixels. Then we multiply the affinity matrix with the feature map, which can selectively increase the weight of the region of interest. Extensive experiments on the public datasets (PASCAL VOC2012, CityScapes) and remote sensing dataset (DLRSD) have been conducted and achieved Mean Iou score 74.5%, 70.3%, and 63.9% respectively. Compared with the current typical algorithms, the proposed method has achieved excellent performance.

scholarly journals Multi-Scale Context Aggregation for Semantic Segmentation of Remote Sensing Images

2020 ◽  
Vol 12 (4) ◽  
pp. 701 ◽  
Author(s):  
Jing Zhang ◽  
Shaofu Lin ◽  
Lei Ding ◽  
Lorenzo Bruzzone
Keyword(s):  
Remote Sensing ◽  
High Resolution ◽  
Semantic Information ◽  
Contextual Information ◽  
Semantic Segmentation ◽  
Spatial Correlations ◽  
Remote Sensing Images ◽  
Localization Accuracy ◽  
Spatial Pooling ◽  
Spatial Size

The semantic segmentation of remote sensing images (RSIs) is important in a variety of applications. Conventional encoder-decoder-based convolutional neural networks (CNNs) use cascade pooling operations to aggregate the semantic information, which results in a loss of localization accuracy and in the preservation of spatial details. To overcome these limitations, we introduce the use of the high-resolution network (HRNet) to produce high-resolution features without the decoding stage. Moreover, we enhance the low-to-high features extracted from different branches separately to strengthen the embedding of scale-related contextual information. The low-resolution features contain more semantic information and have a small spatial size; thus, they are utilized to model the long-term spatial correlations. The high-resolution branches are enhanced by introducing an adaptive spatial pooling (ASP) module to aggregate more local contexts. By combining these context aggregation designs across different levels, the resulting architecture is capable of exploiting spatial context at both global and local levels. The experimental results obtained on two RSI datasets show that our approach significantly improves the accuracy with respect to the commonly used CNNs and achieves state-of-the-art performance.

scholarly journals UnetDVH-Linear: Linear Feature Segmentation by Dilated Convolution with Vertical and Horizontal Kernels

Sensors ◽  
2020 ◽  
Vol 20 (20) ◽  
pp. 5759 ◽  
Author(s):  
Jiacai Liao ◽  
Libo Cao ◽  
Wei Li ◽  
Xiaole Luo ◽  
Xiexing Feng
Keyword(s):  
Feature Extraction ◽  
Contextual Information ◽  
Semantic Segmentation ◽  
Complete Information ◽  
Lane Detection ◽  
Aerial Images ◽  
Linear Feature ◽  
Dilated Convolution ◽  
Linear Feature Extraction ◽  
The Neural Network

Linear feature extraction is crucial for special objects in semantic segmentation networks, such as slot marking and lanes. The objects with linear characteristics have global contextual information dependency. It is very difficult to capture the complete information of these objects in semantic segmentation tasks. To improve the linear feature extraction ability of the semantic segmentation network, we propose introducing the dilated convolution with vertical and horizontal kernels (DVH) into the task of feature extraction in semantic segmentation networks. Meanwhile, we figure out the outcome if we put the different vertical and horizontal kernels on different places in the semantic segmentation networks. Our networks are trained on the basis of the SS dataset, the TuSimple lane dataset and the Massachusetts Roads dataset. These datasets consist of slot marking, lanes, and road images. The research results show that our method improves the accuracy of the slot marking segmentation of the SS dataset by 2%. Compared with other state-of-the-art methods, our UnetDVH-Linear (v1) obtains better accuracy on the TuSimple Benchmark Lane Detection Challenge with a value of 97.53%. To prove the generalization of our models, road segmentation experiments were performed on aerial images. Without data argumentation, the segmentation accuracy of our model on the Massachusetts roads dataset is 95.3%. Moreover, our models perform better than other models when training with the same loss function and experimental settings. The experiment result shows that the dilated convolution with vertical and horizontal kernels will enhance the neural network on linear feature extraction.

Quantify pixel-level detection of dam surface crack using deep learning

2022 ◽  
Author(s):  
Bo Chen ◽  
Hua Zhang ◽  
Yonglong Li ◽  
Shuang Wang ◽  
Huaifang Zhou ◽  
...  
Keyword(s):  
Deep Learning ◽  
Surface Crack ◽  
Crack Detection ◽  
State Of The Art ◽  
Contextual Information ◽  
Semantic Segmentation ◽  
Quantitative Information ◽  
Cross Entropy ◽  
Detection Methods ◽  
Water Conservancy

Abstract An increasing number of detection methods based on computer vision are applied to detect cracks in water conservancy infrastructure. However, most studies directly use existing feature extraction networks to extract cracks information, which are proposed for open-source datasets. As the cracks distribution and pixel features are different from these data, the extracted cracks information is incomplete. In this paper, a deep learning-based network for dam surface crack detection is proposed, which mainly addresses the semantic segmentation of cracks on the dam surface. Particularly, we design a shallow encoding network to extract features of crack images based on the statistical analysis of cracks. Further, to enhance the relevance of contextual information, we introduce an attention module into the decoding network. During the training, we use the sum of Cross-Entropy and Dice Loss as the loss function to overcome data imbalance. The quantitative information of cracks is extracted by the imaging principle after using morphological algorithms to extract the morphological features of the predicted result. We built a manual annotation dataset containing 1577 images to verify the effectiveness of the proposed method. This method achieves the state-of-the-art performance on our dataset. Specifically, the precision, recall, IoU, F1_measure, and accuracy achieve 90.81%, 81.54%, 75.23%, 85.93%, 99.76%, respectively. And the quantization error of cracks is less than 4%.

Sign in / Sign up

Export Citation Format

Share Document