scholarly journals Multi-scale Adaptive Feature Fusion Network for Semantic Segmentation in Remote Sensing Images

2020 ◽  
Vol 12 (5) ◽  
pp. 872 ◽  
Author(s):  
Ronghua Shang ◽  
Jiyu Zhang ◽  
Licheng Jiao ◽  
Yangyang Li ◽  
Naresh Marturi ◽  
...  
Keyword(s):  
Remote Sensing ◽  
Multiple Scales ◽  
Feature Fusion ◽  
Semantic Segmentation ◽  
Semantic Features ◽  
Remote Sensing Images ◽  
Global Features ◽  
Global Average ◽  
Multi Scale ◽  
Context Extraction

Semantic segmentation of high-resolution remote sensing images is highly challenging due to the presence of a complicated background, irregular target shapes, and similarities in the appearance of multiple target categories. Most of the existing segmentation methods that rely only on simple fusion of the extracted multi-scale features often fail to provide satisfactory results when there is a large difference in the target sizes. Handling this problem through multi-scale context extraction and efficient fusion of multi-scale features, in this paper we present an end-to-end multi-scale adaptive feature fusion network (MANet) for semantic segmentation in remote sensing images. It is a coding and decoding structure that includes a multi-scale context extraction module (MCM) and an adaptive fusion module (AFM). The MCM employs two layers of atrous convolutions with different dilatation rates and global average pooling to extract context information at multiple scales in parallel. MANet embeds the channel attention mechanism to fuse semantic features. The high- and low-level semantic information are concatenated to generate global features via global average pooling. These global features are used as channel weights to acquire adaptive weight information of each channel by the fully connected layer. To accomplish an efficient fusion, these tuned weights are applied to the fused features. Performance of the proposed method has been evaluated by comparing it with six other state-of-the-art networks: fully convolutional networks (FCN), U-net, UZ1, Light-weight RefineNet, DeepLabv3+, and APPD. Experiments performed using the publicly available Potsdam and Vaihingen datasets show that the proposed MANet significantly outperforms the other existing networks, with overall accuracy reaching 89.4% and 88.2%, respectively and with average of F1 reaching 90.4% and 86.7% respectively.

scholarly journals A DETECTION METHOD OF ARTIFICIAL AREA FROM HIGH RESOLUTION REMOTE SENSING IMAGES BASED ON MULTI SCALE AND MULTI FEATURE FUSION

2017 ◽  
Vol XLII-1/W1 ◽  
pp. 387-392
Author(s):  
P. Li ◽  
X. Hu ◽  
Y. Hu ◽  
Y. Ding ◽  
L. Wang ◽  
...  
Keyword(s):  
Remote Sensing ◽  
High Resolution ◽  
Multiple Scales ◽  
Feature Fusion ◽  
Visual Saliency ◽  
Geometric Feature ◽  
Remote Sensing Images ◽  
Multi Scale ◽  
Urban City ◽  
Intensity Map

In order to solve the problem of automatic detection of artificial objects in high resolution remote sensing images, a method for detection of artificial areas in high resolution remote sensing images based on multi-scale and multi feature fusion is proposed. Firstly, the geometric features such as corner, straight line and right angle are extracted from the original resolution, and the pseudo corner points, pseudo linear features and pseudo orthogonal angles are filtered out by the self-constraint and mutual restraint between them. Then the radiation intensity map of the image with high geometric characteristics is obtained by the linear inverse distance weighted method. Secondly, the original image is reduced to multiple scales and the visual saliency image of each scale is obtained by adaptive weighting of the orthogonal saliency, the local brightness and contrast which are calculated at the corresponding scale. Then the final visual saliency image is obtained by fusing all scales’ visual saliency images. Thirdly, the visual saliency images of artificial areas based on multi scales and multi features are obtained by fusing the geometric feature energy intensity map and visual saliency image obtained in previous decision level. Finally, the artificial areas can be segmented based on the method called OTSU. Experiments show that the method in this paper not only can detect large artificial areas such as urban city, residential district, but also detect the single family house in the countryside correctly. The detection rate of artificial areas reached 92 %.

scholarly journals DFFAN: Dual Function Feature Aggregation Network for Semantic Segmentation of Land Cover

2021 ◽  
Vol 10 (3) ◽  
pp. 125
Author(s):  
Junqing Huang ◽  
Liguo Weng ◽  
Bingyu Chen ◽  
Min Xia
Keyword(s):  
Remote Sensing ◽  
Land Cover ◽  
Spatial Information ◽  
Feature Fusion ◽  
Semantic Segmentation ◽  
Dual Function ◽  
Context Information ◽  
Remote Sensing Images ◽  
Feature Aggregation ◽  
Image Context

Analyzing land cover using remote sensing images has broad prospects, the precise segmentation of land cover is the key to the application of this technology. Nowadays, the Convolution Neural Network (CNN) is widely used in many image semantic segmentation tasks. However, existing CNN models often exhibit poor generalization ability and low segmentation accuracy when dealing with land cover segmentation tasks. To solve this problem, this paper proposes Dual Function Feature Aggregation Network (DFFAN). This method combines image context information, gathers image spatial information, and extracts and fuses features. DFFAN uses residual neural networks as backbone to obtain different dimensional feature information of remote sensing images through multiple downsamplings. This work designs Affinity Matrix Module (AMM) to obtain the context of each feature map and proposes Boundary Feature Fusion Module (BFF) to fuse the context information and spatial information of an image to determine the location distribution of each image’s category. Compared with existing methods, the proposed method is significantly improved in accuracy. Its mean intersection over union (MIoU) on the LandCover dataset reaches 84.81%.

scholarly journals MSST-Net: A Multi-Scale Adaptive Network for Building Extraction from Remote Sensing Images Based on Swin Transformer

2021 ◽  
Vol 13 (23) ◽  
pp. 4743
Author(s):  
Wei Yuan ◽  
Wenbo Xu
Keyword(s):  
Neural Network ◽  
Remote Sensing ◽  
Convolutional Neural Network ◽  
Network Model ◽  
Remote Sensing Images ◽  
Feature Maps ◽  
Global Features ◽  
Adaptive Network ◽  
Data Set ◽  
Multi Scale

The segmentation of remote sensing images by deep learning technology is the main method for remote sensing image interpretation. However, the segmentation model based on a convolutional neural network cannot capture the global features very well. A transformer, whose self-attention mechanism can supply each pixel with a global feature, makes up for the deficiency of the convolutional neural network. Therefore, a multi-scale adaptive segmentation network model (MSST-Net) based on a Swin Transformer is proposed in this paper. Firstly, a Swin Transformer is used as the backbone to encode the input image. Then, the feature maps of different levels are decoded separately. Thirdly, the convolution is used for fusion, so that the network can automatically learn the weight of the decoding results of each level. Finally, we adjust the channels to obtain the final prediction map by using the convolution with a kernel of 1 × 1. By comparing this with other segmentation network models on a WHU building data set, the evaluation metrics, mIoU, F1-score and accuracy are all improved. The network model proposed in this paper is a multi-scale adaptive network model that pays more attention to the global features for remote sensing segmentation.

scholarly journals Cascaded Residual Attention Enhanced Road Extraction from Remote Sensing Images

2021 ◽  
Vol 11 (1) ◽  
pp. 9
Author(s):  
Shengfu Li ◽  
Cheng Liao ◽  
Yulin Ding ◽  
Han Hu ◽  
Yang Jia ◽  
...  
Keyword(s):  
Remote Sensing ◽  
Spatial Information ◽  
Semantic Segmentation ◽  
Road Extraction ◽  
Remote Sensing Images ◽  
Long Distance ◽  
Features Fusion ◽  
Multi Scale ◽  
Boundary Recognition ◽  
Benchmark Datasets

Efficient and accurate road extraction from remote sensing imagery is important for applications related to navigation and Geographic Information System updating. Existing data-driven methods based on semantic segmentation recognize roads from images pixel by pixel, which generally uses only local spatial information and causes issues of discontinuous extraction and jagged boundary recognition. To address these problems, we propose a cascaded attention-enhanced architecture to extract boundary-refined roads from remote sensing images. Our proposed architecture uses spatial attention residual blocks on multi-scale features to capture long-distance relations and introduce channel attention layers to optimize the multi-scale features fusion. Furthermore, a lightweight encoder-decoder network is connected to adaptively optimize the boundaries of the extracted roads. Our experiments showed that the proposed method outperformed existing methods and achieved state-of-the-art results on the Massachusetts dataset. In addition, our method achieved competitive results on more recent benchmark datasets, e.g., the DeepGlobe and the Huawei Cloud road extraction challenge.

scholarly journals Multi-Scale DenseNets-Based Aircraft Detection from Remote Sensing Images

Sensors ◽  
2019 ◽  
Vol 19 (23) ◽  
pp. 5270
Author(s):  
Yantian Wang ◽  
Haifeng Li ◽  
Peng Jia ◽  
Guo Zhang ◽  
Taoyang Wang ◽  
...  
Keyword(s):  
Remote Sensing ◽  
Detection Methods ◽  
Computational Time ◽  
Semantic Features ◽  
Remote Sensing Images ◽  
Bottom Level ◽  
Multi Scale ◽  
Noticeable Improvement ◽  
Small Targets ◽  
Aircraft Detection

Deep learning-based aircraft detection methods have been increasingly implemented in recent years. However, due to the multi-resolution imaging modes, aircrafts in different images show very wide diversity on size, view and other visual features, which brings great challenges to detection. Although standard deep convolution neural networks (DCNN) can extract rich semantic features, they destroy the bottom-level location information. The features of small targets may also be submerged by redundant top-level features, resulting in poor detection. To address these problems, we proposed a compact multi-scale dense convolutional neural network (MS-DenseNet) for aircraft detection in remote sensing images. Herein, DenseNet was utilized for feature extraction, which enhances the propagation and reuse of the bottom-level high-resolution features. Subsequently, we combined feature pyramid network (FPN) with DenseNet to form a MS-DenseNet for learning multi-scale features, especially features of small objects. Finally, by compressing some of the unnecessary convolution layers of each dense block, we designed three new compact architectures: MS-DenseNet-41, MS-DenseNet-65, and MS-DenseNet-77. Comparative experiments showed that the compact MS-DenseNet-65 obtained a noticeable improvement in detecting small aircrafts and achieved state-of-the-art performance with a recall of 94% and an F1-score of 92.7% and cost less computational time. Furthermore, the experimental results on robustness of UCAS-AOD and RSOD datasets also indicate the good transferability of our method.

Road Extraction from GF-1 Remote Sensing Images Based on Dilated Convolution Residual Network with Multi-Scale Feature Fusion

2021 ◽  
Vol 58 (2) ◽  
pp. 0228001
Author(s):  
马天浩 Ma Tianhao ◽  
谭海 Tan Hai ◽  
李天琪 Li Tianqi ◽  
吴雅男 Wu Yanan ◽  
刘祺 Liu Qi
Keyword(s):  
Remote Sensing ◽  
Feature Fusion ◽  
Road Extraction ◽  
Remote Sensing Images ◽  
Residual Network ◽  
Scale Feature ◽  
Multi Scale ◽  
Dilated Convolution

scholarly journals A Multi-Scale Water Extraction Convolutional Neural Network (MWEN) Method for GaoFen-1 Remote Sensing Images

2020 ◽  
Vol 9 (4) ◽  
pp. 189 ◽  
Author(s):  
Hongxiang Guo ◽  
Guojin He ◽  
Wei Jiang ◽  
Ranyu Yin ◽  
Lei Yan ◽  
...  
Keyword(s):  
Neural Network ◽  
Remote Sensing ◽  
Neural Networks ◽  
Convolutional Neural Network ◽  
Water Body ◽  
Semantic Segmentation ◽  
Water Bodies ◽  
Water Extraction ◽  
Remote Sensing Images ◽  
Multi Scale

Automatic water body extraction method is important for monitoring floods, droughts, and water resources. In this study, a new semantic segmentation convolutional neural network named the multi-scale water extraction convolutional neural network (MWEN) is proposed to automatically extract water bodies from GaoFen-1 (GF-1) remote sensing images. Three convolutional neural networks for semantic segmentation (fully convolutional network (FCN), Unet, and Deeplab V3+) are employed to compare with the water bodies extraction performance of MWEN. Visual comparison and five evaluation metrics are used to evaluate the performance of these convolutional neural networks (CNNs). The results show the following. (1) The results of water body extraction in multiple scenes using the MWEN are better than those of the other comparison methods based on the indicators. (2) The MWEN method has the capability to accurately extract various types of water bodies, such as urban water bodies, open ponds, and plateau lakes. (3) By fusing features extracted at different scales, the MWEN has the capability to extract water bodies with different sizes and suppress noise, such as building shadows and highways. Therefore, MWEN is a robust water extraction algorithm for GaoFen-1 satellite images and has the potential to conduct water body mapping with multisource high-resolution satellite remote sensing data.

scholarly journals Subtask Attention Based Object Detection in Remote Sensing Images

2021 ◽  
Vol 13 (10) ◽  
pp. 1925
Author(s):  
Shengzhou Xiong ◽  
Yihua Tan ◽  
Yansheng Li ◽  
Cai Wen ◽  
Pei Yan
Keyword(s):  
Remote Sensing ◽  
Object Detection ◽  
Feature Fusion ◽  
Detection Task ◽  
Feature Representation ◽  
Detection Accuracy ◽  
Remote Sensing Images ◽  
Attention Network ◽  
Multi Scale ◽  
Automatic Interpretation

Object detection in remote sensing images (RSIs) is one of the basic tasks in the field of remote sensing image automatic interpretation. In recent years, the deep object detection frameworks of natural scene images (NSIs) have been introduced into object detection on RSIs, and the detection performance has improved significantly because of the powerful feature representation. However, there are still many challenges concerning the particularities of remote sensing objects. One of the main challenges is the missed detection of small objects which have less than five percent of the pixels of the big objects. Generally, the existing algorithms choose to deal with this problem by multi-scale feature fusion based on a feature pyramid. However, the benefits of this strategy are limited, considering that the location of small objects in the feature map will disappear when the detection task is processed at the end of the network. In this study, we propose a subtask attention network (StAN), which handles the detection task directly on the shallow layer of the network. First, StAN contains one shared feature branch and two subtask attention branches of a semantic auxiliary subtask and a detection subtask based on the multi-task attention network (MTAN). Second, the detection branch uses only low-level features considering small objects. Third, the attention map guidance mechanism is put forward to optimize the network for keeping the identification ability. Fourth, the multi-dimensional sampling module (MdS), global multi-view channel weights (GMulW) and target-guided pixel attention (TPA) are designed for further improvement of the detection accuracy in complex scenes. The experimental results on the NWPU VHR-10 dataset and DOTA dataset demonstrated that the proposed algorithm achieved the SOTA performance, and the missed detection of small objects decreased. On the other hand, ablation experiments also proved the effects of MdS, GMulW and TPA.

Sign in / Sign up

Export Citation Format

Share Document