CF2PN: A Cross-Scale Feature Fusion Pyramid Network Based Remote Sensing Target Detection

In the wake of developments in remote sensing, the application of target detection of remote sensing is of increasing interest. Unfortunately, unlike natural image processing, remote sensing image processing involves dealing with large variations in object size, which poses a great challenge to researchers. Although traditional multi-scale detection networks have been successful in solving problems with such large variations, they still have certain limitations: (1) The traditional multi-scale detection methods note the scale of features but ignore the correlation between feature levels. Each feature map is represented by a single layer of the backbone network, and the extracted features are not comprehensive enough. For example, the SSD network uses the features extracted from the backbone network at different scales directly for detection, resulting in the loss of a large amount of contextual information. (2) These methods combine with inherent backbone classification networks to perform detection tasks. RetinaNet is just a combination of the ResNet-101 classification network and FPN network to perform the detection tasks; however, there are differences in object classification and detection tasks. To address these issues, a cross-scale feature fusion pyramid network (CF2PN) is proposed. First and foremost, a cross-scale fusion module (CSFM) is introduced to extract sufficiently comprehensive semantic information from features for performing multi-scale fusion. Moreover, a feature pyramid for target detection utilizing thinning U-shaped modules (TUMs) performs the multi-level fusion of the features. Eventually, a focal loss in the prediction section is used to control the large number of negative samples generated during the feature fusion process. The new architecture of the network proposed in this paper is verified by DIOR and RSOD dataset. The experimental results show that the performance of this method is improved by 2–12% in the DIOR dataset and RSOD dataset compared with the current SOTA target detection methods.

Download Full-text

AF-EMS Detector: Improve the Multi-Scale Detection Performance of the Anchor-Free Detector

Remote Sensing ◽

10.3390/rs13020160 ◽

2021 ◽

Vol 13 (2) ◽

pp. 160

Author(s):

Jiangqiao Yan ◽

Liangjin Zhao ◽

Wenhui Diao ◽

Hongqi Wang ◽

Xian Sun

Keyword(s):

Remote Sensing ◽

Object Detection ◽

Feature Fusion ◽

Detection Algorithm ◽

Detection Performance ◽

Natural Image ◽

Detection Model ◽

Multi Scale ◽

Feature Pyramid ◽

Scale Detection

As a precursor step for computer vision algorithms, object detection plays an important role in various practical application scenarios. With the objects to be detected becoming more complex, the problem of multi-scale object detection has attracted more and more attention, especially in the field of remote sensing detection. Early convolutional neural network detection algorithms are mostly based on artificially preset anchor-boxes to divide different regions in the image, and then obtain the prior position of the target. However, the anchor box is difficult to set reasonably and will cause a large amount of computational redundancy, which affects the generality of the detection model obtained under fixed parameters. In the past two years, anchor-free detection algorithm has achieved remarkable development in the field of detection on natural image. However, there is no sufficient research on how to deal with multi-scale detection more effectively in anchor-free framework and use these detectors on remote sensing images. In this paper, we propose a specific-attention Feature Pyramid Network (FPN) module, which is able to generate a feature pyramid, basing on the characteristics of objects with various sizes. In addition, this pyramid suits multi-scale object detection better. Besides, a scale-aware detection head is proposed which contains a multi-receptive feature fusion module and a size-based feature compensation module. The new anchor-free detector can obtain a more effective multi-scale feature expression. Experiments on challenging datasets show that our approach performs favorably against other methods in terms of the multi-scale object detection performance.

Download Full-text

A Single Shot Framework with Multi-Scale Feature Fusion for Geospatial Object Detection

Remote Sensing ◽

10.3390/rs11050594 ◽

2019 ◽

Vol 11 (5) ◽

pp. 594 ◽

Cited By ~ 11

Author(s):

Shuo Zhuang ◽

Ping Wang ◽

Boran Jiang ◽

Gang Wang ◽

Cong Wang

Keyword(s):

Remote Sensing ◽

Object Detection ◽

Large Scale ◽

Feature Fusion ◽

Aerial Images ◽

Detection Methods ◽

Single Shot ◽

Feature Maps ◽

Scale Feature ◽

Multi Scale

With the rapid advances in remote-sensing technologies and the larger number of satellite images, fast and effective object detection plays an important role in understanding and analyzing image information, which could be further applied to civilian and military fields. Recently object detection methods with region-based convolutional neural network have shown excellent performance. However, these two-stage methods contain region proposal generation and object detection procedures, resulting in low computation speed. Because of the expensive manual costs, the quantity of well-annotated aerial images is scarce, which also limits the progress of geospatial object detection in remote sensing. In this paper, on the one hand, we construct and release a large-scale remote-sensing dataset for geospatial object detection (RSD-GOD) that consists of 5 different categories with 18,187 annotated images and 40,990 instances. On the other hand, we design a single shot detection framework with multi-scale feature fusion. The feature maps from different layers are fused together through the up-sampling and concatenation blocks to predict the detection results. High-level features with semantic information and low-level features with fine details are fully explored for detection tasks, especially for small objects. Meanwhile, a soft non-maximum suppression strategy is put into practice to select the final detection results. Extensive experiments have been conducted on two datasets to evaluate the designed network. Results show that the proposed approach achieves a good detection performance and obtains the mean average precision value of 89.0% on a newly constructed RSD-GOD dataset and 83.8% on the Northwestern Polytechnical University very high spatial resolution-10 (NWPU VHR-10) dataset at 18 frames per second (FPS) on a NVIDIA GTX-1080Ti GPU.

Download Full-text

Semantic segmentation of remote sensing images based on dual attention and multi-scale feature fusion

Twelfth International Conference on Graphics and Image Processing (ICGIP 2020) ◽

10.1117/12.2589380 ◽

2021 ◽

Author(s):

Mengqian Weng ◽

Zhibo Hu ◽

Xiaopeng Xie ◽

Yunhong Li ◽

Lei Hu

Keyword(s):

Remote Sensing ◽

Feature Fusion ◽

Semantic Segmentation ◽

Remote Sensing Images ◽

Scale Feature ◽

Multi Scale

Download Full-text

BMF-CNN: an object detection method based on multi-scale feature fusion in VHR remote sensing images

Remote Sensing Letters ◽

10.1080/2150704x.2019.1706007 ◽

2019 ◽

Vol 11 (3) ◽

pp. 215-224

Author(s):

Zhong Dong ◽

Baojun Lin

Keyword(s):

Remote Sensing ◽

Object Detection ◽

Detection Method ◽

Feature Fusion ◽

Remote Sensing Images ◽

Scale Feature ◽

Multi Scale

Download Full-text

Road Extraction from GF-1 Remote Sensing Images Based on Dilated Convolution Residual Network with Multi-Scale Feature Fusion

Laser & Optoelectronics Progress ◽

10.3788/lop202158.0228001 ◽

2021 ◽

Vol 58 (2) ◽

pp. 0228001

Author(s):

马天浩 Ma Tianhao ◽

谭海 Tan Hai ◽

李天琪 Li Tianqi ◽

吴雅男 Wu Yanan ◽

刘祺 Liu Qi

Keyword(s):

Remote Sensing ◽

Feature Fusion ◽

Road Extraction ◽

Remote Sensing Images ◽

Residual Network ◽

Scale Feature ◽

Multi Scale ◽

Dilated Convolution

Download Full-text

Recognition of small targets in remote sensing image using multi-scale feature fusion-based shot multi-box detector

Optics and Precision Engineering ◽

10.37188/ope.20212911.2672 ◽

2021 ◽

Vol 29 (11) ◽

pp. 2672-2682

Author(s):

Xin CHEN ◽

◽

Min-jie WAN ◽

Chao MA ◽

Qian CHEN ◽

...

Keyword(s):

Remote Sensing ◽

Feature Fusion ◽

Remote Sensing Image ◽

Scale Feature ◽

Multi Scale ◽

Small Targets

Download Full-text

Systematic Review of Anomaly Detection in Hyperspectral Remote Sensing Applications

Applied Sciences ◽

10.3390/app11114878 ◽

2021 ◽

Vol 11 (11) ◽

pp. 4878

Author(s):

Ivan Racetin ◽

Andrija Krtalić

Keyword(s):

Remote Sensing ◽

Image Processing ◽

Anomaly Detection ◽

Target Detection ◽

Hyperspectral Image ◽

Hyperspectral Images ◽

Detection Methods ◽

Hyperspectral Image Processing ◽

Sensing Applications ◽

Remote Sensing Applications

Hyperspectral sensors are passive instruments that record reflected electromagnetic radiation in tens or hundreds of narrow and consecutive spectral bands. In the last two decades, the availability of hyperspectral data has sharply increased, propelling the development of a plethora of hyperspectral classification and target detection algorithms. Anomaly detection methods in hyperspectral images refer to a class of target detection methods that do not require any a-priori knowledge about a hyperspectral scene or target spectrum. They are unsupervised learning techniques that automatically discover rare features on hyperspectral images. This review paper is organized into two parts: part A provides a bibliographic analysis of hyperspectral image processing for anomaly detection in remote sensing applications. Development of the subject field is discussed, and key authors and journals are highlighted. In part B an overview of the topic is presented, starting from the mathematical framework for anomaly detection. The anomaly detection methods were generally categorized as techniques that implement structured or unstructured background models and then organized into appropriate sub-categories. Specific anomaly detection methods are presented with corresponding detection statistics, and their properties are discussed. This paper represents the first review regarding hyperspectral image processing for anomaly detection in remote sensing applications.

Download Full-text

Remote Sensing Imagery Super Resolution Based on Adaptive Multi-Scale Feature Fusion Network

Sensors ◽

10.3390/s20041142 ◽

2020 ◽

Vol 20 (4) ◽

pp. 1142

Author(s):

Xinying Wang ◽

Yingdan Wu ◽

Yang Ming ◽

Hui Lv

Keyword(s):

Remote Sensing ◽

Feature Extraction ◽

Feature Fusion ◽

Super Resolution ◽

Convolutional Network ◽

Remote Sensing Imagery ◽

Resolution Image ◽

Scale Feature ◽

Multi Scale ◽

Key Characteristics

Due to increasingly complex factors of image degradation, inferring high-frequency details of remote sensing imagery is more difficult compared to ordinary digital photos. This paper proposes an adaptive multi-scale feature fusion network (AMFFN) for remote sensing image super-resolution. Firstly, the features are extracted from the original low-resolution image. Then several adaptive multi-scale feature extraction (AMFE) modules, the squeeze-and-excited and adaptive gating mechanisms are adopted for feature extraction and fusion. Finally, the sub-pixel convolution method is used to reconstruct the high-resolution image. Experiments are performed on three datasets, the key characteristics, such as the number of AMFEs and the gating connection way are studied, and super-resolution of remote sensing imagery of different scale factors are qualitatively and quantitatively analyzed. The results show that our method outperforms the classic methods, such as Super-Resolution Convolutional Neural Network(SRCNN), Efficient Sub-Pixel Convolutional Network (ESPCN), and multi-scale residual CNN(MSRN).

Download Full-text