Recognition of small targets in remote sensing image using multi-scale feature fusion-based shot multi-box detector

Abstract Aiming at the problem of false detection and missed detection of small targets and occluded targets in the process of pedestrian detection, a pedestrian detection algorithm based on improved multi-scale feature fusion is proposed. First, for the YOLOv4 multi-scale feature fusion module PANet, which does not consider the interaction relationship between scales, PANet is improved to reduce the semantic gap between scales, and the attention mechanism is introduced to learn the importance of different layers to strengthen feature fusion; then, dilated convolution is introduced. Dilated convolution reduces the problem of information loss during the downsampling process; finally, the K-means clustering algorithm is used to redesign the anchor box and modify the loss function to detect a single category. The experimental results show that the improved pedestrian detection algorithm in the INRIA and WiderPerson data sets under different congestion conditions, the AP reaches 96.83% and 59.67%, respectively. Compared with the pedestrian detection results of the YOLOv4 model, the algorithm improves by 2.41% and 1.03%, respectively. The problem of false detection and missed detection of small targets and occlusion has been significantly improved.

Download Full-text

Semantic segmentation of remote sensing images based on dual attention and multi-scale feature fusion

Twelfth International Conference on Graphics and Image Processing (ICGIP 2020) ◽

10.1117/12.2589380 ◽

2021 ◽

Author(s):

Mengqian Weng ◽

Zhibo Hu ◽

Xiaopeng Xie ◽

Yunhong Li ◽

Lei Hu

Keyword(s):

Remote Sensing ◽

Feature Fusion ◽

Semantic Segmentation ◽

Remote Sensing Images ◽

Scale Feature ◽

Multi Scale

Download Full-text

IR remote sensing image registration based on multi-scale feature extraction

2014 International Joint Conference on Neural Networks (IJCNN) ◽

10.1109/ijcnn.2014.6889630 ◽

2014 ◽

Author(s):

Jun Kong ◽

Min Jiang ◽

Jun Kong ◽

Yi-Ning Sun

Keyword(s):

Remote Sensing ◽

Feature Extraction ◽

Image Registration ◽

Remote Sensing Image ◽

Scale Feature ◽

Multi Scale

Download Full-text

BMF-CNN: an object detection method based on multi-scale feature fusion in VHR remote sensing images

Remote Sensing Letters ◽

10.1080/2150704x.2019.1706007 ◽

2019 ◽

Vol 11 (3) ◽

pp. 215-224

Author(s):

Zhong Dong ◽

Baojun Lin

Keyword(s):

Remote Sensing ◽

Object Detection ◽

Detection Method ◽

Feature Fusion ◽

Remote Sensing Images ◽

Scale Feature ◽

Multi Scale

Download Full-text

Road Extraction from GF-1 Remote Sensing Images Based on Dilated Convolution Residual Network with Multi-Scale Feature Fusion

Laser & Optoelectronics Progress ◽

10.3788/lop202158.0228001 ◽

2021 ◽

Vol 58 (2) ◽

pp. 0228001

Author(s):

马天浩 Ma Tianhao ◽

谭海 Tan Hai ◽

李天琪 Li Tianqi ◽

吴雅男 Wu Yanan ◽

刘祺 Liu Qi

Keyword(s):

Remote Sensing ◽

Feature Fusion ◽

Road Extraction ◽

Remote Sensing Images ◽

Residual Network ◽

Scale Feature ◽

Multi Scale ◽

Dilated Convolution

Download Full-text

Remote Sensing Image Change Detection Based on Density Attraction and Multi-Scale and Multi-Feature Fusion

Laser & Optoelectronics Progress ◽

10.3788/lop56.121003 ◽

2019 ◽

Vol 56 (12) ◽

pp. 121003

Author(s):

金秋含 Qiuhan Jin ◽

王阳萍 Yangping Wang ◽

杨景玉 Jingyu Yang

Keyword(s):

Remote Sensing ◽

Change Detection ◽

Feature Fusion ◽

Remote Sensing Image ◽

Multi Scale ◽

Image Change Detection

Download Full-text

CF2PN: A Cross-Scale Feature Fusion Pyramid Network Based Remote Sensing Target Detection

Remote Sensing ◽

10.3390/rs13050847 ◽

2021 ◽

Vol 13 (5) ◽

pp. 847

Author(s):

Wei Huang ◽

Guanyi Li ◽

Qiqiang Chen ◽

Ming Ju ◽

Jiantao Qu

Keyword(s):

Remote Sensing ◽

Image Processing ◽

Target Detection ◽

Feature Fusion ◽

Detection Methods ◽

Natural Image ◽

Scale Feature ◽

Multi Scale ◽

Backbone Network ◽

Scale Detection

In the wake of developments in remote sensing, the application of target detection of remote sensing is of increasing interest. Unfortunately, unlike natural image processing, remote sensing image processing involves dealing with large variations in object size, which poses a great challenge to researchers. Although traditional multi-scale detection networks have been successful in solving problems with such large variations, they still have certain limitations: (1) The traditional multi-scale detection methods note the scale of features but ignore the correlation between feature levels. Each feature map is represented by a single layer of the backbone network, and the extracted features are not comprehensive enough. For example, the SSD network uses the features extracted from the backbone network at different scales directly for detection, resulting in the loss of a large amount of contextual information. (2) These methods combine with inherent backbone classification networks to perform detection tasks. RetinaNet is just a combination of the ResNet-101 classification network and FPN network to perform the detection tasks; however, there are differences in object classification and detection tasks. To address these issues, a cross-scale feature fusion pyramid network (CF2PN) is proposed. First and foremost, a cross-scale fusion module (CSFM) is introduced to extract sufficiently comprehensive semantic information from features for performing multi-scale fusion. Moreover, a feature pyramid for target detection utilizing thinning U-shaped modules (TUMs) performs the multi-level fusion of the features. Eventually, a focal loss in the prediction section is used to control the large number of negative samples generated during the feature fusion process. The new architecture of the network proposed in this paper is verified by DIOR and RSOD dataset. The experimental results show that the performance of this method is improved by 2–12% in the DIOR dataset and RSOD dataset compared with the current SOTA target detection methods.

Download Full-text

A Single Shot Framework with Multi-Scale Feature Fusion for Geospatial Object Detection

Remote Sensing ◽

10.3390/rs11050594 ◽

2019 ◽

Vol 11 (5) ◽

pp. 594 ◽

Cited By ~ 11

Author(s):

Shuo Zhuang ◽

Ping Wang ◽

Boran Jiang ◽

Gang Wang ◽

Cong Wang

Keyword(s):

Remote Sensing ◽

Object Detection ◽

Large Scale ◽

Feature Fusion ◽

Aerial Images ◽

Detection Methods ◽

Single Shot ◽

Feature Maps ◽

Scale Feature ◽

Multi Scale

With the rapid advances in remote-sensing technologies and the larger number of satellite images, fast and effective object detection plays an important role in understanding and analyzing image information, which could be further applied to civilian and military fields. Recently object detection methods with region-based convolutional neural network have shown excellent performance. However, these two-stage methods contain region proposal generation and object detection procedures, resulting in low computation speed. Because of the expensive manual costs, the quantity of well-annotated aerial images is scarce, which also limits the progress of geospatial object detection in remote sensing. In this paper, on the one hand, we construct and release a large-scale remote-sensing dataset for geospatial object detection (RSD-GOD) that consists of 5 different categories with 18,187 annotated images and 40,990 instances. On the other hand, we design a single shot detection framework with multi-scale feature fusion. The feature maps from different layers are fused together through the up-sampling and concatenation blocks to predict the detection results. High-level features with semantic information and low-level features with fine details are fully explored for detection tasks, especially for small objects. Meanwhile, a soft non-maximum suppression strategy is put into practice to select the final detection results. Extensive experiments have been conducted on two datasets to evaluate the designed network. Results show that the proposed approach achieves a good detection performance and obtains the mean average precision value of 89.0% on a newly constructed RSD-GOD dataset and 83.8% on the Northwestern Polytechnical University very high spatial resolution-10 (NWPU VHR-10) dataset at 18 frames per second (FPS) on a NVIDIA GTX-1080Ti GPU.

Download Full-text

Remote Sensing Imagery Super Resolution Based on Adaptive Multi-Scale Feature Fusion Network

Sensors ◽

10.3390/s20041142 ◽

2020 ◽

Vol 20 (4) ◽

pp. 1142

Author(s):

Xinying Wang ◽

Yingdan Wu ◽

Yang Ming ◽

Hui Lv

Keyword(s):

Remote Sensing ◽

Feature Extraction ◽

Feature Fusion ◽

Super Resolution ◽

Convolutional Network ◽

Remote Sensing Imagery ◽

Resolution Image ◽

Scale Feature ◽

Multi Scale ◽

Key Characteristics

Due to increasingly complex factors of image degradation, inferring high-frequency details of remote sensing imagery is more difficult compared to ordinary digital photos. This paper proposes an adaptive multi-scale feature fusion network (AMFFN) for remote sensing image super-resolution. Firstly, the features are extracted from the original low-resolution image. Then several adaptive multi-scale feature extraction (AMFE) modules, the squeeze-and-excited and adaptive gating mechanisms are adopted for feature extraction and fusion. Finally, the sub-pixel convolution method is used to reconstruct the high-resolution image. Experiments are performed on three datasets, the key characteristics, such as the number of AMFEs and the gating connection way are studied, and super-resolution of remote sensing imagery of different scale factors are qualitatively and quantitatively analyzed. The results show that our method outperforms the classic methods, such as Super-Resolution Convolutional Neural Network(SRCNN), Efficient Sub-Pixel Convolutional Network (ESPCN), and multi-scale residual CNN(MSRN).

Download Full-text