Predicting Arbitrary-Oriented Objects as Points in Remote Sensing Images

To detect rotated objects in remote sensing images, researchers have proposed a series of arbitrary-oriented object detection methods, which place multiple anchors with different angles, scales, and aspect ratios on the images. However, a major difference between remote sensing images and natural images is the small probability of overlap between objects in the same category, so the anchor-based design can introduce much redundancy during the detection process. In this paper, we convert the detection problem to a center point prediction problem, where the pre-defined anchors can be discarded. By directly predicting the center point, orientation, and corresponding height and width of the object, our methods can simplify the design of the model and reduce the computations related to anchors. In order to further fuse the multi-level features and get accurate object centers, a deformable feature pyramid network is proposed, to detect objects under complex backgrounds and various orientations of rotated objects. Experiments and analysis on two remote sensing datasets, DOTA and HRSC2016, demonstrate the effectiveness of our approach. Our best model, equipped with Deformable-FPN, achieved 74.75% mAP on DOTA and 96.59% on HRSC2016 with a single-stage model, single-scale training, and testing. By detecting arbitrarily oriented objects from their centers, the proposed model performs competitively against oriented anchor-based methods.

Download Full-text

Self-Adaptive Aspect Ratio Anchor for Oriented Object Detection in Remote Sensing Images

Remote Sensing ◽

10.3390/rs13071318 ◽

2021 ◽

Vol 13 (7) ◽

pp. 1318

Author(s):

Jie-Bo Hou ◽

Xiaobin Zhu ◽

Xu-Cheng Yin

Keyword(s):

Remote Sensing ◽

Aspect Ratio ◽

Object Detection ◽

Detection Methods ◽

Remote Sensing Images ◽

Feature Maps ◽

Aspect Ratios ◽

Feature Pyramid ◽

Oriented Object ◽

Self Adaptive

Object detection is a significant and challenging problem in the study of remote sensing. Since remote sensing images are typically captured with a bird’s-eye view, the aspect ratios of objects in the same category may obey a Gaussian distribution. Generally, existing object detection methods ignore exploring the distribution character of aspect ratios for improving performance in remote sensing tasks. In this paper, we propose a novel Self-Adaptive Aspect Ratio Anchor (SARA) to explicitly explore aspect ratio variations of objects in remote sensing images. To be concrete, our SARA can self-adaptively learn an appropriate aspect ratio for each category. In this way, we can only utilize a simple squared anchor (related to the strides of feature maps in Feature Pyramid Networks) to regress objects in various aspect ratios. Finally, we adopt an Oriented Box Decoder (OBD) to align the feature maps and encode the orientation information of oriented objects. Our method achieves a promising mAP value of 79.91% on the DOTA dataset.

Download Full-text

Weighted Ensemble Object Detection with Optimized Coefficients for Remote Sensing Images

ISPRS International Journal of Geo-Information ◽

10.3390/ijgi9060370 ◽

2020 ◽

Vol 9 (6) ◽

pp. 370

Author(s):

Atakan Körez ◽

Necaattin Barışçı ◽

Aydın Çetin ◽

Uçman Ergün

Keyword(s):

Remote Sensing ◽

High Resolution ◽

Object Detection ◽

Mean Average Precision ◽

Detection Methods ◽

Remote Sensing Images ◽

Average Precision ◽

Proposed Model ◽

Detection Of Objects ◽

Very High

The detection of objects in very high-resolution (VHR) remote sensing images has become increasingly popular with the enhancement of remote sensing technologies. High-resolution images from aircrafts or satellites contain highly detailed and mixed backgrounds that decrease the success of object detection in remote sensing images. In this study, a model that performs weighted ensemble object detection using optimized coefficients is proposed. This model uses the outputs of three different object detection models trained on the same dataset. The model’s structure takes two or more object detection methods as its input and provides an output with an optimized coefficient-weighted ensemble. The Northwestern Polytechnical University Very High Resolution 10 (NWPU-VHR10) and Remote Sensing Object Detection (RSOD) datasets were used to measure the object detection success of the proposed model. Our experiments reveal that the proposed model improved the Mean Average Precision (mAP) performance by 0.78%–16.5% compared to stand-alone models and presents better mean average precision than other state-of-the-art methods (3.55% higher on the NWPU-VHR-10 dataset and 1.49% higher when using the RSOD dataset).

Download Full-text

A Lightweight Keypoint-Based Oriented Object Detection of Remote Sensing Images

Remote Sensing ◽

10.3390/rs13132459 ◽

2021 ◽

Vol 13 (13) ◽

pp. 2459

Author(s):

Yangyang Li ◽

Heting Mao ◽

Ruijiao Liu ◽

Xuan Pei ◽

Licheng Jiao ◽

...

Keyword(s):

Remote Sensing ◽

Object Detection ◽

Large Scale ◽

Detection Methods ◽

Gaussian Kernel ◽

Remote Sensing Images ◽

Computational Overhead ◽

Comparable Performance ◽

Bounding Boxes ◽

Oriented Object

Object detection in remote sensing images has been widely used in military and civilian fields and is a challenging task due to the complex background, large-scale variation, and dense arrangement in arbitrary orientations of objects. In addition, existing object detection methods rely on the increasingly deeper network, which increases a lot of computational overhead and parameters, and is unfavorable to deployment on the edge devices. In this paper, we proposed a lightweight keypoint-based oriented object detector for remote sensing images. First, we propose a semantic transfer block (STB) when merging shallow and deep features, which reduces noise and restores the semantic information. Then, the proposed adaptive Gaussian kernel (AGK) is adapted to objects of different scales, and further improves detection performance. Finally, we propose the distillation loss associated with object detection to obtain a lightweight student network. Experiments on the HRSC2016 and UCAS-AOD datasets show that the proposed method adapts to different scale objects, obtains accurate bounding boxes, and reduces the influence of complex backgrounds. The comparison with mainstream methods proves that our method has comparable performance under lightweight.

Download Full-text

Generating Anchor Boxes Based on Attention Mechanism for Object Detection in Remote Sensing Images

Remote Sensing ◽

10.3390/rs12152416 ◽

2020 ◽

Vol 12 (15) ◽

pp. 2416 ◽

Cited By ~ 1

Author(s):

Zhuangzhuang Tian ◽

Ronghui Zhan ◽

Jiemin Hu ◽

Wei Wang ◽

Zhiqiang He ◽

...

Keyword(s):

Remote Sensing ◽

Object Detection ◽

Attention Mechanism ◽

Detection Methods ◽

Optical Remote Sensing ◽

Remote Sensing Images ◽

Feature Maps ◽

Wide Range ◽

Feature Pyramid ◽

Comprehensive Evaluations

Nowadays, object detection methods based on deep learning are applied more and more to the interpretation of optical remote sensing images. However, the complex background and the wide range of object sizes in remote sensing images increase the difficulty of object detection. In this paper, we improve the detection performance by combining the attention information, and generate adaptive anchor boxes based on the attention map. Specifically, the attention mechanism is introduced into the proposed method to enhance the features of the object regions while reducing the influence of the background. The generated attention map is then used to obtain diverse and adaptable anchor boxes using the guided anchoring method. The generated anchor boxes can match better with the scene and the objects, compared with the traditional proposal boxes. Finally, the modulated feature adaptation module is applied to transform the feature maps to adapt to the diverse anchor boxes. Comprehensive evaluations on the DIOR dataset demonstrate the superiority of the proposed method over the state-of-the-art methods, such as RetinaNet, FCOS and CornerNet. The mean average precision of the proposed method is 4.5% higher than the feature pyramid network. In addition, the ablation experiments are also implemented to further analyze the respective influence of different blocks on the performance improvement.

Download Full-text

CGP Box: An effective direction representation strategy for oriented object detection in remote sensing images

International Journal of Remote Sensing ◽

10.1080/01431161.2021.1941389 ◽

2021 ◽

Vol 42 (17) ◽

pp. 6670-6691

Author(s):

Qiuyu Guan ◽

Zhenshen Qu ◽

Ming Zeng ◽

Jianxiong Shen ◽

Jingda Du

Keyword(s):

Remote Sensing ◽

Object Detection ◽

Remote Sensing Images ◽

Oriented Object

Download Full-text

Improved Oriented Object Detection in Remote Sensing Images Based on a Three-Point Regression Method

Remote Sensing ◽

10.3390/rs13224517 ◽

2021 ◽

Vol 13 (22) ◽

pp. 4517

Author(s):

Falin Wu ◽

Jiaqi He ◽

Guopeng Zhou ◽

Haolun Li ◽

Yushuang Liu ◽

...

Keyword(s):

Remote Sensing ◽

Object Detection ◽

Poor Performance ◽

Regression Method ◽

Remote Sensing Images ◽

Sensing Applications ◽

Bounding Box ◽

Bounding Boxes ◽

Fully Connected ◽

Oriented Object

Object detection in remote sensing images plays an important role in both military and civilian remote sensing applications. Objects in remote sensing images are different from those in natural images. They have the characteristics of scale diversity, arbitrary directivity, and dense arrangement, which causes difficulties in object detection. For objects with a large aspect ratio and that are oblique and densely arranged, using an oriented bounding box can help to avoid deleting some correct detection bounding boxes by mistake. The classic rotational region convolutional neural network (R2CNN) has advantages for text detection. However, R2CNN has poor performance in the detection of slender objects with arbitrary directivity in remote sensing images, and its fault tolerance rate is low. In order to solve this problem, this paper proposes an improved R2CNN based on a double detection head structure and a three-point regression method, namely, TPR-R2CNN. The proposed network modifies the original R2CNN network structure by applying a double fully connected (2-fc) detection head and classification fusion. One detection head is for classification and horizontal bounding box regression, the other is for classification and oriented bounding box regression. The three-point regression method (TPR) is proposed for oriented bounding box regression, which determines the positions of the oriented bounding box by regressing the coordinates of the center point and the first two vertices. The proposed network was validated on the DOTA-v1.5 and HRSC2016 datasets, and it achieved a mean average precision (mAP) of 3.90% and 15.27%, respectively, from feature pyramid network (FPN) baselines with a ResNet-50 backbone.

Download Full-text

Application of an improved oriented object detection algorithm in remote sensing images

10.1109/icwcsg53609.2021.00014 ◽

2021 ◽

Author(s):

Guozhi Miao ◽

Xiaokang Ren ◽

Ruchuan Guo ◽

Zhichao Peng

Keyword(s):

Remote Sensing ◽

Object Detection ◽

Detection Algorithm ◽

Remote Sensing Images ◽

Oriented Object

Download Full-text

Road Extraction from Very-High-Resolution Remote Sensing Images via a Nested SE-Deeplab Model

Remote Sensing ◽

10.3390/rs12182985 ◽

2020 ◽

Vol 12 (18) ◽

pp. 2985 ◽

Cited By ~ 1

Author(s):

Yeneng Lin ◽

Dongyun Xu ◽

Nan Wang ◽

Zhou Shi ◽

Qiuxiao Chen

Keyword(s):

Remote Sensing ◽

Deep Learning ◽

High Resolution ◽

Network Models ◽

Road Extraction ◽

Remote Sensing Images ◽

Proposed Model ◽

Wide Range ◽

Network Modules ◽

Very High

Automatic road extraction from very-high-resolution remote sensing images has become a popular topic in a wide range of fields. Convolutional neural networks are often used for this purpose. However, many network models do not achieve satisfactory extraction results because of the elongated nature and varying sizes of roads in images. To improve the accuracy of road extraction, this paper proposes a deep learning model based on the structure of Deeplab v3. It incorporates squeeze-and-excitation (SE) module to apply weights to different feature channels, and performs multi-scale upsampling to preserve and fuse shallow and deep information. To solve the problems associated with unbalanced road samples in images, different loss functions and backbone network modules are tested in the model’s training process. Compared with cross entropy, dice loss can improve the performance of the model during training and prediction. The SE module is superior to ResNext and ResNet in improving the integrity of the extracted roads. Experimental results obtained using the Massachusetts Roads Dataset show that the proposed model (Nested SE-Deeplab) improves F1-Score by 2.4% and Intersection over Union by 2.0% compared with FC-DenseNet. The proposed model also achieves better segmentation accuracy in road extraction compared with other mainstream deep-learning models including Deeplab v3, SegNet, and UNet.

Download Full-text

Ship Target Automatic Detection Based on Hypercomplex Flourier Transform Saliency Model in High Spatial Resolution Remote-Sensing Images

Sensors ◽

10.3390/s20092536 ◽

2020 ◽

Vol 20 (9) ◽

pp. 2536 ◽

Cited By ~ 1

Author(s):

Jian He ◽

Yongfei Guo ◽

Hangfei Yuan

Keyword(s):

Remote Sensing ◽

Spatial Resolution ◽

Detection Efficiency ◽

Detection System ◽

Automatic Detection ◽

Sea Surface ◽

Detection Methods ◽

Remote Sensing Images ◽

Ship Detection ◽

Saliency Model

Efficient ship detection is essential to the strategies of commerce and military. However, traditional ship detection methods have low detection efficiency and poor reliability due to uncertain conditions of the sea surface, such as the atmosphere, illumination, clouds and islands. Hence, in this study, a novel ship target automatic detection system based on a modified hypercomplex Flourier transform (MHFT) saliency model is proposed for spatial resolution of remote-sensing images. The method first utilizes visual saliency theory to effectively suppress sea surface interference. Then we use OTSU methods to extract regions of interest. After obtaining the candidate ship target regions, we get the candidate target using a method of ship target recognition based on ResNet framework. This method has better accuracy and better performance for the recognition of ship targets than other methods. The experimental results show that the proposed method not only accurately and effectively recognizes ship targets, but also is suitable for spatial resolution of remote-sensing images with complex backgrounds.

Download Full-text