Constraint Loss for Rotated Object Detection in Remote Sensing Images

Rotated object detection is an extension of object detection that uses an oriented bounding box instead of a general horizontal bounding box to define the object position. It is widely used in remote sensing images, scene text, and license plate recognition. The existing rotated object detection methods usually add an angle prediction channel in the bounding box prediction branch, and smooth L1 loss is used as the regression loss function. However, we argue that smooth L1 loss causes a sudden change in loss and slow convergence due to the angle solving mechanism of openCV (the angle between the horizontal line and the first side of the bounding box in the counter-clockwise direction is defined as the rotation angle), and this problem exists in most existing regression loss functions. To solve the above problems, we propose a decoupling modulation mechanism to overcome the problem of sudden changes in loss. On this basis, we also proposed a constraint mechanism, the purpose of which is to accelerate the convergence of the network and ensure optimization toward the ideal direction. In addition, the proposed decoupling modulation mechanism and constraint mechanism can be integrated into the popular regression loss function individually or together, which further improves the performance of the model and makes the model converge faster. The experimental results show that our method achieves 75.2% performance on the aerial image dataset DOTA (OBB task), and saves more than 30% of computing resources. The method also achieves a state-of-the-art performance in HRSC2016, and saved more than 40% of computing resources, which confirms the applicability of the approach.

Download Full-text

Improved Oriented Object Detection in Remote Sensing Images Based on a Three-Point Regression Method

Remote Sensing ◽

10.3390/rs13224517 ◽

2021 ◽

Vol 13 (22) ◽

pp. 4517

Author(s):

Falin Wu ◽

Jiaqi He ◽

Guopeng Zhou ◽

Haolun Li ◽

Yushuang Liu ◽

...

Keyword(s):

Remote Sensing ◽

Object Detection ◽

Poor Performance ◽

Regression Method ◽

Remote Sensing Images ◽

Sensing Applications ◽

Bounding Box ◽

Bounding Boxes ◽

Fully Connected ◽

Oriented Object

Object detection in remote sensing images plays an important role in both military and civilian remote sensing applications. Objects in remote sensing images are different from those in natural images. They have the characteristics of scale diversity, arbitrary directivity, and dense arrangement, which causes difficulties in object detection. For objects with a large aspect ratio and that are oblique and densely arranged, using an oriented bounding box can help to avoid deleting some correct detection bounding boxes by mistake. The classic rotational region convolutional neural network (R2CNN) has advantages for text detection. However, R2CNN has poor performance in the detection of slender objects with arbitrary directivity in remote sensing images, and its fault tolerance rate is low. In order to solve this problem, this paper proposes an improved R2CNN based on a double detection head structure and a three-point regression method, namely, TPR-R2CNN. The proposed network modifies the original R2CNN network structure by applying a double fully connected (2-fc) detection head and classification fusion. One detection head is for classification and horizontal bounding box regression, the other is for classification and oriented bounding box regression. The three-point regression method (TPR) is proposed for oriented bounding box regression, which determines the positions of the oriented bounding box by regressing the coordinates of the center point and the first two vertices. The proposed network was validated on the DOTA-v1.5 and HRSC2016 datasets, and it achieved a mean average precision (mAP) of 3.90% and 15.27%, respectively, from feature pyramid network (FPN) baselines with a ResNet-50 backbone.

Download Full-text

Detection of Collapsed Buildings in Post-Earthquake Remote Sensing Images Based on the Improved YOLOv3

Remote Sensing ◽

10.3390/rs12010044 ◽

2019 ◽

Vol 12 (1) ◽

pp. 44 ◽

Cited By ~ 10

Author(s):

Haojie Ma ◽

Yalan Liu ◽

Yuhuan Ren ◽

Jingxian Yu

Keyword(s):

Remote Sensing ◽

Object Detection ◽

Spatial Resolution ◽

High Spatial Resolution ◽

Detection Methods ◽

Remote Sensing Images ◽

Sensing Technology ◽

Collapsed Buildings ◽

Improved Model ◽

Detection Speed

An important and effective method for the preliminary mitigation and relief of an earthquake is the rapid estimation of building damage via high spatial resolution remote sensing technology. Traditional object detection methods only use artificially designed shallow features on post-earthquake remote sensing images, which are uncertain and complex background environment and time-consuming feature selection. The satisfactory results from them are often difficult. Therefore, this study aims to apply the object detection method You Only Look Once (YOLOv3) based on the convolutional neural network (CNN) to locate collapsed buildings from post-earthquake remote sensing images. Moreover, YOLOv3 was improved to obtain more effective detection results. First, we replaced the Darknet53 CNN in YOLOv3 with the lightweight CNN ShuffleNet v2. Second, the prediction box center point, XY loss, and prediction box width and height, WH loss, in the loss function was replaced with the generalized intersection over union (GIoU) loss. Experiments performed using the improved YOLOv3 model, with high spatial resolution aerial remote sensing images at resolutions of 0.5 m after the Yushu and Wenchuan earthquakes, show a significant reduction in the number of parameters, detection speed of up to 29.23 f/s, and target precision of 90.89%. Compared with the general YOLOv3, the detection speed improved by 5.21 f/s and its precision improved by 5.24%. Moreover, the improved model had stronger noise immunity capabilities, which indicates a significant improvement in the model’s generalization. Therefore, this improved YOLOv3 model is effective for the detection of collapsed buildings in post-earthquake high-resolution remote sensing images.

Download Full-text

VaryBlock: A Novel Approach for Object Detection in Remote Sensed Images

Sensors ◽

10.3390/s19235284 ◽

2019 ◽

Vol 19 (23) ◽

pp. 5284 ◽

Cited By ~ 3

Author(s):

Heng Zhang ◽

Jiayu Wu ◽

Yanli Liu ◽

Jia Yu

Keyword(s):

Remote Sensing ◽

Object Detection ◽

Poor Performance ◽

Detection Methods ◽

Optical Remote Sensing ◽

Remote Sensing Images ◽

Novel Approach ◽

Challenging Tasks ◽

The Mean ◽

General Object

In recent years, the research on optical remote sensing images has received greater and greater attention. Object detection, as one of the most challenging tasks in the area of remote sensing, has been remarkably promoted by convolutional neural network (CNN)-based methods like You Only Look Once (YOLO) and Faster R-CNN. However, due to the complexity of backgrounds and the distinctive object distribution, directly applying these general object detection methods to the remote sensing object detection usually renders poor performance. To tackle this problem, a highly efficient and robust framework based on YOLO is proposed. We devise and integrate VaryBlock to the architecture which effectively offsets some of the information loss caused by downsampling. In addition, some techniques are utilized to facilitate the performance and to avoid overfitting. Experimental results show that our proposed method can enormously improve the mean average precision by a large margin on the NWPU VHR-10 dataset.

Download Full-text

Weighted Ensemble Object Detection with Optimized Coefficients for Remote Sensing Images

ISPRS International Journal of Geo-Information ◽

10.3390/ijgi9060370 ◽

2020 ◽

Vol 9 (6) ◽

pp. 370

Author(s):

Atakan Körez ◽

Necaattin Barışçı ◽

Aydın Çetin ◽

Uçman Ergün

Keyword(s):

Remote Sensing ◽

High Resolution ◽

Object Detection ◽

Mean Average Precision ◽

Detection Methods ◽

Remote Sensing Images ◽

Average Precision ◽

Proposed Model ◽

Detection Of Objects ◽

Very High

The detection of objects in very high-resolution (VHR) remote sensing images has become increasingly popular with the enhancement of remote sensing technologies. High-resolution images from aircrafts or satellites contain highly detailed and mixed backgrounds that decrease the success of object detection in remote sensing images. In this study, a model that performs weighted ensemble object detection using optimized coefficients is proposed. This model uses the outputs of three different object detection models trained on the same dataset. The model’s structure takes two or more object detection methods as its input and provides an output with an optimized coefficient-weighted ensemble. The Northwestern Polytechnical University Very High Resolution 10 (NWPU-VHR10) and Remote Sensing Object Detection (RSOD) datasets were used to measure the object detection success of the proposed model. Our experiments reveal that the proposed model improved the Mean Average Precision (mAP) performance by 0.78%–16.5% compared to stand-alone models and presents better mean average precision than other state-of-the-art methods (3.55% higher on the NWPU-VHR-10 dataset and 1.49% higher when using the RSOD dataset).

Download Full-text

A Lightweight Keypoint-Based Oriented Object Detection of Remote Sensing Images

Remote Sensing ◽

10.3390/rs13132459 ◽

2021 ◽

Vol 13 (13) ◽

pp. 2459

Author(s):

Yangyang Li ◽

Heting Mao ◽

Ruijiao Liu ◽

Xuan Pei ◽

Licheng Jiao ◽

...

Keyword(s):

Remote Sensing ◽

Object Detection ◽

Large Scale ◽

Detection Methods ◽

Gaussian Kernel ◽

Remote Sensing Images ◽

Computational Overhead ◽

Comparable Performance ◽

Bounding Boxes ◽

Oriented Object

Object detection in remote sensing images has been widely used in military and civilian fields and is a challenging task due to the complex background, large-scale variation, and dense arrangement in arbitrary orientations of objects. In addition, existing object detection methods rely on the increasingly deeper network, which increases a lot of computational overhead and parameters, and is unfavorable to deployment on the edge devices. In this paper, we proposed a lightweight keypoint-based oriented object detector for remote sensing images. First, we propose a semantic transfer block (STB) when merging shallow and deep features, which reduces noise and restores the semantic information. Then, the proposed adaptive Gaussian kernel (AGK) is adapted to objects of different scales, and further improves detection performance. Finally, we propose the distillation loss associated with object detection to obtain a lightweight student network. Experiments on the HRSC2016 and UCAS-AOD datasets show that the proposed method adapts to different scale objects, obtains accurate bounding boxes, and reduces the influence of complex backgrounds. The comparison with mainstream methods proves that our method has comparable performance under lightweight.

Download Full-text

Mapping Utility Poles in Aerial Orthoimages Using ATSS Deep Learning Method

Sensors ◽

10.3390/s20216070 ◽

2020 ◽

Vol 20 (21) ◽

pp. 6070

Author(s):

Matheus Gomes ◽

Jonathan Silva ◽

Diogo Gonçalves ◽

Pedro Zamboni ◽

Jader Perez ◽

...

Keyword(s):

Remote Sensing ◽

Object Detection ◽

Urban Areas ◽

Aerial Images ◽

Detection Methods ◽

Ground Sample ◽

Sensing Applications ◽

Bounding Box ◽

Utility Poles ◽

Remote Sensing Applications

Mapping utility poles using side-view images acquired with car-mounted cameras is a time-consuming task, mainly in larger areas due to the need for street-by-street surveying. Aerial images cover larger areas and can be feasible alternatives although the detection and mapping of the utility poles in urban environments using top-view images is challenging. Thus, we propose the use of Adaptive Training Sample Selection (ATSS) for detecting utility poles in urban areas since it is a novel method and has not yet investigated in remote sensing applications. Here, we compared ATSS with Faster Region-based Convolutional Neural Networks (Faster R-CNN) and Focal Loss for Dense Object Detection (RetinaNet ), currently used in remote sensing applications, to assess the performance of the proposed methodology. We used 99,473 patches of 256 × 256 pixels with ground sample distance (GSD) of 10 cm. The patches were divided into training, validation and test datasets in approximate proportions of 60%, 20% and 20%, respectively. As the utility pole labels are point coordinates and the object detection methods require a bounding box, we assessed the influence of the bounding box size on the ATSS method by varying the dimensions from 30×30 to 70×70 pixels. For the proposal task, our findings show that ATSS is, on average, 5% more accurate than Faster R-CNN and RetinaNet. For a bounding box size of 40×40, we achieved Average Precision with intersection over union of 50% (AP50) of 0.913 for ATSS, 0.875 for Faster R-CNN and 0.874 for RetinaNet. Regarding the influence of the bounding box size on ATSS, our results indicate that the AP50 is about 6.5% higher for 60×60 compared to 30×30. For AP75, this margin reaches 23.1% in favor of the 60×60 bounding box size. In terms of computational costs, all the methods tested remain at the same level, with an average processing time around of 0.048 s per patch. Our findings show that ATSS outperforms other methodologies and is suitable for developing operation tools that can automatically detect and map utility poles.

Download Full-text

RADet: Refine Feature Pyramid Network and Multi-Layer Attention Network for Arbitrary-Oriented Object Detection of Remote Sensing Images

Remote Sensing ◽

10.3390/rs12030389 ◽

2020 ◽

Vol 12 (3) ◽

pp. 389 ◽

Cited By ~ 6

Author(s):

Yangyang Li ◽

Qin Huang ◽

Xuan Pei ◽

Licheng Jiao ◽

Ronghua Shang

Keyword(s):

Remote Sensing ◽

Object Detection ◽

Small Object ◽

Remote Sensing Images ◽

Feature Maps ◽

Attention Network ◽

Complex Background ◽

Bounding Box ◽

Feature Pyramid ◽

Public Datasets

Object detection has made significant progress in many real-world scenes. Despite this remarkable progress, the common use case of detection in remote sensing images remains challenging even for leading object detectors, due to the complex background, objects with arbitrary orientation, and large difference in scale of objects. In this paper, we propose a novel rotation detector for remote sensing images, mainly inspired by Mask R-CNN, namely RADet. RADet can obtain the rotation bounding box of objects with shape mask predicted by the mask branch, which is a novel, simple and effective way to get the rotation bounding box of objects. Specifically, a refine feature pyramid network is devised with an improved building block constructing top-down feature maps, to solve the problem of large difference in scales. Meanwhile, the position attention network and the channel attention network are jointly explored by modeling the spatial position dependence between global pixels and highlighting the object feature, for detecting small object surrounded by complex background. Extensive experiments on two remote sensing public datasets, DOTA and NWPUVHR -10, show our method to outperform existing leading object detectors in remote sensing field.

Download Full-text

Generating Anchor Boxes Based on Attention Mechanism for Object Detection in Remote Sensing Images

Remote Sensing ◽

10.3390/rs12152416 ◽

2020 ◽

Vol 12 (15) ◽

pp. 2416 ◽

Cited By ~ 1

Author(s):

Zhuangzhuang Tian ◽

Ronghui Zhan ◽

Jiemin Hu ◽

Wei Wang ◽

Zhiqiang He ◽

...

Keyword(s):

Remote Sensing ◽

Object Detection ◽

Attention Mechanism ◽

Detection Methods ◽

Optical Remote Sensing ◽

Remote Sensing Images ◽

Feature Maps ◽

Wide Range ◽

Feature Pyramid ◽

Comprehensive Evaluations

Nowadays, object detection methods based on deep learning are applied more and more to the interpretation of optical remote sensing images. However, the complex background and the wide range of object sizes in remote sensing images increase the difficulty of object detection. In this paper, we improve the detection performance by combining the attention information, and generate adaptive anchor boxes based on the attention map. Specifically, the attention mechanism is introduced into the proposed method to enhance the features of the object regions while reducing the influence of the background. The generated attention map is then used to obtain diverse and adaptable anchor boxes using the guided anchoring method. The generated anchor boxes can match better with the scene and the objects, compared with the traditional proposal boxes. Finally, the modulated feature adaptation module is applied to transform the feature maps to adapt to the diverse anchor boxes. Comprehensive evaluations on the DIOR dataset demonstrate the superiority of the proposed method over the state-of-the-art methods, such as RetinaNet, FCOS and CornerNet. The mean average precision of the proposed method is 4.5% higher than the feature pyramid network. In addition, the ablation experiments are also implemented to further analyze the respective influence of different blocks on the performance improvement.

Download Full-text

YOLO-Fine: One-Stage Detector of Small Objects Under Various Backgrounds in Remote Sensing Images

Remote Sensing ◽

10.3390/rs12152501 ◽

2020 ◽

Vol 12 (15) ◽

pp. 2501 ◽

Cited By ~ 2

Author(s):

Minh-Tan Pham ◽

Luc Courtrai ◽

Chloé Friguet ◽

Sébastien Lefèvre ◽

Alexandre Baussard

Keyword(s):

Remote Sensing ◽

Deep Learning ◽

Object Detection ◽

High Speed ◽

Domain Adaptation ◽

Experimental Studies ◽

Data Availability ◽

Detection Methods ◽

Remote Sensing Images ◽

One Stage

Object detection from aerial and satellite remote sensing images has been an active research topic over the past decade. Thanks to the increase in computational resources and data availability, deep learning-based object detection methods have achieved numerous successes in computer vision, and more recently in remote sensing. However, the ability of current detectors to deal with (very) small objects still remains limited. In particular, the fast detection of small objects from a large observed scene is still an open question. In this work, we address this challenge and introduce an enhanced one-stage deep learning-based detection model, called You Only Look Once (YOLO)-fine, which is based on the structure of YOLOv3. Our detector is designed to be capable of detecting small objects with high accuracy and high speed, allowing further real-time applications within operational contexts. We also investigate its robustness to the appearance of new backgrounds in the validation set, thus tackling the issue of domain adaptation that is critical in remote sensing. Experimental studies that were conducted on both aerial and satellite benchmark datasets show some significant improvement of YOLO-fine as compared to other state-of-the art object detectors.

Download Full-text

Object Detection in Remote Sensing Images Based on Improved Bounding Box Regression and Multi-Level Features Fusion

Remote Sensing ◽

10.3390/rs12010143 ◽

2020 ◽

Vol 12 (1) ◽

pp. 143 ◽

Cited By ~ 6

Author(s):

Xiaoliang Qian ◽

Sheng Lin ◽

Gong Cheng ◽

Xiwen Yao ◽

Hangli Ren ◽

...

Keyword(s):

Remote Sensing ◽

Object Detection ◽

State Of The Art ◽

Remote Sensing Images ◽

Baseline Method ◽

Deep Network ◽

Features Fusion ◽

Bounding Box ◽

Multi Level ◽

Bounding Boxes

The objective of detection in remote sensing images is to determine the location and category of all targets in these images. The anchor based methods are the most prevalent deep learning based methods, and still have some problems that need to be addressed. First, the existing metric (i.e., intersection over union (IoU)) could not measure the distance between two bounding boxes when they are nonoverlapping. Second, the exsiting bounding box regression loss could not directly optimize the metric in the training process. Third, the existing methods which adopt a hierarchical deep network only choose a single level feature layer for the feature extraction of region proposals, meaning they do not take full use of the advantage of multi-level features. To resolve the above problems, a novel object detection method for remote sensing images based on improved bounding box regression and multi-level features fusion is proposed in this paper. First, a new metric named generalized IoU is applied, which can quantify the distance between two bounding boxes, regardless of whether they are overlapping or not. Second, a novel bounding box regression loss is proposed, which can not only optimize the new metric (i.e., generalized IoU) directly but also overcome the problem that existing bounding box regression loss based on the new metric cannot adaptively change the gradient based on the metric value. Finally, a multi-level features fusion module is proposed and incorporated into the existing hierarchical deep network, which can make full use of the multi-level features for each region proposal. The quantitative comparisons between the proposed method and baseline method on the large scale dataset DIOR demonstrate that incorporating the proposed bounding box regression loss, multi-level features fusion module, and a combination of both into the baseline method can obtain an absolute gain of 0.7%, 1.4%, and 2.2% or so in terms of mAP, respectively. Comparing this with the state-of-the-art methods demonstrates that the proposed method has achieved a state-of-the-art performance. The curves of average precision with different thresholds show that the advantage of the proposed method is more evident when the threshold of generalized IoU (or IoU) is relatively high, which means that the proposed method can improve the precision of object localization. Similar conclusions can be obtained on a NWPU VHR-10 dataset.

Download Full-text