Fast and Accurate Object Detection in Remote Sensing Images Based on Lightweight Deep Neural Network

Deep learning-based object detection in remote sensing images is an important yet challenging task due to a series of difficulties, such as complex geometry scene, dense target quantity, and large variant in object distributions and scales. Moreover, algorithm designers also have to make a trade-off between model’s complexity and accuracy to meet the real-world deployment requirements. To deal with these challenges, we proposed a lightweight YOLO-like object detector with the ability to detect objects in remote sensing images with high speed and high accuracy. The detector is constructed with efficient channel attention layers to improve the channel information sensitivity. Differential evolution was also developed to automatically find the optimal anchor configurations to address issue of large variant in object scales. Comprehensive experiment results show that the proposed network outperforms state-of-the-art lightweight models by 5.13% and 3.58% in accuracy on the RSOD and DIOR dataset, respectively. The deployed model on an NVIDIA Jetson Xavier NX embedded board can achieve a detection speed of 58 FPS with less than 10W power consumption, which makes the proposed detector very suitable for low-cost low-power remote sensing application scenarios.

Download Full-text

IoU-Adaptive Deformable R-CNN: Make Full Use of IoU for Multi-Class Object Detection in Remote Sensing Imagery

Remote Sensing ◽

10.3390/rs11030286 ◽

2019 ◽

Vol 11 (3) ◽

pp. 286 ◽

Cited By ~ 24

Author(s):

Jiangqiao Yan ◽

Hongqi Wang ◽

Menglong Yan ◽

Wenhui Diao ◽

Xian Sun ◽

...

Keyword(s):

Neural Network ◽

Remote Sensing ◽

Object Detection ◽

Convolutional Neural Network ◽

State Of The Art ◽

Ground Truth ◽

Detection Performance ◽

Candidate Region ◽

Detection Accuracy ◽

Remote Sensing Images

Recently, methods based on Faster region-based convolutional neural network (R-CNN)have been popular in multi-class object detection in remote sensing images due to their outstandingdetection performance. The methods generally propose candidate region of interests (ROIs) througha region propose network (RPN), and the regions with high enough intersection-over-union (IoU)values against ground truth are treated as positive samples for training. In this paper, we find thatthe detection result of such methods is sensitive to the adaption of different IoU thresholds. Specially,detection performance of small objects is poor when choosing a normal higher threshold, while alower threshold will result in poor location accuracy caused by a large quantity of false positives.To address the above issues, we propose a novel IoU-Adaptive Deformable R-CNN framework formulti-class object detection. Specially, by analyzing the different roles that IoU can play in differentparts of the network, we propose an IoU-guided detection framework to reduce the loss of small objectinformation during training. Besides, the IoU-based weighted loss is designed, which can learn theIoU information of positive ROIs to improve the detection accuracy effectively. Finally, the class aspectratio constrained non-maximum suppression (CARC-NMS) is proposed, which further improves theprecision of the results. Extensive experiments validate the effectiveness of our approach and weachieve state-of-the-art detection performance on the DOTA dataset.

Download Full-text

Compact Cloud Detection with Bidirectional Self-Attention Knowledge Distillation

Remote Sensing ◽

10.3390/rs12172770 ◽

2020 ◽

Vol 12 (17) ◽

pp. 2770 ◽

Cited By ~ 1

Author(s):

Yajie Chai ◽

Kun Fu ◽

Xian Sun ◽

Wenhui Diao ◽

Zhiyuan Yan ◽

...

Keyword(s):

Remote Sensing ◽

Large Scale ◽

Semantic Information ◽

State Of The Art ◽

High Accuracy ◽

Compact Model ◽

Cloud Detection ◽

Significant Progress ◽

Remote Sensing Images ◽

Knowledge Distillation

The deep convolutional neural network has made significant progress in cloud detection. However, the compromise between having a compact model and high accuracy has always been a challenging task in cloud detection for large-scale remote sensing imagery. A promising method to tackle this problem is knowledge distillation, which usually lets the compact model mimic the cumbersome model’s output to get better generalization. However, vanilla knowledge distillation methods cannot properly distill the characteristics of clouds in remote sensing images. In this paper, we propose a novel self-attention knowledge distillation approach for compact and accurate cloud detection, named Bidirectional Self-Attention Distillation (Bi-SAD). Bi-SAD lets a model learn from itself without adding additional parameters or supervision. With bidirectional layer-wise features learning, the model can get a better representation of the cloud’s textural information and semantic information, so that the cloud’s boundaries become more detailed and the predictions become more reliable. Experiments on a dataset acquired by GaoFen-1 satellite show that our Bi-SAD has a great balance between compactness and accuracy, and outperforms vanilla distillation methods. Compared with state-of-the-art cloud detection models, the parameter size and FLOPs are reduced by 100 times and 400 times, respectively, with a small drop in accuracy.

Download Full-text

An Oil Well Dataset Derived from Satellite-Based Remote Sensing

Remote Sensing ◽

10.3390/rs13061132 ◽

2021 ◽

Vol 13 (6) ◽

pp. 1132

Author(s):

Zhibao Wang ◽

Lu Bai ◽

Guangfu Song ◽

Jie Zhang ◽

Jinhua Tao ◽

...

Keyword(s):

Remote Sensing ◽

Deep Learning ◽

Object Detection ◽

State Of The Art ◽

Google Earth ◽

Oil Wells ◽

Oil Well ◽

Optical Remote Sensing ◽

Learning Models ◽

Remote Sensing Images

Estimation of the number and geo-location of oil wells is important for policy holders considering their impact on energy resource planning. With the recent development in optical remote sensing, it is possible to identify oil wells from satellite images. Moreover, the recent advancement in deep learning frameworks for object detection in remote sensing makes it possible to automatically detect oil wells from remote sensing images. In this paper, we collected a dataset named Northeast Petroleum University–Oil Well Object Detection Version 1.0 (NEPU–OWOD V1.0) based on high-resolution remote sensing images from Google Earth Imagery. Our database includes 1192 oil wells in 432 images from Daqing City, which has the largest oilfield in China. In this study, we compared nine different state-of-the-art deep learning models based on algorithms for object detection from optical remote sensing images. Experimental results show that the state-of-the-art deep learning models achieve high precision on our collected dataset, which demonstrate the great potential for oil well detection in remote sensing.

Download Full-text

Detection of Schools in Remote Sensing Images Based on Attention-Guided Dense Network

ISPRS International Journal of Geo-Information ◽

10.3390/ijgi10110736 ◽

2021 ◽

Vol 10 (11) ◽

pp. 736

Author(s):

Han Fu ◽

Xiangtao Fan ◽

Zhenzhen Yan ◽

Xiaoping Du

Keyword(s):

Remote Sensing ◽

Object Detection ◽

Feature Fusion ◽

State Of The Art ◽

Feature Representation ◽

Detection Accuracy ◽

Dense Network ◽

Remote Sensing Images ◽

Composite Object ◽

Detection Algorithms

The detection of primary and secondary schools (PSSs) is a meaningful task for composite object detection in remote sensing images (RSIs). As a typical composite object in RSIs, PSSs have diverse appearances with complex backgrounds, which makes it difficult to effectively extract their features using the existing deep-learning-based object detection algorithms. Aiming at the challenges of PSSs detection, we propose an end-to-end framework called the attention-guided dense network (ADNet), which can effectively improve the detection accuracy of PSSs. First, a dual attention module (DAM) is designed to enhance the ability in representing complex characteristics and alleviate distractions in the background. Second, a dense feature fusion module (DFFM) is built to promote attention cues flow into low layers, which guides the generation of hierarchical feature representation. Experimental results demonstrate that our proposed method outperforms the state-of-the-art methods and achieves 79.86% average precision. The study proves the effectiveness of our proposed method on PSSs detection.

Download Full-text

YOLO-Fine: One-Stage Detector of Small Objects Under Various Backgrounds in Remote Sensing Images

Remote Sensing ◽

10.3390/rs12152501 ◽

2020 ◽

Vol 12 (15) ◽

pp. 2501 ◽

Cited By ~ 2

Author(s):

Minh-Tan Pham ◽

Luc Courtrai ◽

Chloé Friguet ◽

Sébastien Lefèvre ◽

Alexandre Baussard

Keyword(s):

Remote Sensing ◽

Deep Learning ◽

Object Detection ◽

High Speed ◽

Domain Adaptation ◽

Experimental Studies ◽

Data Availability ◽

Detection Methods ◽

Remote Sensing Images ◽

One Stage

Object detection from aerial and satellite remote sensing images has been an active research topic over the past decade. Thanks to the increase in computational resources and data availability, deep learning-based object detection methods have achieved numerous successes in computer vision, and more recently in remote sensing. However, the ability of current detectors to deal with (very) small objects still remains limited. In particular, the fast detection of small objects from a large observed scene is still an open question. In this work, we address this challenge and introduce an enhanced one-stage deep learning-based detection model, called You Only Look Once (YOLO)-fine, which is based on the structure of YOLOv3. Our detector is designed to be capable of detecting small objects with high accuracy and high speed, allowing further real-time applications within operational contexts. We also investigate its robustness to the appearance of new backgrounds in the validation set, thus tackling the issue of domain adaptation that is critical in remote sensing. Experimental studies that were conducted on both aerial and satellite benchmark datasets show some significant improvement of YOLO-fine as compared to other state-of-the art object detectors.

Download Full-text

Multi-Sector Oriented Object Detector for Accurate Localization in Optical Remote Sensing Images

Remote Sensing ◽

10.3390/rs13101921 ◽

2021 ◽

Vol 13 (10) ◽

pp. 1921

Author(s):

Xu He ◽

Shiping Ma ◽

Linyuan He ◽

Le Ru ◽

Chen Wang

Keyword(s):

Remote Sensing ◽

Object Detection ◽

State Of The Art ◽

Optical Remote Sensing ◽

Remote Sensing Images ◽

Accurate Localization ◽

Angular Sector ◽

Detection Score ◽

Four Quadrant ◽

Oriented Object

Oriented object detection in optical remote sensing images (ORSIs) is a challenging task since the targets in ORSIs are displayed in an arbitrarily oriented manner and on small scales, and are densely packed. Current state-of-the-art oriented object detection models used in ORSIs primarily evolved from anchor-based and direct regression-based detection paradigms. Nevertheless, they still encounter a design difficulty from handcrafted anchor definitions and learning complexities in direct localization regression. To tackle these issues, in this paper, we proposed a novel multi-sector oriented object detection framework called MSO2-Det, which quantizes the scales and orientation prediction of targets in ORSIs via an anchor-free classification-to-regression approach. Specifically, we first represented the arbitrarily oriented bounding box as four scale offsets and angles in four quadrant sectors of the corresponding Cartesian coordinate system. Then, we divided the scales and angle space into multiple discrete sectors and obtained more accurate localization information by a coarse-granularity classification to fine-grained regression strategy. In addition, to decrease the angular-sector classification loss and accelerate the network’s convergence, we designed a smooth angular-sector label (SASL) that smoothly distributes label values with a definite tolerance radius. Finally, we proposed a localization-aided detection score (LADS) to better represent the confidence of a detected box by combining the category-classification score and the sector-selection score. The proposed MSO2-Det achieves state-of-the-art results on three widely used benchmarks, including the DOTA, HRSC2016, and UCAS-AOD data sets.

Download Full-text

Object Detection in Remote Sensing Images Based on Improved Bounding Box Regression and Multi-Level Features Fusion

Remote Sensing ◽

10.3390/rs12010143 ◽

2020 ◽

Vol 12 (1) ◽

pp. 143 ◽

Cited By ~ 6

Author(s):

Xiaoliang Qian ◽

Sheng Lin ◽

Gong Cheng ◽

Xiwen Yao ◽

Hangli Ren ◽

...

Keyword(s):

Remote Sensing ◽

Object Detection ◽

State Of The Art ◽

Remote Sensing Images ◽

Baseline Method ◽

Deep Network ◽

Features Fusion ◽

Bounding Box ◽

Multi Level ◽

Bounding Boxes

The objective of detection in remote sensing images is to determine the location and category of all targets in these images. The anchor based methods are the most prevalent deep learning based methods, and still have some problems that need to be addressed. First, the existing metric (i.e., intersection over union (IoU)) could not measure the distance between two bounding boxes when they are nonoverlapping. Second, the exsiting bounding box regression loss could not directly optimize the metric in the training process. Third, the existing methods which adopt a hierarchical deep network only choose a single level feature layer for the feature extraction of region proposals, meaning they do not take full use of the advantage of multi-level features. To resolve the above problems, a novel object detection method for remote sensing images based on improved bounding box regression and multi-level features fusion is proposed in this paper. First, a new metric named generalized IoU is applied, which can quantify the distance between two bounding boxes, regardless of whether they are overlapping or not. Second, a novel bounding box regression loss is proposed, which can not only optimize the new metric (i.e., generalized IoU) directly but also overcome the problem that existing bounding box regression loss based on the new metric cannot adaptively change the gradient based on the metric value. Finally, a multi-level features fusion module is proposed and incorporated into the existing hierarchical deep network, which can make full use of the multi-level features for each region proposal. The quantitative comparisons between the proposed method and baseline method on the large scale dataset DIOR demonstrate that incorporating the proposed bounding box regression loss, multi-level features fusion module, and a combination of both into the baseline method can obtain an absolute gain of 0.7%, 1.4%, and 2.2% or so in terms of mAP, respectively. Comparing this with the state-of-the-art methods demonstrates that the proposed method has achieved a state-of-the-art performance. The curves of average precision with different thresholds show that the advantage of the proposed method is more evident when the threshold of generalized IoU (or IoU) is relatively high, which means that the proposed method can improve the precision of object localization. Similar conclusions can be obtained on a NWPU VHR-10 dataset.

Download Full-text