Automatic Extraction of Damaged Houses by Earthquake Based on Improved YOLOv5: A Case Study in Yangbi

Efficiently and automatically acquiring information on earthquake damage through remote sensing has posed great challenges because the classical methods of detecting houses damaged by destructive earthquakes are often both time consuming and low in accuracy. A series of deep-learning-based techniques have been developed and recent studies have demonstrated their high intelligence for automatic target extraction for natural and remote sensing images. For the detection of small artificial targets, current studies show that You Only Look Once (YOLO) has a good performance in aerial and Unmanned Aerial Vehicle (UAV) images. However, less work has been conducted on the extraction of damaged houses. In this study, we propose a YOLOv5s-ViT-BiFPN-based neural network for the detection of rural houses. Specifically, to enhance the feature information of damaged houses from the global information of the feature map, we introduce the Vision Transformer into the feature extraction network. Furthermore, regarding the scale differences for damaged houses in UAV images due to the changes in flying height, we apply the Bi-Directional Feature Pyramid Network (BiFPN) for multi-scale feature fusion to aggregate features with different resolutions and test the model. We took the 2021 Yangbi earthquake with a surface wave magnitude (Ms) of 6.4 in Yunan, China, as an example; the results show that the proposed model presents a better performance, with the average precision (AP) being increased by 9.31% and 1.23% compared to YOLOv3 and YOLOv5s, respectively, and a detection speed of 80 FPS, which is 2.96 times faster than YOLOv3. In addition, the transferability test for five other areas showed that the average accuracy was 91.23% and the total processing time was 4 min, while 100 min were needed for professional visual interpreters. The experimental results demonstrate that the YOLOv5s-ViT-BiFPN model can automatically detect damaged rural houses due to destructive earthquakes in UAV images with a good performance in terms of accuracy and timeliness, as well as being robust and transferable.

Download Full-text

AF-EMS Detector: Improve the Multi-Scale Detection Performance of the Anchor-Free Detector

Remote Sensing ◽

10.3390/rs13020160 ◽

2021 ◽

Vol 13 (2) ◽

pp. 160

Author(s):

Jiangqiao Yan ◽

Liangjin Zhao ◽

Wenhui Diao ◽

Hongqi Wang ◽

Xian Sun

Keyword(s):

Remote Sensing ◽

Object Detection ◽

Feature Fusion ◽

Detection Algorithm ◽

Detection Performance ◽

Natural Image ◽

Detection Model ◽

Multi Scale ◽

Feature Pyramid ◽

Scale Detection

As a precursor step for computer vision algorithms, object detection plays an important role in various practical application scenarios. With the objects to be detected becoming more complex, the problem of multi-scale object detection has attracted more and more attention, especially in the field of remote sensing detection. Early convolutional neural network detection algorithms are mostly based on artificially preset anchor-boxes to divide different regions in the image, and then obtain the prior position of the target. However, the anchor box is difficult to set reasonably and will cause a large amount of computational redundancy, which affects the generality of the detection model obtained under fixed parameters. In the past two years, anchor-free detection algorithm has achieved remarkable development in the field of detection on natural image. However, there is no sufficient research on how to deal with multi-scale detection more effectively in anchor-free framework and use these detectors on remote sensing images. In this paper, we propose a specific-attention Feature Pyramid Network (FPN) module, which is able to generate a feature pyramid, basing on the characteristics of objects with various sizes. In addition, this pyramid suits multi-scale object detection better. Besides, a scale-aware detection head is proposed which contains a multi-receptive feature fusion module and a size-based feature compensation module. The new anchor-free detector can obtain a more effective multi-scale feature expression. Experiments on challenging datasets show that our approach performs favorably against other methods in terms of the multi-scale object detection performance.

Download Full-text

UNMANNED AERIAL VEHICLE APPLICATIONS OF 3D MODELLING, VISUALIZATION AND PARAMETER CALCULATIONS

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xlii-5-649-2018 ◽

2018 ◽

Vol XLII-5 ◽

pp. 649-652 ◽

Cited By ~ 1

Author(s):

V. Lambey ◽

A. D. Prasad

Keyword(s):

Point Cloud ◽

Low Cost ◽

3D Modelling ◽

Aerial Photogrammetry ◽

Average Accuracy ◽

Aerial Vehicle ◽

High Resolution Images ◽

Institute Of Technology ◽

Uav Images ◽

Economical Alternative

<p><strong>Abstract.</strong> Photogrammetric surveying with the use of Unmanned Aerial Vehicles (UAV) have gained vast popularity in short span. UAV have the potential to provide imagery at an extraordinary spatial and temporal resolution when coupled with remote sensing. Currently, UAV platforms are fastest and easiest source of data for mapping and 3D modelling. It is to be considered as a low-cost substitute to the traditional airborne photogrammetry. In the present study, UAV applications are explored in terms of 3D modelling, visualization and parameter calculations. National Institute of Technology Raipur, Raipur is chosen as study area and high resolution images are acquired from the UAV with 85% overlap. 3D model is processed through the point cloud generated for the UAV images. The results are compared with traditional methods for validation. The average accuracy obtained for elevation points and area is 97.99% and 97.75%. The study proves that UAV based surveying is an economical alternative in terms of money, time and resources, when compared to the classical aerial photogrammetry methods.</p>

Download Full-text

Semantic segmentation of remote sensing images based on dual attention and multi-scale feature fusion

Twelfth International Conference on Graphics and Image Processing (ICGIP 2020) ◽

10.1117/12.2589380 ◽

2021 ◽

Author(s):

Mengqian Weng ◽

Zhibo Hu ◽

Xiaopeng Xie ◽

Yunhong Li ◽

Lei Hu

Keyword(s):

Remote Sensing ◽

Feature Fusion ◽

Semantic Segmentation ◽

Remote Sensing Images ◽

Scale Feature ◽

Multi Scale

Download Full-text

Multi scale switchable atrous convolution for target detection based on feature pyramid

MATEC Web of Conferences ◽

10.1051/matecconf/202235503011 ◽

2022 ◽

Vol 355 ◽

pp. 03011

Author(s):

Cheng Fang ◽

Ziqiang Hao ◽

Jiaxin Chen

Keyword(s):

Feature Extraction ◽

Field Of View ◽

Convolution Kernel ◽

Data Set ◽

Sample Distribution ◽

Multi Scale ◽

Average Accuracy ◽

Repeated Observation ◽

Feature Pyramid ◽

Low Efficiency

Repeated observation mechanism can effectively solve the problem of low efficiency of feature extraction. By extracting features for many times to strengthen target features, this paper proposed a multi-scale switchable atrous convolution based on feature pyramid, SPC. The head of the detector adopted pyramid convolution mode, constructs 3-D convolution in the feature pyramid, and detected the same target in different pyramid levels by using the shared convolution with different stride changes, which realized the repeated observation of target features on multi-scale. The module optimized the convolution layer, extracted the features of the same image by convolution check of different sizes, and then selected and integrated the extracted results by using switch function, which effectively expanded the field of view of convolution kernel. In this paper, we choosed retinanet as the baseline network, and improved the loss function of focal loss proposed by retinanet to further solved the problem of unbalanced number of samples and sample distribution in the network model. The proposed method performed well on MS coco data set, improved the average accuracy of 9.8% on the basis of retinanet to 48.9%, and achieved FPS of 5.1 in 1333 * 800 images.

Download Full-text

SAR Ship Detection Based on Improved Libra RetinaNet

10.20944/preprints202108.0509.v1 ◽

2021 ◽

Author(s):

Haomiao Liu ◽

Haizhou Xu ◽

Lei Zhang ◽

Weigang Lu ◽

Fei Yang ◽

...

Keyword(s):

Feature Fusion ◽

Weather Conditions ◽

Maritime Transportation ◽

Detection Methods ◽

Sar Images ◽

Style Transfer ◽

Ship Detection ◽

Multi Scale ◽

Optical Images ◽

Average Accuracy

Maritime ship monitoring plays an important role in maritime transportation. Fast and accurate detection of maritime ship is the key to maritime ship monitoring. The main sources of marine ship images are optical images and synthetic aperture radar (SAR) images. Different from natural images, SAR images are independent to daylight and weather conditions. Traditional ship detection methods of SAR images mainly depend on the statistical distribution of sea clutter, which leads to poor robustness. As a deep learning detector, RetinaNet can break this obstacle, and the problem of imbalance on feature level and objective level can be further solved by combining with Libra R-CNN algorithm. In this paper, we modify the feature fusion part of Libra RetinaNet by adding a bottom-up path augmentation structure to better preserve the low-level feature information, and we expand the dataset through style transfer. We evaluate our method on the publicly available SAR dataset of ship detection with complex backgrounds. The experimental results show that the improved Libra RetinaNet can effectively detect multi-scale ships through expansion of the dataset, with an average accuracy of 97.38%.

Download Full-text

Multi-Scale and Occlusion Aware Network for Vehicle Detection and Segmentation on UAV Aerial Images

Remote Sensing ◽

10.3390/rs12111760 ◽

2020 ◽

Vol 12 (11) ◽

pp. 1760 ◽

Cited By ~ 2

Author(s):

Wang Zhang ◽

Chunsheng Liu ◽

Faliang Chang ◽

Ye Song

Keyword(s):

Feature Fusion ◽

Vehicle Detection ◽

Aerial Images ◽

Feature Maps ◽

Vehicle Monitoring ◽

Multi Scale ◽

Adaptive Fusion ◽

Feature Pyramid ◽

Vehicle Information ◽

Vehicle Segmentation

With the advantage of high maneuverability, Unmanned Aerial Vehicles (UAVs) have been widely deployed in vehicle monitoring and controlling. However, processing the images captured by UAV for the extracting vehicle information is hindered by some challenges including arbitrary orientations, huge scale variations and partial occlusion. In seeking to address these challenges, we propose a novel Multi-Scale and Occlusion Aware Network (MSOA-Net) for UAV based vehicle segmentation, which consists of two parts including a Multi-Scale Feature Adaptive Fusion Network (MSFAF-Net) and a Regional Attention based Triple Head Network (RATH-Net). In MSFAF-Net, a self-adaptive feature fusion module is proposed, which can adaptively aggregate hierarchical feature maps from multiple levels to help Feature Pyramid Network (FPN) deal with the scale change of vehicles. The RATH-Net with a self-attention mechanism is proposed to guide the location-sensitive sub-networks to enhance the vehicle of interest and suppress background noise caused by occlusions. In this study, we release a large comprehensive UAV based vehicle segmentation dataset (UVSD), which is the first public dataset for UAV based vehicle detection and segmentation. Experiments are conducted on the challenging UVSD dataset. Experimental results show that the proposed method is efficient in detecting and segmenting vehicles, and outperforms the compared state-of-the-art works.

Download Full-text

Geospatial Object Detection on High Resolution Remote Sensing Imagery Based on Double Multi-Scale Feature Pyramid Network

Remote Sensing ◽

10.3390/rs11070755 ◽

2019 ◽

Vol 11 (7) ◽

pp. 755 ◽

Cited By ~ 20

Author(s):

Xiaodong Zhang ◽

Kun Zhu ◽

Guanzhou Chen ◽

Xiaoliang Tan ◽

Lifei Zhang ◽

...

Keyword(s):

Remote Sensing ◽

High Resolution ◽

Object Detection ◽

Large Scale ◽

Training Data ◽

Validation Dataset ◽

Remote Sensing Imagery ◽

Scale Feature ◽

Multi Scale ◽

Feature Pyramid

Object detection on very-high-resolution (VHR) remote sensing imagery has attracted a lot of attention in the field of image automatic interpretation. Region-based convolutional neural networks (CNNs) have been vastly promoted in this domain, which first generate candidate regions and then accurately classify and locate the objects existing in these regions. However, the overlarge images, the complex image backgrounds and the uneven size and quantity distribution of training samples make the detection tasks more challenging, especially for small and dense objects. To solve these problems, an effective region-based VHR remote sensing imagery object detection framework named Double Multi-scale Feature Pyramid Network (DM-FPN) was proposed in this paper, which utilizes inherent multi-scale pyramidal features and combines the strong-semantic, low-resolution features and the weak-semantic, high-resolution features simultaneously. DM-FPN consists of a multi-scale region proposal network and a multi-scale object detection network, these two modules share convolutional layers and can be trained end-to-end. We proposed several multi-scale training strategies to increase the diversity of training data and overcome the size restrictions of the input images. We also proposed multi-scale inference and adaptive categorical non-maximum suppression (ACNMS) strategies to promote detection performance, especially for small and dense objects. Extensive experiments and comprehensive evaluations on large-scale DOTA dataset demonstrate the effectiveness of the proposed framework, which achieves mean average precision (mAP) value of 0.7927 on validation dataset and the best mAP value of 0.793 on testing dataset.

Download Full-text

Multi-scale Adaptive Feature Fusion Network for Semantic Segmentation in Remote Sensing Images

Remote Sensing ◽

10.3390/rs12050872 ◽

2020 ◽

Vol 12 (5) ◽

pp. 872 ◽

Cited By ~ 3

Author(s):

Ronghua Shang ◽

Jiyu Zhang ◽

Licheng Jiao ◽

Yangyang Li ◽

Naresh Marturi ◽

...

Keyword(s):

Remote Sensing ◽

Multiple Scales ◽

Feature Fusion ◽

Semantic Segmentation ◽

Semantic Features ◽

Remote Sensing Images ◽

Global Features ◽

Global Average ◽

Multi Scale ◽

Context Extraction

Semantic segmentation of high-resolution remote sensing images is highly challenging due to the presence of a complicated background, irregular target shapes, and similarities in the appearance of multiple target categories. Most of the existing segmentation methods that rely only on simple fusion of the extracted multi-scale features often fail to provide satisfactory results when there is a large difference in the target sizes. Handling this problem through multi-scale context extraction and efficient fusion of multi-scale features, in this paper we present an end-to-end multi-scale adaptive feature fusion network (MANet) for semantic segmentation in remote sensing images. It is a coding and decoding structure that includes a multi-scale context extraction module (MCM) and an adaptive fusion module (AFM). The MCM employs two layers of atrous convolutions with different dilatation rates and global average pooling to extract context information at multiple scales in parallel. MANet embeds the channel attention mechanism to fuse semantic features. The high- and low-level semantic information are concatenated to generate global features via global average pooling. These global features are used as channel weights to acquire adaptive weight information of each channel by the fully connected layer. To accomplish an efficient fusion, these tuned weights are applied to the fused features. Performance of the proposed method has been evaluated by comparing it with six other state-of-the-art networks: fully convolutional networks (FCN), U-net, UZ1, Light-weight RefineNet, DeepLabv3+, and APPD. Experiments performed using the publicly available Potsdam and Vaihingen datasets show that the proposed MANet significantly outperforms the other existing networks, with overall accuracy reaching 89.4% and 88.2%, respectively and with average of F1 reaching 90.4% and 86.7% respectively.

Download Full-text