Benchmarking Human Performance for Visual Search of Aerial Images

Aerial images are frequently used in geospatial analysis to inform responses to crises and disasters but can pose unique challenges for visual search when they contain low resolution, degraded information about color, and small object sizes. Aerial image analysis is often performed by humans, but machine learning approaches are being developed to complement manual analysis. To date, however, relatively little work has explored how humans perform visual search on these tasks, and understanding this could ultimately help enable human-machine teaming. We designed a set of studies to understand what features of an aerial image make visual search difficult for humans and what strategies humans use when performing these tasks. Across two experiments, we tested human performance on a counting task with a series of aerial images and examined the influence of features such as target size, location, color, clarity, and number of targets on accuracy and search strategies. Both experiments presented trials consisting of an aerial satellite image; participants were asked to find all instances of a search template in the image. Target size was consistently a significant predictor of performance, influencing not only accuracy of selections but the order in which participants selected target instances in the trial. Experiment 2 demonstrated that the clarity of the target instance and the match between the color of the search template and the color of the target instance also predicted accuracy. Furthermore, color also predicted the order of selecting instances in the trial. These experiments establish not only a benchmark of typical human performance on visual search of aerial images but also identify several features that can influence the task difficulty level for humans. These results have implications for understanding human visual search on real-world tasks and when humans may benefit from automated approaches.

Download Full-text

Multiple-Oriented and Small Object Detection with Convolutional Neural Networks for Aerial Image

Remote Sensing ◽

10.3390/rs11182176 ◽

2019 ◽

Vol 11 (18) ◽

pp. 2176 ◽

Cited By ~ 3

Author(s):

Chen ◽

Zhong ◽

Tan

Keyword(s):

Neural Networks ◽

Object Detection ◽

Convolutional Neural Networks ◽

Aerial Images ◽

Superior Performance ◽

Aerial Image ◽

Detection Accuracy ◽

Small Object ◽

Data Set ◽

Orientation Information

Detecting objects in aerial images is a challenging task due to multiple orientations and relatively small size of the objects. Although many traditional detection models have demonstrated an acceptable performance by using the imagery pyramid and multiple templates in a sliding-window manner, such techniques are inefficient and costly. Recently, convolutional neural networks (CNNs) have successfully been used for object detection, and they have demonstrated considerably superior performance than that of traditional detection methods; however, this success has not been expanded to aerial images. To overcome such problems, we propose a detection model based on two CNNs. One of the CNNs is designed to propose many object-like regions that are generated from the feature maps of multi scales and hierarchies with the orientation information. Based on such a design, the positioning of small size objects becomes more accurate, and the generated regions with orientation information are more suitable for the objects arranged with arbitrary orientations. Furthermore, another CNN is designed for object recognition; it first extracts the features of each generated region and subsequently makes the final decisions. The results of the extensive experiments performed on the vehicle detection in aerial imagery (VEDAI) and overhead imagery research data set (OIRDS) datasets indicate that the proposed model performs well in terms of not only the detection accuracy but also the detection speed.

Download Full-text

Land Cover Mapping from Colorized CORONA Archived Greyscale Satellite Data and Feature Extraction Classification

Land ◽

10.3390/land10080771 ◽

2021 ◽

Vol 10 (8) ◽

pp. 771

Author(s):

Athos Agapiou

Keyword(s):

Land Cover ◽

Urban Areas ◽

Common Ground ◽

Salt Lake ◽

Satellite Image ◽

Aerial Images ◽

Aerial Image ◽

Support Vector ◽

Svm Classifier ◽

Land Cover Mapping

Land cover mapping is often performed via satellite or aerial multispectral/hyperspectral datasets. This paper explores new potentials for the characterisation of land cover from archive greyscale satellite sources by using classification analysis of colourised images. In particular, a CORONA satellite image over Larnaca city in Cyprus was used for this study. The DeOldify Deep learning method embedded in the MyHeritage platform was initially applied to colourise the CORONA image. The new image was then compared against the original greyscale image across various quality metric methods. Then, the geometric correction of the CORONA coloured image was performed using common ground control points taken for aerial images. Later a segmentation process of the image was completed, while segments were selected and characterised for training purposes during the classification process. The latest was performed using the support vector machine (SVM) classifier. Five main land cover classes were selected: land, water, salt lake, vegetation, and urban areas. The overall results of the classification process were then evaluated. The results were very promising (>85 classification accuracy, 0.91 kappa coefficient). The outcomes show that this method can be implemented in any archive greyscale satellite or aerial image to characterise preview landscapes. These results are improved compared to other methods, such as using texture filters.

Download Full-text

HDFNet: Hierarchical Dynamic Fusion Network for Change Detection in Optical Aerial Images

Remote Sensing ◽

10.3390/rs13081440 ◽

2021 ◽

Vol 13 (8) ◽

pp. 1440

Author(s):

Yi Zhang ◽

Lei Fu ◽

Ying Li ◽

Yanning Zhang

Keyword(s):

Deep Learning ◽

Change Detection ◽

Aerial Images ◽

Aerial Image ◽

Sufficient Information ◽

Learning Approaches ◽

Learning Techniques ◽

Benchmark Datasets ◽

Deep Learning Features ◽

Global And Local

Accurate change detection in optical aerial images by using deep learning techniques has been attracting lots of research efforts in recent years. Correct change-detection results usually involve both global and local deep learning features. Existing deep learning approaches have achieved good performance on this task. However, under the scenarios of containing multiscale change areas within a bi-temporal image pair, existing methods still have shortcomings in adapting these change areas, such as false detection and limited completeness in detected areas. To deal with these problems, we design a hierarchical dynamic fusion network (HDFNet) to implement the optical aerial image-change detection task. Specifically, we propose a change-detection framework with hierarchical fusion strategy to provide sufficient information encouraging for change detection and introduce dynamic convolution modules to self-adaptively learn from this information. Also, we use a multilevel supervision strategy with multiscale loss functions to supervise the training process. Comprehensive experiments are conducted on two benchmark datasets, LEBEDEV and LEVIR-CD, to verify the effectiveness of the proposed method and the experimental results show that our model achieves state-of-the-art performance.

Download Full-text

ECAP-YOLO: Efficient Channel Attention Pyramid YOLO for Small Object Detection in Aerial Image

Remote Sensing ◽

10.3390/rs13234851 ◽

2021 ◽

Vol 13 (23) ◽

pp. 4851

Author(s):

Munhyeong Kim ◽

Jongmin Jeong ◽

Sungho Kim

Keyword(s):

Object Detection ◽

High Performance ◽

Difficult Problem ◽

Aerial Images ◽

Aerial Image ◽

Small Object ◽

Detection Technology ◽

Small Ship ◽

Small Object Detection ◽

Small Targets

Detection of small targets in aerial images is still a difficult problem due to the low resolution and background-like targets. With the recent development of object detection technology, efficient and high-performance detector techniques have been developed. Among them, the YOLO series is a representative method of object detection that is light and has good performance. In this paper, we propose a method to improve the performance of small target detection in aerial images by modifying YOLOv5. The backbone is was modified by applying the first efficient channel attention module, and the channel attention pyramid method was proposed. We propose an efficient channel attention pyramid YOLO (ECAP-YOLO). Second, in order to optimize the detection of small objects, we eliminated the module for detecting large objects and added a detect layer to find smaller objects, reducing the computing power used for detecting small targets and improving the detection rate. Finally, we use transposed convolution instead of upsampling. Comparing the method proposed in this paper to the original YOLOv5, the performance improvement for the mAP was 6.9% when using the VEDAI dataset, 5.4% when detecting small cars in the xView dataset, 2.7% when detecting small vehicle and small ship classes from the DOTA dataset, and approximately 2.4% when finding small cars in the Arirang dataset.

Download Full-text

Fig Plant Segmentation from Aerial Images Using a Deep Convolutional Encoder-Decoder Network

Remote Sensing ◽

10.3390/rs11101157 ◽

2019 ◽

Vol 11 (10) ◽

pp. 1157 ◽

Cited By ~ 8

Author(s):

Jorge Fuentes-Pacheco ◽

Juan Torres-Olivares ◽

Edgar Roman-Rangel ◽

Salvador Cervantes ◽

Porfirio Juarez-Lopez ◽

...

Keyword(s):

Precision Agriculture ◽

Image Data ◽

Ground Truth ◽

Aerial Images ◽

Aerial Image ◽

Data Set ◽

Visual Appearance ◽

Aerial Robots ◽

Lighting Conditions ◽

Convolutional Encoder

Crop segmentation is an important task in Precision Agriculture, where the use of aerial robots with an on-board camera has contributed to the development of new solution alternatives. We address the problem of fig plant segmentation in top-view RGB (Red-Green-Blue) images of a crop grown under open-field difficult circumstances of complex lighting conditions and non-ideal crop maintenance practices defined by local farmers. We present a Convolutional Neural Network (CNN) with an encoder-decoder architecture that classifies each pixel as crop or non-crop using only raw colour images as input. Our approach achieves a mean accuracy of 93.85% despite the complexity of the background and a highly variable visual appearance of the leaves. We make available our CNN code to the research community, as well as the aerial image data set and a hand-made ground truth segmentation with pixel precision to facilitate the comparison among different algorithms.

Download Full-text

A Multi-Task Network with Distance–Mask–Boundary Consistency Constraints for Building Extraction from Aerial Images

Remote Sensing ◽

10.3390/rs13142656 ◽

2021 ◽

Vol 13 (14) ◽

pp. 2656

Author(s):

Furong Shi ◽

Tong Zhang

Keyword(s):

Distance Estimation ◽

Image Data ◽

Learning Technologies ◽

Aerial Images ◽

Superior Performance ◽

Aerial Image ◽

Great Success ◽

Building Extraction ◽

Shape Information ◽

Multi Scale

Deep-learning technologies, especially convolutional neural networks (CNNs), have achieved great success in building extraction from areal images. However, shape details are often lost during the down-sampling process, which results in discontinuous segmentation or inaccurate segmentation boundary. In order to compensate for the loss of shape information, two shape-related auxiliary tasks (i.e., boundary prediction and distance estimation) were jointly learned with building segmentation task in our proposed network. Meanwhile, two consistency constraint losses were designed based on the multi-task network to exploit the duality between the mask prediction and two shape-related information predictions. Specifically, an atrous spatial pyramid pooling (ASPP) module was appended to the top of the encoder of a U-shaped network to obtain multi-scale features. Based on the multi-scale features, one regression loss and two classification losses were used for predicting the distance-transform map, segmentation, and boundary. Two inter-task consistency-loss functions were constructed to ensure the consistency between distance maps and masks, and the consistency between masks and boundary maps. Experimental results on three public aerial image data sets showed that our method achieved superior performance over the recent state-of-the-art models.

Download Full-text

Road Detection Based on Edge Feature with GAC Model in Aerial Image

International Journal of Pattern Recognition and Artificial Intelligence ◽

10.1142/s021800142154032x ◽

2021 ◽

Author(s):

Linying Zhou ◽

Zhou Zhou ◽

Hang Ning

Keyword(s):

Transformation Method ◽

Aerial Images ◽

Aerial Image ◽

Initial Contour ◽

Detection Accuracy ◽

Road Detection ◽

Canny Operator ◽

The Road ◽

Gradient Magnitude ◽

Edge Feature

Road detection from aerial images still is a challenging task since it is heavily influenced by spectral reflectance, shadows and occlusions. In order to increase the road detection accuracy, a proposed method for road detection by GAC model with edge feature extraction and segmentation is studied in this paper. First, edge feature can be extracted using the proposed gradient magnitude with Canny operator. Then, a reconstructed gradient map is applied in watershed transformation method, which is segmented for the next initial contour. Last, with the combination of edge feature and initial contour, the boundary stopping function is applied in the GAC model. The road boundary result can be accomplished finally. Experimental results show, by comparing with other methods in [Formula: see text]-measure system, that the proposed method can achieve satisfying results.

Download Full-text

A MODIFIED MULTIPHASE LEVEL SET EVOLUTION SCHEME FOR AERIAL IMAGE SEGMENTATION

International Journal of Pattern Recognition and Artificial Intelligence ◽

10.1142/s0218001407005909 ◽

2007 ◽

Vol 21 (07) ◽

pp. 1195-1212 ◽

Cited By ~ 3

Author(s):

WANG WEI ◽

YANG XIN

Keyword(s):

Level Set ◽

Geometric Analysis ◽

Aerial Images ◽

Contourlet Transform ◽

Aerial Image ◽

Global Features ◽

Precise Shape ◽

Level Set Evolution ◽

Weighting Coefficients ◽

Evolution Scheme

This paper describes an innovative aerial images segmentation algorithm. The algorithm is based upon the knowledge of image multiscale geometric analysis using contourlet transform, which can extract the image's intrinsic geometrical structure efficiently. The contourlet transform is introduced to represent the most distinguished and the rotation invariant features of the image. A modified Mumford–Shah model is applied to segment the aerial image by a multifeature level set evolution. To avoid possible local minima in the level set evolution, we adjust the weighting coefficients of the multiscale features in different evolution periods, i.e. the global features have bigger weighting coefficients at the beginning stages which roughly define the shape of the contour, then bigger weighting coefficients are assigned to the detailed features for segmenting the precise shape. When the algorithm is applied to segment the aerial images with several classes, satisfied experimental results are achieved by the proposed method.

Download Full-text

ANALYZING SPECTRAL CHARACTERISTICS OF SHADOW AREA FROM ADS-40 HIGH RADIOMETRIC RESOLUTION AERIAL IMAGES

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprsarchives-xli-b7-223-2016 ◽

2016 ◽

Vol XLI-B7 ◽

pp. 223-227 ◽

Cited By ~ 2

Author(s):

Yi-Ta Hsieh ◽

Shou-Tsung Wu ◽

Chaur-Tzuhn Chen ◽

Jan-Chang Chen

Keyword(s):

Remote Sensing ◽

Land Cover ◽

Vegetation Index ◽

Spectral Characteristics ◽

Aerial Images ◽

Aerial Image ◽

Optical Remote Sensing ◽

Shadow Area ◽

Vegetation Indexes ◽

Research Findings

The shadows in optical remote sensing images are regarded as image nuisances in numerous applications. The classification and interpretation of shadow area in a remote sensing image are a challenge, because of the reduction or total loss of spectral information in those areas. In recent years, airborne multispectral aerial image devices have been developed 12-bit or higher radiometric resolution data, including Leica ADS-40, Intergraph DMC. The increased radiometric resolution of digital imagery provides more radiometric details of potential use in classification or interpretation of land cover of shadow areas. Therefore, the objectives of this study are to analyze the spectral properties of the land cover in the shadow areas by ADS-40 high radiometric resolution aerial images, and to investigate the spectral and vegetation index differences between the various shadow and non-shadow land covers. According to research findings of spectral analysis of ADS-40 image: (i) The DN values in shadow area are much lower than in nonshadow area; (ii) DN values received from shadowed areas that will also be affected by different land cover, and it shows the possibility of land cover property retrieval as in nonshadow area; (iii) The DN values received from shadowed regions decrease in the visible band from short to long wavelengths due to scattering; (iv) The shadow area NIR of vegetation category also shows a strong reflection; (v) Generally, vegetation indexes (NDVI) still have utility to classify the vegetation and non-vegetation in shadow area. The spectral data of high radiometric resolution images (ADS-40) is potential for the extract land cover information of shadow areas.

Download Full-text

EXPLORING ALS AND DIM DATA FOR SEMANTIC SEGMENTATION USING CNNS

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xlii-1-347-2018 ◽

2018 ◽

Vol XLII-1 ◽

pp. 347-354 ◽

Cited By ~ 5

Author(s):

F. Politz ◽

M. Sester

Keyword(s):

Point Cloud ◽

Laser Scanning ◽

Semantic Segmentation ◽

Point Clouds ◽

Good Alternative ◽

Aerial Images ◽

Learning Approaches ◽

Advantages And Disadvantages ◽

Sensing Applications ◽

High Level

Abstract. Over the past years, the algorithms for dense image matching (DIM) to obtain point clouds from aerial images improved significantly. Consequently, DIM point clouds are now a good alternative to the established Airborne Laser Scanning (ALS) point clouds for remote sensing applications. In order to derive high-level applications such as digital terrain models or city models, each point within a point cloud must be assigned a class label. Usually, ALS and DIM are labelled with different classifiers due to their varying characteristics. In this work, we explore both point cloud types in a fully convolutional encoder-decoder network, which learns to classify ALS as well as DIM point clouds. As input, we project the point clouds onto a 2D image raster plane and calculate the minimal, average and maximal height values for each raster cell. The network then differentiates between the classes ground, non-ground, building and no data. We test our network in six training setups using only one point cloud type, both point clouds as well as several transfer-learning approaches. We quantitatively and qualitatively compare all results and discuss the advantages and disadvantages of all setups. The best network achieves an overall accuracy of 96% in an ALS and 83% in a DIM test set.

Download Full-text