A NEW PARADIGM FOR MATCHING UAV- AND AERIAL IMAGES

This paper investigates the performance of SIFT-based image matching regarding large differences in image scaling and rotation, as this is usually the case when trying to match images captured from UAVs and airplanes. This task represents an essential step for image registration and 3d-reconstruction applications. Various real world examples presented in this paper show that SIFT, as well as A-SIFT perform poorly or even fail in this matching scenario. Even if the scale difference in the images is known and eliminated beforehand, the matching performance suffers from too few feature point detections, ambiguous feature point orientations and rejection of many correct matches when applying the ratio-test afterwards. Therefore, a new feature matching method is provided that overcomes these problems and offers thousands of matches by a novel feature point detection strategy, applying a one-to-many matching scheme and substitute the ratio-test by adding geometric constraints to achieve geometric correct matches at repetitive image regions. This method is designed for matching almost nadir-directed images with low scene depth, as this is typical in UAV and aerial image matching scenarios. We tested the proposed method on different real world image pairs. While standard SIFT failed for most of the datasets, plenty of geometrical correct matches could be found using our approach. Comparing the estimated fundamental matrices and homographies with ground-truth solutions, mean errors of few pixels can be achieved.

Download Full-text

Fig Plant Segmentation from Aerial Images Using a Deep Convolutional Encoder-Decoder Network

Remote Sensing ◽

10.3390/rs11101157 ◽

2019 ◽

Vol 11 (10) ◽

pp. 1157 ◽

Cited By ~ 8

Author(s):

Jorge Fuentes-Pacheco ◽

Juan Torres-Olivares ◽

Edgar Roman-Rangel ◽

Salvador Cervantes ◽

Porfirio Juarez-Lopez ◽

...

Keyword(s):

Precision Agriculture ◽

Image Data ◽

Ground Truth ◽

Aerial Images ◽

Aerial Image ◽

Data Set ◽

Visual Appearance ◽

Aerial Robots ◽

Lighting Conditions ◽

Convolutional Encoder

Crop segmentation is an important task in Precision Agriculture, where the use of aerial robots with an on-board camera has contributed to the development of new solution alternatives. We address the problem of fig plant segmentation in top-view RGB (Red-Green-Blue) images of a crop grown under open-field difficult circumstances of complex lighting conditions and non-ideal crop maintenance practices defined by local farmers. We present a Convolutional Neural Network (CNN) with an encoder-decoder architecture that classifies each pixel as crop or non-crop using only raw colour images as input. Our approach achieves a mean accuracy of 93.85% despite the complexity of the background and a highly variable visual appearance of the leaves. We make available our CNN code to the research community, as well as the aerial image data set and a hand-made ground truth segmentation with pixel precision to facilitate the comparison among different algorithms.

Download Full-text

CRITICAL ASSESSMENT OF OBJECT SEGMENTATION IN AERIAL IMAGE USING GEO-HAUSDORFF DISTANCE

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprsarchives-xli-b4-187-2016 ◽

2016 ◽

Vol XLI-B4 ◽

pp. 187-194 ◽

Cited By ~ 1

Author(s):

H. Sun ◽

Y. Ding ◽

Y. Huang ◽

G. Wang

Keyword(s):

Hausdorff Distance ◽

Object Segmentation ◽

Signal To Noise Ratio ◽

Limited Resource ◽

Ground Truth ◽

Aerial Images ◽

Aerial Image ◽

Spatial Distance ◽

Valuable Insight ◽

Spatial Correspondence

Aerial Image records the large-range earth objects with the ever-improving spatial and radiometric resolution. It becomes a powerful tool for earth observation, land-coverage survey, geographical census, etc., and helps delineating the boundary of different kinds of objects on the earth both manually and automatically. In light of the geo-spatial correspondence between the pixel locations of aerial image and the spatial coordinates of ground objects, there is an increasing need of super-pixel segmentation and high-accuracy positioning of objects in aerial image. Besides the commercial software package of eCognition and ENVI, many algorithms have also been developed in the literature to segment objects of aerial images. But how to evaluate the segmentation results remains a challenge, especially in the context of the geo-spatial correspondence. The Geo-Hausdorff Distance (GHD) is proposed to measure the geo-spatial distance between the results of various object segmentation that can be done with the manual ground truth or with the automatic algorithms.Based on the early-breaking and random-sampling design, the GHD calculates the geographical Hausdorff distance with nearly-linear complexity. Segmentation results of several state-of-the-art algorithms, including those of the commercial packages, are evaluated with a diverse set of aerial images. They have different signal-to-noise ratio around the object boundaries and are hard to trace correctly even for human operators. The GHD value is analyzed to comprehensively measure the suitability of different object segmentation methods for aerial images of different spatial resolution. By critically assessing the strengths and limitations of the existing algorithms, the paper provides valuable insight and guideline for extensive research in automating object detection and classification of aerial image in the nation-wide geographic census. It is also promising for the optimal design of operational specification of remote sensing interpretation under the constraints of limited resource.

Download Full-text

Feature Matching Based on Triangle Guidance and Constraints

International Journal of Pattern Recognition and Artificial Intelligence ◽

10.1142/s0218001418550145 ◽

2018 ◽

Vol 32 (08) ◽

pp. 1855014 ◽

Cited By ~ 2

Author(s):

Hongmin Liu ◽

Hongya Zhang ◽

Zhiheng Wang ◽

Yiming Zheng

Keyword(s):

Feature Matching ◽

High Accuracy ◽

Geometric Constraints ◽

Feature Point ◽

Feature Points ◽

Matching Method ◽

Shared Feature ◽

Triangular Structure ◽

Circle Center ◽

Image Transformations

For images with distortions or repetitive patterns, the existing matching methods usually work well just on one of the two kinds of images. In this paper, we present novel triangle guidance and constraints (TGC)-based feature matching method, which can achieve good results on both kinds of images. We first extract stable matched feature points and combine these points into triangles as the initial matched triangles, and triangles combined by feature points are as the candidates to be matched. Then, triangle guidance based on the connection relationship via the shared feature point between the matched triangles and the candidates is defined to find the potential matching triangles. Triangle constraints, specially the location of a vertex relative to the inscribed circle center of the triangle, the scale represented by the ratio of corresponding side lengths of two matching triangles and the included angles between the sides of two triangles with connection relationship, are subsequently used to verify the potential matches and obtain the correct ones. Comparative experiments show that the proposed TGC can increase the number of the matched points with high accuracy under various image transformations, especially more effective on images with distortions or repetitive patterns due to the fact that the triangular structure are not only stable to image transformations but also provides more geometric constraints.

Download Full-text

Feature Point Matching Method for Aerial Image Based on Recursive Diffusion Algorithm

Symmetry ◽

10.3390/sym13030407 ◽

2021 ◽

Vol 13 (3) ◽

pp. 407

Author(s):

Jiayan Shen ◽

Xiucheng Guo ◽

Wenzong Zhou ◽

Yiming Zhang ◽

Juchen Li

Keyword(s):

Large Scale ◽

High Density ◽

Feature Point ◽

Aerial Images ◽

Image Feature ◽

Aerial Image ◽

Experimental Comparison ◽

Matching Accuracy ◽

Point Matching ◽

Feature Point Matching

Aerial images are large-scale and susceptible to light. Traditional image feature point matching algorithms cannot achieve satisfactory matching accuracy for aerial images. This paper proposes a recursive diffusion algorithm, which is scale-invariant and can be used to extract symmetrical areas of different images. This narrows the matching range of feature points by extracting high-density areas of the image and improving the matching accuracy through correlation analysis of high-density areas. Through experimental comparison, it can be found that the recursive diffusion algorithm has more advantages compared to the correlation coefficient method and the mean shift algorithm when matching accuracy of aerial images, especially when the light of aerial images changes greatly.

Download Full-text

Performance Validation of High Resolution Digital Surface Models Generated by Dense Image Matching with the Aerial Images

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprsarchives-xl-1-429-2014 ◽

2014 ◽

Vol XL-1 ◽

pp. 429-433

Author(s):

N. Yastikli ◽

H. Bayraktar ◽

Z. Erisir

Keyword(s):

High Resolution ◽

Image Matching ◽

Automatic Generation ◽

Aerial Images ◽

Aerial Image ◽

Ground Sample ◽

Dense Image ◽

Surface Models ◽

Performance Validation ◽

Sample Distance

The digital surface models (DSM) are the most popular products to determine visible surface of Earth which includes all non-terrain objects such as vegetation, forest, and man-made constructions. The airborne light detection and ranging (LiDAR) is the preferred technique for high resolution DSM generation in local coverage. The automatic generation of the high resolution DSM is also possible with stereo image matching using the aerial images. The image matching algorithms usually rely on the feature based matching for DSM generation. First, feature points are extracted and then corresponding features are searched in the overlapping images. These image matching algorithms face with the problems in the areas which have repetitive pattern such as urban structure and forest. <br><br> The recent innovation in camera technology and image matching algorithm enabled the automatic dense DSM generation for large scale city and environment modelling. The new pixel-wise matching approaches are generates very high resolution DSMs which corresponds to the ground sample distance (GSD) of the original images. The numbers of the research institutes and photogrammetric software vendors are currently developed software tools for dense DSM generation using the aerial images. This new approach can be used high resolution DSM generation for the larger cities, rural areas and forest even Nation-wide applications. In this study, the performance validation of high resolution DSM generated by pixel-wise dense image matching in part of Istanbul was aimed. The study area in Istanbul is including different land classes such as open areas, forest and built-up areas to test performance of dense image matching in different land classes. The obtained result from this performance validation in Istanbul test area showed that, high resolution DSM which corresponds to the ground sample distance (GSD) of original aerial image can be generated successfully by pixel-wise dense image matching using commercial and research institution’s software.

Download Full-text

Reliable and Efficient UAV Image Matching via Geometric Constraints Structured by Delaunay Triangulation

Remote Sensing ◽

10.3390/rs12203390 ◽

2020 ◽

Vol 12 (20) ◽

pp. 3390

Author(s):

San Jiang ◽

Wanshou Jiang ◽

Lelin Li ◽

Lizhe Wang ◽

Wei Huang

Keyword(s):

Delaunay Triangulation ◽

Image Matching ◽

Feature Matching ◽

Geometric Constraints ◽

Affine Invariant ◽

Two Stage ◽

Outlier Removal ◽

Core Idea ◽

Uav Image ◽

Image Orientation

Outlier removal is a crucial step in local feature-based unmanned aerial vehicle (UAV) image matching. Inspired by our previous work, this paper proposes a method for reliable and efficient outlier removal in UAV image matching. The inputs of the method are only two images without any other auxiliary data. The core idea is to design local geometric constraints within the neighboring structure via the Delaunay triangulation and use a two-stage method for outlier removal and match refinement. In the filter stage, initial matches are first organized as the Delaunay triangulation (DT) and its corresponding graph, and their dissimilarity scores are computed from the affine-invariant spatial angular order (SAO), which is used to achieve hierarchical outlier removal. In addition, by using the triangle constraint between the refined Delaunay triangulation and its corresponding graph, missed inliers are resumed from match expansion. In the verification stage, retained matches are refined using the RANSAC-based global geometric constraint. Therefore, the two-stage algorithm is termed DTSAO-RANSAC. Finally, using four datasets, DTSAO-RANSAC is comprehensively analyzed and compared with other methods in feature matching and image orientation tests. The experimental results demonstrate that compared with the LO-RANSAC algorithm, DTSAO-RANSAC can achieve efficient outlier removal with speedup ratios ranging from 4 to 16 and, it can provide reliable matching results for image orientation of UAV datasets.

Download Full-text

CRITICAL ASSESSMENT OF OBJECT SEGMENTATION IN AERIAL IMAGE USING GEO-HAUSDORFF DISTANCE

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xli-b4-187-2016 ◽

2016 ◽

Vol XLI-B4 ◽

pp. 187-194 ◽

Cited By ~ 1

Author(s):

H. Sun ◽

Y. Ding ◽

Y. Huang ◽

G. Wang

Keyword(s):

Hausdorff Distance ◽

Object Segmentation ◽

Signal To Noise Ratio ◽

Limited Resource ◽

Ground Truth ◽

Aerial Images ◽

Aerial Image ◽

Spatial Distance ◽

Valuable Insight ◽

Spatial Correspondence

Aerial Image records the large-range earth objects with the ever-improving spatial and radiometric resolution. It becomes a powerful tool for earth observation, land-coverage survey, geographical census, etc., and helps delineating the boundary of different kinds of objects on the earth both manually and automatically. In light of the geo-spatial correspondence between the pixel locations of aerial image and the spatial coordinates of ground objects, there is an increasing need of super-pixel segmentation and high-accuracy positioning of objects in aerial image. Besides the commercial software package of eCognition and ENVI, many algorithms have also been developed in the literature to segment objects of aerial images. But how to evaluate the segmentation results remains a challenge, especially in the context of the geo-spatial correspondence. The Geo-Hausdorff Distance (GHD) is proposed to measure the geo-spatial distance between the results of various object segmentation that can be done with the manual ground truth or with the automatic algorithms.Based on the early-breaking and random-sampling design, the GHD calculates the geographical Hausdorff distance with nearly-linear complexity. Segmentation results of several state-of-the-art algorithms, including those of the commercial packages, are evaluated with a diverse set of aerial images. They have different signal-to-noise ratio around the object boundaries and are hard to trace correctly even for human operators. The GHD value is analyzed to comprehensively measure the suitability of different object segmentation methods for aerial images of different spatial resolution. By critically assessing the strengths and limitations of the existing algorithms, the paper provides valuable insight and guideline for extensive research in automating object detection and classification of aerial image in the nation-wide geographic census. It is also promising for the optimal design of operational specification of remote sensing interpretation under the constraints of limited resource.

Download Full-text

Study on Reconstruction and Feature Tracking of Silicone Heart 3D Surface

Sensors ◽

10.3390/s21227570 ◽

2021 ◽

Vol 21 (22) ◽

pp. 7570

Author(s):

Ziyan Zhang ◽

Yan Liu ◽

Jiawei Tian ◽

Shan Liu ◽

Bo Yang ◽

...

Keyword(s):

Surface Reconstruction ◽

Feature Tracking ◽

Feature Matching ◽

Negative Impact ◽

Three Dimensional ◽

Geometric Constraints ◽

Feature Point ◽

Feature Points ◽

3D Surface Reconstruction ◽

3D Surface

At present, feature-based 3D reconstruction and tracking technology is widely applied in the medical field. In minimally invasive surgery, the surgeon can achieve three-dimensional reconstruction through the images obtained by the endoscope in the human body, restore the three-dimensional scene of the area to be operated on, and track the motion of the soft tissue surface. This enables doctors to have a clearer understanding of the location depth of the surgical area, greatly reducing the negative impact of 2D image defects and ensuring smooth operation. In this study, firstly, the 3D coordinates of each feature point are calculated by using the parameters of the parallel binocular endoscope and the spatial geometric constraints. At the same time, the discrete feature points are divided into multiple triangles using the Delaunay triangulation method. Then, the 3D coordinates of feature points and the division results of each triangle are combined to complete the 3D surface reconstruction. Combined with the feature matching method based on convolutional neural network, feature tracking is realized by calculating the three-dimensional coordinate changes of the same feature point in different frames. Finally, experiments are carried out on the endoscope image to complete the 3D surface reconstruction and feature tracking.

Download Full-text

A Method of Insulator Faults Detection in Aerial Images for High-Voltage Transmission Lines Inspection

Applied Sciences ◽

10.3390/app9102009 ◽

2019 ◽

Vol 9 (10) ◽

pp. 2009 ◽

Cited By ~ 7

Author(s):

Jiaming Han ◽

Zhong Yang ◽

Qiuyan Zhang ◽

Cong Chen ◽

Hongchen Li ◽

...

Keyword(s):

Fault Detection ◽

High Voltage ◽

Transmission Lines ◽

Ground Truth ◽

Aerial Images ◽

Aerial Image ◽

Detection Methods ◽

Convolutional Network ◽

High Voltage Transmission ◽

The Impact

Insulator faults detection is an important task for high-voltage transmission line inspection. However, current methods often suffer from the lack of accuracy and robustness. Moreover, these methods can only detect one fault in the insulator string, but cannot detect a multi-fault. In this paper, a novel method is proposed for insulator one fault and multi-fault detection in UAV-based aerial images, the backgrounds of which usually contain much complex interference. The shapes of the insulators also vary obviously due to the changes in filming angle and distance. To reduce the impact of complex interference on insulator faults detection, we make full use of the deep neural network to distinguish between insulators and background interference. First of all, plenty of insulator aerial images with manually labelled ground-truth are collected to construct a standard insulator detection dataset ‘InST_detection’. Secondly, a new convolutional network is proposed to obtain accurate insulator string positions in the aerial image. Finally, a novel fault detection method is proposed that can detect both insulator one fault and multi-fault in aerial images. Experimental results on a large number of aerial images show that our proposed method is more effective and efficient than the state-of-the-art insulator fault detection methods.

Download Full-text