scholarly journals Stereo Dense Image Matching by Adaptive Fusion of Multiple-Window Matching Results

2020 ◽  
Vol 12 (19) ◽  
pp. 3138
Author(s):  
Yilong Han ◽  
Wei Liu ◽  
Xu Huang ◽  
Shugen Wang ◽  
Rongjun Qin
Keyword(s):  
Energy Function ◽  
Image Matching ◽  
Global Energy ◽  
Dense Image ◽  
Adaptive Fusion ◽  
Matching Cost ◽  
Fine Structures ◽  
Matching Window ◽  
Multiple Window ◽  
Image Datasets

Traditional stereo dense image matching (DIM) methods normally predefine a fixed window to compute matching cost, while their performances are limited by the matching window sizes. A large matching window usually achieves robust matching results in weak-textured regions, while it may cause over-smoothness problems in disparity jumps and fine structures. A small window can recover sharp boundaries and fine structures, while it contains high matching uncertainties in weak-textured regions. To address the issue above, we respectively compute matching results with different matching window sizes and then proposes an adaptive fusion method of these matching results so that a better matching result can be generated. The core algorithm designs a Convolutional Neural Network (CNN) to predict the probabilities of large and small windows for each pixel and then refines these probabilities by imposing a global energy function. A compromised solution of the global energy function is utilized by breaking the optimization into sub-optimizations of each pixel in one-dimensional (1D) paths. Finally, the matching results of large and small windows are fused by taking the refined probabilities as weights for more accurate matching. We test our method on aerial image datasets, satellite image datasets, and Middlebury benchmark with different matching cost metrics. Experiments show that our proposed adaptive fusion of multiple-window matching results method has a good transferability across different datasets and outperforms the small windows, the median windows, the large windows, and some state-of-the-art matching window selection methods.

scholarly journals TEXTURE-AWARE DENSE IMAGE MATCHING USING TERNARY CENSUS TRANSFORM

2016 ◽  
Vol III-3 ◽  
pp. 59-66 ◽  
Author(s):  
Han Hu ◽  
Chongtai Chen ◽  
Bo Wu ◽  
Xiaoxia Yang ◽  
Qing Zhu ◽  
...  
Keyword(s):  
Image Matching ◽  
Urban Areas ◽  
Point Clouds ◽  
Parameter Tuning ◽  
Census Transform ◽  
Urban Scenes ◽  
Dense Image ◽  
Global Matching ◽  
Matching Cost ◽  
Sharp Edges

Textureless and geometric discontinuities are major problems in state-of-the-art dense image matching methods, as they can cause visually significant noise and the loss of sharp features. Binary census transform is one of the best matching cost methods but in textureless areas, where the intensity values are similar, it suffers from small random noises. Global optimization for disparity computation is inherently sensitive to parameter tuning in complex urban scenes, and must compromise between smoothness and discontinuities. The aim of this study is to provide a method to overcome these issues in dense image matching, by extending the industry proven Semi-Global Matching through 1) developing a ternary census transform, which takes three outputs in a single order comparison and encodes the results in two bits rather than one, and also 2) by using texture-information to self-tune the parameters, which both preserves sharp edges and enforces smoothness when necessary. Experimental results using various datasets from different platforms have shown that the visual qualities of the triangulated point clouds in urban areas can be largely improved by these proposed methods.

scholarly journals STATE OF THE ART IN DENSE IMAGE MATCHING COST COMPUTATION FOR HIGH-RESOLUTION SATELLITE STEREO

2021 ◽  
Vol XLIII-B2-2021 ◽  
pp. 109-114
Author(s):  
Y. Wang ◽  
D. Gong ◽  
H. Hu ◽  
S. Wang ◽  
Y. Han ◽  
...  
Keyword(s):  
High Resolution ◽  
Image Matching ◽  
Large Scale ◽  
Satellite Image ◽  
Ground Truth ◽  
Surface Model ◽  
Dense Image ◽  
High Resolution Satellite Images ◽  
Matching Cost ◽  
End To End

Abstract. Large-scale Digital Surface Model (DSM) generated with high-resolution satellite images (HRSI) are comparable, cheaper, and more accessible when comparing to Light Detection and Ranging (LiDAR) data and aerial remotely sensed images. Several photogrammetric commercial/open-source software packages are being developed for satellite image-based 3D reconstruction, in which, most of them adopt a modified version of Semi-Global Matching (SGM) algorithm for dense image matching. With the continuous development of matching cost computation methods, the existing methods can be divided into classical (low-level) and learning-based algorithms (non-end-to-end learning and end-to-end learning methods). On Middlebury and KITTI datasets, learning-based algorithms has shown their superiority compared to SGM derived methods. In this context, we assume that matching cost is the key factor of DIM. This paper reviews and evaluates Census Transform, and MC-CNN on a WorldView-3 typical city scene satellite stereo images on the premise that the overall SGM framework remains unchanged, providing a preliminary comparison for academic and industrial. We first compute the cost valume of these two methods, obtains the final DSM after semi-global optimization, and compares their gemetric accuracy with the corresponding LiDAR derived ground truth. We presented our comparison and findings in the experimental section.

scholarly journals TEXTURE-AWARE DENSE IMAGE MATCHING USING TERNARY CENSUS TRANSFORM

2016 ◽  
Vol III-3 ◽  
pp. 59-66
Author(s):  
Han Hu ◽  
Chongtai Chen ◽  
Bo Wu ◽  
Xiaoxia Yang ◽  
Qing Zhu ◽  
...  
Keyword(s):  
Image Matching ◽  
Urban Areas ◽  
Point Clouds ◽  
Parameter Tuning ◽  
Census Transform ◽  
Urban Scenes ◽  
Dense Image ◽  
Global Matching ◽  
Matching Cost ◽  
Sharp Edges

Textureless and geometric discontinuities are major problems in state-of-the-art dense image matching methods, as they can cause visually significant noise and the loss of sharp features. Binary census transform is one of the best matching cost methods but in textureless areas, where the intensity values are similar, it suffers from small random noises. Global optimization for disparity computation is inherently sensitive to parameter tuning in complex urban scenes, and must compromise between smoothness and discontinuities. The aim of this study is to provide a method to overcome these issues in dense image matching, by extending the industry proven Semi-Global Matching through 1) developing a ternary census transform, which takes three outputs in a single order comparison and encodes the results in two bits rather than one, and also 2) by using texture-information to self-tune the parameters, which both preserves sharp edges and enforces smoothness when necessary. Experimental results using various datasets from different platforms have shown that the visual qualities of the triangulated point clouds in urban areas can be largely improved by these proposed methods.

scholarly journals EFFECT OF IMAGE MATCHING WINDOW SIZE ON SATELLITE JITTER FREQUENCY DETECTION

2020 ◽  
Vol XLIII-B1-2020 ◽  
pp. 149-155
Author(s):  
H. Zhang ◽  
S. Liu ◽  
Z. Ye ◽  
X. Tong ◽  
H. Xie ◽  
...  
Keyword(s):  
Image Matching ◽  
Time Lag ◽  
Window Size ◽  
Dynamic Imaging ◽  
Frequency Detection ◽  
Imaging Model ◽  
Dense Image ◽  
Multi Temporal ◽  
Short Time ◽  
Matching Window

Abstract. Satellite attitude jitter is a common and complex phenomenon for high-resolution satellites and it is detectable by multi-temporal image matching. This paper analyses the effect of image matching window size on jitter frequency detection. First, two simulation images with a given short time lag and line scanning frequency affected by a modelled jitter are generated based on the principle of dynamic imaging model. Then, the relative image distortions are obtained by dense image matching with different matching window size and the frequency is estimated through spectrum analysis of the obtained image distortions. The experimental results demonstrated the feasibility and reliability of high frequency jitter detection based on dense image matching, and the results indicated that the maximum detectable frequency is almost not affected by the change of image matching window size, which provided useful demonstration of image-based satellite jitter detection capacity.

scholarly journals A CASE STUDY ON THROUGH-WATER DENSE IMAGE MATCHING

2018 ◽  
Vol XLII-2 ◽  
pp. 659-666 ◽  
Author(s):  
G. Mandlburger
Keyword(s):  
Image Matching ◽  
Water Surface ◽  
Point Clouds ◽  
Water Clarity ◽  
Aerial Images ◽  
Mean Deviation ◽  
Range Image ◽  
Surface Mapping ◽  
Dense Image

In the last years, the tremendous progress in image processing and camera technology has reactivated the interest in photogrammetrybased surface mapping. With the advent of Dense Image Matching (DIM), the derivation of height values on a per-pixel basis became feasible, allowing the derivation of Digital Elevation Models (DEM) with a spatial resolution in the range of the ground sampling distance of the aerial images, which is often below 10 cm today. While mapping topography and vegetation constitutes the primary field of application for image based surface reconstruction, multi-spectral images also allow to see through the water surface to the bottom underneath provided sufficient water clarity. In this contribution, the feasibility of through-water dense image matching for mapping shallow water bathymetry using off-the-shelf software is evaluated. In a case study, the SURE software is applied to three different coastal and inland water bodies. After refraction correction, the DIM point clouds and the DEMs derived thereof are compared to concurrently acquired laser bathymetry data. The results confirm the general suitability of through-water dense image matching, but sufficient bottom texture and favorable environmental conditions (clear water, calm water surface) are a preconditions for achieving accurate results. Water depths of up to 5 m could be mapped with a mean deviation between laser and trough-water DIM in the dm-range. Image based water depth estimates, however, become unreliable in case of turbid or wavy water and poor bottom texture.

scholarly journals AN AUTOMATIC FILTER ALGORITHM FOR DENSE IMAGE MATCHING POINT CLOUDS

2017 ◽  
Vol XLII-2/W7 ◽  
pp. 703-709 ◽  
Author(s):  
Y. Q. Dong ◽  
L. Zhang ◽  
X. M. Cui ◽  
H. B. Ai
Keyword(s):  
Image Matching ◽  
Iterative Process ◽  
Point Clouds ◽  
High Density ◽  
Aerial Images ◽  
Dense Image ◽  
Seed Points ◽  
Filter Algorithms

Although many filter algorithms have been presented over past decades, these algorithms are usually designed for the Lidar point clouds and can’t separate the ground points from the DIM (dense image matching, DIM) point clouds derived from the oblique aerial images owing to the high density and variation of the DIM point clouds completely. To solve this problem, a new automatic filter algorithm is developed on the basis of adaptive TIN models. At first, the differences between Lidar and DIM point clouds which influence the filtering results are analysed in this paper. To avoid the influences of the plants which can’t be penetrated by the DIM point clouds in the searching seed pointes process, the algorithm makes use of the facades of buildings to get ground points located on the roads as seed points and construct the initial TIN. Then a new densification strategy is applied to deal with the problem that the densification thresholds do not change as described in other methods in each iterative process. Finally, we use the DIM point clouds located in Potsdam produced by Photo-Scan to evaluate the method proposed in this paper. The experiment results show that the method proposed in this paper can not only separate the ground points from the DIM point clouds completely but also obtain the better filter results compared with TerraSolid. 1.

scholarly journals THE POSSIBILITY OF USING IMAGES OBTAINED FROM THE UAS IN CADASTRAL WORKS

2016 ◽  
Vol XLI-B1 ◽  
pp. 909-915 ◽  
Author(s):  
Z. Kurczynski ◽  
K. Bakuła ◽  
M. Karabin ◽  
M. Kowalczyk ◽  
J. S. Markiewicz ◽  
...  
Keyword(s):  
Image Matching ◽  
Bundle Adjustment ◽  
Aerial Photographs ◽  
Surface Model ◽  
Digital Surface Model ◽  
Reference Field ◽  
Alternative Source ◽  
Dense Image ◽  
Low Altitude ◽  
Very High

Updating the cadastre requires much work carried out by surveying companies in countries that have still not solved the problem of updating the cadastral data. In terms of the required precision, these works are among the most accurate. This raises the question: to what extent may modern digital photogrammetric methods be useful in this process? The capabilities of photogrammetry have increased significantly after the introduction of digital aerial cameras and digital technologies. For the registration of cadastral objects, i.e., land parcels’ boundaries and the outlines of buildings, very high-resolution aerial photographs can be used. The paper relates an attempt to use an alternative source of data for this task - the development of images acquired from UAS platforms. Multivariate mapping of cadastral parcels was implemented to determine the scope of the suitability of low altitude photos for the cadastre. In this study, images obtained from UAS with the GSD of 3 cm were collected for an area of a few square kilometres. Bundle adjustment of these data was processed with sub-pixel accuracy. This led to photogrammetric measurements being carried out and the provision of an orthophotomap (orthogonalized with a digital surface model from dense image matching of UAS images). Geometric data related to buildings were collected with two methods: stereoscopic and multi-photo measurements. Data related to parcels’ boundaries were measured with monoplotting on an orthophotomap from low-altitude images. As reference field surveying data were used. The paper shows the potential and limits of the use of UAS in a process of updating cadastral data. It also gives recommendations when performing photogrammetric missions and presents the possible accuracy of this type of work.

scholarly journals UNSUPERVISED MULTI-CONSTRAINT DEEP NEURAL NETWORK FOR DENSE IMAGE MATCHING

2020 ◽  
Vol XLIII-B2-2020 ◽  
pp. 163-167
Author(s):  
W. Yuan ◽  
Z. Fan ◽  
X. Yuan ◽  
J. Gong ◽  
R. Shibasaki
Keyword(s):  
Neural Network ◽  
Object Detection ◽  
Image Matching ◽  
Deep Neural Network ◽  
Aerial Images ◽  
Surface Model ◽  
Learning Technology ◽  
Ground Sample ◽  
Volume Calculation ◽  
Dense Image

Abstract. Dense image matching is essential to photogrammetry applications, including Digital Surface Model (DSM) generation, three dimensional (3D) reconstruction, and object detection and recognition. The development of an efficient and robust method for dense image matching has been one of the technical challenges due to high variations in illumination and ground features of aerial images of large areas. Nowadays, due to the development of deep learning technology, deep neural network-based algorithms outperform traditional methods on a variety of tasks such as object detection, semantic segmentation and stereo matching. The proposed network includes cost-volume computation, cost-volume aggregation, and disparity prediction. It starts with a pre-trained VGG-16 network as a backend and using the U-net architecture with nine layers for feature map extraction and a correlation layer for cost volume calculation, after that a guided filter based cost aggregation is adopted for cost volume filtering and finally the soft Argmax function is utilized for disparity prediction. The experimental conducted on a UAV dataset demonstrated that the proposed method achieved the RMSE (root mean square error) of the reprojection error better than 1 pixel in image coordinate and in-ground positioning accuracy within 2.5 ground sample distance. The comparison experiments on KITTI 2015 dataset shows the proposed unsupervised method even comparably with other supervised methods.

scholarly journals AN INTEGRATED PHOTOGRAMMETRIC AND PHOTOCLINOMETRIC APPROACH FOR PIXEL-RESOLUTION 3D MODELLING OF LUNAR SURFACE

2018 ◽  
Vol XLII-3 ◽  
pp. 1117-1122
Author(s):  
W. C. Liu ◽  
B. Wu
Keyword(s):  
High Resolution ◽  
Lunar Surface ◽  
Image Matching ◽  
Point Cloud ◽  
3D Modelling ◽  
Conjugate Points ◽  
Stereo Pair ◽  
Pixel Resolution ◽  
Dense Image ◽  
Lunar Reconnaissance Orbiter

High-resolution 3D modelling of lunar surface is important for lunar scientific research and exploration missions. Photogrammetry is known for 3D mapping and modelling from a pair of stereo images based on dense image matching. However dense matching may fail in poorly textured areas and in situations when the image pair has large illumination differences. As a result, the actual achievable spatial resolution of the 3D model from photogrammetry is limited by the performance of dense image matching. On the other hand, photoclinometry (i.e., shape from shading) is characterised by its ability to recover pixel-wise surface shapes based on image intensity and imaging conditions such as illumination and viewing directions. More robust shape reconstruction through photoclinometry can be achieved by incorporating images acquired under different illumination conditions (i.e., photometric stereo). Introducing photoclinometry into photogrammetric processing can therefore effectively increase the achievable resolution of the mapping result while maintaining its overall accuracy. This research presents an integrated photogrammetric and photoclinometric approach for pixel-resolution 3D modelling of the lunar surface. First, photoclinometry is interacted with stereo image matching to create robust and spatially well distributed dense conjugate points. Then, based on the 3D point cloud derived from photogrammetric processing of the dense conjugate points, photoclinometry is further introduced to derive the 3D positions of the unmatched points and to refine the final point cloud. The approach is able to produce one 3D point for each image pixel within the overlapping area of the stereo pair so that to obtain pixel-resolution 3D models. Experiments using the Lunar Reconnaissance Orbiter Camera - Narrow Angle Camera (LROC NAC) images show the superior performances of the approach compared with traditional photogrammetric technique. The results and findings from this research contribute to optimal exploitation of image information for high-resolution 3D modelling of the lunar surface, which is of significance for the advancement of lunar and planetary mapping.

scholarly journals Area-Based Dense Image Matching with Subpixel Accuracy for Remote Sensing Applications: Practical Analysis and Comparative Study

2020 ◽  
Vol 12 (4) ◽  
pp. 696 ◽  
Author(s):  
Zhen Ye ◽  
Yusheng Xu ◽  
Hao Chen ◽  
Jingwei Zhu ◽  
Xiaohua Tong ◽  
...  
Keyword(s):  
Remote Sensing ◽  
Comparative Study ◽  
Image Matching ◽  
Systematic Errors ◽  
Disparity Estimation ◽  
Sensing Applications ◽  
Subpixel Accuracy ◽  
Dense Image ◽  
Remote Sensing Applications ◽  
Practical Analysis

Dense image matching is a crucial step in many image processing tasks. Subpixel accuracy and fractional measurement are commonly pursued, considering the image resolution and application requirement, especially in the field of remote sensing. In this study, we conducted a practical analysis and comparative study on area-based dense image matching with subpixel accuracy for remote sensing applications, with a specific focus on the subpixel capability and robustness. Twelve representative matching algorithms with two types of correlation-based similarity measures and seven types of subpixel methods were selected. The existing matching algorithms were compared and evaluated in a simulated experiment using synthetic image pairs with varying amounts of aliasing and two real applications of attitude jitter detection and disparity estimation. The experimental results indicate that there are two types of systematic errors: displacement-dependent errors, depending on the fractional values of displacement, and displacement-independent errors represented as unexpected wave artifacts in this study. In addition, the strengths and limitations of different matching algorithms on the robustness to these two types of systematic errors were investigated and discussed.

Sign in / Sign up

Export Citation Format

Share Document