scholarly journals SEMI-GLOBAL MATCHING WITH SELF-ADJUSTING PENALTIES

2017 ◽  
Vol XLII-2/W3 ◽  
pp. 353-360
Author(s):  
E. Karkalou ◽  
C. Stentoumis ◽  
G. Karras
Keyword(s):  
Computational Efficiency ◽  
Image Matching ◽  
3D Models ◽  
Stereo Pair ◽  
Dense Image ◽  
Wide Range ◽  
Global Matching ◽  
Active Research ◽  
Disparity Space Image ◽  
Disparity Space

The demand for 3D models of various scales and precisions is strong for a wide range of applications, among which cultural heritage recording is particularly important and challenging. In this context, dense image matching is a fundamental task for processes which involve image-based reconstruction of 3D models. Despite the existence of commercial software, the need for complete and accurate results under different conditions, as well as for computational efficiency under a variety of hardware, has kept image-matching algorithms as one of the most active research topics. Semi-global matching (SGM) is among the most popular optimization algorithms due to its accuracy, computational efficiency, and simplicity. A challenging aspect in SGM implementation is the determination of smoothness constraints, i.e. penalties P1, P2 for disparity changes and discontinuities. In fact, penalty adjustment is needed for every particular stereo-pair and cost computation. In this work, a novel formulation of <i>self-adjusting penalties</i> is proposed: <i>SGM penalties can be estimated solely from the statistical properties of the initial disparity space image</i>. The proposed method of self-adjusting penalties (SGM-SAP) is evaluated using typical cost functions on stereo-pairs from the recent Middlebury dataset of interior scenes, as well as from the EPFL Herz-Jesu architectural scenes. Results are competitive against the original SGM estimates. The significant aspects of self-adjusting penalties are: (i) the time-consuming tuning process is avoided; (ii) SGM can be used in image collections with limited number of stereo-pairs; and (iii) no heuristic user intervention is needed.

scholarly journals AUTOMATIC DETECTION OF BUILDING POINTS FROM LIDAR AND DENSE IMAGE MATCHING POINT CLOUDS

2015 ◽  
Vol II-3/W5 ◽  
pp. 33-40 ◽  
Author(s):  
E. Maltezos ◽  
C. Ioannidis
Keyword(s):  
Image Matching ◽  
Point Cloud ◽  
Vegetation Index ◽  
Point Clouds ◽  
3D Models ◽  
Northern Greece ◽  
Dense Image ◽  
Global Matching ◽  
Scan Line ◽  
Morphological Operator

This study aims to detect automatically building points: (a) from LIDAR point cloud using simple techniques of filtering that enhance the geometric properties of each point, and (b) from a point cloud which is extracted applying dense image matching at high resolution colour-infrared (CIR) digital aerial imagery using the stereo method semi-global matching (SGM). At first step, the removal of the vegetation is carried out. At the LIDAR point cloud, two different methods are implemented and evaluated using initially the normals and the roughness values afterwards: (1) the proposed scan line smooth filtering and a thresholding process, and (2) a bilateral filtering and a thresholding process. For the case of the CIR point cloud, a variation of the normalized differential vegetation index (NDVI) is computed for the same purpose. Afterwards, the bare-earth is extracted using a morphological operator and removed from the rest scene so as to maintain the buildings points. The results of the extracted buildings applying each approach at an urban area in northern Greece are evaluated using an existing orthoimage as reference; also, the results are compared with the corresponding classified buildings extracted from two commercial software. Finally, in order to verify the utility and functionality of the extracted buildings points that achieved the best accuracy, the 3D models in terms of Level of Detail 1 (LoD 1) and a 3D building change detection process are indicatively performed on a sub-region of the overall scene.

scholarly journals AN INTEGRATED PHOTOGRAMMETRIC AND PHOTOCLINOMETRIC APPROACH FOR PIXEL-RESOLUTION 3D MODELLING OF LUNAR SURFACE

2018 ◽  
Vol XLII-3 ◽  
pp. 1117-1122
Author(s):  
W. C. Liu ◽  
B. Wu
Keyword(s):  
High Resolution ◽  
Lunar Surface ◽  
Image Matching ◽  
Point Cloud ◽  
3D Modelling ◽  
Conjugate Points ◽  
Stereo Pair ◽  
Pixel Resolution ◽  
Dense Image ◽  
Lunar Reconnaissance Orbiter

High-resolution 3D modelling of lunar surface is important for lunar scientific research and exploration missions. Photogrammetry is known for 3D mapping and modelling from a pair of stereo images based on dense image matching. However dense matching may fail in poorly textured areas and in situations when the image pair has large illumination differences. As a result, the actual achievable spatial resolution of the 3D model from photogrammetry is limited by the performance of dense image matching. On the other hand, photoclinometry (i.e., shape from shading) is characterised by its ability to recover pixel-wise surface shapes based on image intensity and imaging conditions such as illumination and viewing directions. More robust shape reconstruction through photoclinometry can be achieved by incorporating images acquired under different illumination conditions (i.e., photometric stereo). Introducing photoclinometry into photogrammetric processing can therefore effectively increase the achievable resolution of the mapping result while maintaining its overall accuracy. This research presents an integrated photogrammetric and photoclinometric approach for pixel-resolution 3D modelling of the lunar surface. First, photoclinometry is interacted with stereo image matching to create robust and spatially well distributed dense conjugate points. Then, based on the 3D point cloud derived from photogrammetric processing of the dense conjugate points, photoclinometry is further introduced to derive the 3D positions of the unmatched points and to refine the final point cloud. The approach is able to produce one 3D point for each image pixel within the overlapping area of the stereo pair so that to obtain pixel-resolution 3D models. Experiments using the Lunar Reconnaissance Orbiter Camera - Narrow Angle Camera (LROC NAC) images show the superior performances of the approach compared with traditional photogrammetric technique. The results and findings from this research contribute to optimal exploitation of image information for high-resolution 3D modelling of the lunar surface, which is of significance for the advancement of lunar and planetary mapping.

scholarly journals TEXTURE-AWARE DENSE IMAGE MATCHING USING TERNARY CENSUS TRANSFORM

2016 ◽  
Vol III-3 ◽  
pp. 59-66 ◽  
Author(s):  
Han Hu ◽  
Chongtai Chen ◽  
Bo Wu ◽  
Xiaoxia Yang ◽  
Qing Zhu ◽  
...  
Keyword(s):  
Image Matching ◽  
Urban Areas ◽  
Point Clouds ◽  
Parameter Tuning ◽  
Census Transform ◽  
Urban Scenes ◽  
Dense Image ◽  
Global Matching ◽  
Matching Cost ◽  
Sharp Edges

Textureless and geometric discontinuities are major problems in state-of-the-art dense image matching methods, as they can cause visually significant noise and the loss of sharp features. Binary census transform is one of the best matching cost methods but in textureless areas, where the intensity values are similar, it suffers from small random noises. Global optimization for disparity computation is inherently sensitive to parameter tuning in complex urban scenes, and must compromise between smoothness and discontinuities. The aim of this study is to provide a method to overcome these issues in dense image matching, by extending the industry proven Semi-Global Matching through 1) developing a ternary census transform, which takes three outputs in a single order comparison and encodes the results in two bits rather than one, and also 2) by using texture-information to self-tune the parameters, which both preserves sharp edges and enforces smoothness when necessary. Experimental results using various datasets from different platforms have shown that the visual qualities of the triangulated point clouds in urban areas can be largely improved by these proposed methods.

scholarly journals PRECISION 3D SURFACE RECONSTRUCTION FROM LRO NAC IMAGES USING SEMI-GLOBAL MATCHING WITH COUPLED EPIPOLAR RECTIFICATION

2017 ◽  
Vol XLII-3/W1 ◽  
pp. 55-61
Author(s):  
H. Hu ◽  
B. Wu
Keyword(s):  
Surface Reconstruction ◽  
Image Matching ◽  
Reference Image ◽  
Stereo Pair ◽  
Stereo Images ◽  
3D Surface Reconstruction ◽  
3D Surface ◽  
Dense Matching ◽  
Lunar Reconnaissance Orbiter ◽  
Global Matching

The Narrow-Angle Camera (NAC) on board the Lunar Reconnaissance Orbiter (LRO) comprises of a pair of closely attached high-resolution push-broom sensors, in order to improve the swath coverage. However, the two image sensors do not share the same lenses and cannot be modelled geometrically using a single physical model. Thus, previous works on dense matching of stereo pairs of NAC images would generally create two to four stereo models, each with an irregular and overlapping region of varying size. Semi-Global Matching (SGM) is a well-known dense matching method and has been widely used for image-based 3D surface reconstruction. SGM is a global matching algorithm relying on global inference in a larger context rather than individual pixels to establish stable correspondences. The stereo configuration of LRO NAC images causes severe problem for image matching methods such as SGM, which emphasizes global matching strategy. Aiming at using SGM for image matching of LRO NAC stereo pairs for precision 3D surface reconstruction, this paper presents a coupled epipolar rectification methods for LRO NAC stereo images, which merges the image pair in the disparity space and in this way, only one stereo model will be estimated. For a stereo pair (four) of NAC images, the method starts with the boresight calibration by finding correspondence in the small overlapping stripe between each pair of NAC images and bundle adjustment of the stereo pair, in order to clean the vertical disparities. Then, the dominate direction of the images are estimated by project the center of the coverage area to the reference image and back-projected to the bounding box plane determined by the image orientation parameters iteratively. The dominate direction will determine an affine model, by which the pair of NAC images are warped onto the object space with a given ground resolution and in the meantime, a mask is produced indicating the owner of each pixel. SGM is then used to generate a disparity map for the stereo pair and each correspondence is transformed back to the owner and 3D points are derived through photogrammetric space intersection. Experimental results reveal that the proposed method is able to reduce gaps and inconsistencies caused by the inaccurate boresight offsets between the two NAC cameras and the irregular overlapping regions, and finally generate precise and consistent 3D surface models from the NAC stereo images automatically.

scholarly journals AUTOMATIC 3D BUILDING RECONSTRUCTION FROM A DENSE IMAGE MATCHING DATASET

2016 ◽  
Vol XLI-B3 ◽  
pp. 641-648 ◽  
Author(s):  
Andrew P. McClune ◽  
Jon P. Mills ◽  
Pauline E. Miller ◽  
David A. Holland
Keyword(s):  
Image Matching ◽  
Three Dimensional ◽  
Stereo Pair ◽  
Building Models ◽  
Dense Image ◽  
Corner Points ◽  
Fitting Procedures ◽  
Recent Developments ◽  
Airborne Sensors ◽  
Building Reconstruction

Over the last 20 years the demand for three dimensional (3D) building models has resulted in a vast amount of research being conducted in attempts to automate the extraction and reconstruction of models from airborne sensors. Recent results have shown that current methods tend to favour planar fitting procedures from lidar data, which are able to successfully reconstruct simple roof structures automatically but fail to reconstruct more complex structures or roofs with small artefacts. Current methods have also not fully explored the potential of recent developments in digital photogrammetry. Large format digital aerial cameras can now capture imagery with increased overlap and a higher spatial resolution, increasing the number of pixel correspondences between images. Every pixel in each stereo pair can also now be matched using per-pixel algorithms, which has given rise to the approach known as dense image matching. This paper presents an approach to 3D building reconstruction to try and overcome some of the limitations of planar fitting procedures. Roof vertices, extracted from true-orthophotos using edge detection, are refined and converted to roof corner points. By determining the connection between extracted corner points, a roof plane can be defined as a closed-cycle of points. Presented results demonstrate the potential of this method for the reconstruction of complex 3D building models at CityGML LoD2 specification.

scholarly journals LOCK-FREE MULTITHREADED SEMI-GLOBAL MATCHING WITH AN ARBITRARY NUMBER OF PATH DIRECTIONS

2020 ◽  
Vol V-2-2020 ◽  
pp. 143-150
Author(s):  
D. Frommholz
Keyword(s):  
Arbitrary Number ◽  
Ground Truth ◽  
Multiple Data ◽  
Reduced Height ◽  
Global Matching ◽  
Path Cost ◽  
Disparity Space Image ◽  
The Cost ◽  
Disparity Space ◽  
Space Image

Abstract. This paper describes an efficient implementation of the semi-global matching (SGM) algorithm on multi-core processors that allows a nearly arbitrary number of path directions for the cost aggregation stage. The scanlines for each orientation are discretized iteratively once, and the regular substructures of the obtained template are reused and shifted to concurrently sum up the path cost in at most two sweeps per direction over the disparity space image. Since path overlaps do not occur at any time, no expensive thread synchronization will be needed. To further reduce the runtime on high counts of path directions, pixel-wise disparity gating is applied, and both the cost function and disparity loop of SGM are optimized using current single instruction multiple data (SIMD) intrinsics for two major CPU architectures. Performance evaluation of the proposed implementation on synthetic ground truth reveals a reduced height error if the number of aggregation directions is significantly increased or when the paths start with an angular offset. Overall runtime shows a speedup that is nearly linear to the number of available processors.

scholarly journals IMPROVING SEMI-GLOBAL MATCHING: COST AGGREGATION AND CONFIDENCE MEASURE

2016 ◽  
Vol XLI-B1 ◽  
pp. 299-304
Author(s):  
Pablo d’Angelo
Keyword(s):  
Belief Propagation ◽  
Stereo Pair ◽  
Confidence Measure ◽  
Remotely Sensed Imagery ◽  
Cost Aggregation ◽  
Digital Elevation ◽  
Dense Image ◽  
Global Matching ◽  
Matching Cost ◽  
Data Term

Digital elevation models are one of the basic products that can be generated from remotely sensed imagery. The Semi Global Matching (SGM) algorithm is a robust and practical algorithm for dense image matching. The connection between SGM and Belief Propagation was recently developed, and based on that improvements such as correction of over-counting the data term, and a new confidence measure have been proposed. Later the MGM algorithm has been proposed, it aims at improving the regularization step of SGM, but has only been evaluated on the Middlebury stereo benchmark so far. This paper evaluates these proposed improvements on the ISPRS satellite stereo benchmark, using a Pleiades Triplet and a Cartosat-1 Stereo pair. The over-counting correction slightly improves matching density, at the expense of adding a few outliers. The MGM cost aggregation shows leads to a slight increase of accuracy.

scholarly journals TEXTURE-AWARE DENSE IMAGE MATCHING USING TERNARY CENSUS TRANSFORM

2016 ◽  
Vol III-3 ◽  
pp. 59-66
Author(s):  
Han Hu ◽  
Chongtai Chen ◽  
Bo Wu ◽  
Xiaoxia Yang ◽  
Qing Zhu ◽  
...  
Keyword(s):  
Image Matching ◽  
Urban Areas ◽  
Point Clouds ◽  
Parameter Tuning ◽  
Census Transform ◽  
Urban Scenes ◽  
Dense Image ◽  
Global Matching ◽  
Matching Cost ◽  
Sharp Edges

Textureless and geometric discontinuities are major problems in state-of-the-art dense image matching methods, as they can cause visually significant noise and the loss of sharp features. Binary census transform is one of the best matching cost methods but in textureless areas, where the intensity values are similar, it suffers from small random noises. Global optimization for disparity computation is inherently sensitive to parameter tuning in complex urban scenes, and must compromise between smoothness and discontinuities. The aim of this study is to provide a method to overcome these issues in dense image matching, by extending the industry proven Semi-Global Matching through 1) developing a ternary census transform, which takes three outputs in a single order comparison and encodes the results in two bits rather than one, and also 2) by using texture-information to self-tune the parameters, which both preserves sharp edges and enforces smoothness when necessary. Experimental results using various datasets from different platforms have shown that the visual qualities of the triangulated point clouds in urban areas can be largely improved by these proposed methods.

scholarly journals IMPROVING SEMI-GLOBAL MATCHING: COST AGGREGATION AND CONFIDENCE MEASURE

2016 ◽  
Vol XLI-B1 ◽  
pp. 299-304 ◽  
Author(s):  
Pablo d’Angelo
Keyword(s):  
Belief Propagation ◽  
Stereo Pair ◽  
Confidence Measure ◽  
Remotely Sensed Imagery ◽  
Cost Aggregation ◽  
Digital Elevation ◽  
Dense Image ◽  
Global Matching ◽  
Matching Cost ◽  
Data Term

Digital elevation models are one of the basic products that can be generated from remotely sensed imagery. The Semi Global Matching (SGM) algorithm is a robust and practical algorithm for dense image matching. The connection between SGM and Belief Propagation was recently developed, and based on that improvements such as correction of over-counting the data term, and a new confidence measure have been proposed. Later the MGM algorithm has been proposed, it aims at improving the regularization step of SGM, but has only been evaluated on the Middlebury stereo benchmark so far. This paper evaluates these proposed improvements on the ISPRS satellite stereo benchmark, using a Pleiades Triplet and a Cartosat-1 Stereo pair. The over-counting correction slightly improves matching density, at the expense of adding a few outliers. The MGM cost aggregation shows leads to a slight increase of accuracy.

Sign in / Sign up

Export Citation Format

Share Document