TEXTURE-AWARE DENSE IMAGE MATCHING USING TERNARY CENSUS TRANSFORM

Textureless and geometric discontinuities are major problems in state-of-the-art dense image matching methods, as they can cause visually significant noise and the loss of sharp features. Binary census transform is one of the best matching cost methods but in textureless areas, where the intensity values are similar, it suffers from small random noises. Global optimization for disparity computation is inherently sensitive to parameter tuning in complex urban scenes, and must compromise between smoothness and discontinuities. The aim of this study is to provide a method to overcome these issues in dense image matching, by extending the industry proven Semi-Global Matching through 1) developing a ternary census transform, which takes three outputs in a single order comparison and encodes the results in two bits rather than one, and also 2) by using texture-information to self-tune the parameters, which both preserves sharp edges and enforces smoothness when necessary. Experimental results using various datasets from different platforms have shown that the visual qualities of the triangulated point clouds in urban areas can be largely improved by these proposed methods.

Download Full-text

QUALITY ASPECTS OF TRUE ORTHOPHOTO IN URBAN AREAS

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xliii-b5-2020-191-2020 ◽

2020 ◽

Vol XLIII-B5-2020 ◽

pp. 191-198

Author(s):

W. Ostrowski ◽

V. D. Gulli ◽

K. Bakula ◽

Z. Kurczyński

Keyword(s):

Image Matching ◽

Urban Areas ◽

Quantitative Methods ◽

Quality Evaluation ◽

Point Clouds ◽

Dense Image ◽

Quality Aspects ◽

Surface Models ◽

Recent Developments ◽

2D Data

Abstract. Orthophotos are one of the most popular photogrammetric products and have been a leading source of up-to-date 2D data of urban areas for years. In the last few years, together with innovations in the area of Dense Image Matching, Digital Surface Models created with dense image matching start to be utilized as the height source during orthorectification. Recently this production workflow of true orthophotos were adopted to production standard in many countries. The aim of the presented research was to evaluate recent developments in the area of automatic true orthophoto generation for urban areas and to define factors which have the main influence on the quality of the final product. Obtained results showed that besides of the image overlap, the main factors which have direct influence on the resulted true orthophoto are the occurrence of shadows and vegetation (trees). One of the outcomes of the presented research was that the quantitative methods develop for quality evaluation of Digital Surface Models and Point Clouds are not directly transferable on the quality evaluation of true orthophotos.

Download Full-text

LIDAR vs dense image matching point clouds in complex urban scenes

10.1117/12.2240475 ◽

2016 ◽

Cited By ~ 1

Author(s):

Evangelos Maltezos ◽

Athanasia Kyrkou ◽

Charalabos Ioannidis

Keyword(s):

Image Matching ◽

Point Clouds ◽

Urban Scenes ◽

Dense Image

Download Full-text

SEMANTIC SEGMENTATION OF BUILDING IN AIRBORNE IMAGES

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xlii-2-w13-35-2019 ◽

2019 ◽

Vol XLII-2/W13 ◽

pp. 35-42 ◽

Cited By ~ 1

Author(s):

S. Huang ◽

F. Nex ◽

Y. Lin ◽

M. Y. Yang

Keyword(s):

Neural Networks ◽

Image Matching ◽

Urban Areas ◽

Deep Neural Networks ◽

Semantic Information ◽

Semantic Segmentation ◽

Point Clouds ◽

Dense Image ◽

2D And 3D ◽

Rgb Image

Abstract. Building is a key component to the reconstructing of LoD3 city modelling. Compared to terrestrial view, airborne datasets have more occlusions at street level but can cover larger area in the urban areas. With the popularity of the Deep Learning, many tasks in the field of computer vision can be solved in easier and efficiency way. In this paper, we propose a method to apply deep neural networks to building façade segmentation. In particular, the FC-DenseNet and the DeepLabV3+ algorithms are used to segment the building from airborne images and get semantic information such as, wall, roof, balcony and opening area. The patch-wise segmentation is used in the training and testing process in order to get information at pixel level. Different typologies of input have been considered: beside the conventional 2D information (i.e. RGB image), we combined 2D information with 3D features extracted from dense image matching point clouds to improve the performance of the segmentation. Results show that FC-DenseNet trained with 2D and 3D features achieves the best result, IoU up to 64.41%, it increases 5.13% compared to the result of the same model trained without 3D features.

Download Full-text

AUTOMATIC DETECTION OF BUILDING POINTS FROM LIDAR AND DENSE IMAGE MATCHING POINT CLOUDS

ISPRS Annals of Photogrammetry Remote Sensing and Spatial Information Sciences ◽

10.5194/isprsannals-ii-3-w5-33-2015 ◽

2015 ◽

Vol II-3/W5 ◽

pp. 33-40 ◽

Cited By ~ 10

Author(s):

E. Maltezos ◽

C. Ioannidis

Keyword(s):

Image Matching ◽

Point Cloud ◽

Vegetation Index ◽

Point Clouds ◽

3D Models ◽

Northern Greece ◽

Dense Image ◽

Global Matching ◽

Scan Line ◽

Morphological Operator

This study aims to detect automatically building points: (a) from LIDAR point cloud using simple techniques of filtering that enhance the geometric properties of each point, and (b) from a point cloud which is extracted applying dense image matching at high resolution colour-infrared (CIR) digital aerial imagery using the stereo method semi-global matching (SGM). At first step, the removal of the vegetation is carried out. At the LIDAR point cloud, two different methods are implemented and evaluated using initially the normals and the roughness values afterwards: (1) the proposed scan line smooth filtering and a thresholding process, and (2) a bilateral filtering and a thresholding process. For the case of the CIR point cloud, a variation of the normalized differential vegetation index (NDVI) is computed for the same purpose. Afterwards, the bare-earth is extracted using a morphological operator and removed from the rest scene so as to maintain the buildings points. The results of the extracted buildings applying each approach at an urban area in northern Greece are evaluated using an existing orthoimage as reference; also, the results are compared with the corresponding classified buildings extracted from two commercial software. Finally, in order to verify the utility and functionality of the extracted buildings points that achieved the best accuracy, the 3D models in terms of Level of Detail 1 (LoD 1) and a 3D building change detection process are indicatively performed on a sub-region of the overall scene.

Download Full-text

A CASE STUDY ON THROUGH-WATER DENSE IMAGE MATCHING

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xlii-2-659-2018 ◽

2018 ◽

Vol XLII-2 ◽

pp. 659-666 ◽

Cited By ~ 5

Author(s):

G. Mandlburger

Keyword(s):

Image Matching ◽

Water Surface ◽

Point Clouds ◽

Water Clarity ◽

Aerial Images ◽

Mean Deviation ◽

Range Image ◽

Surface Mapping ◽

Dense Image

In the last years, the tremendous progress in image processing and camera technology has reactivated the interest in photogrammetrybased surface mapping. With the advent of Dense Image Matching (DIM), the derivation of height values on a per-pixel basis became feasible, allowing the derivation of Digital Elevation Models (DEM) with a spatial resolution in the range of the ground sampling distance of the aerial images, which is often below 10&thinsp;cm today. While mapping topography and vegetation constitutes the primary field of application for image based surface reconstruction, multi-spectral images also allow to see through the water surface to the bottom underneath provided sufficient water clarity. In this contribution, the feasibility of through-water dense image matching for mapping shallow water bathymetry using off-the-shelf software is evaluated. In a case study, the SURE software is applied to three different coastal and inland water bodies. After refraction correction, the DIM point clouds and the DEMs derived thereof are compared to concurrently acquired laser bathymetry data. The results confirm the general suitability of through-water dense image matching, but sufficient bottom texture and favorable environmental conditions (clear water, calm water surface) are a preconditions for achieving accurate results. Water depths of up to 5&thinsp;m could be mapped with a mean deviation between laser and trough-water DIM in the dm-range. Image based water depth estimates, however, become unreliable in case of turbid or wavy water and poor bottom texture.

Download Full-text

AN IMPROVED AUTOMATIC POINTWISE SEMANTIC SEGMENTATION OF A 3D URBAN SCENE FROM MOBILE TERRESTRIAL AND AIRBORNE LIDAR POINT CLOUDS: A MACHINE LEARNING APPROACH

ISPRS Annals of Photogrammetry Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-annals-iv-4-w8-139-2019 ◽

2019 ◽

Vol IV-4/W8 ◽

pp. 139-146

Author(s):

X.-F. Xing ◽

M. A. Mostafavi ◽

G. Edwards ◽

N. Sabo

Keyword(s):

Machine Learning ◽

Urban Areas ◽

Learning Algorithm ◽

Semantic Segmentation ◽

Point Clouds ◽

Airborne Lidar ◽

Urban Scenes ◽

Machine Learning Approach ◽

Urban Scene ◽

Point Level

Abstract. Automatic semantic segmentation of point clouds observed in a 3D complex urban scene is a challenging issue. Semantic segmentation of urban scenes based on machine learning algorithm requires appropriate features to distinguish objects from mobile terrestrial and airborne LiDAR point clouds in point level. In this paper, we propose a pointwise semantic segmentation method based on our proposed features derived from Difference of Normal and the features “directional height above” that compare height difference between a given point and neighbors in eight directions in addition to the features based on normal estimation. Random forest classifier is chosen to classify points in mobile terrestrial and airborne LiDAR point clouds. The results obtained from our experiments show that the proposed features are effective for semantic segmentation of mobile terrestrial and airborne LiDAR point clouds, especially for vegetation, building and ground classes in an airborne LiDAR point clouds in urban areas.

Download Full-text

AN AUTOMATIC FILTER ALGORITHM FOR DENSE IMAGE MATCHING POINT CLOUDS

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xlii-2-w7-703-2017 ◽

2017 ◽

Vol XLII-2/W7 ◽

pp. 703-709 ◽

Cited By ~ 1

Author(s):

Y. Q. Dong ◽

L. Zhang ◽

X. M. Cui ◽

H. B. Ai

Keyword(s):

Image Matching ◽

Iterative Process ◽

Point Clouds ◽

High Density ◽

Aerial Images ◽

Dense Image ◽

Seed Points ◽

Filter Algorithms

Although many filter algorithms have been presented over past decades, these algorithms are usually designed for the Lidar point clouds and can’t separate the ground points from the DIM (dense image matching, DIM) point clouds derived from the oblique aerial images owing to the high density and variation of the DIM point clouds completely. To solve this problem, a new automatic filter algorithm is developed on the basis of adaptive TIN models. At first, the differences between Lidar and DIM point clouds which influence the filtering results are analysed in this paper. To avoid the influences of the plants which can’t be penetrated by the DIM point clouds in the searching seed pointes process, the algorithm makes use of the facades of buildings to get ground points located on the roads as seed points and construct the initial TIN. Then a new densification strategy is applied to deal with the problem that the densification thresholds do not change as described in other methods in each iterative process. Finally, we use the DIM point clouds located in Potsdam produced by Photo-Scan to evaluate the method proposed in this paper. The experiment results show that the method proposed in this paper can not only separate the ground points from the DIM point clouds completely but also obtain the better filter results compared with TerraSolid. 1.

Download Full-text