From 3D Point Clouds to Pose-Normalised Depth Maps

We present a workflow for the automatic generation of building models with levels of detail (LOD) 1 to 3 according to the CityGML standard (Gröger et al., 2012). We start with orienting unsorted image sets employing (Mayer et al., 2012), we compute depth maps using semi-global matching (SGM) (Hirschmüller, 2008), and fuse these depth maps to reconstruct dense 3D point clouds (Kuhn et al., 2014). Based on planes segmented from these point clouds, we have developed a stochastic method for roof model selection (Nguatem et al., 2013) and window model selection (Nguatem et al., 2014). We demonstrate our workflow up to the export into CityGML.

Download Full-text

AUTOMATED MOSAICKING OF MULTIPLE 3D POINT CLOUDS GENERATED FROM A DEPTH CAMERA

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xli-b3-269-2016 ◽

2016 ◽

Vol XLI-B3 ◽

pp. 269-272

Author(s):

H. Kim ◽

W. Yoon ◽

T. Kim

Keyword(s):

Depth Map ◽

Point Clouds ◽

Mapping Method ◽

Transformation Model ◽

Depth Camera ◽

Depth Data ◽

Depth Maps ◽

3D Point Clouds ◽

Intensity Map ◽

Time Of Flight Method

In this paper, we propose a method for automated mosaicking of multiple 3D point clouds generated from a depth camera. A depth camera generates depth data by using ToF (Time of Flight) method and intensity data by using intensity of returned signal. The depth camera used in this paper was a SR4000 from MESA Imaging. This camera generates a depth map and intensity map of 176 x 44 pixels. Generated depth map saves physical depth data with mm of precision. Generated intensity map contains texture data with many noises. We used texture maps for extracting tiepoints and depth maps for assigning z coordinates to tiepoints and point cloud mosaicking. There are four steps in the proposed mosaicking method. In the first step, we acquired multiple 3D point clouds by rotating depth camera and capturing data per rotation. In the second step, we estimated 3D-3D transformation relationships between subsequent point clouds. For this, 2D tiepoints were extracted automatically from the corresponding two intensity maps. They were converted into 3D tiepoints using depth maps. We used a 3D similarity transformation model for estimating the 3D-3D transformation relationships. In the third step, we converted local 3D-3D transformations into a global transformation for all point clouds with respect to a reference one. In the last step, the extent of single depth map mosaic was calculated and depth values per mosaic pixel were determined by a ray tracing method. For experiments, 8 depth maps and intensity maps were used. After the four steps, an output mosaicked depth map of 454x144 was generated. It is expected that the proposed method would be useful for developing an effective 3D indoor mapping method in future.

Download Full-text

Recognition of Human Activities Using Depth Maps and the Viewpoint Feature Histogram Descriptor

Sensors ◽

10.3390/s20102940 ◽

2020 ◽

Vol 20 (10) ◽

pp. 2940 ◽

Cited By ~ 1

Author(s):

Kamil Sidor ◽

Marian Wysocki

Keyword(s):

Human Activities ◽

Short Term Memory ◽

Point Clouds ◽

Human Action ◽

Learning Networks ◽

University Of Texas ◽

Multiple Networks ◽

Depth Maps ◽

3D Point Clouds ◽

Dynamic Time

In this paper we propose a way of using depth maps transformed into 3D point clouds to classify human activities. The activities are described as time sequences of feature vectors based on the Viewpoint Feature Histogram descriptor (VFH) computed using the Point Cloud Library. Recognition is performed by two types of classifiers: (i) k-NN nearest neighbors’ classifier with Dynamic Time Warping measure, (ii) bidirectional long short-term memory (BiLSTM) deep learning networks. Reduction of classification time for the k-NN by introducing a two tier model and improvement of BiLSTM-based classification via transfer learning and combining multiple networks by fuzzy integral are discussed. Our classification results obtained on two representative datasets: University of Texas at Dallas Multimodal Human Action Dataset and Mining Software Repositories Action 3D Dataset are comparable or better than the current state of the art.

Download Full-text

Pedestrian Detection Based on Panoramic Depth Map Transformed from 3D-LiDAR Data

Periodica Polytechnica Electrical Engineering and Computer Science ◽

10.3311/ppee.14960 ◽

2020 ◽

Author(s):

Guoqiang Chen ◽

Zhuangzhuang Mao ◽

Huailong Yi ◽

Xiaofeng Li ◽

Bingxin Bai ◽

...

Keyword(s):

Pedestrian Detection ◽

Depth Map ◽

Point Clouds ◽

Autonomous Driving ◽

Detection Algorithm ◽

Lidar Data ◽

Average Precision ◽

Depth Maps ◽

3D Point Clouds ◽

3D Lidar

Object detection is a crucial task of autonomous driving. This paper addresses an effective algorithm for pedestrian detection of the panoramic depth map transformed from the 3D-LiDAR data. Firstly, the 3D point clouds are transformed into panoramic depth maps, and then the panoramic depth maps are enhanced. Secondly, the grounds of the 3D point clouds are removed. The remaining point clouds are clustered, filtered and projected onto the previously generated panoramic depth maps, and new panoramic depth maps are obtained. Finally, the new panoramic depth maps are jointed to generate depth maps with different sizes, which are used as input of the improved PVANET for pedestrian detection. The 2D image of the panoramic depth map applied to the proposed algorithm is transformed from 3D point cloud, effectively containing the panorama of the sensor, and is more suitable for the environment perception of autonomous driving. Compared with the detection algorithm based on RGB images, the proposed algorithm cannot be affected by light, and can maintain the normal average precision of pedestrian detection at night. In order to increase the robustness of detecting small objects like pedestrians, the network structure based on the original PVANET is modified in this paper. A new dataset is built by processing the 3D-LiDAR data and the model trained on the new dataset perform well. The experimental results show that the proposed algorithm achieves high accuracy and robustness in pedestrian detection under different illumination conditions. Furthermore, when trained on the new dataset, the model exhibits average precision improvements of 2.8–5.1 % over the original PVANET, making it more suitable for autonomous driving applications.

Download Full-text

AUTOMATED MOSAICKING OF MULTIPLE 3D POINT CLOUDS GENERATED FROM A DEPTH CAMERA

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprsarchives-xli-b3-269-2016 ◽

2016 ◽

Vol XLI-B3 ◽

pp. 269-272

Author(s):

H. Kim ◽

W. Yoon ◽

T. Kim

Keyword(s):

Depth Map ◽

Point Clouds ◽

Mapping Method ◽

Transformation Model ◽

Depth Camera ◽

Depth Data ◽

Depth Maps ◽

3D Point Clouds ◽

Intensity Map ◽

Time Of Flight Method

In this paper, we propose a method for automated mosaicking of multiple 3D point clouds generated from a depth camera. A depth camera generates depth data by using ToF (Time of Flight) method and intensity data by using intensity of returned signal. The depth camera used in this paper was a SR4000 from MESA Imaging. This camera generates a depth map and intensity map of 176 x 44 pixels. Generated depth map saves physical depth data with mm of precision. Generated intensity map contains texture data with many noises. We used texture maps for extracting tiepoints and depth maps for assigning z coordinates to tiepoints and point cloud mosaicking. There are four steps in the proposed mosaicking method. In the first step, we acquired multiple 3D point clouds by rotating depth camera and capturing data per rotation. In the second step, we estimated 3D-3D transformation relationships between subsequent point clouds. For this, 2D tiepoints were extracted automatically from the corresponding two intensity maps. They were converted into 3D tiepoints using depth maps. We used a 3D similarity transformation model for estimating the 3D-3D transformation relationships. In the third step, we converted local 3D-3D transformations into a global transformation for all point clouds with respect to a reference one. In the last step, the extent of single depth map mosaic was calculated and depth values per mosaic pixel were determined by a ray tracing method. For experiments, 8 depth maps and intensity maps were used. After the four steps, an output mosaicked depth map of 454x144 was generated. It is expected that the proposed method would be useful for developing an effective 3D indoor mapping method in future.

Download Full-text

AUTOMATIC GENERATION OF BUILDING MODELS WITH LEVELS OF DETAIL 1-3

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xli-b3-649-2016 ◽

2016 ◽

Vol XLI-B3 ◽

pp. 649-654

Author(s):

W. Nguatem ◽

M. Drauschke ◽

H. Mayer

Keyword(s):

Model Selection ◽

Point Clouds ◽

Automatic Generation ◽

Stochastic Method ◽

Depth Maps ◽

Levels Of Detail ◽

3D Point Clouds ◽

Building Models ◽

Global Matching

We present a workflow for the automatic generation of building models with levels of detail (LOD) 1 to 3 according to the CityGML standard (Gröger et al., 2012). We start with orienting unsorted image sets employing (Mayer et al., 2012), we compute depth maps using semi-global matching (SGM) (Hirschmüller, 2008), and fuse these depth maps to reconstruct dense 3D point clouds (Kuhn et al., 2014). Based on planes segmented from these point clouds, we have developed a stochastic method for roof model selection (Nguatem et al., 2013) and window model selection (Nguatem et al., 2014). We demonstrate our workflow up to the export into CityGML.

Download Full-text

Multi-scale Information Extraction Residual Network for 3D Point Clouds Classification

2020 Chinese Automation Congress (CAC) ◽

10.1109/cac51589.2020.9327126 ◽

2020 ◽

Author(s):

Yumei Li ◽

Shan Meng ◽

Daoyuan Liang

Keyword(s):

Information Extraction ◽

Point Clouds ◽

Residual Network ◽

Multi Scale ◽

3D Point Clouds

Download Full-text

Symmetry Analysis of Oriental Polygonal Pagodas Using 3D Point Clouds for Cultural Heritage

Sensors ◽

10.3390/s21041228 ◽

2021 ◽

Vol 21 (4) ◽

pp. 1228

Author(s):

Ting On Chan ◽

Linyuan Xia ◽

Yimin Chen ◽

Wei Lang ◽

Tingting Chen ◽

...

Keyword(s):

Cultural Heritage ◽

Unmanned Aerial Vehicle ◽

Point Cloud ◽

Geometric Model ◽

Digital Camera ◽

Point Clouds ◽

Symmetry Analysis ◽

3D Point Clouds ◽

Transmission Towers ◽

Aerial Vehicle

Ancient pagodas are usually parts of hot tourist spots in many oriental countries due to their unique historical backgrounds. They are usually polygonal structures comprised by multiple floors, which are separated by eaves. In this paper, we propose a new method to investigate both the rotational and reflectional symmetry of such polygonal pagodas through developing novel geometric models to fit to the 3D point clouds obtained from photogrammetric reconstruction. The geometric model consists of multiple polygonal pyramid/prism models but has a common central axis. The method was verified by four datasets collected by an unmanned aerial vehicle (UAV) and a hand-held digital camera. The results indicate that the models fit accurately to the pagodas’ point clouds. The symmetry was realized by rotating and reflecting the pagodas’ point clouds after a complete leveling of the point cloud was achieved using the estimated central axes. The results show that there are RMSEs of 5.04 cm and 5.20 cm deviated from the perfect (theoretical) rotational and reflectional symmetries, respectively. This concludes that the examined pagodas are highly symmetric, both rotationally and reflectionally. The concept presented in the paper not only work for polygonal pagodas, but it can also be readily transformed and implemented for other applications for other pagoda-like objects such as transmission towers.

Download Full-text

Using Least-Squares Residuals to Assess the Stochasticity of Measurements—Example: Terrestrial Laser Scanner and Surface Modeling

Engineering Proceedings ◽

10.3390/engproc2021005059 ◽

2021 ◽

Vol 5 (1) ◽

pp. 59

Author(s):

Gaël Kermarrec ◽

Niklas Schild ◽

Jan Hartmann

Keyword(s):

Least Squares ◽

Laser Scanner ◽

Real Data ◽

Point Clouds ◽

Statistical Testing ◽

Normal Vector ◽

Engineering Structures ◽

3D Point Clouds ◽

Least Squares Adjustment ◽

Correlation Parameters

Terrestrial laser scanners (TLS) capture a large number of 3D points rapidly, with high precision and spatial resolution. These scanners are used for applications as diverse as modeling architectural or engineering structures, but also high-resolution mapping of terrain. The noise of the observations cannot be assumed to be strictly corresponding to white noise: besides being heteroscedastic, correlations between observations are likely to appear due to the high scanning rate. Unfortunately, if the variance can sometimes be modeled based on physical or empirical considerations, the latter are more often neglected. Trustworthy knowledge is, however, mandatory to avoid the overestimation of the precision of the point cloud and, potentially, the non-detection of deformation between scans recorded at different epochs using statistical testing strategies. The TLS point clouds can be approximated with parametric surfaces, such as planes, using the Gauss–Helmert model, or the newly introduced T-splines surfaces. In both cases, the goal is to minimize the squared distance between the observations and the approximated surfaces in order to estimate parameters, such as normal vector or control points. In this contribution, we will show how the residuals of the surface approximation can be used to derive the correlation structure of the noise of the observations. We will estimate the correlation parameters using the Whittle maximum likelihood and use comparable simulations and real data to validate our methodology. Using the least-squares adjustment as a “filter of the geometry” paves the way for the determination of a correlation model for many sensors recording 3D point clouds.

Download Full-text