Registration of 3D Point Cloud of Human Body Based on the Range Images and RGB Images

2015 ◽  
Vol 738-739 ◽  
pp. 656-661
Author(s):  
Yan Wan ◽  
Jun Lu ◽  
Ao Qiong Li
Keyword(s):  
Point Cloud ◽  
High Efficiency ◽  
Low Cost ◽  
Point Clouds ◽  
The Body ◽  
Background Information ◽  
Range Images ◽  
Scale Invariant ◽  
Rgb Images ◽  
Body Point

The existing 3D reconstruction techniques rarely can be easily used in people's daily life, and the traditional registration algorithms have the drawback of massive calculation. In this paper it presented a registration algorithm of body point cloud based on RGB images and Range images. First, it used kinect to obtain the RGB images and Range images from different perspectives. Then it extracted the pairs of 2D feature points on RGB images using scale-invariant feature transform (SIFT) and random sample consensus (RANSAC) algorithm application for the coarse registration and used the improved iterative closest point (ICP) algorithm for the fine registration. Second, it eliminated the background information and the noise points of the model edges. Finally it completed the registration process. Experimental results show that the algorithm can accurately accomplish the body point clouds registration using the low-cost instrument and has a relatively high efficiency.

scholarly journals Integrating UAV and TLS Approaches for Environmental Management: A Case Study of a Waste Stockpile Area

2020 ◽  
Vol 12 (10) ◽  
pp. 1615 ◽  
Author(s):  
Seung Woo Son ◽  
Dong Woo Kim ◽  
Woong Gi Sung ◽  
Jae Jin Yu
Keyword(s):  
Environmental Management ◽  
Point Cloud ◽  
Laser Scanning ◽  
High Efficiency ◽  
Point Clouds ◽  
Fusion Model ◽  
Spatial Features ◽  
Computation Efficiency ◽  
Volume Computation ◽  
Mean Square Errors

A methodology for optimal volume computation for the environmental management of waste stockpiles was derived by integrating the terrestrial laser scanning (TLS) and unmanned aerial vehicle (UAV) technologies. Among the UAV-based point clouds generated under various flight scenarios, the most accurate point cloud was selected for analysis. The root mean square errors (RMSEs) of the TLS- and UAV-based methods were 0.202 and 0.032 m, respectively, and the volume computation yielded 41,226 and 41,526 m3, respectively. Both techniques showed high accuracy but also exhibited drawbacks in terms of their spatial features and efficiency. The TLS and UAV methods required 800 and 340 min, respectively, demonstrating the high efficiency of the UAV method. The RMSE and volume obtained using the TLS/UAV fusion model were calculated as 0.030 m and 41,232 m3, respectively. The UAV approach generally yielded high point cloud accuracy and volume computation efficiency.

scholarly journals Accuracy assessment of building point clouds automatically generated from iphone images

2014 ◽  
Vol XL-5 ◽  
pp. 547-552 ◽  
Author(s):  
B. Sirmacek ◽  
R. Lindenbergh
Keyword(s):  
Measurement Errors ◽  
Point Cloud ◽  
Model Updating ◽  
Accuracy Assessment ◽  
Low Cost ◽  
Point Clouds ◽  
3D Models ◽  
Map Updating ◽  
Cloud Points ◽  
Point To Point

Low-cost sensor generated 3D models can be useful for quick 3D urban model updating, yet the quality of the models is questionable. In this article, we evaluate the reliability of an automatic point cloud generation method using multi-view iPhone images or an iPhone video file as an input. We register such automatically generated point cloud on a TLS point cloud of the same object to discuss accuracy, advantages and limitations of the iPhone generated point clouds. For the chosen example showcase, we have classified 1.23% of the iPhone point cloud points as outliers, and calculated the mean of the point to point distances to the TLS point cloud as 0.11 m. Since a TLS point cloud might also include measurement errors and noise, we computed local noise values for the point clouds from both sources. Mean (μ) and standard deviation (&amp;sigma;) of roughness histograms are calculated as (μ<sub>1</sub> = 0.44 m., &amp;sigma;<sub>1</sub> = 0.071 m.) and (μ<sub>2</sub> = 0.025 m., &amp;sigma;<sub>2</sub> = 0.037 m.) for the iPhone and TLS point clouds respectively. Our experimental results indicate possible usage of the proposed automatic 3D model generation framework for 3D urban map updating, fusion and detail enhancing, quick and real-time change detection purposes. However, further insights should be obtained first on the circumstances that are needed to guarantee a successful point cloud generation from smartphone images.

scholarly journals COMBINATION OF TLS POINT CLOUDS AND 3D DATA FROM KINECT V2 SENSOR TO COMPLETE INDOOR MODELS

2016 ◽  
Vol XLI-B5 ◽  
pp. 659-666 ◽  
Author(s):  
E. Lachat ◽  
T. Landes ◽  
P. Grussenmeyer
Keyword(s):  
Point Cloud ◽  
Low Cost ◽  
Measurement Techniques ◽  
Laser Scanner ◽  
Point Clouds ◽  
Final Model ◽  
Kinect V2 ◽  
Multiple Sensors ◽  
3D Data ◽  
Final Data

The combination of data coming from multiple sensors is more and more applied for remote sensing issues (multi-sensor imagery) but also in cultural heritage or robotics, since it often results in increased robustness and accuracy of the final data. In this paper, the reconstruction of building elements such as window frames or door jambs scanned thanks to a low cost 3D sensor (Kinect v2) is presented. Their combination within a global point cloud of an indoor scene acquired with a terrestrial laser scanner (TLS) is considered. If the added elements acquired with the Kinect sensor enable to reach a better level of detail of the final model, an adapted acquisition protocol may also provide several benefits as for example time gain. The paper aims at analyzing whether the two measurement techniques can be complementary in this context. The limitations encountered during the acquisition and reconstruction steps are also investigated.

scholarly journals A Two-Phase Cross-Modality Fusion Network for Robust 3D Object Detection

Sensors ◽  
2020 ◽  
Vol 20 (21) ◽  
pp. 6043
Author(s):  
Yujun Jiao ◽  
Zhishuai Yin
Keyword(s):  
Object Detection ◽  
Point Cloud ◽  
Point Clouds ◽  
Second Phase ◽  
Two Phase ◽  
3D Object ◽  
Rgb Images ◽  
Fusion Scheme ◽  
3D Object Detection ◽  
Level Fusion

A two-phase cross-modality fusion detector is proposed in this study for robust and high-precision 3D object detection with RGB images and LiDAR point clouds. First, a two-stream fusion network is built into the framework of Faster RCNN to perform accurate and robust 2D detection. The visible stream takes the RGB images as inputs, while the intensity stream is fed with the intensity maps which are generated by projecting the reflection intensity of point clouds to the front view. A multi-layer feature-level fusion scheme is designed to merge multi-modal features across multiple layers in order to enhance the expressiveness and robustness of the produced features upon which region proposals are generated. Second, a decision-level fusion is implemented by projecting 2D proposals to the space of the point cloud to generate 3D frustums, on the basis of which the second-phase 3D detector is built to accomplish instance segmentation and 3D-box regression on the filtered point cloud. The results on the KITTI benchmark show that features extracted from RGB images and intensity maps complement each other, and our proposed detector achieves state-of-the-art performance on 3D object detection with a substantially lower running time as compared to available competitors.

scholarly journals Efficient Coarse Registration of Pairwise TLS Point Clouds Using Ortho Projected Feature Images

2020 ◽  
Vol 9 (4) ◽  
pp. 255
Author(s):  
Hua Liu ◽  
Xiaoming Zhang ◽  
Yuancheng Xu ◽  
Xiaoyong Chen
Keyword(s):  
Parameter Estimation ◽  
Degrees Of Freedom ◽  
High Efficiency ◽  
Point Clouds ◽  
Azimuth Angle ◽  
Translation Vector ◽  
Scale Invariant ◽  
Registration Method ◽  
Coarse Registration ◽  
Vertical Translation

The degree of automation and efficiency are among the most important factors that influence the availability of Terrestrial light detection and ranging (LiDAR) Scanning (TLS) registration algorithms. This paper proposes an Ortho Projected Feature Images (OPFI) based 4 Degrees of Freedom (DOF) coarse registration method, which is fully automated and with high efficiency, for TLS point clouds acquired using leveled or inclination compensated LiDAR scanners. The proposed 4DOF registration algorithm decomposes the parameter estimation into two parts: (1) the parameter estimation of horizontal translation vector and azimuth angle; and (2) the parameter estimation of the vertical translation vector. The parameter estimation of the horizontal translation vector and the azimuth angle is achieved by ortho projecting the TLS point clouds into feature images and registering the ortho projected feature images by Scale Invariant Feature Transform (SIFT) key points and descriptors. The vertical translation vector is estimated using the height difference of source points and target points in the overlapping regions after horizontally aligned. Three real TLS datasets captured by the Riegl VZ-400 and the Trimble SX10 and one simulated dataset were used to validate the proposed method. The proposed method was compared with four state-of-the-art 4DOF registration methods. The experimental results showed that: (1) the accuracy of the proposed coarse registration method ranges from 0.02 m to 0.07 m in horizontal and 0.01 m to 0.02 m in elevation, which is at centimeter-level and sufficient for fine registration; and (2) as many as 120 million points can be registered in less than 50 s, which is much faster than the compared methods.

scholarly journals Direct local building inundation depth determination in 3D point clouds generated from user-generated flood images

2017 ◽  
Author(s):  
Luisa Griesbaum ◽  
Sabrina Marx ◽  
Bernhard Höfle
Keyword(s):  
Point Cloud ◽  
Low Cost ◽  
Point Clouds ◽  
Urban Environments ◽  
Extreme Weather Events ◽  
Inundation Depth ◽  
Area Of Interest ◽  
3D Point Clouds ◽  
Flood Level ◽  
Flood Elevation

Abstract. In recent years, the number of people affected by flooding caused by extreme weather events has increased considerably. In order to provide support in disaster recovery or to develop mitigation plans, accurate flood information is necessary. Particularly pluvial urban floods, characterized by high temporal and spatial variations, are not well documented. This study proposes a new, low-cost approach to determining local flood elevation and inundation depth of buildings based on user-generated flood images. It first applies close-range digital photogrammetry to generate a geo-referenced 3D point cloud. Second, based on estimated camera orientation parameters, the flood level captured in a single flood image is mapped to the previously derived point cloud. The local flood elevation and the building inundation depth can then be derived automatically from the point cloud. The proposed method is carried out once for each of 66 different flood images showing the same building façade. An overall accuracy of 0.05 m  with an uncertainty of ±0.13 m for the derived flood elevation within the area of interest and an accuracy of 0.13 m  ± 0.10 m for the determined building inundation depth is achieved. Our results demonstrate that the proposed method can provide reliable flood information on a local scale using user-generated flood images as input. The approach can thus allow inundation depth maps to be derived even in complex urban environments with relatively high accuracies.

scholarly journals A Density-Based Approach for Leaf Area Index Assessment in a Complex Forest Environment Using a Terrestrial Laser Scanner

2019 ◽  
Vol 11 (15) ◽  
pp. 1791 ◽  
Author(s):  
Ali Rouzbeh Kargar ◽  
Richard MacKenzie ◽  
Gregory P. Asner ◽  
Jan van Aardt
Keyword(s):  
Leaf Area Index ◽  
Leaf Area ◽  
Point Cloud ◽  
Forest Canopy ◽  
Low Cost ◽  
Point Clouds ◽  
Grid Size ◽  
Density Estimator ◽  
Forest Environment ◽  
Area Index

Forests are an important part natural ecosystems, by for example providing food, fiber, habitat, and biodiversity, all of which contribute to stable natural systems. Assessing and modeling the structure and characteristics of forests, e.g., Leaf Area Index (LAI), volume, biomass, etc., can lead to a better understanding and management of these resources. In recent years, Terrestrial Laser Scanning (TLS) has been recognized as a tool that addresses many of the limitations of manual and traditional forest data collection methods. In this study, we propose a density-based approach for estimating the LAI in a structurally-complex forest environment, which contains variable and diverse structural attributes, e.g., non-circular stem forms, dense canopy and below-canopy vegetation cover, and a diverse species composition. In addition, 242 TLS scans were collected using a portable low-cost scanner, the Compact Biomass Lidar (CBL), in the Hawaii Volcanoes National Park (HAVO), Hawaii Island, USA. LAI also was measured for 242 plots in the site, using an AccuPAR LP-80 ceptometer. The first step after cleaning the point cloud involved detecting the higher forest canopy in the light detection and ranging (lidar) point clouds, using normal change rate assessment. We then estimated Leaf Area Density (LAD), using a voxel-based approach, and divided the canopy point cloud into five layers in the Z (vertical) direction. These five layers subsequently were divided into voxels in the X direction, where the size of these voxels were obtained based on inter-quartile analysis and the number of points in each voxel. We hypothesized that the intensity returned to the lidar system from woody materials, like branches, would be higher than from leaves, due to the liquid water absorption feature of the leaves and higher reflectance for woody material at the 905 nm laser wavelength. We also differentiated between foliar and woody materials using edge detection in the images from projected point clouds and evaluated the density of these regions to support our hypothesis. Density of points, or the number of points divided by the volume of a grid, in a 3D grid size of 0.1 m, was calculated for each of the voxels. The grid size was determined by investigating the size of the branches in the lower portion of the canopy. Subsequently, we fitted a Kernel Density Estimator (KDE) to these values, with the threshold set based on half of the area under the curve in each of the density distributions. All the grids with a density below the threshold were labeled as leaves, while those grids above the threshold were identified as non-leaves. Finally, we modeled LAI using the point densities derived from the TLS point clouds and the listed analysis steps. This model resulted in an R 2 value of 0.88. We also estimated the LAI directly from lidar data using the point densities and calculating LAD, which is defined as the total one-sided leaf area per unit volume. LAI can be obtained as the sum of the LAD values in all the voxels. The accuracy of LAI estimation was 90%, with an RMSE value of 0.31, and an average overestimation of 9 % in TLS-derived LAI, when compared to field-measured LAI. Algorithm performance mainly was affected by the vegetation density and complexity of the canopy structures. It is worth noting that, since the LAI values cannot be considered spatially independent throughout all the plots in this site, we performed semivariogram analysis on the field-measured LAI data. This analysis showed that the LAI values can be assumed to be independent in plots that are at least 30 m apart. As a result, we divided the data into six subsets in which the plots were 30 m spaced. The R 2 values for these subsets, based on modeling of the field-measured LAI using leaf point density values, ranged between 0.84–0.96. The results bode well for using this method for efficient, automatic, and accurate/precise estimation of LAI values in complex forest environments, using a low-cost, rapid-scan TLS.

scholarly journals Automatic 3D Landmark Extraction System Based on an Encoder–Decoder Using Fusion of Vision and LiDAR

2020 ◽  
Vol 12 (7) ◽  
pp. 1142
Author(s):  
Jeonghoon Kwak ◽  
Yunsick Sung
Keyword(s):  
Point Cloud ◽  
Point Clouds ◽  
Depth Image ◽  
3D Point Cloud ◽  
Digital World ◽  
Depth Images ◽  
3D Point Clouds ◽  
Rgb Images ◽  
Rgb Image ◽  
3D Landmarks

To provide a realistic environment for remote sensing applications, point clouds are used to realize a three-dimensional (3D) digital world for the user. Motion recognition of objects, e.g., humans, is required to provide realistic experiences in the 3D digital world. To recognize a user’s motions, 3D landmarks are provided by analyzing a 3D point cloud collected through a light detection and ranging (LiDAR) system or a red green blue (RGB) image collected visually. However, manual supervision is required to extract 3D landmarks as to whether they originate from the RGB image or the 3D point cloud. Thus, there is a need for a method for extracting 3D landmarks without manual supervision. Herein, an RGB image and a 3D point cloud are used to extract 3D landmarks. The 3D point cloud is utilized as the relative distance between a LiDAR and a user. Because it cannot contain all information the user’s entire body due to disparities, it cannot generate a dense depth image that provides the boundary of user’s body. Therefore, up-sampling is performed to increase the density of the depth image generated based on the 3D point cloud; the density depends on the 3D point cloud. This paper proposes a system for extracting 3D landmarks using 3D point clouds and RGB images without manual supervision. A depth image provides the boundary of a user’s motion and is generated by using 3D point cloud and RGB image collected by a LiDAR and an RGB camera, respectively. To extract 3D landmarks automatically, an encoder–decoder model is trained with the generated depth images, and the RGB images and 3D landmarks are extracted from these images with the trained encoder model. The method of extracting 3D landmarks using RGB depth (RGBD) images was verified experimentally, and 3D landmarks were extracted to evaluate the user’s motions with RGBD images. In this manner, landmarks could be extracted according to the user’s motions, rather than by extracting them using the RGB images. The depth images generated by the proposed method were 1.832 times denser than the up-sampling-based depth images generated with bilateral filtering.

scholarly journals FINDING A GOOD FEATURE DETECTOR-DESCRIPTOR COMBINATION FOR THE 2D KEYPOINT-BASED REGISTRATION OF TLS POINT CLOUDS

2015 ◽  
Vol II-3/W5 ◽  
pp. 121-128 ◽  
Author(s):  
S. Urban ◽  
M. Weinmann
Keyword(s):  
Point Cloud ◽  
Laser Scanning ◽  
Point Clouds ◽  
Scale Invariant ◽  
Point Cloud Registration ◽  
Speeded Up Robust Features ◽  
Extensive Evaluation ◽  
Feature Correspondences ◽  
Intensity Images ◽  
Keypoint Detector

The automatic and accurate registration of terrestrial laser scanning (TLS) data is a topic of great interest in the domains of city modeling, construction surveying or cultural heritage. While numerous of the most recent approaches focus on keypoint-based point cloud registration relying on forward-projected 2D keypoints detected in panoramic intensity images, little attention has been paid to the selection of appropriate keypoint detector-descriptor combinations. Instead, keypoints are commonly detected and described by applying well-known methods such as the Scale Invariant Feature Transform (SIFT) or Speeded-Up Robust Features (SURF). In this paper, we present a framework for evaluating the influence of different keypoint detector-descriptor combinations on the results of point cloud registration. For this purpose, we involve five different approaches for extracting local features from the panoramic intensity images and exploit the range information of putative feature correspondences in order to define bearing vectors which, in turn, may be exploited to transfer the task of point cloud registration from the object space to the observation space. With an extensive evaluation of our framework on a standard benchmark TLS dataset, we clearly demonstrate that replacing SIFT and SURF detectors and descriptors by more recent approaches significantly alleviates point cloud registration in terms of accuracy, efficiency and robustness.

scholarly journals PROCESSING UAV AND LIDAR POINT CLOUDS IN GRASS GIS

2016 ◽  
Vol XLI-B7 ◽  
pp. 945-952 ◽  
Author(s):  
V. Petras ◽  
A. Petrasova ◽  
J. Jeziorska ◽  
H. Mitasova
Keyword(s):  
Open Source ◽  
Point Cloud ◽  
Low Cost ◽  
Point Clouds ◽  
Physical Models ◽  
Microsoft Kinect ◽  
Surface Model ◽  
3D Scanner ◽  
Grass Gis ◽  
Point Data

Today’s methods of acquiring Earth surface data, namely lidar and unmanned aerial vehicle (UAV) imagery, non-selectively collect or generate large amounts of points. Point clouds from different sources vary in their properties such as number of returns, density, or quality. We present a set of tools with applications for different types of points clouds obtained by a lidar scanner, structure from motion technique (SfM), and a low-cost 3D scanner. To take advantage of the vertical structure of multiple return lidar point clouds, we demonstrate tools to process them using 3D raster techniques which allow, for example, the development of custom vegetation classification methods. Dense point clouds obtained from UAV imagery, often containing redundant points, can be decimated using various techniques before further processing. We implemented and compared several decimation techniques in regard to their performance and the final digital surface model (DSM). Finally, we will describe the processing of a point cloud from a low-cost 3D scanner, namely Microsoft Kinect, and its application for interaction with physical models. All the presented tools are open source and integrated in GRASS GIS, a multi-purpose open source GIS with remote sensing capabilities. The tools integrate with other open source projects, specifically Point Data Abstraction Library (PDAL), Point Cloud Library (PCL), and OpenKinect libfreenect2 library to benefit from the open source point cloud ecosystem. The implementation in GRASS GIS ensures long term maintenance and reproducibility by the scientific community but also by the original authors themselves.

Sign in / Sign up

Export Citation Format

Share Document