scholarly journals CLASSIFICATION AND INTEGRATION OF MASSIVE 3D POINTS CLOUDS IN A VIRTUAL REALITY (VR) ENVIRONMENT

2019 ◽  
Vol XLII-2/W17 ◽  
pp. 165-171 ◽  
Author(s):  
A. Kharroubi ◽  
R. Hajji ◽  
R. Billen ◽  
F. Poux
Keyword(s):  
Virtual Reality ◽  
Open Source ◽  
Point Cloud ◽  
Mixed Reality ◽  
Automatic Segmentation ◽  
Point Clouds ◽  
3D Point Clouds ◽  
3D Data ◽  
Segmentation Approach ◽  
And Performance

Abstract. With the increasing volume of 3D applications using immersive technologies such as virtual, augmented and mixed reality, it is very interesting to create better ways to integrate unstructured 3D data such as point clouds as a source of data. Indeed, this can lead to an efficient workflow from 3D capture to 3D immersive environment creation without the need to derive 3D model, and lengthy optimization pipelines. In this paper, the main focus is on the direct classification and integration of massive 3D point clouds in a virtual reality (VR) environment. The emphasis is put on leveraging open-source frameworks for an easy replication of the findings. First, we develop a semi-automatic segmentation approach to provide semantic descriptors (mainly classes) to groups of points. We then build an octree data structure leveraged through out-of-core algorithms to load in real time and continuously only the points that are in the VR user's field of view. Then, we provide an open-source solution using Unity with a user interface for VR point cloud interaction and visualisation. Finally, we provide a full semantic VR data integration enhanced through developed shaders for future spatio-semantic queries. We tested our approach on several datasets of which a point cloud composed of 2.3 billion points, representing the heritage site of the castle of Jehay (Belgium). The results underline the efficiency and performance of the solution for visualizing classifieds massive point clouds in virtual environments with more than 100 frame per second.

scholarly journals Leaf Segmentation on Dense Plant Point Clouds with Facet Region Growing

Sensors ◽  
2018 ◽  
Vol 18 (11) ◽  
pp. 3625 ◽  
Author(s):  
Dawei Li ◽  
Yan Cao ◽  
Xue-song Tang ◽  
Siyuan Yan ◽  
Xin Cai
Keyword(s):  
Point Cloud ◽  
Active Material ◽  
Region Growing ◽  
Point Clouds ◽  
Phenotypic Traits ◽  
Individual Leaf ◽  
3D Point Clouds ◽  
Segmentation Approach ◽  
Monstera Deliciosa ◽  
Leaf Segmentation

Leaves account for the largest proportion of all organ areas for most kinds of plants, and are comprise the main part of the photosynthetically active material in a plant. Observation of individual leaves can help to recognize their growth status and measure complex phenotypic traits. Current image-based leaf segmentation methods have problems due to highly restricted species and vulnerability toward canopy occlusion. In this work, we propose an individual leaf segmentation approach for dense plant point clouds using facet over-segmentation and facet region growing. The approach can be divided into three steps: (1) point cloud pre-processing, (2) facet over-segmentation, and (3) facet region growing for individual leaf segmentation. The experimental results show that the proposed method is effective and efficient in segmenting individual leaves from 3D point clouds of greenhouse ornamentals such as Epipremnum aureum, Monstera deliciosa, and Calathea makoyana, and the average precision and recall are both above 90%. The results also reveal the wide applicability of the proposed methodology for point clouds scanned from different kinds of 3D imaging systems, such as stereo vision and Kinect v2. Moreover, our method is potentially applicable in a broad range of applications that aim at segmenting regular surfaces and objects from a point cloud.

scholarly journals Comparison of 3D interest point detectors and descriptors for point cloud fusion

2014 ◽  
Vol II-3 ◽  
pp. 57-64 ◽  
Author(s):  
R. Hänsch ◽  
T. Weber ◽  
O. Hellwich
Keyword(s):  
Point Cloud ◽  
Point Clouds ◽  
Specific Method ◽  
Scanner Data ◽  
3D Scanner ◽  
Strong Reduction ◽  
Interest Point Detectors ◽  
3D Point Clouds ◽  
3D Data ◽  
2D Images

The extraction and description of keypoints as salient image parts has a long tradition within processing and analysis of 2D images. Nowadays, 3D data gains more and more importance. This paper discusses the benefits and limitations of keypoints for the task of fusing multiple 3D point clouds. For this goal, several combinations of 3D keypoint detectors and descriptors are tested. The experiments are based on 3D scenes with varying properties, including 3D scanner data as well as Kinect point clouds. The obtained results indicate that the specific method to extract and describe keypoints in 3D data has to be carefully chosen. In many cases the accuracy suffers from a too strong reduction of the available points to keypoints.

scholarly journals HYBRID GEOREFERENCING, ENHANCEMENT AND CLASSIFICATION OF ULTRA-HIGH RESOLUTION UAV LIDAR AND IMAGE POINT CLOUDS FOR MONITORING APPLICATIONS

2020 ◽  
Vol V-2-2020 ◽  
pp. 727-734
Author(s):  
N. Haala ◽  
M. Kölle ◽  
M. Cramer ◽  
D. Laupheimer ◽  
G. Mandlburger ◽  
...  
Keyword(s):  
Point Cloud ◽  
Image Data ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Airborne Lidar ◽  
Image Texture ◽  
Image Point ◽  
Joint Orientation ◽  
3D Point Clouds ◽  
3D Data

Abstract. This paper presents a study on the potential of ultra-high accurate UAV-based 3D data capture by combining both imagery and LiDAR data. Our work is motivated by a project aiming at the monitoring of subsidence in an area of mixed use. Thus, it covers built-up regions in a village with a ship lock as the main object of interest as well as regions of agricultural use. In order to monitor potential subsidence in the order of 10 mm/year, we aim at sub-centimeter accuracies of the respective 3D point clouds. We show that hybrid georeferencing helps to increase the accuracy of the adjusted LiDAR point cloud by integrating results from photogrammetric block adjustment to improve the time-dependent trajectory corrections. As our main contribution, we demonstrate that joint orientation of laser scans and images in a hybrid adjustment framework significantly improves the relative and absolute height accuracies. By these means, accuracies corresponding to the GSD of the integrated imagery can be achieved. Image data can also help to enhance the LiDAR point clouds. As an example, integrating results from Multi-View Stereo potentially increases the point density from airborne LiDAR. Furthermore, image texture can support 3D point cloud classification. This semantic segmentation discussed in the final part of the paper is a prerequisite for further enhancement and analysis of the captured point cloud.

scholarly journals FUSION OF 3D POINT CLOUDS WITH TIR IMAGES FOR INDOOR SCENE RECONSTRUCTION

2018 ◽  
Vol XLII-1 ◽  
pp. 189-194 ◽  
Author(s):  
L. Hoegner ◽  
T. Abmayr ◽  
D. Tosic ◽  
S. Turzer ◽  
U. Stilla
Keyword(s):  
Point Cloud ◽  
Near Infrared ◽  
Moving Objects ◽  
Infrared Camera ◽  
Point Clouds ◽  
Thermal Infrared ◽  
Active Sensor ◽  
3D Point Clouds ◽  
3D Data ◽  
3D Scene

<p><strong>Abstract.</strong> Obtaining accurate 3D descriptions in the thermal infrared (TIR) is a quite challenging task due to the low geometric resolutions of TIR cameras and the low number of strong features in TIR images. Combining the radiometric information of the thermal infrared with 3D data from another sensor is able to overcome most of the limitations in the 3D geometric accuracy. In case of dynamic scenes with moving objects or a moving sensor system, a combination with RGB cameras and profile laserscanners is suitable. As a laserscanner is an active sensor in the visible red or near infrared (NIR) and the thermal infrared camera captures the radiation emitted by the objects in the observed scene, the combination of these two sensors for close range applications are independent from external illumination or textures in the scene. This contribution focusses on the fusion of point clouds from terrestrial laserscanners and RGB cameras with images from thermal infrared mounted together on a robot for indoor 3D reconstruction. The system is geometrical calibrated including the lever arm between the different sensors. As the field of view is different for the sensors, the different sensors record the same scene points not exactly at the same time. Thus, the 3D scene points of the laserscanner and the photogrammetric point cloud from the RGB camera have to be synchronized before point cloud fusion and adding the thermal channel to the 3D points.</p>

Building-Scale Virtual Reality

2019 ◽  
Vol 10 (1) ◽  
pp. 1-21
Author(s):  
Katashi Nagao ◽  
Menglong Yang ◽  
Yusuke Miyakawa
Keyword(s):  
Virtual Reality ◽  
Real World ◽  
Point Cloud ◽  
Point Clouds ◽  
3D Point Cloud ◽  
Pose Tracking ◽  
3D Point Clouds ◽  
Localization And Mapping ◽  
Real World Applications ◽  
Using Data

A method is presented that extends the real world into all buildings. This building-scale virtual reality (VR) method differs from augmented reality (AR) in that it uses automatically generated 3D point cloud maps of building interiors. It treats an entire indoor area a pose tracking area by using data collected using an RGB-D camera mounted on a VR headset and using deep learning to build a model from the data. It modifies the VR space in accordance with its intended usage by using segmentation and replacement of the 3D point clouds. This is difficult to do with AR but is essential if VR is to be used for actual real-world applications, such as disaster simulation including simulation of fires and flooding in buildings. 3D pose tracking in the building-scale VR is more accurate than conventional RGB-D simultaneous localization and mapping.

scholarly journals DETERMINATION OF PARKING SPACE AND ITS CONCURRENT USAGE OVER TIME USING SEMANTICALLY SEGMENTED MOBILE MAPPING DATA

2021 ◽  
Vol XLIII-B2-2021 ◽  
pp. 185-192
Author(s):  
A. Leichter ◽  
U. Feuerhake ◽  
M. Sester
Keyword(s):  
Public Space ◽  
Point Cloud ◽  
Point Clouds ◽  
Parking Space ◽  
3D Point Cloud ◽  
Cloud Data ◽  
3D Point Clouds ◽  
3D Data ◽  
Rgb Images ◽  
Bounding Boxes

Abstract. Public space is a scarce good in cities. There are many concurrent usages, which makes an adequate allocation of space both difficult and highly attractive. A lot of space is allocated by parking cars – even if the parking spaces are not occupied by cars all the time. In this work, we analyze space demand and usage by parking cars, in order to evaluate, when this space could be used for other purposes. The analysis is based on 3D point clouds acquired at several times during a day. We propose a processing pipeline to extract car bounding boxes from a given 3D point cloud. For the car extraction we utilize a label transfer technique for transfers from semantically segmented 2D RGB images to 3D point cloud data. This semantically segmented 3D data allows us to identify car instances. Subsequently, we aggregate and analyze information about parking cars. We present an exemplary analysis of the urban area where we extracted 15.000 cars at five different points in time. Based on this aggregated we present analytical results for time dependent parking behavior, parking space availability and utilization.

scholarly journals Symmetry Analysis of Oriental Polygonal Pagodas Using 3D Point Clouds for Cultural Heritage

Sensors ◽  
2021 ◽  
Vol 21 (4) ◽  
pp. 1228
Author(s):  
Ting On Chan ◽  
Linyuan Xia ◽  
Yimin Chen ◽  
Wei Lang ◽  
Tingting Chen ◽  
...  
Keyword(s):  
Cultural Heritage ◽  
Unmanned Aerial Vehicle ◽  
Point Cloud ◽  
Geometric Model ◽  
Digital Camera ◽  
Point Clouds ◽  
Symmetry Analysis ◽  
3D Point Clouds ◽  
Transmission Towers ◽  
Aerial Vehicle

Ancient pagodas are usually parts of hot tourist spots in many oriental countries due to their unique historical backgrounds. They are usually polygonal structures comprised by multiple floors, which are separated by eaves. In this paper, we propose a new method to investigate both the rotational and reflectional symmetry of such polygonal pagodas through developing novel geometric models to fit to the 3D point clouds obtained from photogrammetric reconstruction. The geometric model consists of multiple polygonal pyramid/prism models but has a common central axis. The method was verified by four datasets collected by an unmanned aerial vehicle (UAV) and a hand-held digital camera. The results indicate that the models fit accurately to the pagodas’ point clouds. The symmetry was realized by rotating and reflecting the pagodas’ point clouds after a complete leveling of the point cloud was achieved using the estimated central axes. The results show that there are RMSEs of 5.04 cm and 5.20 cm deviated from the perfect (theoretical) rotational and reflectional symmetries, respectively. This concludes that the examined pagodas are highly symmetric, both rotationally and reflectionally. The concept presented in the paper not only work for polygonal pagodas, but it can also be readily transformed and implemented for other applications for other pagoda-like objects such as transmission towers.

scholarly journals MATLAB Virtual Toolbox for Retrospective Rockfall Source Detection and Volume Estimation Using 3D Point Clouds: A Case Study of a Subalpine Molasse Cliff

Geosciences ◽  
2021 ◽  
Vol 11 (2) ◽  
pp. 75
Author(s):  
Dario Carrea ◽  
Antonio Abellan ◽  
Marc-Henri Derron ◽  
Neal Gauvin ◽  
Michel Jaboyedoff
Keyword(s):  
Point Cloud ◽  
Natural Hazard ◽  
Point Clouds ◽  
Volume Estimation ◽  
Natural Phenomena ◽  
Frequency Distributions ◽  
Erosion Rates ◽  
3D Point Clouds

The use of 3D point clouds to improve the understanding of natural phenomena is currently applied in natural hazard investigations, including the quantification of rockfall activity. However, 3D point cloud treatment is typically accomplished using nondedicated (and not optimal) software. To fill this gap, we present an open-source, specific rockfall package in an object-oriented toolbox developed in the MATLAB® environment. The proposed package offers a complete and semiautomatic 3D solution that spans from extraction to identification and volume estimations of rockfall sources using state-of-the-art methods and newly implemented algorithms. To illustrate the capabilities of this package, we acquired a series of high-quality point clouds in a pilot study area referred to as the La Cornalle cliff (West Switzerland), obtained robust volume estimations at different volumetric scales, and derived rockfall magnitude–frequency distributions, which assisted in the assessment of rockfall activity and long-term erosion rates. An outcome of the case study shows the influence of the volume computation on the magnitude–frequency distribution and ensuing erosion process interpretation.

scholarly journals Comparison of Depth Camera and Terrestrial Laser Scanner in Monitoring Structural Deflections

Sensors ◽  
2020 ◽  
Vol 21 (1) ◽  
pp. 201
Author(s):  
Michael Bekele Maru ◽  
Donghwan Lee ◽  
Kassahun Demissie Tola ◽  
Seunghee Park
Keyword(s):  
Optical Sensors ◽  
Point Cloud ◽  
Three Dimensional ◽  
Laser Scanner ◽  
Point Clouds ◽  
Depth Camera ◽  
Terrestrial Laser Scanner ◽  
Cloud Data ◽  
3D Point Clouds ◽  
3D Information

Modeling a structure in the virtual world using three-dimensional (3D) information enhances our understanding, while also aiding in the visualization, of how a structure reacts to any disturbance. Generally, 3D point clouds are used for determining structural behavioral changes. Light detection and ranging (LiDAR) is one of the crucial ways by which a 3D point cloud dataset can be generated. Additionally, 3D cameras are commonly used to develop a point cloud containing many points on the external surface of an object around it. The main objective of this study was to compare the performance of optical sensors, namely a depth camera (DC) and terrestrial laser scanner (TLS) in estimating structural deflection. We also utilized bilateral filtering techniques, which are commonly used in image processing, on the point cloud data for enhancing their accuracy and increasing the application prospects of these sensors in structure health monitoring. The results from these sensors were validated by comparing them with the outputs from a linear variable differential transformer sensor, which was mounted on the beam during an indoor experiment. The results showed that the datasets obtained from both the sensors were acceptable for nominal deflections of 3 mm and above because the error range was less than ±10%. However, the result obtained from the TLS were better than those obtained from the DC.

Sign in / Sign up

Export Citation Format

Share Document