scholarly journals MARKER-LESS MOBILE AUGMENTED REALITY APPLICATION FOR MASSIVE 3D POINT CLOUDS AND SEMANTICS

2020 ◽  
Vol XLIII-B2-2020 ◽  
pp. 255-261 ◽  
Author(s):  
A. Kharroubi ◽  
R. Billen ◽  
F. Poux
Keyword(s):  
Augmented Reality ◽  
Real Time ◽  
Point Cloud ◽  
Point Clouds ◽  
Mobile Augmented Reality ◽  
3D Tracking ◽  
Cloud Data ◽  
Network Connection ◽  
3D Point Clouds ◽  
Significant Research

Abstract. Mobile Augmented Reality (MAR) attracts significant research and development efforts from both the industry and academia, but rarely integrate massive 3D dataset’s interactions. The emergence of dedicated AR devices and powerful Software Development Kit (ARCore for android and ARKit for iOS) improves performance on mobile devices (Smartphones and tablets). This is aided by new sensor integration and advances in computer vision that fuels the development of MAR. In this paper, we propose a direct integration of massive 3D point clouds with semantics in a web-based marker-less mobile Augmented Reality (AR) application for real-time visualization. We specifically investigate challenges linked to point cloud data structure and semantic injection. Our solution consolidates some of the overarching principles of AR, of which pose estimation, registration and 3D tracking. The developed AR system is tested on mobile phones web-browsers providing clear insights on the performance of the system. Promising results highlight a number of frame per second varying between 27 and 60 for a real-time point budget of 4.3 million points. The point cloud tested is composed of 29 million points and shows how our indexation strategy permits the integration of massive point clouds aiming at the point budget. The results also gives research directions concerning the dependence and delay related to the quality of the network connection, and the battery consumption since portable sensors are used all the time.

scholarly journals Comparison of Depth Camera and Terrestrial Laser Scanner in Monitoring Structural Deflections

Sensors ◽  
2020 ◽  
Vol 21 (1) ◽  
pp. 201
Author(s):  
Michael Bekele Maru ◽  
Donghwan Lee ◽  
Kassahun Demissie Tola ◽  
Seunghee Park
Keyword(s):  
Optical Sensors ◽  
Point Cloud ◽  
Three Dimensional ◽  
Laser Scanner ◽  
Point Clouds ◽  
Depth Camera ◽  
Terrestrial Laser Scanner ◽  
Cloud Data ◽  
3D Point Clouds ◽  
3D Information

Modeling a structure in the virtual world using three-dimensional (3D) information enhances our understanding, while also aiding in the visualization, of how a structure reacts to any disturbance. Generally, 3D point clouds are used for determining structural behavioral changes. Light detection and ranging (LiDAR) is one of the crucial ways by which a 3D point cloud dataset can be generated. Additionally, 3D cameras are commonly used to develop a point cloud containing many points on the external surface of an object around it. The main objective of this study was to compare the performance of optical sensors, namely a depth camera (DC) and terrestrial laser scanner (TLS) in estimating structural deflection. We also utilized bilateral filtering techniques, which are commonly used in image processing, on the point cloud data for enhancing their accuracy and increasing the application prospects of these sensors in structure health monitoring. The results from these sensors were validated by comparing them with the outputs from a linear variable differential transformer sensor, which was mounted on the beam during an indoor experiment. The results showed that the datasets obtained from both the sensors were acceptable for nominal deflections of 3 mm and above because the error range was less than ±10%. However, the result obtained from the TLS were better than those obtained from the DC.

scholarly journals Fast and Robust Flight Altitude Estimation of Multirotor UAVs in Dynamic Unstructured Environments Using 3D Point Cloud Sensors

Aerospace ◽  
2018 ◽  
Vol 5 (3) ◽  
pp. 94 ◽  
Author(s):  
Hriday Bavle ◽  
Jose Sanchez-Lopez ◽  
Paloma Puente ◽  
Alejandro Rodriguez-Ramos ◽  
Carlos Sampedro ◽  
...  
Keyword(s):  
Point Cloud ◽  
Point Clouds ◽  
Estimation Algorithm ◽  
Indoor Environments ◽  
3D Point Cloud ◽  
Flight Altitude ◽  
Cloud Data ◽  
3D Point Clouds ◽  
Laser Altimeters ◽  
Altitude Estimation

This paper presents a fast and robust approach for estimating the flight altitude of multirotor Unmanned Aerial Vehicles (UAVs) using 3D point cloud sensors in cluttered, unstructured, and dynamic indoor environments. The objective is to present a flight altitude estimation algorithm, replacing the conventional sensors such as laser altimeters, barometers, or accelerometers, which have several limitations when used individually. Our proposed algorithm includes two stages: in the first stage, a fast clustering of the measured 3D point cloud data is performed, along with the segmentation of the clustered data into horizontal planes. In the second stage, these segmented horizontal planes are mapped based on the vertical distance with respect to the point cloud sensor frame of reference, in order to provide a robust flight altitude estimation even in presence of several static as well as dynamic ground obstacles. We validate our approach using the IROS 2011 Kinect dataset available in the literature, estimating the altitude of the RGB-D camera using the provided 3D point clouds. We further validate our approach using a point cloud sensor on board a UAV, by means of several autonomous real flights, closing its altitude control loop using the flight altitude estimated by our proposed method, in presence of several different static as well as dynamic ground obstacles. In addition, the implementation of our approach has been integrated in our open-source software framework for aerial robotics called Aerostack.

scholarly journals VISUALIZATION OF POINT CLOUD MODELS IN MOBILE AUGMENTED REALITY USING CONTINUOUS LEVEL OF DETAIL METHOD

2020 ◽  
Vol XLIV-4/W1-2020 ◽  
pp. 167-170
Author(s):  
L. Zhang ◽  
P. van Oosterom ◽  
H. Liu
Keyword(s):  
Augmented Reality ◽  
Point Cloud ◽  
Cloud Model ◽  
Point Clouds ◽  
Level Of Detail ◽  
Mobile Augmented Reality ◽  
Cloud Models ◽  
Point Cloud Model ◽  
Visual Artifacts ◽  
Mobile Ar

Abstract. Point clouds have become one of the most popular sources of data in geospatial fields due to their availability and flexibility. However, because of the large amount of data and the limited resources of mobile devices, the use of point clouds in mobile Augmented Reality applications is still quite limited. Many current mobile AR applications of point clouds lack fluent interactions with users. In our paper, a cLoD (continuous level-of-detail) method is introduced to filter the number of points to be rendered considerably, together with an adaptive point size rendering strategy, thus improve the rendering performance and remove visual artifacts of mobile AR point cloud applications. Our method uses a cLoD model that has an ideal distribution over LoDs, with which can remove unnecessary points without sudden changes in density as present in the commonly used discrete level-of-detail approaches. Besides, camera position, orientation and distance from the camera to point cloud model is taken into consideration as well. With our method, good interactive visualization of point clouds can be realized in the mobile AR environment, with both nice visual quality and proper resource consumption.

scholarly journals 3D Point Cloud Data in Conveying Information for Local Green Factor Assessment

2021 ◽  
Vol 10 (11) ◽  
pp. 762
Author(s):  
Kaisa Jaalama ◽  
Heikki Kauhanen ◽  
Aino Keitaanniemi ◽  
Toni Rantanen ◽  
Juho-Pekka Virtanen ◽  
...  
Keyword(s):  
Green Infrastructure ◽  
Point Cloud ◽  
Point Clouds ◽  
Urban Ecosystem ◽  
Small Scale ◽  
Point Cloud Data ◽  
Multidisciplinary Research ◽  
Cloud Data ◽  
3D Point Clouds ◽  
Geometrical Form

The importance of ensuring the adequacy of urban ecosystem services and green infrastructure has been widely highlighted in multidisciplinary research. Meanwhile, the consolidation of cities has been a dominant trend in urban development and has led to the development and implementation of the green factor tool in cities such as Berlin, Melbourne, and Helsinki. In this study, elements of the green factor tool were monitored with laser-scanned and photogrammetrically derived point cloud datasets encompassing a yard in Espoo, Finland. The results show that with the support of 3D point clouds, it is possible to support the monitoring of the local green infrastructure, including elements of smaller size in green areas and yards. However, point clouds generated by distinct means have differing abilities in conveying information on green elements, and canopy covers, for example, might hinder these abilities. Additionally, some green factor elements are more promising for 3D measurement-based monitoring than others, such as those with clear geometrical form. The results encourage the involvement of 3D measuring technologies for monitoring local urban green infrastructure (UGI), also of small scale.

An Improved Method for Restoring the Shape of 3D Point Cloud Surfaces

2018 ◽  
Vol 9 (2) ◽  
pp. 37-53
Author(s):  
Sinh Van Nguyen ◽  
Ha Manh Tran ◽  
Minh Khai Tran
Keyword(s):  
Geometric Modeling ◽  
Point Cloud ◽  
Point Clouds ◽  
Tangent Plane ◽  
Surface Patch ◽  
3D Point Cloud ◽  
Improved Method ◽  
Initial Shape ◽  
Cloud Data ◽  
3D Point Clouds

Building 3D objects or reconstructing their surfaces from 3D point cloud data are researched activities in the field of geometric modeling and computer graphics. In the recent years, they are also studied and used in some fields such as: graph models and simulation; image processing or restoration of digital heritages. This article presents an improved method for restoring the shape of 3D point cloud surfaces. The method is a combination of creating a Bezier surface patch and computing tangent plane of 3D points to fill holes on a surface of 3D point clouds. This method is described as follows: at first, a boundary for each hole on the surface is identified. The holes are then filled by computing Bezier curves of surface patches to find missing points. After that, the holes are refined based on two steps (rough and elaborate) to adjust the inserted points and preserve the local curvature of the holes. The contribution of the proposed method has been shown in processing time and the novelty of combined computation in this method has preserved the initial shape of the surface

6-DOF Localization for a Mobile Robot Using Outdoor 3D Point Clouds

2010 ◽  
Vol 22 (2) ◽  
pp. 158-166 ◽  
Author(s):  
Taro Suzuki ◽  
◽  
Yoshiharu Amano ◽  
Takumi Hashizume
Keyword(s):  
Mobile Robot ◽  
Point Cloud ◽  
Field Experiments ◽  
Point Clouds ◽  
Measurement Model ◽  
3D Point Cloud ◽  
Point Cloud Data ◽  
Cloud Data ◽  
Continuous Contact ◽  
3D Point Clouds

This paper describes outdoor localization for a mobile robot using a laser scanner and three-dimensional (3D) point cloud data. A Mobile Mapping System (MMS) measures outdoor 3D point clouds easily and precisely. The full six-dimensional state of a mobile robot is estimated combining dead reckoning and 3D point cloud data. Two-dimensional (2D) position and orientation are extended to 3D using 3D point clouds assuming that the mobile robot remains in continuous contact with the road surface. Our approach applies a particle filter to correct position error in the laser measurement model in 3D point cloud space. Field experiments were conducted to evaluate the accuracy of our proposal. As the result of the experiment, it was confirmed that a localization precision of 0.2 m (RMS) is possible using our proposal.

scholarly journals Automatic Grapevine Trunk Detection on UAV-Based Point Cloud

2020 ◽  
Vol 12 (18) ◽  
pp. 3043 ◽  
Author(s):  
Juan M. Jurado ◽  
Luís Pádua ◽  
Francisco R. Feito ◽  
Joaquim J. Sousa
Keyword(s):  
Point Cloud ◽  
High Performance ◽  
Point Clouds ◽  
Geometrical Parameters ◽  
3D Point Cloud ◽  
Cloud Data ◽  
3D Point Clouds ◽  
Geometrical Features ◽  
Vegetative Cycle ◽  
Automated Methods

The optimisation of vineyards management requires efficient and automated methods able to identify individual plants. In the last few years, Unmanned Aerial Vehicles (UAVs) have become one of the main sources of remote sensing information for Precision Viticulture (PV) applications. In fact, high resolution UAV-based imagery offers a unique capability for modelling plant’s structure making possible the recognition of significant geometrical features in photogrammetric point clouds. Despite the proliferation of innovative technologies in viticulture, the identification of individual grapevines relies on image-based segmentation techniques. In that way, grapevine and non-grapevine features are separated and individual plants are estimated usually considering a fixed distance between them. In this study, an automatic method for grapevine trunk detection, using 3D point cloud data, is presented. The proposed method focuses on the recognition of key geometrical parameters to ensure the existence of every plant in the 3D model. The method was tested in different commercial vineyards and to push it to its limit a vineyard characterised by several missing plants along the vine rows, irregular distances between plants and occluded trunks by dense vegetation in some areas, was also used. The proposed method represents a disruption in relation to the state of the art, and is able to identify individual trunks, posts and missing plants based on the interpretation and analysis of a 3D point cloud. Moreover, a validation process was carried out allowing concluding that the method has a high performance, especially when it is applied to 3D point clouds generated in phases in which the leaves are not yet very dense (January to May). However, if correct flight parametrizations are set, the method remains effective throughout the entire vegetative cycle.

scholarly journals A Mobile Augmented Reality System for the Real-Time Visualization of Pipes in Point Cloud Data with a Depth Sensor

Electronics ◽  
2020 ◽  
Vol 9 (5) ◽  
pp. 836 ◽  
Author(s):  
Young-Hoon Jin ◽  
In-Tae Hwang ◽  
Won-Hyung Lee
Keyword(s):  
Augmented Reality ◽  
Real Time ◽  
Point Cloud ◽  
Depth Image ◽  
Visualization Method ◽  
Point Cloud Data ◽  
Matching Method ◽  
The Real ◽  
Cloud Data ◽  
Real Time Visualization

Augmented reality (AR) is a useful visualization technology that displays information by adding virtual images to the real world. In AR systems that require three-dimensional information, point cloud data is easy to use after real-time acquisition, however, it is difficult to measure and visualize real-time objects due to the large amount of data and a matching process. In this paper we explored a method of estimating pipes from point cloud data and visualizing them in real-time through augmented reality devices. In general, pipe estimation in a point cloud uses a Hough transform and is performed through a preprocessing process, such as noise filtering, normal estimation, or segmentation. However, there is a disadvantage in that the execution time is slow due to a large amount of computation. Therefore, for the real-time visualization in augmented reality devices, the fast cylinder matching method using random sample consensus (RANSAC) is required. In this paper, we proposed parallel processing, multiple frames, adjustable scale, and error correction for real-time visualization. The real-time visualization method through the augmented reality device obtained a depth image from the sensor and configured a uniform point cloud using a voxel grid algorithm. The constructed data was analyzed according to the fast cylinder matching method using RANSAC. The real-time visualization method through augmented reality devices is expected to be used to identify problems, such as the sagging of pipes, through real-time measurements at plant sites due to the spread of various AR devices.

scholarly journals A Novel Preprocessing Method for Dynamic Point-Cloud Compression

2021 ◽  
Vol 11 (13) ◽  
pp. 5941
Author(s):  
Mun-yong Lee ◽  
Sang-ha Lee ◽  
Kye-dong Jung ◽  
Seung-hyun Lee ◽  
Soon-chul Kwon
Keyword(s):  
Data Processing ◽  
Real Time ◽  
Point Cloud ◽  
High Performance ◽  
Three Dimensional ◽  
Point Clouds ◽  
3D Models ◽  
File Size ◽  
3D Point Clouds ◽  
Computer Based

Computer-based data processing capabilities have evolved to handle a lot of information. As such, the complexity of three-dimensional (3D) models (e.g., animations or real-time voxels) containing large volumes of information has increased exponentially. This rapid increase in complexity has led to problems with recording and transmission. In this study, we propose a method of efficiently managing and compressing animation information stored in the 3D point-clouds sequence. A compressed point-cloud is created by reconfiguring the points based on their voxels. Compared with the original point-cloud, noise caused by errors is removed, and a preprocessing procedure that achieves high performance in a redundant processing algorithm is proposed. The results of experiments and rendering demonstrate an average file-size reduction of 40% using the proposed algorithm. Moreover, 13% of the over-lap data are extracted and removed, and the file size is further reduced.

scholarly journals CONTEXTUAL CLASSIFICATION OF POINT CLOUD DATA BY EXPLOITING INDIVIDUAL 3D NEIGBOURHOODS

2015 ◽  
Vol II-3/W4 ◽  
pp. 271-278 ◽  
Author(s):  
M. Weinmann ◽  
A. Schmidt ◽  
C. Mallet ◽  
S. Hinz ◽  
F. Rottensteiner ◽  
...  
Keyword(s):  
Feature Extraction ◽  
Point Cloud ◽  
Contextual Information ◽  
Point Clouds ◽  
Optimal Size ◽  
Point Cloud Data ◽  
Cloud Data ◽  
Semantic Class ◽  
3D Point Clouds

The fully automated analysis of 3D point clouds is of great importance in photogrammetry, remote sensing and computer vision. For reliably extracting objects such as buildings, road inventory or vegetation, many approaches rely on the results of a point cloud classification, where each 3D point is assigned a respective semantic class label. Such an assignment, in turn, typically involves statistical methods for feature extraction and machine learning. Whereas the different components in the processing workflow have extensively, but separately been investigated in recent years, the respective connection by sharing the results of crucial tasks across all components has not yet been addressed. This connection not only encapsulates the interrelated issues of neighborhood selection and feature extraction, but also the issue of how to involve spatial context in the classification step. In this paper, we present a novel and generic approach for 3D scene analysis which relies on (<i>i</i>) individually optimized 3D neighborhoods for (<i>ii</i>) the extraction of distinctive geometric features and (<i>iii</i>) the contextual classification of point cloud data. For a labeled benchmark dataset, we demonstrate the beneficial impact of involving contextual information in the classification process and that using individual 3D neighborhoods of optimal size significantly increases the quality of the results for both pointwise and contextual classification.

Sign in / Sign up

Export Citation Format

Share Document