scholarly journals Pointwise Rotation-Invariant Network with Adaptive Sampling and 3D Spherical Voxel Convolution

2020 ◽  
Vol 34 (07) ◽  
pp. 12717-12724
Author(s):  
Yang You ◽  
Yujing Lou ◽  
Qi Liu ◽  
Yu-Wing Tai ◽  
Lizhuang Ma ◽  
...  
Keyword(s):  
Adaptive Sampling ◽  
Point Cloud ◽  
Data Augmentation ◽  
Feature Matching ◽  
State Of The Art ◽  
Point Clouds ◽  
Rotation Invariant ◽  
Learning Framework ◽  
Point Set ◽  
Part Segmentation

Point cloud analysis without pose priors is very challenging in real applications, as the orientations of point clouds are often unknown. In this paper, we propose a brand new point-set learning framework PRIN, namely, Pointwise Rotation-Invariant Network, focusing on rotation-invariant feature extraction in point clouds analysis. We construct spherical signals by Density Aware Adaptive Sampling to deal with distorted point distributions in spherical space. In addition, we propose Spherical Voxel Convolution and Point Re-sampling to extract rotation-invariant features for each point. Our network can be applied to tasks ranging from object classification, part segmentation, to 3D feature matching and label alignment. We show that, on the dataset with randomly rotated point clouds, PRIN demonstrates better performance than state-of-the-art methods without any data augmentation. We also provide theoretical analysis for the rotation-invariance achieved by our methods.

scholarly journals Triple-GAIL: A Multi-Modal Imitation Learning Framework with Generative Adversarial Nets

2020 ◽  
Author(s):  
Cong Fei ◽  
Bin Wang ◽  
Yuzheng Zhuang ◽  
Zongzhang Zhang ◽  
Jianye Hao ◽  
...  
Keyword(s):  
Real World ◽  
Autonomous Vehicles ◽  
Data Augmentation ◽  
State Of The Art ◽  
Robot Learning ◽  
Imitation Learning ◽  
Proper Understanding ◽  
Learning Framework ◽  
Strategy Game ◽  
Real Time Strategy Game

Generative adversarial imitation learning (GAIL) has shown promising results by taking advantage of generative adversarial nets, especially in the field of robot learning. However, the requirement of isolated single modal demonstrations limits the scalability of the approach to real world scenarios such as autonomous vehicles' demand for a proper understanding of human drivers' behavior. In this paper, we propose a novel multi-modal GAIL framework, named Triple-GAIL, that is able to learn skill selection and imitation jointly from both expert demonstrations and continuously generated experiences with data augmentation purpose by introducing an auxiliary selector. We provide theoretical guarantees on the convergence to optima for both of the generator and the selector respectively. Experiments on real driver trajectories and real-time strategy game datasets demonstrate that Triple-GAIL can better fit multi-modal behaviors close to the demonstrators and outperforms state-of-the-art methods.

scholarly journals VISUALIZATION OF THE CONSTRUCTION OF ANCIENT ROMAN BUILDINGS IN OSTIA USING POINT CLOUD DATA

2017 ◽  
Vol XLII-2/W3 ◽  
pp. 345-352
Author(s):  
Y. Hori ◽  
T. Ogawa
Keyword(s):  
Point Cloud ◽  
Laser Scanning ◽  
Feature Matching ◽  
Three Dimensional ◽  
Point Clouds ◽  
Final Report ◽  
Point Cloud Data ◽  
Cloud Data ◽  
Coverage Accuracy ◽  
New Research

The implementation of laser scanning in the field of archaeology provides us with an entirely new dimension in research and surveying. It allows us to digitally recreate individual objects, or entire cities, using millions of three-dimensional points grouped together in what is referred to as "point clouds". In addition, the visualization of the point cloud data, which can be used in the final report by archaeologists and architects, should usually be produced as a JPG or TIFF file. Not only the visualization of point cloud data, but also re-examination of older data and new survey of the construction of Roman building applying remote-sensing technology for precise and detailed measurements afford new information that may lead to revising drawings of ancient buildings which had been adduced as evidence without any consideration of a degree of accuracy, and finally can provide new research of ancient buildings. We used laser scanners at fields because of its speed, comprehensive coverage, accuracy and flexibility of data manipulation. Therefore, we “skipped” many of post-processing and focused on the images created from the meta-data simply aligned using a tool which extended automatic feature-matching algorithm and a popular renderer that can provide graphic results.

scholarly journals A comprehensive study of the rate-distortion performance in MPEG point cloud compression

2019 ◽  
Vol 8 ◽  
Author(s):  
Evangelos Alexiou ◽  
Irene Viola ◽  
Tomás M. Borges ◽  
Tiago A. Fonseca ◽  
Ricardo L. de Queiroz ◽  
...  
Keyword(s):  
Point Cloud ◽  
State Of The Art ◽  
Subjective Evaluation ◽  
Rate Distortion ◽  
Point Clouds ◽  
Current Approach ◽  
User Engagement ◽  
Viable Solution ◽  
Wide Range ◽  
Objective Quality

Abstract Recent trends in multimedia technologies indicate the need for richer imaging modalities to increase user engagement with the content. Among other alternatives, point clouds denote a viable solution that offers an immersive content representation, as witnessed by current activities in JPEG and MPEG standardization committees. As a result of such efforts, MPEG is at the final stages of drafting an emerging standard for point cloud compression, which we consider as the state-of-the-art. In this study, the entire set of encoders that have been developed in the MPEG committee are assessed through an extensive and rigorous analysis of quality. We initially focus on the assessment of encoding configurations that have been defined by experts in MPEG for their core experiments. Then, two additional experiments are designed and carried to address some of the identified limitations of current approach. As part of the study, state-of-the-art objective quality metrics are benchmarked to assess their capability to predict visual quality of point clouds under a wide range of radically different compression artifacts. To carry the subjective evaluation experiments, a web-based renderer is developed and described. The subjective and objective quality scores along with the rendering software are made publicly available, to facilitate and promote research on the field.

scholarly journals FAST PROBABILISTIC FUSION OF 3D POINT CLOUDS VIA OCCUPANCY GRIDS FOR SCENE CLASSIFICATION

2016 ◽  
Vol III-3 ◽  
pp. 325-332
Author(s):  
Andreas Kuhn ◽  
Hai Huang ◽  
Martin Drauschke ◽  
Helmut Mayer
Keyword(s):  
High Resolution ◽  
Point Cloud ◽  
State Of The Art ◽  
Point Clouds ◽  
Automatic Generation ◽  
Scene Classification ◽  
Semantic Classification ◽  
3D Point Clouds ◽  
Occupancy Grids ◽  
High Scalability

High resolution consumer cameras on Unmanned Aerial Vehicles (UAVs) allow for cheap acquisition of highly detailed images, e.g., of urban regions. Via image registration by means of Structure from Motion (SfM) and Multi View Stereo (MVS) the automatic generation of huge amounts of 3D points with a relative accuracy in the centimeter range is possible. Applications such as semantic classification have a need for accurate 3D point clouds, but do not benefit from an extremely high resolution/density. In this paper, we, therefore, propose a fast fusion of high resolution 3D point clouds based on occupancy grids. The result is used for semantic classification. In contrast to state-of-the-art classification methods, we accept a certain percentage of outliers, arguing that they can be considered in the classification process when a per point belief is determined in the fusion process. To this end, we employ an octree-based fusion which allows for the derivation of outlier probabilities. The probabilities give a belief for every 3D point, which is essential for the semantic classification to consider measurement noise. For an example point cloud with half a billion 3D points (cf. Figure 1), we show that our method can reduce runtime as well as improve classification accuracy and offers high scalability for large datasets.

scholarly journals Point Set Multi-Level Aggregation Feature Extraction Based on Multi-Scale Max Pooling and LDA for Point Cloud Classification

2019 ◽  
Vol 11 (23) ◽  
pp. 2846 ◽  
Author(s):  
Tong ◽  
Li ◽  
Zhang ◽  
Chen ◽  
Zhang ◽  
...  
Keyword(s):  
Point Cloud ◽  
Laser Scanning ◽  
Single Point ◽  
Point Clouds ◽  
Max Pooling ◽  
Multi Scale ◽  
Cloud Classification ◽  
Point Set ◽  
Multi Level ◽  
Point Cloud Classification

Accurate and effective classification of lidar point clouds with discriminative features expression is a challenging task for scene understanding. In order to improve the accuracy and the robustness of point cloud classification based on single point features, we propose a novel point set multi-level aggregation features extraction and fusion method based on multi-scale max pooling and latent Dirichlet allocation (LDA). To this end, in the hierarchical point set feature extraction, point sets of different levels and sizes are first adaptively generated through multi-level clustering. Then, more effective sparse representation is implemented by locality-constrained linear coding (LLC) based on single point features, which contributes to the extraction of discriminative individual point set features. Next, the local point set features are extracted by combining the max pooling method and the multi-scale pyramid structure constructed by the point’s coordinates within each point set. The global and the local features of the point sets are effectively expressed by the fusion of multi-scale max pooling features and global features constructed by the point set LLC-LDA model. The point clouds are classified by using the point set multi-level aggregation features. Our experiments on two scenes of airborne laser scanning (ALS) point clouds—a mobile laser scanning (MLS) scene point cloud and a terrestrial laser scanning (TLS) scene point cloud—demonstrate the effectiveness of the proposed point set multi-level aggregation features for point cloud classification, and the proposed method outperforms other related and compared algorithms.

scholarly journals DERIVING 3D POINT CLOUDS FROM TERRESTRIAL PHOTOGRAPHS - COMPARISON OF DIFFERENT SENSORS AND SOFTWARE

2016 ◽  
Vol XLI-B5 ◽  
pp. 685-692 ◽  
Author(s):  
Robert Niederheiser ◽  
Martin Mokroš ◽  
Julia Lange ◽  
Helene Petschko ◽  
Günther Prasicek ◽  
...  
Keyword(s):  
Point Cloud ◽  
State Of The Art ◽  
User Interaction ◽  
Point Clouds ◽  
Rock Face ◽  
3D Point Clouds ◽  
Software Packages ◽  
User Friendly ◽  
Insight Into

Terrestrial photogrammetry nowadays offers a reasonably cheap, intuitive and effective approach to 3D-modelling. However, the important choice, which sensor and which software to use is not straight forward and needs consideration as the choice will have effects on the resulting 3D point cloud and its derivatives. <br><br> We compare five different sensors as well as four different state-of-the-art software packages for a single application, the modelling of a vegetated rock face. The five sensors represent different resolutions, sensor sizes and price segments of the cameras. The software packages used are: (1) Agisoft PhotoScan Pro (1.16), (2) Pix4D (2.0.89), (3) a combination of Visual SFM (V0.5.22) and SURE (1.2.0.286), and (4) MicMac (1.0). We took photos of a vegetated rock face from identical positions with all sensors. Then we compared the results of the different software packages regarding the ease of the workflow, visual appeal, similarity and quality of the point cloud. <br><br> While PhotoScan and Pix4D offer the user-friendliest workflows, they are also “black-box” programmes giving only little insight into their processing. Unsatisfying results may only be changed by modifying settings within a module. The combined workflow of Visual SFM, SURE and CloudCompare is just as simple but requires more user interaction. MicMac turned out to be the most challenging software as it is less user-friendly. However, MicMac offers the most possibilities to influence the processing workflow. The resulting point-clouds of PhotoScan and MicMac are the most appealing.

scholarly journals COMPARISON OF 2D AND 3D APPROACHES FOR THE ALIGNMENT OF UAV AND LIDAR POINT CLOUDS

2017 ◽  
Vol XLII-2/W6 ◽  
pp. 275-279
Author(s):  
R. A. Persad ◽  
C. Armenakis
Keyword(s):  
Prior Knowledge ◽  
Point Cloud ◽  
Feature Matching ◽  
Point Clouds ◽  
Airborne Lidar ◽  
Target Point ◽  
Coordinate Systems ◽  
3D Point Clouds ◽  
Transformation Parameters ◽  
Matching Techniques

The automatic alignment of 3D point clouds acquired or generated from different sensors is a challenging problem. The objective of the alignment is to estimate the 3D similarity transformation parameters, including a global scale factor, 3 rotations and 3 translations. To do so, corresponding anchor features are required in both data sets. There are two main types of alignment: i) Coarse alignment and ii) Refined Alignment. Coarse alignment issues include lack of any prior knowledge of the respective coordinate systems for a source and target point cloud pair and the difficulty to extract and match corresponding control features (e.g., points, lines or planes) co-located on both point cloud pairs to be aligned. With the increasing use of UAVs, there is a need to automatically co-register their generated point cloud-based digital surface models with those from other data acquisition systems such as terrestrial or airborne lidar point clouds. This works presents a comparative study of two independent feature matching techniques for addressing 3D conformal point cloud alignment of UAV and lidar data in different 3D coordinate systems without any prior knowledge of the seven transformation parameters.

scholarly journals Effective Planar Cluster Detection in Point Clouds Using Histogram-Driven Kd-Like Partition and Shifted Mahalanobis Distance Based Regression

2019 ◽  
Vol 11 (21) ◽  
pp. 2465 ◽  
Author(s):  
Jakub Walczak ◽  
Tadeusz Poreda ◽  
Adam Wojciechowski
Keyword(s):  
Mahalanobis Distance ◽  
Point Cloud ◽  
State Of The Art ◽  
Point Clouds ◽  
Rejection Rate ◽  
Regular Space ◽  
Planar Surface ◽  
Point Cloud Segmentation ◽  
Partition Method ◽  
Excessive Number

Point cloud segmentation for planar surface detection is a valid problem of automatic laser scans analysis. It is widely exploited for many industrial remote sensing tasks, such as LIDAR city scanning, creating inventories of buildings, or object reconstruction. Many current methods rely on robustly calculated covariance and centroid for plane model estimation or global energy optimization. This is coupled with point cloud division strategies, based on uniform or regular space subdivision. These approaches result in many redundant divisions, plane maladjustments caused by outliers, and excessive number of processing iterations. In this paper, a new robust method of point clouds segmentation, based on histogram-driven hierarchical space division, inspired by kd-tree is presented. The proposed partition method produces results with a smaller oversegmentation rate. Moreover, state-of-the-art partitions often lead to nodes of low cardinality, which results in the rejection of many points. In the proposed method, the point rejection rate was reduced. Point cloud subdivision is followed by resilient plane estimation, using Mahalanobis distance with respect to seven cardinal points. These points were established based on eigenvectors of the covariance matrix of the considered point cluster. The proposed method shows high robustness and yields good quality metrics, much faster than a FAST-MCD approach. The overall results indicate improvements in terms of plane precision, plane recall, under-, and the over- segmentation rate with respect to the reference benchmark methods. Plane precision for the S3DIS dataset increased on average by 2.6pp and plane recall- by 3pp. Both over- and under- segmentation rates fell by 3.2pp and 4.3pp.

scholarly journals Dynamic polygon clouds: representation and compression for VR/AR

2018 ◽  
Vol 7 ◽  
Author(s):  
Eduardo Pavez ◽  
Philip A. Chou ◽  
Ricardo L. de Queiroz ◽  
Antonio Ortega
Keyword(s):  
Point Cloud ◽  
State Of The Art ◽  
Three Dimensional ◽  
Point Clouds ◽  
Polygonal Meshes ◽  
Triangle Meshes ◽  
Previous State ◽  
Temporal Redundancy ◽  
Inter Frame ◽  
Robust To Noise

We introduce the polygon cloud, a compressible representation of three-dimensional geometry (including attributes, such as color), intermediate between polygonal meshes and point clouds. Dynamic polygon clouds, like dynamic polygonal meshes and dynamic point clouds, can take advantage of temporal redundancy for compression. In this paper, we propose methods for compressing both static and dynamic polygon clouds, specifically triangle clouds. We compare triangle clouds to both triangle meshes and point clouds in terms of compression, for live captured dynamic colored geometry. We find that triangle clouds can be compressed nearly as well as triangle meshes, while being more robust to noise and other structures typically found in live captures, which violate the assumption of a smooth surface manifold, such as lines, points, and ragged boundaries. We also find that triangle clouds can be used to compress point clouds with significantly better performance than previously demonstrated point cloud compression methods. For intra-frame coding of geometry, our method improves upon octree-based intra-frame coding by a factor of 5–10 in bit rate. Inter-frame coding improves this by another factor of 2–5. Overall, our proposed method improves over the previous state-of-the-art in dynamic point cloud compression by 33% or more.

scholarly journals Detail Preserved Surface Reconstruction from Point Cloud

Sensors ◽  
2019 ◽  
Vol 19 (6) ◽  
pp. 1278 ◽  
Author(s):  
Yang Zhou ◽  
Shuhan Shen ◽  
Zhanyi Hu
Keyword(s):  
Total Energy ◽  
Surface Reconstruction ◽  
Point Cloud ◽  
State Of The Art ◽  
Point Clouds ◽  
Experimental Result ◽  
Line Of Sight ◽  
Noise Filtering ◽  
Energy Term ◽  
Object Edge

In this paper, we put forward a new method for surface reconstruction from image-based point clouds. In particular, we introduce a new visibility model for each line of sight to preserve scene details without decreasing the noise filtering ability. To make the proposed method suitable for point clouds with heavy noise, we introduce a new likelihood energy term to the total energy of the binary labeling problem of Delaunay tetrahedra, and we give its s-t graph implementation. Besides, we further improve the performance of the proposed method with the dense visibility technique, which helps to keep the object edge sharp. The experimental result shows that the proposed method rivalled the state-of-the-art methods in terms of accuracy and completeness, and performed better with reference to detail preservation.

Sign in / Sign up

Export Citation Format

Share Document