scholarly journals Tracklet Proposal Network for Multi-Object Tracking on Point Clouds

2021 ◽  
Author(s):  
Hai Wu ◽  
Qing Li ◽  
Chenglu Wen ◽  
Xin Li ◽  
Xiaoliang Fan ◽  
...  
Keyword(s):  
Object Tracking ◽  
Point Cloud ◽  
Point Clouds ◽  
Temporal Features ◽  
Object Proposal ◽  
Multiple Frames

This paper proposes the first tracklet proposal network, named PC-TCNN, for Multi-Object Tracking (MOT) on point clouds. Our pipeline first generates tracklet proposals, then refines these tracklets and associates them to generate long trajectories. Specifically, object proposal generation and motion regression are first performed on a point cloud sequence to generate tracklet candidates. Then, spatial-temporal features of each tracklet are exploited and their consistency is used to refine the tracklet proposal. Finally, the refined tracklets across multiple frames are associated to perform MOT on the point cloud sequence. The PC-TCNN significantly improves the MOT performance by introducing the tracklet proposal design. On the KITTI tracking benchmark, it attains an MOTA of 91.75%, outperforming all submitted results on the online leaderboard.

Object Tracking With a Range Camera for Augmented Reality Assembly Assistance

2016 ◽  
Vol 16 (1) ◽  
Author(s):  
Rafael Radkowski
Keyword(s):  
Augmented Reality ◽  
Object Tracking ◽  
Point Cloud ◽  
Sampling Methods ◽  
Point Clouds ◽  
Reference Object ◽  
Key Factors ◽  
Tracking Method ◽  
Uniform Sampling ◽  
Feature Descriptors

This paper introduces a 3D object tracking method for an augmented reality (AR) assembly assistance application. The tracking method relies on point clouds; it uses 3D feature descriptors and point cloud matching with the iterative closest points (ICP) algorithm. The feature descriptors identify an object in a point cloud; ICP align a reference object with this point cloud. The challenge is to achieve high fidelity while maintaining camera frame rates. The point cloud and reference object sampling density are one of the key factors to meet this challenge. In this research, three-point sampling methods and two-point cloud search algorithms were compared to assess their fidelity when tracking typical products of mechanical engineering. The results indicate that a uniform sampling maintains the best fidelity at camera frame rates.

Poisson Mesh Reconstruction for Accurate Object Tracking With Low-Fidelity Point Clouds

2016 ◽  
Vol 17 (1) ◽  
Author(s):  
Timothy Garrett ◽  
Saverio Debernardis ◽  
James Oliver ◽  
Rafael Radkowski
Keyword(s):  
Object Tracking ◽  
Point Cloud ◽  
Point Clouds ◽  
Point Cloud Data ◽  
Reconstruction Algorithms ◽  
Icp Algorithm ◽  
Cloud Data ◽  
Reconstruction Operator ◽  
Position And Orientation ◽  
Mesh Reconstruction

Tracking refers to a set of techniques that allows one to calculate the position and orientation of an object with respect to a global reference coordinate system in real time. A common method for tracking with point clouds is the iterative closest point (ICP) algorithm, which relies on the continuous matching of sequential sampled point clouds with a reference point cloud. Modern commodity range cameras provide point cloud data that can be used for that purpose. However, this point cloud data is generally considered as low-fidelity and insufficient for accurate object tracking. Mesh reconstruction algorithms can improve the fidelity of the point cloud by reconstructing the overall shape of the object. This paper explores the potential for point cloud fidelity improvement via the Poisson mesh reconstruction (PMR) algorithm and compares the accuracy with a common ICP-based tracking technique and a local mesh reconstruction operator. The results of an offline simulation are promising.

Registration and Fusion of UAV LiDAR System Sequence Images and Laser Point Clouds

2020 ◽  
Author(s):  
Jiayong Yu ◽  
Longchen Ma ◽  
Maoyi Tian, ◽  
Xiushan Lu
Keyword(s):  
Point Cloud ◽  
Image Data ◽  
Point Clouds ◽  
Optical Image ◽  
Point Cloud Data ◽  
Feature Points ◽  
Lidar System ◽  
Registration Accuracy ◽  
Cloud Data ◽  
Intensity Image

The unmanned aerial vehicle (UAV)-mounted mobile LiDAR system (ULS) is widely used for geomatics owing to its efficient data acquisition and convenient operation. However, due to limited carrying capacity of a UAV, sensors integrated in the ULS should be small and lightweight, which results in decrease in the density of the collected scanning points. This affects registration between image data and point cloud data. To address this issue, the authors propose a method for registering and fusing ULS sequence images and laser point clouds, wherein they convert the problem of registering point cloud data and image data into a problem of matching feature points between the two images. First, a point cloud is selected to produce an intensity image. Subsequently, the corresponding feature points of the intensity image and the optical image are matched, and exterior orientation parameters are solved using a collinear equation based on image position and orientation. Finally, the sequence images are fused with the laser point cloud, based on the Global Navigation Satellite System (GNSS) time index of the optical image, to generate a true color point cloud. The experimental results show the higher registration accuracy and fusion speed of the proposed method, thereby demonstrating its accuracy and effectiveness.

scholarly journals 3D Thermal Monitoring of Jointed Rock Masses through Infrared Thermography and Photogrammetry

2021 ◽  
Vol 13 (5) ◽  
pp. 957
Author(s):  
Guglielmo Grechi ◽  
Matteo Fiorucci ◽  
Gian Marco Marmoni ◽  
Salvatore Martino
Keyword(s):  
Surface Temperature ◽  
Infrared Thermography ◽  
Point Cloud ◽  
Engineering Geology ◽  
Point Clouds ◽  
Jointed Rock ◽  
Geometric Accuracy ◽  
Rock Masses ◽  
Natural Systems ◽  
Temperature Distributions

The study of strain effects in thermally-forced rock masses has gathered growing interest from engineering geology researchers in the last decade. In this framework, digital photogrammetry and infrared thermography have become two of the most exploited remote surveying techniques in engineering geology applications because they can provide useful information concerning geomechanical and thermal conditions of these complex natural systems where the mechanical role of joints cannot be neglected. In this paper, a methodology is proposed for generating point clouds of rock masses prone to failure, combining the high geometric accuracy of RGB optical images and the thermal information derived by infrared thermography surveys. Multiple 3D thermal point clouds and a high-resolution RGB point cloud were separately generated and co-registered by acquiring thermograms at different times of the day and in different seasons using commercial software for Structure from Motion and point cloud analysis. Temperature attributes of thermal point clouds were merged with the reference high-resolution optical point cloud to obtain a composite 3D model storing accurate geometric information and multitemporal surface temperature distributions. The quality of merged point clouds was evaluated by comparing temperature distributions derived by 2D thermograms and 3D thermal models, with a view to estimating their accuracy in describing surface thermal fields. Moreover, a preliminary attempt was made to test the feasibility of this approach in investigating the thermal behavior of complex natural systems such as jointed rock masses by analyzing the spatial distribution and temporal evolution of surface temperature ranges under different climatic conditions. The obtained results show that despite the low resolution of the IR sensor, the geometric accuracy and the correspondence between 2D and 3D temperature measurements are high enough to consider 3D thermal point clouds suitable to describe surface temperature distributions and adequate for monitoring purposes of jointed rock mass.

scholarly journals Linear-Based Incremental Co-Registration of MLS and Photogrammetric Point Clouds

2021 ◽  
Vol 13 (11) ◽  
pp. 2195
Author(s):  
Shiming Li ◽  
Xuming Ge ◽  
Shengfu Li ◽  
Bo Xu ◽  
Zhendong Wang
Keyword(s):  
Point Cloud ◽  
Laser Scanning ◽  
Point Clouds ◽  
Registration Method ◽  
Cloud Data ◽  
Linear Feature ◽  
Data Registration ◽  
Source Point ◽  
Urban Remote Sensing ◽  
Single Data

Today, mobile laser scanning and oblique photogrammetry are two standard urban remote sensing acquisition methods, and the cross-source point-cloud data obtained using these methods have significant differences and complementarity. Accurate co-registration can make up for the limitations of a single data source, but many existing registration methods face critical challenges. Therefore, in this paper, we propose a systematic incremental registration method that can successfully register MLS and photogrammetric point clouds in the presence of a large number of missing data, large variations in point density, and scale differences. The robustness of this method is due to its elimination of noise in the extracted linear features and its 2D incremental registration strategy. There are three main contributions of our work: (1) the development of an end-to-end automatic cross-source point-cloud registration method; (2) a way to effectively extract the linear feature and restore the scale; and (3) an incremental registration strategy that simplifies the complex registration process. The experimental results show that this method can successfully achieve cross-source data registration, while other methods have difficulty obtaining satisfactory registration results efficiently. Moreover, this method can be extended to more point-cloud sources.

scholarly journals Parametric Surface Modelling for Tea Leaf Point Cloud Based on Non-Uniform Rational Basis Spline Technique

Sensors ◽  
2021 ◽  
Vol 21 (4) ◽  
pp. 1304
Author(s):  
Wenchao Wu ◽  
Yongguang Hu ◽  
Yongzong Lu
Keyword(s):  
Point Cloud ◽  
Principal Component ◽  
Point Clouds ◽  
Parametric Surface ◽  
B Spline ◽  
Surface Modelling ◽  
Modelling Method ◽  
3D Architecture ◽  
Tea Leaf ◽  
Fitting Error

Plant leaf 3D architecture changes during growth and shows sensitive response to environmental stresses. In recent years, acquisition and segmentation methods of leaf point cloud developed rapidly, but 3D modelling leaf point clouds has not gained much attention. In this study, a parametric surface modelling method was proposed for accurately fitting tea leaf point cloud. Firstly, principal component analysis was utilized to adjust posture and position of the point cloud. Then, the point cloud was sliced into multiple sections, and some sections were selected to generate a point set to be fitted (PSF). Finally, the PSF was fitted into non-uniform rational B-spline (NURBS) surface. Two methods were developed to generate the ordered PSF and the unordered PSF, respectively. The PSF was firstly fitted as B-spline surface and then was transformed to NURBS form by minimizing fitting error, which was solved by particle swarm optimization (PSO). The fitting error was specified as weighted sum of the root-mean-square error (RMSE) and the maximum value (MV) of Euclidean distances between fitted surface and a subset of the point cloud. The results showed that the proposed modelling method could be used even if the point cloud is largely simplified (RMSE < 1 mm, MV < 2 mm, without performing PSO). Future studies will model wider range of leaves as well as incomplete point cloud.

scholarly journals Learning the Incremental Warp for 3D Vehicle Tracking in LiDAR Point Clouds

2021 ◽  
Vol 13 (14) ◽  
pp. 2770
Author(s):  
Shengjing Tian ◽  
Xiuping Liu ◽  
Meng Liu ◽  
Yuhao Bian ◽  
Junbin Gao ◽  
...  
Keyword(s):  
Point Cloud ◽  
Point Clouds ◽  
Vehicle Tracking ◽  
Complex Nature ◽  
Global Feature ◽  
Tracking Accuracy ◽  
State Estimator ◽  
Latent Space ◽  
3D Shapes ◽  
Cloud Tracking

Object tracking from LiDAR point clouds, which are always incomplete, sparse, and unstructured, plays a crucial role in urban navigation. Some existing methods utilize a learned similarity network for locating the target, immensely limiting the advancements in tracking accuracy. In this study, we leveraged a powerful target discriminator and an accurate state estimator to robustly track target objects in challenging point cloud scenarios. Considering the complex nature of estimating the state, we extended the traditional Lucas and Kanade (LK) algorithm to 3D point cloud tracking. Specifically, we propose a state estimation subnetwork that aims to learn the incremental warp for updating the coarse target state. Moreover, to obtain a coarse state, we present a simple yet efficient discrimination subnetwork. It can project 3D shapes into a more discriminatory latent space by integrating the global feature into each point-wise feature. Experiments on KITTI and PandaSet datasets showed that compared with the most advanced of other methods, our proposed method can achieve significant improvements—in particular, up to 13.68% on KITTI.

scholarly journals Symmetry Analysis of Oriental Polygonal Pagodas Using 3D Point Clouds for Cultural Heritage

Sensors ◽  
2021 ◽  
Vol 21 (4) ◽  
pp. 1228
Author(s):  
Ting On Chan ◽  
Linyuan Xia ◽  
Yimin Chen ◽  
Wei Lang ◽  
Tingting Chen ◽  
...  
Keyword(s):  
Cultural Heritage ◽  
Unmanned Aerial Vehicle ◽  
Point Cloud ◽  
Geometric Model ◽  
Digital Camera ◽  
Point Clouds ◽  
Symmetry Analysis ◽  
3D Point Clouds ◽  
Transmission Towers ◽  
Aerial Vehicle

Ancient pagodas are usually parts of hot tourist spots in many oriental countries due to their unique historical backgrounds. They are usually polygonal structures comprised by multiple floors, which are separated by eaves. In this paper, we propose a new method to investigate both the rotational and reflectional symmetry of such polygonal pagodas through developing novel geometric models to fit to the 3D point clouds obtained from photogrammetric reconstruction. The geometric model consists of multiple polygonal pyramid/prism models but has a common central axis. The method was verified by four datasets collected by an unmanned aerial vehicle (UAV) and a hand-held digital camera. The results indicate that the models fit accurately to the pagodas’ point clouds. The symmetry was realized by rotating and reflecting the pagodas’ point clouds after a complete leveling of the point cloud was achieved using the estimated central axes. The results show that there are RMSEs of 5.04 cm and 5.20 cm deviated from the perfect (theoretical) rotational and reflectional symmetries, respectively. This concludes that the examined pagodas are highly symmetric, both rotationally and reflectionally. The concept presented in the paper not only work for polygonal pagodas, but it can also be readily transformed and implemented for other applications for other pagoda-like objects such as transmission towers.

scholarly journals Matching Point Clouds with STL Models by Using the Principle Component Analysis and a Decomposition into Geometric Primitives

2021 ◽  
Vol 11 (5) ◽  
pp. 2268
Author(s):  
Erika Straková ◽  
Dalibor Lukáš ◽  
Zdenko Bobovský ◽  
Tomáš Kot ◽  
Milan Mihola ◽  
...  
Keyword(s):  
Point Cloud ◽  
Principal Component ◽  
Point Clouds ◽  
Component Analysis ◽  
Language Models ◽  
Computational Time ◽  
Dominant Eigenvalue ◽  
Eigenvalues And Eigenvectors ◽  
Correct Model ◽  
Geometric Primitives

While repairing industrial machines or vehicles, recognition of components is a critical and time-consuming task for a human. In this paper, we propose to automatize this task. We start with a Principal Component Analysis (PCA), which fits the scanned point cloud with an ellipsoid by computing the eigenvalues and eigenvectors of a 3-by-3 covariant matrix. In case there is a dominant eigenvalue, the point cloud is decomposed into two clusters to which the PCA is applied recursively. In case the matching is not unique, we continue to distinguish among several candidates. We decompose the point cloud into planar and cylindrical primitives and assign mutual features such as distance or angle to them. Finally, we refine the matching by comparing the matrices of mutual features of the primitives. This is a more computationally demanding but very robust method. We demonstrate the efficiency and robustness of the proposed methodology on a collection of 29 real scans and a database of 389 STL (Standard Triangle Language) models. As many as 27 scans are uniquely matched to their counterparts from the database, while in the remaining two cases, there is only one additional candidate besides the correct model. The overall computational time is about 10 min in MATLAB.

scholarly journals A Novel Vegetation Point Cloud Density Tree-Segmentation Model for Overlapping Crowns Using UAV LiDAR

2021 ◽  
Vol 13 (8) ◽  
pp. 1442
Author(s):  
Kaisen Ma ◽  
Yujiu Xiong ◽  
Fugen Jiang ◽  
Song Chen ◽  
Hua Sun
Keyword(s):  
Point Cloud ◽  
Local Maximum ◽  
Window Size ◽  
Point Clouds ◽  
High Density ◽  
Watershed Algorithm ◽  
Canopy Area ◽  
Individual Trees ◽  
Optimal Window ◽  
Cloud Density

Detecting and segmenting individual trees in forest ecosystems with high-density and overlapping crowns often results in bias due to the limitations of the commonly used canopy height model (CHM). To address such limitations, this paper proposes a new method to segment individual trees and extract tree structural parameters. The method involves the following key steps: (1) unmanned aerial vehicle (UAV)-scanned, high-density laser point clouds were classified, and a vegetation point cloud density model (VPCDM) was established by analyzing the spatial density distribution of the classified vegetation point cloud in the plane projection; and (2) a local maximum algorithm with an optimal window size was used to detect tree seed points and to extract tree heights, and an improved watershed algorithm was used to extract the tree crowns. The proposed method was tested at three sites with different canopy coverage rates in a pine-dominated forest in northern China. The results showed that (1) the kappa coefficient between the proposed VPCDM and the commonly used CHM was 0.79, indicating that performance of the VPCDM is comparable to that of the CHM; (2) the local maximum algorithm with the optimal window size could be used to segment individual trees and obtain optimal single-tree segmentation accuracy and detection rate results; and (3) compared with the original watershed algorithm, the improved watershed algorithm significantly increased the accuracy of canopy area extraction. In conclusion, the proposed VPCDM may provide an innovative data segmentation model for light detection and ranging (LiDAR)-based high-density point clouds and enhance the accuracy of parameter extraction.

Sign in / Sign up

Export Citation Format

Share Document