Object Tracking With a Range Camera for Augmented Reality Assembly Assistance

2016 ◽  
Vol 16 (1) ◽  
Author(s):  
Rafael Radkowski
Keyword(s):  
Augmented Reality ◽  
Object Tracking ◽  
Point Cloud ◽  
Sampling Methods ◽  
Point Clouds ◽  
Reference Object ◽  
Key Factors ◽  
Tracking Method ◽  
Uniform Sampling ◽  
Feature Descriptors

This paper introduces a 3D object tracking method for an augmented reality (AR) assembly assistance application. The tracking method relies on point clouds; it uses 3D feature descriptors and point cloud matching with the iterative closest points (ICP) algorithm. The feature descriptors identify an object in a point cloud; ICP align a reference object with this point cloud. The challenge is to achieve high fidelity while maintaining camera frame rates. The point cloud and reference object sampling density are one of the key factors to meet this challenge. In this research, three-point sampling methods and two-point cloud search algorithms were compared to assess their fidelity when tracking typical products of mechanical engineering. The results indicate that a uniform sampling maintains the best fidelity at camera frame rates.

scholarly journals VISUALIZATION OF POINT CLOUD MODELS IN MOBILE AUGMENTED REALITY USING CONTINUOUS LEVEL OF DETAIL METHOD

2020 ◽  
Vol XLIV-4/W1-2020 ◽  
pp. 167-170
Author(s):  
L. Zhang ◽  
P. van Oosterom ◽  
H. Liu
Keyword(s):  
Augmented Reality ◽  
Point Cloud ◽  
Cloud Model ◽  
Point Clouds ◽  
Level Of Detail ◽  
Mobile Augmented Reality ◽  
Cloud Models ◽  
Point Cloud Model ◽  
Visual Artifacts ◽  
Mobile Ar

Abstract. Point clouds have become one of the most popular sources of data in geospatial fields due to their availability and flexibility. However, because of the large amount of data and the limited resources of mobile devices, the use of point clouds in mobile Augmented Reality applications is still quite limited. Many current mobile AR applications of point clouds lack fluent interactions with users. In our paper, a cLoD (continuous level-of-detail) method is introduced to filter the number of points to be rendered considerably, together with an adaptive point size rendering strategy, thus improve the rendering performance and remove visual artifacts of mobile AR point cloud applications. Our method uses a cLoD model that has an ideal distribution over LoDs, with which can remove unnecessary points without sudden changes in density as present in the commonly used discrete level-of-detail approaches. Besides, camera position, orientation and distance from the camera to point cloud model is taken into consideration as well. With our method, good interactive visualization of point clouds can be realized in the mobile AR environment, with both nice visual quality and proper resource consumption.

Comparison of Natural Feature Descriptors for Rigid-Object Tracking for Real-Time Augmented Reality

2014 ◽  
Author(s):  
Francely Franco Bermudez ◽  
Christian Santana Diaz ◽  
Sheneeka Ward ◽  
Rafael Radkowski ◽  
Timothy Garrett ◽  
...  
Keyword(s):  
Augmented Reality ◽  
Object Tracking ◽  
Real Time ◽  
Virtual Object ◽  
Interest Points ◽  
Rigid Object ◽  
Binary Descriptors ◽  
Rigid Objects ◽  
Natural Feature ◽  
Feature Descriptors

This paper presents a comparison of natural feature descriptors for rigid object tracking for augmented reality (AR) applications. AR relies on object tracking in order to identify a physical object and to superimpose virtual object on an object. Natural feature tracking (NFT) is one approach for computer vision-based object tracking. NFT utilizes interest points of a physcial object, represents them as descriptors, and matches the descriptors against reference descriptors in order to identify a phsical object to track. In this research, we investigate four different natural feature descriptors (SIFT, SURF, FREAK, ORB) and their capability to track rigid objects. Rigid objects need robust descriptors since they need to describe the objects in a 3D space. AR applications are also real-time application, thus, fast feature matching is mandatory. FREAK and ORB are binary descriptors, which promise a higher performance in comparison to SIFT and SURF. We deployed a test in which we match feature descriptors to artificial rigid objects. The results indicate that the SIFT descriptor is the most promising solution in our addressed domain, AR-based assembly training.

scholarly journals Tracklet Proposal Network for Multi-Object Tracking on Point Clouds

2021 ◽  
Author(s):  
Hai Wu ◽  
Qing Li ◽  
Chenglu Wen ◽  
Xin Li ◽  
Xiaoliang Fan ◽  
...  
Keyword(s):  
Object Tracking ◽  
Point Cloud ◽  
Point Clouds ◽  
Temporal Features ◽  
Object Proposal ◽  
Multiple Frames

This paper proposes the first tracklet proposal network, named PC-TCNN, for Multi-Object Tracking (MOT) on point clouds. Our pipeline first generates tracklet proposals, then refines these tracklets and associates them to generate long trajectories. Specifically, object proposal generation and motion regression are first performed on a point cloud sequence to generate tracklet candidates. Then, spatial-temporal features of each tracklet are exploited and their consistency is used to refine the tracklet proposal. Finally, the refined tracklets across multiple frames are associated to perform MOT on the point cloud sequence. The PC-TCNN significantly improves the MOT performance by introducing the tracklet proposal design. On the KITTI tracking benchmark, it attains an MOTA of 91.75%, outperforming all submitted results on the online leaderboard.

scholarly journals GACM: A Graph Attention Capsule Model for the Registration of TLS Point Clouds in the Urban Scene

2021 ◽  
Vol 13 (22) ◽  
pp. 4497
Author(s):  
Jianjun Zou ◽  
Zhenxin Zhang ◽  
Dong Chen ◽  
Qinghua Li ◽  
Lan Sun ◽  
...  
Keyword(s):  
Point Cloud ◽  
Laser Scanning ◽  
Three Dimensional ◽  
Point Clouds ◽  
Autonomous Driving ◽  
Feature Descriptors ◽  
Local Point ◽  
Urban Scene ◽  
Pairwise Registration ◽  
Structure Complexity

Point cloud registration is the foundation and key step for many vital applications, such as digital city, autonomous driving, passive positioning, and navigation. The difference of spatial objects and the structure complexity of object surfaces are the main challenges for the registration problem. In this paper, we propose a graph attention capsule model (named as GACM) for the efficient registration of terrestrial laser scanning (TLS) point cloud in the urban scene, which fuses graph attention convolution and a three-dimensional (3D) capsule network to extract local point cloud features and obtain 3D feature descriptors. These descriptors can take into account the differences of spatial structure and point density in objects and make the spatial features of ground objects more prominent. During the training progress, we used both matched points and non-matched points to train the model. In the test process of the registration, the points in the neighborhood of each keypoint were sent to the trained network, in order to obtain feature descriptors and calculate the rotation and translation matrix after constructing a K-dimensional (KD) tree and random sample consensus (RANSAC) algorithm. Experiments show that the proposed method achieves more efficient registration results and higher robustness than other frontier registration methods in the pairwise registration of point clouds.

Object tracking method based on joint global and local feature descriptor of 3D LIDAR point cloud

2020 ◽  
Vol 18 (6) ◽  
pp. 061001
Author(s):  
Qishu Qian ◽  
Yihua Hu ◽  
Nanxiang Zhao ◽  
Minle Li ◽  
Fucai Shao ◽  
...  
Keyword(s):  
Object Tracking ◽  
Point Cloud ◽  
Local Feature ◽  
Feature Descriptor ◽  
Tracking Method ◽  
Global And Local ◽  
3D Lidar ◽  
Local Feature Descriptor

scholarly journals A Benchmark of Popular Indoor 3D Reconstruction Technologies: Comparison of ARCore and RTAB-Map

Electronics ◽  
2020 ◽  
Vol 9 (12) ◽  
pp. 2091
Author(s):  
Ádám Wolf ◽  
Péter Troll ◽  
Stefan Romeder-Finger ◽  
Andreas Archenti ◽  
Károly Széll ◽  
...  
Keyword(s):  
Standard Deviation ◽  
Augmented Reality ◽  
3D Reconstruction ◽  
Cost Effective ◽  
Point Clouds ◽  
Depth Image ◽  
Reference Object ◽  
Use Case ◽  
Large Standard Deviation ◽  
Fast Evolution

The fast evolution in computational and sensor technologies brings previously niche solutions to a wider userbase. As such, 3D reconstruction technologies are reaching new use-cases in scientific and everyday areas where they were not present before. Cost-effective and easy-to-use solutions include camera-based 3D scanning techniques, such as photogrammetry. This paper provides an overview of the available solutions and discusses in detail the depth-image based Real-time Appearance-based Mapping (RTAB-Map) technique as well as a smartphone-based solution that utilises ARCore, the Augmented Reality (AR) framework of Google. To qualitatively compare the two 3D reconstruction technologies, a simple length measurement-based method was applied with a purpose-designed reference object. The captured data were then analysed by a processing algorithm. In addition to the experimental results, specific case studies are briefly discussed, evaluating the applicability based on the capabilities of the technologies. As such, the paper presents the use-case of interior surveying in an automated laboratory as well as an example for using the discussed techniques for landmark surveying. The major findings are that point clouds created with these technologies provide a direction- and shape-accurate model, but those contain mesh continuity errors, and the estimated scale factor has a large standard deviation.

scholarly journals A study on recognizing multi-real world object and estimating 3D position in augmented reality

2021 ◽  
Author(s):  
Taemin Lee ◽  
Changhun Jung ◽  
Kyungtaek Lee ◽  
Sanghyun Seo
Keyword(s):  
Augmented Reality ◽  
Real World ◽  
Visual Information ◽  
Point Cloud ◽  
Three Dimensional ◽  
Point Clouds ◽  
Location Estimation ◽  
Virtual Space ◽  
Camera Tracking ◽  
The Real

AbstractAs augmented reality technologies develop, real-time interactions between objects present in the real world and virtual space are required. Generally, recognition and location estimation in augmented reality are carried out using tracking techniques, typically markers. However, using markers creates spatial constraints in simultaneous tracking of space and objects. Therefore, we propose a system that enables camera tracking in the real world and visualizes virtual visual information through the recognition and positioning of objects. We scanned the space using an RGB-D camera. A three-dimensional (3D) dense point cloud map is created using point clouds generated through video images. Among the generated point cloud information, objects are detected and retrieved based on the pre-learned data. Finally, using the predicted pose of the detected objects, other information may be augmented. Our system estimates object recognition and 3D pose based on simple camera information, enabling the viewing of virtual visual information based on object location.

TrackingExpert: A Versatile Tracking Toolbox for Augmented Reality

2016 ◽  
Author(s):  
Rafael Radkowski ◽  
Timothy Garrett ◽  
Jarid Ingebrand ◽  
David Wehr
Keyword(s):  
Augmented Reality ◽  
Object Tracking ◽  
Pose Estimation ◽  
Software Package ◽  
Point Cloud ◽  
Promising Candidate ◽  
Rigid Object ◽  
Virtual Objects ◽  
Research Outcomes ◽  
Functional Areas

This paper presents a toolbox for rigid object tracking with a focus on augmented reality applications. Augmented reality relies on tracking to superimpose virtual objects on physical objects. Object tracking is usually based on registration and pose estimation techniques. Many different approaches have already been introduced. Our research focuses on tracking for application areas such as assembly assistance and the most promising candidate is rigid object tracking based on point cloud registration. Our work advances the robustness of point cloud-based tracking as well as the performance. One product of our research is our tracking tool TrackingExpert, which integrates all our research outcomes into one versatile software package. This paper introduces TrackingExpert covering functional areas such as the registration, visualizations, and experiment support. We also highlight several aspects which facilitate data analysis and ease our research.

Sign in / Sign up

Export Citation Format

Share Document