scholarly journals Point Cloud Hand–Object Segmentation Using Multimodal Imaging with Thermal and Color Data for Safe Robotic Object Handover

Sensors ◽  
2021 ◽  
Vol 21 (16) ◽  
pp. 5676
Author(s):  
Yan Zhang ◽  
Steffen Müller ◽  
Benedict Stephan ◽  
Horst-Michael Gross ◽  
Gunther Notni
Keyword(s):  
Point Cloud ◽  
Near Infrared ◽  
Object Segmentation ◽  
Sensor System ◽  
Calibration Target ◽  
Single Class ◽  
Cloud Data ◽  
3D Data ◽  
Object Handover ◽  
Color Data

This paper presents an application of neural networks operating on multimodal 3D data (3D point cloud, RGB, thermal) to effectively and precisely segment human hands and objects held in hand to realize a safe human–robot object handover. We discuss the problems encountered in building a multimodal sensor system, while the focus is on the calibration and alignment of a set of cameras including RGB, thermal, and NIR cameras. We propose the use of a copper–plastic chessboard calibration target with an internal active light source (near-infrared and visible light). By brief heating, the calibration target could be simultaneously and legibly captured by all cameras. Based on the multimodal dataset captured by our sensor system, PointNet, PointNet++, and RandLA-Net are utilized to verify the effectiveness of applying multimodal point cloud data for hand–object segmentation. These networks were trained on various data modes (XYZ, XYZ-T, XYZ-RGB, and XYZ-RGB-T). The experimental results show a significant improvement in the segmentation performance of XYZ-RGB-T (mean Intersection over Union: 82.8% by RandLA-Net) compared with the other three modes (77.3% by XYZ-RGB, 35.7% by XYZ-T, 35.7% by XYZ), in which it is worth mentioning that the Intersection over Union for the single class of hand achieves 92.6%.

A Novel Pre-Processing Approach of Point Cloud in Reverse Engineering

2013 ◽  
Vol 331 ◽  
pp. 631-635
Author(s):  
Ci Zhang ◽  
Guo Fan Hu ◽  
Xu Bing Chen
Keyword(s):  
Reverse Engineering ◽  
Point Cloud ◽  
Triangular Grid ◽  
Point Cloud Data ◽  
Cloud Data ◽  
Engineering Data ◽  
Processing Approach ◽  
3D Data ◽  
Cloud Data Processing

In reverse engineering, data pre-processing has played an increasingly important role for rebuilding the original 3D model. However, it is usually complex, time-consuming, and difficult to realize, as there are huge amounts of redundant 3D data existed in the gained point cloud. To find a solution for this issue, point cloud data processing and streamlining technologies are reviewed firstly. Secondly, a novel pre-processing approach is proposed in three steps: point cloud registration, regional 3D triangular mesh construction and point cloud filtering. And then, the projected hexagonal area and the closest projected point are defined. At last, a parabolic antenna model is employed as a case study. After pre-processing, the number of points are decreased from 4,066,282 to 449,806 under the constraint of triangular grid size h equaling to 2mm, i.e. about 1/9 size of the original point cloud. The result demonstrates its feasibility and efficiency.

scholarly journals COMPARATIVE ANALYSIS OF DATA STRUCTURES FOR STORING MASSIVE TINS IN A DBMS

2016 ◽  
Vol XLI-B2 ◽  
pp. 123-130
Author(s):  
K. Kumar ◽  
H. Ledoux ◽  
J. Stoter
Keyword(s):  
Data Structures ◽  
Point Cloud ◽  
Laser Scanning ◽  
Compact Star ◽  
Point Clouds ◽  
Oracle Spatial ◽  
Cloud Data ◽  
3D Data ◽  
Edge Based ◽  
Elevation Model

Point cloud data are an important source for 3D geoinformation. Modern day 3D data acquisition and processing techniques such as airborne laser scanning and multi-beam echosounding generate billions of 3D points for simply an area of few square kilometers. With the size of the point clouds exceeding the billion mark for even a small area, there is a need for their efficient storage and management. These point clouds are sometimes associated with attributes and constraints as well. Storing billions of 3D points is currently possible which is confirmed by the initial implementations in Oracle Spatial SDO PC and the PostgreSQL Point Cloud extension. But to be able to analyse and extract useful information from point clouds, we need more than just points i.e. we require the surface defined by these points in space. There are different ways to represent surfaces in GIS including grids, TINs, boundary representations, etc. In this study, we investigate the database solutions for the storage and management of massive TINs. The classical (face and edge based) and compact (star based) data structures are discussed at length with reference to their structure, advantages and limitations in handling massive triangulations and are compared with the current solution of PostGIS Simple Feature. The main test dataset is the TIN generated from third national elevation model of the Netherlands (AHN3) with a point density of over 10 points/m<sup>2</sup>. PostgreSQL/PostGIS DBMS is used for storing the generated TIN. The data structures are tested with the generated TIN models to account for their geometry, topology, storage, indexing, and loading time in a database. Our study is useful in identifying what are the limitations of the existing data structures for storing massive TINs and what is required to optimise these structures for managing massive triangulations in a database.

scholarly journals A Lightweight, Robust Exploitation System for Temporal Stacks of UAS Data: Use Case for Forward-Deployed Military or Emergency Responders

Drones ◽  
2019 ◽  
Vol 3 (1) ◽  
pp. 29 ◽  
Author(s):  
Andrew Marx ◽  
Yu-Hsi Chou ◽  
Kevin Mercy ◽  
Richard Windisch
Keyword(s):  
Urban Areas ◽  
Point Cloud ◽  
Point Clouds ◽  
Temporal Analysis ◽  
Unmanned Aerial Systems ◽  
Point Cloud Data ◽  
High Quality ◽  
Cloud Data ◽  
3D Data ◽  
Exploitation System

The availability and precision of unmanned aerial systems (UAS) permit the repeated collection of very-high quality three-dimensional (3D) data to monitor high-interest areas, such as dams, urban areas, or erosion-prone coastlines. However, challenges exist in the temporal analysis of this data, specifically in conducting change-detection analysis on the high-quality point cloud data. These files are very large in size and contain points in varying locations that do not align between scenes. These large file sizes also limit the use of this data for individuals with low computational resources, such as first responders or forward-deployed soldiers. In response, this manuscript presents an approach that aggregates data spatially into voxels to provide the user with a lightweight, web-based exploitation system coupled with a flexible backend database. The system creates a robust set of tools to analyze large temporal stacks of 3D data and reduces data size by 78%, all while being able to query the original point cloud data. This approach offers a solution for organizations analyzing high-resolution, temporal point-clouds, as well as a possible solution for operations in areas with poor computational and connectivity resources requiring high-quality, 3D data for decision support and planning.

A CAD Modeling Method of Dental Surface Based on Scanning Data

2012 ◽  
Vol 151 ◽  
pp. 111-115
Author(s):  
Li Cheng Fan ◽  
Feng Feng Zhang
Keyword(s):  
Point Cloud ◽  
Cnc Machining ◽  
Tooth Surface ◽  
Iteration Algorithm ◽  
Point Cloud Data ◽  
Scanning Method ◽  
Cad Modeling ◽  
Cloud Data ◽  
Data Registration ◽  
3D Data

The measurement of the teeth surface and the CAD modeling of the point cloud data are the key basics for the following CNC machining, and the complete data can only be obtained through multi-perspective scanning method. Using ICP iteration algorithm that based on point-to-line to the multi-perspective scanning data, specific to the features of layering scanning, retrograde the 3D data registration to 2D planar registration, and provide the cutting and splicing algorithm for registered tooth data, obtain precise and integrated tooth surface point cloud data, which serves as the CAD model for the following CNC machining.

scholarly journals Classification Airbrone LiDAR Point cloud Data

2021 ◽  
Vol 7 (01) ◽  
pp. 36-39
Author(s):  
Naga Madhavi lavanya Gandi
Keyword(s):  
High Resolution ◽  
Land Cover ◽  
Point Cloud ◽  
Near Infrared ◽  
Lidar Data ◽  
Point Cloud Data ◽  
Mid Infrared ◽  
Cloud Data ◽  
Classification Information

Land cover classification information plays a very important role in various applications. Airborne Light detection and Ranging (LiDAR) data is widely used in remote sensing application for the classification of land cover. The present study presents a Spatial classification method using Terrasoild macros . The data used in this study are a LiDAR point cloud data with the wavelength of green:532nm, near infrared:1064nm and mid-infrared-1550nm and High Resolution RGB data. The classification is carried in TERRASCAN Module with twelve land cover classes. The classification accuracies were assessed using high resolution RGB data. From the results it is concluded that the LiDAR data classification with overall accuracy and kappa coefficient 85.2% and 0.7562.

scholarly journals FUSION OF 3D POINT CLOUDS WITH TIR IMAGES FOR INDOOR SCENE RECONSTRUCTION

2018 ◽  
Vol XLII-1 ◽  
pp. 189-194 ◽  
Author(s):  
L. Hoegner ◽  
T. Abmayr ◽  
D. Tosic ◽  
S. Turzer ◽  
U. Stilla
Keyword(s):  
Point Cloud ◽  
Near Infrared ◽  
Moving Objects ◽  
Infrared Camera ◽  
Point Clouds ◽  
Thermal Infrared ◽  
Active Sensor ◽  
3D Point Clouds ◽  
3D Data ◽  
3D Scene

<p><strong>Abstract.</strong> Obtaining accurate 3D descriptions in the thermal infrared (TIR) is a quite challenging task due to the low geometric resolutions of TIR cameras and the low number of strong features in TIR images. Combining the radiometric information of the thermal infrared with 3D data from another sensor is able to overcome most of the limitations in the 3D geometric accuracy. In case of dynamic scenes with moving objects or a moving sensor system, a combination with RGB cameras and profile laserscanners is suitable. As a laserscanner is an active sensor in the visible red or near infrared (NIR) and the thermal infrared camera captures the radiation emitted by the objects in the observed scene, the combination of these two sensors for close range applications are independent from external illumination or textures in the scene. This contribution focusses on the fusion of point clouds from terrestrial laserscanners and RGB cameras with images from thermal infrared mounted together on a robot for indoor 3D reconstruction. The system is geometrical calibrated including the lever arm between the different sensors. As the field of view is different for the sensors, the different sensors record the same scene points not exactly at the same time. Thus, the 3D scene points of the laserscanner and the photogrammetric point cloud from the RGB camera have to be synchronized before point cloud fusion and adding the thermal channel to the 3D points.</p>

scholarly journals MODELLING PERMAFROST TERRAIN USING KINEMATIC, DUAL-WAVELENGTH LASER SCANNING

2020 ◽  
Vol V-2-2020 ◽  
pp. 749-756
Author(s):  
A. Kukko ◽  
H. Kaartinen ◽  
G. Osinski ◽  
J. Hyyppä
Keyword(s):  
Standard Deviation ◽  
Data Processing ◽  
Point Cloud ◽  
Laser Scanning ◽  
Base Station ◽  
Dual Wavelength ◽  
The Arctic ◽  
Point Cloud Data ◽  
Cloud Data ◽  
3D Data

Abstract. In this paper we introduce the first dual-wavelength, kinematic backpack laser scanning system and its application on high resolution 3D terrain modelling of permafrost landforms. We discuss the data processing pipeline from acquisition to preparation, system calibration and terrain model process. Topographic information is vital for planning and monitoring tasks in urban planning, road construction for mass calculations, and mitigation of flood and wind related risks by structural design in coastal areas. 3D data gives possibility to understand natural processes inducing changes in the terrain, such as the cycles of thaw-freeze in permafrost regions. Through an application case on permafrost landforms in the Arctic we present the field practices and data processing applied, characterize the data output and discuss the precision and accuracy of the base station, trajectory and point cloud data. Two pulsed time of flight ranging, high performance mobile laser scanners were used in combination with a near navigation grade GNSS-IMU positioning on a kinematic backpack platform. The study shows that with a high-end system 15 mm absolute accuracy of 3D data could be achieved using PPP processing for the GNSS base station and multi-pass differential trajectory post-processing. The PPP solution shows millimetre level agreement (Easting 6 mm, Northing 4 mm, and elevation 8 mm standard deviations) for the base station coordinates over an 11 day period. The point cloud residual standard deviation for angular boresight misalignment was 27 mm. The absolute distance between ground surfaces from interactive analysis was 17 mm with 13 mm standard deviation (n = 64). The proposed backpack laser scanning provides accurate and precise 3D data and performance over considerable land surface area for detailed elevation modelling and analysis of the morphology of features of interest. The high density point cloud data permits fusion of the dual-wavelength lidar reflectance data into spectral products.

scholarly journals A Hierarchical Machine Learning Approach for Multi-Level and Multi-Resolution 3D Point Cloud Classification

2020 ◽  
Vol 12 (16) ◽  
pp. 2598
Author(s):  
Simone Teruggi ◽  
Eleonora Grilli ◽  
Michele Russo ◽  
Francesco Fassi ◽  
Fabio Remondino
Keyword(s):  
Machine Learning ◽  
Point Cloud ◽  
Large Scale ◽  
3D Point Cloud ◽  
Cloud Data ◽  
Cloud Classification ◽  
3D Data ◽  
Structured Information ◽  
Multi Level ◽  
Point Cloud Classification

The recent years saw an extensive use of 3D point cloud data for heritage documentation, valorisation and visualisation. Although rich in metric quality, these 3D data lack structured information such as semantics and hierarchy between parts. In this context, the introduction of point cloud classification methods can play an essential role for better data usage, model definition, analysis and conservation. The paper aims to extend a machine learning (ML) classification method with a multi-level and multi-resolution (MLMR) approach. The proposed MLMR approach improves the learning process and optimises 3D classification results through a hierarchical concept. The MLMR procedure is tested and evaluated on two large-scale and complex datasets: the Pomposa Abbey (Italy) and the Milan Cathedral (Italy). Classification results show the reliability and replicability of the developed method, allowing the identification of the necessary architectural classes at each geometric resolution.

Sign in / Sign up

Export Citation Format

Share Document