scholarly journals Reconstruction of High-Precision Semantic Map

Sensors ◽  
2020 ◽  
Vol 20 (21) ◽  
pp. 6264
Author(s):  
Xinyuan Tu ◽  
Jian Zhang ◽  
Runhao Luo ◽  
Kai Wang ◽  
Qingji Zeng ◽  
...  
Keyword(s):  
Real Time ◽  
Point Cloud ◽  
Semantic Information ◽  
Feature Fusion ◽  
Three Dimensional ◽  
Semantic Segmentation ◽  
Line Of Sight ◽  
Lidar Data ◽  
Implicit Surface ◽  
Robotic Mapping

We present a real-time Truncated Signed Distance Field (TSDF)-based three-dimensional (3D) semantic reconstruction for LiDAR point cloud, which achieves incremental surface reconstruction and highly accurate semantic segmentation. The high-precise 3D semantic reconstruction in real time on LiDAR data is important but challenging. Lighting Detection and Ranging (LiDAR) data with high accuracy is massive for 3D reconstruction. We so propose a line-of-sight algorithm to update implicit surface incrementally. Meanwhile, in order to use more semantic information effectively, an online attention-based spatial and temporal feature fusion method is proposed, which is well integrated into the reconstruction system. We implement parallel computation in the reconstruction and semantic fusion process, which achieves real-time performance. We demonstrate our approach on the CARLA dataset, Apollo dataset, and our dataset. When compared with the state-of-art mapping methods, our method has a great advantage in terms of both quality and speed, which meets the needs of robotic mapping and navigation.

scholarly journals Point Cloud Semantic Segmentation Network Based on Multi-Scale Feature Fusion

Sensors ◽  
2021 ◽  
Vol 21 (5) ◽  
pp. 1625
Author(s):  
Jing Du ◽  
Zuning Jiang ◽  
Shangfeng Huang ◽  
Zongyue Wang ◽  
Jinhe Su ◽  
...  
Keyword(s):  
Network Optimization ◽  
Point Cloud ◽  
Semantic Information ◽  
Feature Fusion ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Scale Feature ◽  
Multi Scale ◽  
Sensing Applications ◽  
Remote Sensing Applications

The semantic segmentation of small objects in point clouds is currently one of the most demanding tasks in photogrammetry and remote sensing applications. Multi-resolution feature extraction and fusion can significantly enhance the ability of object classification and segmentation, so it is widely used in the image field. For this motivation, we propose a point cloud semantic segmentation network based on multi-scale feature fusion (MSSCN) to aggregate the feature of a point cloud with different densities and improve the performance of semantic segmentation. In our method, random downsampling is first applied to obtain point clouds of different densities. A Spatial Aggregation Net (SAN) is then employed as the backbone network to extract local features from these point clouds, followed by concatenation of the extracted feature descriptors at different scales. Finally, a loss function is used to combine the different semantic information from point clouds of different densities for network optimization. Experiments were conducted on the S3DIS and ScanNet datasets, and our MSSCN achieved accuracies of 89.80% and 86.3%, respectively, on these datasets. Our method showed better performance than the recent methods PointNet, PointNet++, PointCNN, PointSIFT, and SAN.

Point cloud augmented virtual reality environment with haptic constraints for teleoperation

2018 ◽  
Vol 40 (15) ◽  
pp. 4091-4104 ◽  
Author(s):  
Dejing Ni ◽  
AYC Nee ◽  
SK Ong ◽  
Huijun Li ◽  
Chengcheng Zhu ◽  
...  
Keyword(s):  
Virtual Reality ◽  
Force Field ◽  
Real Time ◽  
Point Cloud ◽  
Three Dimensional ◽  
Implicit Surface ◽  
Virtual Reality Environment ◽  
Constraint Generation ◽  
Remote Manipulation ◽  
Unstructured Environment

Remote manipulation of a robot without assistance in an unstructured environment is a challenging task for operators. In this paper, a novel methodology for haptic constraints in a point cloud augmented virtual reality environment is proposed to address this human operation limitation. The proposed method generates haptic constraints in real time for an unstructured environment, including regional constraints and guidance constraints. A modified implicit surface method is applied for regional constraint generation for the entire point cloud. Additionally, the isosurface derived from the implicit surface is proposed for real-time three-dimensional artificial force field estimation. For guidance constraint generation, a new incremental prediction and local artificial force field generation method based on the modified sigmoid model is proposed in an unstructured point cloud virtual reality environment. With the generated haptic constraints, the operator can control the robot to realize obstacle avoidance and easily reach the target tasks. System evaluation is conducted, and the result demonstrates the effectiveness of the proposed method. In addition, a 10-participant study with users who control the robot to three specific targets shows that the system can enhance human operation efficiency and reduce time costs by at least 59% compared with no-haptic-constraint operations. Additionally, the designed questionnaire also demonstrates that the proposed methodology can reduce the workload during human operations.

CALCULATION OF THE DEPENDENCE OF THE DISTANCE TO THE LOCATED OBJECT FROM THE CORNER POSITION OF THE VEHICLE LINE

2018 ◽  
pp. 14-18
Author(s):  
V. V. Artyushenko ◽  
A. V. Nikulin
Keyword(s):  
Real Time ◽  
Statistical Physics ◽  
Angular Position ◽  
Three Dimensional ◽  
Line Of Sight ◽  
Two Dimensional ◽  
Radar Station ◽  
Flight Mode ◽  
Vector Algebra ◽  
Analytical Expressions

To simulate echoes from the earth’s surface in the low flight mode, it is necessary to reproduce reliably the delayed reflected sounding signal of the radar in real time. For this, it is necessary to be able to calculate accurately and quickly the dependence of the distance to the object being measured from the angular position of the line of sight of the radar station. Obviously, the simplest expressions for calculating the range can be obtained for a segment or a plane. In the text of the article, analytical expressions for the calculation of range for two-dimensional and three-dimensional cases are obtained. Methods of statistical physics, vector algebra, and the theory of the radar of extended objects were used. Since the calculation of the dependence of the range of the object to the target from the angular position of the line of sight is carried out on the analytical expressions found in the paper, the result obtained is accurate, and due to the relative simplicity of the expressions obtained, the calculation does not require much time.

RSNet: Rail semantic segmentation network for extracting aerial railroad images

2021 ◽  
pp. 1-18
Author(s):  
R.S. Rampriya ◽  
Sabarinathan ◽  
R. Suganya
Keyword(s):  
Real Time ◽  
Visual Processing ◽  
Feature Fusion ◽  
Semantic Segmentation ◽  
Vital Role ◽  
Obstacle Detection ◽  
Aerial Images ◽  
Computationally Efficient ◽  
Fusion Algorithm ◽  
Uav Images

In the near future, combo of UAV (Unmanned Aerial Vehicle) and computer vision will play a vital role in monitoring the condition of the railroad periodically to ensure passenger safety. The most significant module involved in railroad visual processing is obstacle detection, in which caution is obstacle fallen near track gage inside or outside. This leads to the importance of detecting and segment the railroad as three key regions, such as gage inside, rails, and background. Traditional railroad segmentation methods depend on either manual feature selection or expensive dedicated devices such as Lidar, which is typically less reliable in railroad semantic segmentation. Also, cameras mounted on moving vehicles like a drone can produce high-resolution images, so segmenting precise pixel information from those aerial images has been challenging due to the railroad surroundings chaos. RSNet is a multi-level feature fusion algorithm for segmenting railroad aerial images captured by UAV and proposes an attention-based efficient convolutional encoder for feature extraction, which is robust and computationally efficient and modified residual decoder for segmentation which considers only essential features and produces less overhead with higher performance even in real-time railroad drone imagery. The network is trained and tested on a railroad scenic view segmentation dataset (RSSD), which we have built from real-time UAV images and achieves 0.973 dice coefficient and 0.94 jaccard on test data that exhibits better results compared to the existing approaches like a residual unit and residual squeeze net.

scholarly journals Study of the Effect of Exploiting 3D Semantic Segmentation in LiDAR Odometry

2020 ◽  
Vol 10 (16) ◽  
pp. 5657
Author(s):  
Francisco Miguel Moreno ◽  
Carlos Guindel ◽  
José María Armingol ◽  
Fernando García
Keyword(s):  
Point Cloud ◽  
Semantic Information ◽  
Semantic Segmentation ◽  
Raw Data ◽  
Special Relevance ◽  
Original Point

This paper presents a study of how the performance of LiDAR odometry is affected by the preprocessing of the point cloud through the use of 3D semantic segmentation. The study analyzed the estimated trajectories when the semantic information is exploited to filter the original raw data. Different filtering configurations were tested: raw (original point cloud), dynamic (dynamic obstacles are removed from the point cloud), dynamic vehicles (vehicles are removed), far (distant points are removed), ground (the points belonging to the ground are removed) and structure (only structures and objects are kept in the point cloud). The experiments were performed using the KITTI and SemanticKITTI datasets, which feature different scenarios that allowed identifying the implications and relevance of each element of the environment in LiDAR odometry algorithms. The conclusions obtained from this work are of special relevance for improving the efficiency of LiDAR odometry algorithms in all kinds of scenarios.

scholarly journals Voxel-based 3D Point Cloud Semantic Segmentation: Unsupervised Geometric and Relationship Featuring vs Deep Learning Methods

2019 ◽  
Vol 8 (5) ◽  
pp. 213 ◽  
Author(s):  
Florent Poux ◽  
Roland Billen
Keyword(s):  
Deep Learning ◽  
Point Cloud ◽  
Semantic Representation ◽  
Structural Connectivity ◽  
Three Dimensional ◽  
Strong Support ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Learning Methods ◽  
Cloud Data

Automation in point cloud data processing is central in knowledge discovery within decision-making systems. The definition of relevant features is often key for segmentation and classification, with automated workflows presenting the main challenges. In this paper, we propose a voxel-based feature engineering that better characterize point clusters and provide strong support to supervised or unsupervised classification. We provide different feature generalization levels to permit interoperable frameworks. First, we recommend a shape-based feature set (SF1) that only leverages the raw X, Y, Z attributes of any point cloud. Afterwards, we derive relationship and topology between voxel entities to obtain a three-dimensional (3D) structural connectivity feature set (SF2). Finally, we provide a knowledge-based decision tree to permit infrastructure-related classification. We study SF1/SF2 synergy on a new semantic segmentation framework for the constitution of a higher semantic representation of point clouds in relevant clusters. Finally, we benchmark the approach against novel and best-performing deep-learning methods while using the full S3DIS dataset. We highlight good performances, easy-integration, and high F1-score (> 85%) for planar-dominant classes that are comparable to state-of-the-art deep learning.

scholarly journals ROAD SURFACE DETECTION FROM MOBILE LIDAR DATA

2018 ◽  
Vol IV-5 ◽  
pp. 95-101 ◽  
Author(s):  
M. Yadav ◽  
B. Lohani ◽  
A. K. Singh
Keyword(s):  
Point Cloud ◽  
Three Dimensional ◽  
Test Site ◽  
Road Surface ◽  
Lidar Data ◽  
Rural Roads ◽  
The Road ◽  
Surface Detection ◽  
Rural Road ◽  
Data Points

<p><strong>Abstract.</strong> The accurate three-dimensional road surface information is highly useful for health assessment and maintenance of roads. It is basic information for further analysis in several applications including road surface settlement, pavement condition assessment and slope collapse. Mobile LiDAR system (MLS) is frequently used now a days to collect detail road surface and its surrounding information in terms three-dimensional (3D) point cloud. Extraction of road surface from volumetric point cloud data is still in infancy stage because of heavy data processing requirement and the complexity in the road environment. The extraction of roads especially rural road, where road-curb is not present is very tedious job especially in Indian roadway settings. Only a few studies are available, and none for Indian roads, in the literature for rural road detection. The limitations of existing studies are in terms of their lower accuracy, very slow speed of data processing and detection of other objects having similar characteristics as the road surface. A fast and accurate method is proposed for LiDAR data points of road surface detection, keeping in mind the essence of road surface extraction especially for Indian rural roads. The Mobile LiDAR data in <i>XYZI</i> format is used as input in the proposed method. First square gridding is performed and ground points are roughly extracted. Then planar surface detection using mathematical framework of principal component analysis (PCA) is performed and further road surface points are detected using similarity in intensity and height difference of road surface pointe in their neighbourhood.</p><p>A case study was performed on the MLS data points captured along wide-street (two-lane road without curb) of 156<span class="thinspace"></span>m length along rural roadway site in the outskirt of Bengaluru city (South-West of India). The proposed algorithm was implemented on the MLS data of test site and its performance was evaluated it terms of recall, precision and overall accuracy that were 95.27%, 98.85% and 94.23%, respectively. The algorithm was found computationally time efficient. A 7.6 million MLS data points of size 27.1<span class="thinspace"></span>MB from test site were processed in 24 minutes using the available computational resources. The proposed method is found to work even for worst case scenarios, i.e., complex road environments and rural roads, where road boundary is not clear and generally merged with road-side features.</p>

Sign in / Sign up

Export Citation Format

Share Document