scholarly journals Fast 3D Semantic Mapping on Naturalistic Road Scenes

2019 ◽  
Author(s):  
Xuanpeng Li ◽  
Dong Wang ◽  
Huanxuan Ao ◽  
Rachid Belaroussi ◽  
Dominique Gruyer
Keyword(s):  
Autonomous Navigation ◽  
Semantic Information ◽  
Reference Frames ◽  
Computational Cost ◽  
Semantic Segmentation ◽  
Semantic Mapping ◽  
Semantic Inference ◽  
Spatial Consistency ◽  
Mapping System ◽  
Speed Up

Fast 3D reconstruction with semantic information on road scenes is of great requirements for autonomous navigation. It involves issues of geometry and appearance in the field of computer vision. In this work, we propose a method of fast 3D semantic mapping based on the monocular vision. At present, due to the inexpensive price and easy installation, monocular cameras are widely equipped on recent vehicles for the advanced driver assistance and it is possible to acquire semantic information and 3D map. The monocular visual sequence is used to estimate the camera pose, calculate the depth, predict the semantic segmentation, and finally realize the 3D semantic mapping by combination of the techniques of localization, mapping and scene parsing. Our method recovers the 3D semantic mapping by incrementally transferring 2D semantic information to 3D point cloud. And a global optimization is explored to improve the accuracy of the semantic mapping in light of the spatial consistency. In our framework, there is no need to make semantic inference on each frame of the sequence, since the mesh data with semantic information is corresponding to sparse reference frames. It saves amounts of the computational cost and allows our mapping system to perform online. We evaluate the system on naturalistic road scenes, e.g., KITTI and observe a significant speed-up in the inference stage by labeling on the mesh.

scholarly journals Fast 3D Semantic Mapping in Road Scenes

2019 ◽  
Vol 9 (4) ◽  
pp. 631 ◽  
Author(s):  
Xuanpeng Li ◽  
Dong Wang ◽  
Huanxuan Ao ◽  
Rachid Belaroussi ◽  
Dominique Gruyer
Keyword(s):  
3D Reconstruction ◽  
Autonomous Navigation ◽  
Point Cloud ◽  
Semantic Information ◽  
Computational Cost ◽  
Semantic Segmentation ◽  
Semantic Mapping ◽  
3D Point Cloud ◽  
Semantic Inference ◽  
Mapping System

Fast 3D reconstruction with semantic information in road scenes is of great requirements for autonomous navigation. It involves issues of geometry and appearance in the field of computer vision. In this work, we propose a fast 3D semantic mapping system based on the monocular vision by fusion of localization, mapping, and scene parsing. From visual sequences, it can estimate the camera pose, calculate the depth, predict the semantic segmentation, and finally realize the 3D semantic mapping. Our system consists of three modules: a parallel visual Simultaneous Localization And Mapping (SLAM) and semantic segmentation module, an incrementally semantic transfer from 2D image to 3D point cloud, and a global optimization based on Conditional Random Field (CRF). It is a heuristic approach that improves the accuracy of the 3D semantic labeling in light of the spatial consistency on each step of 3D reconstruction. In our framework, there is no need to make semantic inference on each frame of sequence, since the 3D point cloud data with semantic information is corresponding to sparse reference frames. It saves on the computational cost and allows our mapping system to perform online. We evaluate the system on road scenes, e.g., KITTI, and observe a significant speed-up in the inference stage by labeling on the 3D point cloud.

scholarly journals Fast Image Registration for Spacecraft Autonomous Navigation Using Natural Landmarks

2018 ◽  
Vol 2018 ◽  
pp. 1-12
Author(s):  
Yun-Hua Wu ◽  
Lin-Lin Ge ◽  
Feng Wang ◽  
Bing Hua ◽  
Zhi-Ming Chen ◽  
...  
Keyword(s):  
Image Registration ◽  
Autonomous Navigation ◽  
Computational Cost ◽  
Segmentation Algorithm ◽  
Speeded Up Robust Features ◽  
Speed Up ◽  
Random Sample Consensus ◽  
Invariant Properties ◽  
Low Computational Cost ◽  
Ransac Algorithm

In order to satisfy the real-time requirement of spacecraft autonomous navigation using natural landmarks, a novel algorithm called CSA-SURF (chessboard segmentation algorithm and speeded up robust features) is proposed to improve the speed without loss of repeatability performance of image registration progress. It is a combination of chessboard segmentation algorithm and SURF. Here, SURF is used to extract the features from satellite images because of its scale- and rotation-invariant properties and low computational cost. CSA is based on image segmentation technology, aiming to find representative blocks, which will be allocated to different tasks to speed up the image registration progress. To illustrate the advantages of the proposed algorithm, PCA-SURF, which is the combination of principle component analysis and SURF, is also analyzed in this paper for comparison. Furthermore, random sample consensus (RANSAC) algorithm is applied to eliminate the false matches for further accuracy improvement. The simulation results show that the proposed strategy obtains good results, especially in scaling and rotation variation. Besides, CSA-SURF decreased 50% of the time in extraction and 90% of the time in matching without losing the repeatability performance by comparing with SURF algorithm. The proposed method has been demonstrated as an alternative way for image registration of spacecraft autonomous navigation using natural landmarks.

scholarly journals Towards a Meaningful 3D Map Using a 3D Lidar and a Camera

Sensors ◽  
2018 ◽  
Vol 18 (8) ◽  
pp. 2571 ◽  
Author(s):  
Jongmin Jeong ◽  
Tae Yoon ◽  
Jin Park
Keyword(s):  
Large Scale ◽  
Semantic Information ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Semantic Mapping ◽  
Alignment Error ◽  
Semantic Labeling ◽  
Moving Vehicles ◽  
3D Lidar

Semantic 3D maps are required for various applications including robot navigation and surveying, and their importance has significantly increased. Generally, existing studies on semantic mapping were camera-based approaches that could not be operated in large-scale environments owing to their computational burden. Recently, a method of combining a 3D Lidar with a camera was introduced to address this problem, and a 3D Lidar and a camera were also utilized for semantic 3D mapping. In this study, our algorithm consists of semantic mapping and map refinement. In the semantic mapping, a GPS and an IMU are integrated to estimate the odometry of the system, and subsequently, the point clouds measured from a 3D Lidar are registered by using this information. Furthermore, we use the latest CNN-based semantic segmentation to obtain semantic information on the surrounding environment. To integrate the point cloud with semantic information, we developed incremental semantic labeling including coordinate alignment, error minimization, and semantic information fusion. Additionally, to improve the quality of the generated semantic map, the map refinement is processed in a batch. It enhances the spatial distribution of labels and removes traces produced by moving vehicles effectively. We conduct experiments on challenging sequences to demonstrate that our algorithm outperforms state-of-the-art methods in terms of accuracy and intersection over union.

scholarly journals Semantic Mapping for Mobile Robots in Indoor Scenes: A Survey

Information ◽  
2021 ◽  
Vol 12 (2) ◽  
pp. 92
Author(s):  
Xiaoning Han ◽  
Shuailong Li ◽  
Xiaohui Wang ◽  
Weijia Zhou
Keyword(s):  
Mobile Robots ◽  
Semantic Information ◽  
Semantic Mapping ◽  
Human Beings ◽  
Future Directions ◽  
Essential Requirement ◽  
Semantic Map ◽  
Semantic Maps ◽  
Great Progress ◽  
Indoor Scenes

Sensing and mapping its surroundings is an essential requirement for a mobile robot. Geometric maps endow robots with the capacity of basic tasks, e.g., navigation. To co-exist with human beings in indoor scenes, the need to attach semantic information to a geometric map, which is called a semantic map, has been realized in the last two decades. A semantic map can help robots to behave in human rules, plan and perform advanced tasks, and communicate with humans on the conceptual level. This survey reviews methods about semantic mapping in indoor scenes. To begin with, we answered the question, what is a semantic map for mobile robots, by its definitions. After that, we reviewed works about each of the three modules of semantic mapping, i.e., spatial mapping, acquisition of semantic information, and map representation, respectively. Finally, though great progress has been made, there is a long way to implement semantic maps in advanced tasks for robots, thus challenges and potential future directions are discussed before a conclusion at last.

scholarly journals Deep Learning Based Pavement Inspection Using Self-Reconfigurable Robot

Sensors ◽  
2021 ◽  
Vol 21 (8) ◽  
pp. 2595
Author(s):  
Balakrishnan Ramalingam ◽  
Abdullah Aamir Hayat ◽  
Mohan Rajesh Elara ◽  
Braulio Félix Gómez ◽  
Lim Yi ◽  
...  
Keyword(s):  
Deep Learning ◽  
Semantic Segmentation ◽  
High Accuracy ◽  
Experimental Results ◽  
Mobile Mapping ◽  
Mapping System ◽  
Mobile Mapping System ◽  
Reconfigurable Robot ◽  
Nvidia Gpu ◽  
Inspection Task

The pavement inspection task, which mainly includes crack and garbage detection, is essential and carried out frequently. The human-based or dedicated system approach for inspection can be easily carried out by integrating with the pavement sweeping machines. This work proposes a deep learning-based pavement inspection framework for self-reconfigurable robot named Panthera. Semantic segmentation framework SegNet was adopted to segment the pavement region from other objects. Deep Convolutional Neural Network (DCNN) based object detection is used to detect and localize pavement defects and garbage. Furthermore, Mobile Mapping System (MMS) was adopted for the geotagging of the defects. The proposed system was implemented and tested with the Panthera robot having NVIDIA GPU cards. The experimental results showed that the proposed technique identifies the pavement defects and litters or garbage detection with high accuracy. The experimental results on the crack and garbage detection are presented. It is found that the proposed technique is suitable for deployment in real-time for garbage detection and, eventually, sweeping or cleaning tasks.

RGB-D Semantic Segmentation and Label-oriented Voxelgrid Fusion for Actuate 3D Semantic Mapping

2021 ◽  
pp. 1-1
Author(s):  
Wenjun Shi ◽  
Jingwei Xu ◽  
Dongchen Zhu ◽  
Guanghui Zhang ◽  
Xianshun Wang ◽  
...  
Keyword(s):  
Semantic Segmentation ◽  
Semantic Mapping

scholarly journals Dynamic maintenance model for high average-utility pattern mining with deletion operation

2021 ◽  
Author(s):  
Jimmy Ming-Tai Wu ◽  
Qian Teng ◽  
Shahab Tayeb ◽  
Jerry Chun-Wei Lin
Keyword(s):  
Pattern Mining ◽  
Computational Cost ◽  
Practical Applications ◽  
Itemset Mining ◽  
Dynamic Databases ◽  
Speed Up ◽  
Dynamic Maintenance ◽  
Average Utility ◽  
High Utility ◽  
Maintenance Model

AbstractThe high average-utility itemset mining (HAUIM) was established to provide a fair measure instead of genetic high-utility itemset mining (HUIM) for revealing the satisfied and interesting patterns. In practical applications, the database is dynamically changed when insertion/deletion operations are performed on databases. Several works were designed to handle the insertion process but fewer studies focused on processing the deletion process for knowledge maintenance. In this paper, we then develop a PRE-HAUI-DEL algorithm that utilizes the pre-large concept on HAUIM for handling transaction deletion in the dynamic databases. The pre-large concept is served as the buffer on HAUIM that reduces the number of database scans while the database is updated particularly in transaction deletion. Two upper-bound values are also established here to reduce the unpromising candidates early which can speed up the computational cost. From the experimental results, the designed PRE-HAUI-DEL algorithm is well performed compared to the Apriori-like model in terms of runtime, memory, and scalability in dynamic databases.

scholarly journals Combining Acceleration Techniques for Low-Dose X-Ray Cone Beam Computed Tomography Image Reconstruction

2017 ◽  
Vol 2017 ◽  
pp. 1-10
Author(s):  
Hsuan-Ming Huang ◽  
Ing-Tsung Hsiao
Keyword(s):  
Computed Tomography ◽  
Image Reconstruction ◽  
Low Dose ◽  
Computational Cost ◽  
Reconstruction Algorithm ◽  
Reconstruction Algorithms ◽  
Acceleration Techniques ◽  
Reconstruction Methods ◽  
Speed Up ◽  
Phantom Studies

Background and Objective. Over the past decade, image quality in low-dose computed tomography has been greatly improved by various compressive sensing- (CS-) based reconstruction methods. However, these methods have some disadvantages including high computational cost and slow convergence rate. Many different speed-up techniques for CS-based reconstruction algorithms have been developed. The purpose of this paper is to propose a fast reconstruction framework that combines a CS-based reconstruction algorithm with several speed-up techniques.Methods. First, total difference minimization (TDM) was implemented using the soft-threshold filtering (STF). Second, we combined TDM-STF with the ordered subsets transmission (OSTR) algorithm for accelerating the convergence. To further speed up the convergence of the proposed method, we applied the power factor and the fast iterative shrinkage thresholding algorithm to OSTR and TDM-STF, respectively.Results. Results obtained from simulation and phantom studies showed that many speed-up techniques could be combined to greatly improve the convergence speed of a CS-based reconstruction algorithm. More importantly, the increased computation time (≤10%) was minor as compared to the acceleration provided by the proposed method.Conclusions. In this paper, we have presented a CS-based reconstruction framework that combines several acceleration techniques. Both simulation and phantom studies provide evidence that the proposed method has the potential to satisfy the requirement of fast image reconstruction in practical CT.

scholarly journals Real-Time Path Planning Based on Harmonic Functions under a Proper Generalized Decomposition-Based Framework

Sensors ◽  
2021 ◽  
Vol 21 (12) ◽  
pp. 3943
Author(s):  
Nicolas Montés ◽  
Francisco Chinesta ◽  
Marta C. Mora ◽  
Antonio Falcó ◽  
Lucia Hilario ◽  
...  
Keyword(s):  
Path Planning ◽  
Real Time ◽  
Harmonic Functions ◽  
Autonomous Navigation ◽  
Numerical Technique ◽  
Computational Cost ◽  
Main Idea ◽  
Main Property ◽  
Proper Generalized Decomposition ◽  
Computational Vademecum

This paper presents a real-time global path planning method for mobile robots using harmonic functions, such as the Poisson equation, based on the Proper Generalized Decomposition (PGD) of these functions. The main property of the proposed technique is that the computational cost is negligible in real-time, even if the robot is disturbed or the goal is changed. The main idea of the method is the off-line generation, for a given environment, of the whole set of paths from any start and goal configurations of a mobile robot, namely the computational vademecum, derived from a harmonic potential field in order to use it on-line for decision-making purposes. Up until now, the resolution of the Laplace or Poisson equations has been based on traditional numerical techniques unfeasible for real-time calculation. This drawback has prevented the extensive use of harmonic functions in autonomous navigation, despite their powerful properties. The numerical technique that reverses this situation is the Proper Generalized Decomposition. To demonstrate and validate the properties of the PGD-vademecum in a potential-guided path planning framework, both real and simulated implementations have been developed. Simulated scenarios, such as an L-Shaped corridor and a benchmark bug trap, are used, and a real navigation of a LEGO®MINDSTORMS robot running in static environments with variable start and goal configurations is shown. This device has been selected due to its computational and memory-restricted capabilities, and it is a good example of how its properties could help the development of social robots.

scholarly journals Finite element method completely implemented for graphic processor units using parallel algorithm libraries

2017 ◽  
Vol 33 (1) ◽  
pp. 53-66 ◽  
Author(s):  
Franz Pichler ◽  
Gundolf Haase
Keyword(s):  
Finite Element ◽  
Graphics Processing Unit ◽  
Computational Cost ◽  
Processing Unit ◽  
Time Step ◽  
Device Architecture ◽  
Transient Problems ◽  
Speed Up ◽  
Automotive Batteries ◽  
Graphics Processing

A finite element code is developed in which all of the computationally expensive steps are performed on a graphics processing unit via the THRUST and the PARALUTION libraries. The code focuses on the simulation of transient problems where the repeated computations per time-step create the computational cost. It is used to solve partial and ordinary differential equations as they arise in thermal-runaway simulations of automotive batteries. The speed-up obtained by utilizing the graphics processing unit for every critical step is compared against the single core and the multi-threading solutions which are also supported by the chosen libraries. This way a high total speed-up on the graphics processing unit is achieved without the need for programming a single classical Compute Unified Device Architecture kernel.

Sign in / Sign up

Export Citation Format

Share Document