scholarly journals Semantic 3D Mapping from Deep Image Segmentation

2021 ◽  
Vol 11 (4) ◽  
pp. 1953
Author(s):  
Francisco Martín ◽  
Fernando González ◽  
José Miguel Guerrero ◽  
Manuel Fernández ◽  
Jonatan Ginés
Keyword(s):  
Point Cloud ◽  
Autonomous Mobile Robots ◽  
Semantic Mapping ◽  
Indoor Environments ◽  
3D Mapping ◽  
Direct Mapping ◽  
Learning Techniques ◽  
3D Camera ◽  
Deep Image ◽  
Boundary Pixel

The perception and identification of visual stimuli from the environment is a fundamental capacity of autonomous mobile robots. Current deep learning techniques make it possible to identify and segment objects of interest in an image. This paper presents a novel algorithm to segment the object’s space from a deep segmentation of an image taken by a 3D camera. The proposed approach solves the boundary pixel problem that appears when a direct mapping from segmented pixels to their correspondence in the point cloud is used. We validate our approach by comparing baseline approaches using real images taken by a 3D camera, showing that our method outperforms their results in terms of accuracy and reliability. As an application of the proposed algorithm, we present a semantic mapping approach for a mobile robot’s indoor environments.

scholarly journals SGC-VSLAM: A Semantic and Geometric Constraints VSLAM for Dynamic Indoor Environments

Sensors ◽  
2020 ◽  
Vol 20 (8) ◽  
pp. 2432
Author(s):  
Shiqiang Yang ◽  
Guohao Fan ◽  
Lele Bai ◽  
Cheng Zhao ◽  
Dexin Li
Keyword(s):  
Mobile Robots ◽  
Point Cloud ◽  
Feature Detection ◽  
Geometric Constraints ◽  
Autonomous Mobile Robots ◽  
Selection Strategy ◽  
Dynamic Feature ◽  
Indoor Environments ◽  
Dynamic Features ◽  
Static Point

As one of the core technologies for autonomous mobile robots, Visual Simultaneous Localization and Mapping (VSLAM) has been widely researched in recent years. However, most state-of-the-art VSLAM adopts a strong scene rigidity assumption for analytical convenience, which limits the utility of these algorithms for real-world environments with independent dynamic objects. Hence, this paper presents a semantic and geometric constraints VSLAM (SGC-VSLAM), which is built on the RGB-D mode of ORB-SLAM2 with the addition of dynamic detection and static point cloud map construction modules. In detail, a novel improved quadtree-based method was adopted for SGC-VSLAM to enhance the performance of the feature extractor in ORB-SLAM (Oriented FAST and Rotated BRIEF-SLAM). Moreover, a new dynamic feature detection method called semantic and geometric constraints was proposed, which provided a robust and fast way to filter dynamic features. The semantic bounding box generated by YOLO v3 (You Only Look Once, v3) was used to calculate a more accurate fundamental matrix between adjacent frames, which was then used to filter all of the truly dynamic features. Finally, a static point cloud was estimated by using a new drawing key frame selection strategy. Experiments on the public TUM RGB-D (Red-Green-Blue Depth) dataset were conducted to evaluate the proposed approach. This evaluation revealed that the proposed SGC-VSLAM can effectively improve the positioning accuracy of the ORB-SLAM2 system in high-dynamic scenarios and was also able to build a map with the static parts of the real environment, which has long-term application value for autonomous mobile robots.

scholarly journals Indoor Path-Planning Algorithm for UAV-Based Contact Inspection

Sensors ◽  
2021 ◽  
Vol 21 (2) ◽  
pp. 642
Author(s):  
Luis Miguel González de Santos ◽  
Ernesto Frías Nores ◽  
Joaquín Martínez Sánchez ◽  
Higinio González Jorge
Keyword(s):  
Path Planning ◽  
Real Time ◽  
Execution Time ◽  
Point Cloud ◽  
Indoor Environments ◽  
Calculation Algorithm ◽  
Time Restrictions ◽  
Planning Algorithm ◽  
Inspection Tasks ◽  
Path Planning Algorithm

Nowadays, unmanned aerial vehicles (UAVs) are extensively used for multiple purposes, such as infrastructure inspections or surveillance. This paper presents a real-time path planning algorithm in indoor environments designed to perform contact inspection tasks using UAVs. The only input used by this algorithm is the point cloud of the building where the UAV is going to navigate. The algorithm is divided into two main parts. The first one is the pre-processing algorithm that processes the point cloud, segmenting it into rooms and discretizing each room. The second part is the path planning algorithm that has to be executed in real time. In this way, all the computational load is in the first step, which is pre-processed, making the path calculation algorithm faster. The method has been tested in different buildings, measuring the execution time for different paths calculations. As can be seen in the results section, the developed algorithm is able to calculate a new path in 8–9 milliseconds. The developed algorithm fulfils the execution time restrictions, and it has proven to be reliable for route calculation.

scholarly journals Multi-sensor point cloud data fusion for precise 3D mapping

2021 ◽  
Author(s):  
Mohamed Abdelazeem ◽  
Ahmed Elamin ◽  
Akram Afifi ◽  
Ahmed El-Rabbany
Keyword(s):  
Data Fusion ◽  
Point Cloud ◽  
3D Mapping ◽  
Point Cloud Data ◽  
Cloud Data

Semantic Mapping Based on Image Feature Fusion in Indoor Environments

2021 ◽  
Author(s):  
Cong Jin ◽  
Armagan Elibol ◽  
Pengfei Zhu ◽  
Nak Young Chong
Keyword(s):  
Feature Fusion ◽  
Image Feature ◽  
Semantic Mapping ◽  
Indoor Environments

scholarly journals Dense RGB-D Semantic Mapping with Pixel-Voxel Neural Network

Sensors ◽  
2018 ◽  
Vol 18 (9) ◽  
pp. 3099 ◽  
Author(s):  
Cheng Zhao ◽  
Li Sun ◽  
Pulak Purkait ◽  
Tom Duckett ◽  
Rustam Stolkin
Keyword(s):  
Point Cloud ◽  
Contextual Information ◽  
Semantic Category ◽  
Semantic Mapping ◽  
Confidence Levels ◽  
Time Performance ◽  
Weighted Fusion ◽  
Geometrical Shapes ◽  
Rgb Images ◽  
High Level

In this paper, a novel Pixel-Voxel network is proposed for dense 3D semantic mapping, which can perform dense 3D mapping while simultaneously recognizing and labelling the semantic category each point in the 3D map. In our approach, we fully leverage the advantages of different modalities. That is, the PixelNet can learn the high-level contextual information from 2D RGB images, and the VoxelNet can learn 3D geometrical shapes from the 3D point cloud. Unlike the existing architecture that fuses score maps from different modalities with equal weights, we propose a softmax weighted fusion stack that adaptively learns the varying contributions of PixelNet and VoxelNet and fuses the score maps according to their respective confidence levels. Our approach achieved competitive results on both the SUN RGB-D and NYU V2 benchmarks, while the runtime of the proposed system is boosted to around 13 Hz, enabling near-real-time performance using an i7 eight-cores PC with a single Titan X GPU.

scholarly journals Fast and Robust Flight Altitude Estimation of Multirotor UAVs in Dynamic Unstructured Environments Using 3D Point Cloud Sensors

Aerospace ◽  
2018 ◽  
Vol 5 (3) ◽  
pp. 94 ◽  
Author(s):  
Hriday Bavle ◽  
Jose Sanchez-Lopez ◽  
Paloma Puente ◽  
Alejandro Rodriguez-Ramos ◽  
Carlos Sampedro ◽  
...  
Keyword(s):  
Point Cloud ◽  
Point Clouds ◽  
Estimation Algorithm ◽  
Indoor Environments ◽  
3D Point Cloud ◽  
Flight Altitude ◽  
Cloud Data ◽  
3D Point Clouds ◽  
Laser Altimeters ◽  
Altitude Estimation

This paper presents a fast and robust approach for estimating the flight altitude of multirotor Unmanned Aerial Vehicles (UAVs) using 3D point cloud sensors in cluttered, unstructured, and dynamic indoor environments. The objective is to present a flight altitude estimation algorithm, replacing the conventional sensors such as laser altimeters, barometers, or accelerometers, which have several limitations when used individually. Our proposed algorithm includes two stages: in the first stage, a fast clustering of the measured 3D point cloud data is performed, along with the segmentation of the clustered data into horizontal planes. In the second stage, these segmented horizontal planes are mapped based on the vertical distance with respect to the point cloud sensor frame of reference, in order to provide a robust flight altitude estimation even in presence of several static as well as dynamic ground obstacles. We validate our approach using the IROS 2011 Kinect dataset available in the literature, estimating the altitude of the RGB-D camera using the provided 3D point clouds. We further validate our approach using a point cloud sensor on board a UAV, by means of several autonomous real flights, closing its altitude control loop using the flight altitude estimated by our proposed method, in presence of several different static as well as dynamic ground obstacles. In addition, the implementation of our approach has been integrated in our open-source software framework for aerial robotics called Aerostack.

scholarly journals Robust Visual Compass Using Hybrid Features for Indoor Environments

Electronics ◽  
2019 ◽  
Vol 8 (2) ◽  
pp. 220 ◽  
Author(s):  
Ruibin Guo ◽  
Keju Peng ◽  
Dongxiang Zhou ◽  
Yunhui Liu
Keyword(s):  
Autonomous Navigation ◽  
Estimation Method ◽  
Rotation Matrix ◽  
Indoor Environments ◽  
3D Mapping ◽  
Orientation Estimation ◽  
Hybrid Features ◽  
Depth Images ◽  
Improved Accuracy ◽  
Crucial Part

Orientation estimation is a crucial part of robotics tasks such as motion control, autonomous navigation, and 3D mapping. In this paper, we propose a robust visual-based method to estimate robots’ drift-free orientation with RGB-D cameras. First, we detect and track hybrid features (i.e., plane, line, and point) from color and depth images, which provides reliable constraints even in uncharacteristic environments with low texture or no consistent lines. Then, we construct a cost function based on these features and, by minimizing this function, we obtain the accurate rotation matrix of each captured frame with respect to its reference keyframe. Furthermore, we present a vanishing direction-estimation method to extract the Manhattan World (MW) axes; by aligning the current MW axes with the global MW axes, we refine the aforementioned rotation matrix of each keyframe and achieve drift-free orientation. Experiments on public RGB-D datasets demonstrate the robustness and accuracy of the proposed algorithm for orientation estimation. In addition, we have applied our proposed visual compass to pose estimation, and the evaluation on public sequences shows improved accuracy.

scholarly journals A JOINT EFFORT OF SPEEDED-UP ROBUST FEATURES ALGORITHM AND A DISPARITY-BASED MODEL FOR 3D INDOOR MAPPING USING RGB-D DATA

2018 ◽  
Vol 24 (3) ◽  
pp. 351-366
Author(s):  
Marcos Aurélio Basso ◽  
Daniel Rodrigues dos Santos
Keyword(s):  
Indoor Environments ◽  
3D Mapping ◽  
Joint Effort ◽  
Plane Model ◽  
Loop Closure ◽  
Speeded Up Robust Features ◽  
Speed Up ◽  
Coarse To Fine ◽  
Fine Registration ◽  
Different Levels

Abstract In this paper, we present a method for 3D mapping of indoor environments using RGB-D data. The contribution of our proposed method is two-fold. First, our method exploits a joint effort of the speed-up robust features (SURF) algorithm and a disparity-to-plane model for a coarse-to-fine registration procedure. Once the coarse-to-fine registration task accumulates errors, the same features can appear in two different locations of the map. This is known as the loop closure problem. Then, the variance-covariance matrix that describes the uncertainty of transformation parameters (3D rotation and 3D translation) for view-based loop closure detection followed by a graph-based optimization are proposed to achieve a 3D consistent indoor map. To demonstrate and evaluate the effectiveness of the proposed method, experimental datasets obtained in three indoor environments with different levels of details are used. The experimental results shown that the proposed framework can create 3D indoor maps with an error of 11,97 cm into object space that corresponds to a positional imprecision around 1,5% at the distance of 9 m travelled by sensor.

Sign in / Sign up

Export Citation Format

Share Document