Robust Semantic Mapping in Challenging Environments

Robotica ◽  
2019 ◽  
Vol 38 (2) ◽  
pp. 256-270 ◽  
Author(s):  
Jiyu Cheng ◽  
Yuxiang Sun ◽  
Max Q.-H. Meng
Keyword(s):  
Semantic Information ◽  
Moving Objects ◽  
Dynamic Environments ◽  
Semantic Mapping ◽  
Visual Slam ◽  
The Public ◽  
Semantic Map ◽  
Recent Success ◽  
3D Space ◽  
Localization And Mapping

SummaryVisual simultaneous localization and mapping (visual SLAM) has been well developed in recent decades. To facilitate tasks such as path planning and exploration, traditional visual SLAM systems usually provide mobile robots with the geometric map, which overlooks the semantic information. To address this problem, inspired by the recent success of the deep neural network, we combine it with the visual SLAM system to conduct semantic mapping. Both the geometric and semantic information will be projected into the 3D space for generating a 3D semantic map. We also use an optical-flow-based method to deal with the moving objects such that our method is capable of working robustly in dynamic environments. We have performed our experiments in the public TUM dataset and our recorded office dataset. Experimental results demonstrate the feasibility and impressive performance of the proposed method.

scholarly journals An Improved Deep Residual Network-Based Semantic Simultaneous Localization and Mapping Method for Monocular Vision Robot

2020 ◽  
Vol 2020 ◽  
pp. 1-14 ◽  
Author(s):  
Jianjun Ni ◽  
Tao Gong ◽  
Yafei Gu ◽  
Jinxiu Zhu ◽  
Xinnan Fan
Keyword(s):  
Semantic Information ◽  
Hot Spot ◽  
Simultaneous Localization And Mapping ◽  
Mapping Method ◽  
Monocular Vision ◽  
Visual Slam ◽  
Camera Tracking ◽  
Residual Network ◽  
Semantic Map ◽  
Localization And Mapping

The robot simultaneous localization and mapping (SLAM) is a very important and useful technology in the robotic field. However, the environmental map constructed by the traditional visual SLAM method contains little semantic information, which cannot satisfy the needs of complex applications. The semantic map can deal with this problem efficiently, which has become a research hot spot. This paper proposed an improved deep residual network- (ResNet-) based semantic SLAM method for monocular vision robots. In the proposed approach, an improved image matching algorithm based on feature points is presented, to enhance the anti-interference ability of the algorithm. Then, the robust feature point extraction method is adopted in the front-end module of the SLAM system, which can effectively reduce the probability of camera tracking loss. In addition, the improved key frame insertion method is introduced in the visual SLAM system to enhance the stability of the system during the turning and moving of the robot. Furthermore, an improved ResNet model is proposed to extract the semantic information of the environment to complete the construction of the semantic map of the environment. Finally, various experiments are conducted and the results show that the proposed method is effective.

scholarly journals Semantic Mapping for Mobile Robots in Indoor Scenes: A Survey

Information ◽  
2021 ◽  
Vol 12 (2) ◽  
pp. 92
Author(s):  
Xiaoning Han ◽  
Shuailong Li ◽  
Xiaohui Wang ◽  
Weijia Zhou
Keyword(s):  
Mobile Robots ◽  
Semantic Information ◽  
Semantic Mapping ◽  
Human Beings ◽  
Future Directions ◽  
Essential Requirement ◽  
Semantic Map ◽  
Semantic Maps ◽  
Great Progress ◽  
Indoor Scenes

Sensing and mapping its surroundings is an essential requirement for a mobile robot. Geometric maps endow robots with the capacity of basic tasks, e.g., navigation. To co-exist with human beings in indoor scenes, the need to attach semantic information to a geometric map, which is called a semantic map, has been realized in the last two decades. A semantic map can help robots to behave in human rules, plan and perform advanced tasks, and communicate with humans on the conceptual level. This survey reviews methods about semantic mapping in indoor scenes. To begin with, we answered the question, what is a semantic map for mobile robots, by its definitions. After that, we reviewed works about each of the three modules of semantic mapping, i.e., spatial mapping, acquisition of semantic information, and map representation, respectively. Finally, though great progress has been made, there is a long way to implement semantic maps in advanced tasks for robots, thus challenges and potential future directions are discussed before a conclusion at last.

scholarly journals Sensor Fusion-Based Approach to Eliminating Moving Objects for SLAM in Dynamic Environments

Sensors ◽  
2021 ◽  
Vol 21 (1) ◽  
pp. 230
Author(s):  
Xiangwei Dang ◽  
Zheng Rong ◽  
Xingdong Liang
Keyword(s):  
State Estimation ◽  
Autonomous Navigation ◽  
Moving Objects ◽  
Physical Simulation ◽  
Point Clouds ◽  
Dynamic Environments ◽  
Accurate Localization ◽  
Novel Approach ◽  
Localization And Mapping ◽  
Real World Datasets

Accurate localization and reliable mapping is essential for autonomous navigation of robots. As one of the core technologies for autonomous navigation, Simultaneous Localization and Mapping (SLAM) has attracted widespread attention in recent decades. Based on vision or LiDAR sensors, great efforts have been devoted to achieving real-time SLAM that can support a robot’s state estimation. However, most of the mature SLAM methods generally work under the assumption that the environment is static, while in dynamic environments they will yield degenerate performance or even fail. In this paper, first we quantitatively evaluate the performance of the state-of-the-art LiDAR-based SLAMs taking into account different pattens of moving objects in the environment. Through semi-physical simulation, we observed that the shape, size, and distribution of moving objects all can impact the performance of SLAM significantly, and obtained instructive investigation results by quantitative comparison between LOAM and LeGO-LOAM. Secondly, based on the above investigation, a novel approach named EMO to eliminating the moving objects for SLAM fusing LiDAR and mmW-radar is proposed, towards improving the accuracy and robustness of state estimation. The method fully uses the advantages of different characteristics of two sensors to realize the fusion of sensor information with two different resolutions. The moving objects can be efficiently detected based on Doppler effect by radar, accurately segmented and localized by LiDAR, then filtered out from the point clouds through data association and accurate synchronized in time and space. Finally, the point clouds representing the static environment are used as the input of SLAM. The proposed approach is evaluated through experiments using both semi-physical simulation and real-world datasets. The results demonstrate the effectiveness of the method at improving SLAM performance in accuracy (decrease by 30% at least in absolute position error) and robustness in dynamic environments.

scholarly journals 3D Semantic VSLAM of Indoor Environment Based on Mask Scoring RCNN

2020 ◽  
Vol 2020 ◽  
pp. 1-14
Author(s):  
Chongben Tao ◽  
Yufeng Jin ◽  
Feng Cao ◽  
Zufeng Zhang ◽  
Chunguang Li ◽  
...  
Keyword(s):  
Indoor Environment ◽  
Semantic Information ◽  
Moving Objects ◽  
Experimental Comparison ◽  
Indoor Environments ◽  
Feature Points ◽  
Current Frame ◽  
Semantic Map ◽  
Key Frame ◽  
The Impact

In view of existing Visual SLAM (VSLAM) algorithms when constructing semantic map of indoor environment, there are problems with low accuracy and low label classification accuracy when feature points are sparse. This paper proposed a 3D semantic VSLAM algorithm called BMASK-RCNN based on Mask Scoring RCNN. Firstly, feature points of images are extracted by Binary Robust Invariant Scalable Keypoints (BRISK) algorithm. Secondly, map points of reference key frame are projected to current frame for feature matching and pose estimation, and an inverse depth filter is used to estimate scene depth of created key frame to obtain camera pose changes. In order to achieve object detection and semantic segmentation for both static objects and dynamic objects in indoor environments and then construct dense 3D semantic map with VSLAM algorithm, a Mask Scoring RCNN is used to adjust its structure partially, where a TUM RGB-D SLAM dataset for transfer learning is employed. Semantic information of independent targets in scenes provides semantic information including categories, which not only provides high accuracy of localization but also realizes the probability update of semantic estimation by marking movable objects, thereby reducing the impact of moving objects on real-time mapping. Through simulation and actual experimental comparison with other three algorithms, results show the proposed algorithm has better robustness, and semantic information used in 3D semantic mapping can be accurately obtained.

scholarly journals Object-Level Semantic Map Construction for Dynamic Scenes

2021 ◽  
Vol 11 (2) ◽  
pp. 645
Author(s):  
Xujie Kang ◽  
Jing Li ◽  
Xiangtao Fan ◽  
Hongdeng Jian ◽  
Chen Xu
Keyword(s):  
Moving Objects ◽  
Dynamic Environments ◽  
Dynamic Object ◽  
Dynamic Scenes ◽  
Current Frame ◽  
Semantic Map ◽  
Pose Tracking ◽  
Camera Pose ◽  
Object Level ◽  
Instance Segmentation

Visual simultaneous localization and mapping (SLAM) is challenging in dynamic environments as moving objects can impair camera pose tracking and mapping. This paper introduces a method for robust dense bject-level SLAM in dynamic environments that takes a live stream of RGB-D frame data as input, detects moving objects, and segments the scene into different objects while simultaneously tracking and reconstructing their 3D structures. This approach provides a new method of dynamic object detection, which integrates prior knowledge of the object model database constructed, object-oriented 3D tracking against the camera pose, and the association between the instance segmentation results on the current frame data and an object database to find dynamic objects in the current frame. By leveraging the 3D static model for frame-to-model alignment, as well as dynamic object culling, the camera motion estimation reduced the overall drift. According to the camera pose accuracy and instance segmentation results, an object-level semantic map representation was constructed for the world map. The experimental results obtained using the TUM RGB-D dataset, which compares the proposed method to the related state-of-the-art approaches, demonstrating that our method achieves similar performance in static scenes and improved accuracy and robustness in dynamic scenes.

scholarly journals Visual SLAM Based on Dynamic Object Removal

2020 ◽  
Author(s):  
Guoliang Liu
Keyword(s):  
Moving Objects ◽  
State Of The Art ◽  
Service Robot ◽  
Visual Slam ◽  
Dynamic Object ◽  
Dynamic Features ◽  
Dynamic Scenes ◽  
The Core ◽  
Localization And Mapping ◽  
Strong Hypothesis

Visual simultaneous localization and mapping (SLAM) is the core of intelligent robot navigation system. Many traditional SLAM algorithms assume that the scene is static. When a dynamic object appears in the environment, the accuracy of visual SLAM can degrade due to the interference of dynamic features of moving objects. This strong hypothesis limits the SLAM applications for service robot or driverless car intherealdynamicenvironment.Inthispaper,adynamicobject removal algorithm that combines object recognition and optical flow techniques is proposed in the visual SLAM framework for dynamic scenes. The experimental results show that our new method can detect moving object effectively and improve the SLAM performance compared to the state of the art methods.<br>

scholarly journals IMU-Assisted 2D SLAM Method for Low-Texture and Dynamic Environments

2018 ◽  
Vol 8 (12) ◽  
pp. 2534 ◽  
Author(s):  
Zhongli Wang ◽  
Yan Chen ◽  
Yue Mei ◽  
Kuo Yang ◽  
Baigen Cai
Keyword(s):  
Closed Loop ◽  
Moving Objects ◽  
Interpolation Method ◽  
Dynamic Environments ◽  
Open Loop ◽  
Indoor Environments ◽  
Matching Problem ◽  
Scan Matching ◽  
Localization And Mapping ◽  
Loop Detection

Generally, the key issues of 2D LiDAR-based simultaneous localization and mapping (SLAM) for indoor application include data association (DA) and closed-loop detection. Particularly, a low-texture environment, which refers to no obvious changes between two consecutive scanning outputs, with moving objects existing in the environment will bring great challenges on DA and the closed-loop detection, and the accuracy and consistency of SLAM may be badly affected. There is not much literature that addresses this issue. In this paper, a mapping strategy is firstly exploited to improve the performance of the 2D SLAM in dynamic environments. Secondly, a fusion method which combines the IMU sensor with a 2D LiDAR, based on framework of extended Kalman Filter (EKF), is proposed to enhance the performance under low-texture environments. In the front-end of the proposed SLAM method, initial motion estimation is obtained from the output of EKF, and it can be taken as the initial pose for the scan matching problem. Then the scan matching problem can be optimized by the Levenberg–Marquardt (LM) algorithm. For the back-end optimization, a sparse pose adjustment (SPA) method is employed. To improve the accuracy, the grid map is updated with the bicubic interpolation method for derivative computing. With the improvements both in the DA process and the back-end optimization stage, the accuracy and consistency of SLAM results in low-texture environments is enhanced. Qualitative and quantitative experiments with open-loop and closed-loop cases have been conducted and the results are analyzed, confirming that the proposed method is effective in low-texture and dynamic indoor environments.

scholarly journals The Method of Static Semantic Map Construction Based on Instance Segmentation and Dynamic Point Elimination

Electronics ◽  
2021 ◽  
Vol 10 (16) ◽  
pp. 1883
Author(s):  
Jingyu Li ◽  
Rongfen Zhang ◽  
Yuhong Liu ◽  
Zaiteng Zhang ◽  
Runze Fan ◽  
...  
Keyword(s):  
Semantic Information ◽  
Semantic Map ◽  
Average Value ◽  
Map Construction ◽  
Dense Optical Flow ◽  
Semantic Maps ◽  
Localization And Mapping ◽  
Dynamic Objects ◽  
High Level ◽  
Static Semantic

Semantic information usually contains a description of the environment content, which enables mobile robot to understand the environment and improves its ability to interact with the environment. In high-level human–computer interaction application, the Simultaneous Localization and Mapping (SLAM) system not only needs higher accuracy and robustness, but also has the ability to construct a static semantic map of the environment. However, traditional visual SLAM lacks semantic information. Furthermore, in an actual scene, dynamic objects will reduce the system performance and also generate redundancy when constructing map. these all directly affect the robot’s ability to perceive and understand the surrounding environment. Based on ORB-SLAM3, this article proposes a new Algorithm that uses semantic information and the global dense optical flow as constraints to generate dynamic-static mask and eliminate dynamic objects. then, to further construct a static 3D semantic map under indoor dynamic environments, a fusion of 2D semantic information and 3D point cloud is carried out. the experimental results on different types of dataset sequences show that, compared with original ORB-SLAM3, both Absolute Pose Error (APE) and Relative Pose Error (RPE) have been ameliorated to varying degrees, especially on freiburg3-walking-xyz, the APE reduced by 97.78% from the original average value of 0.523, and RPE reduced by 52.33% from the original average value of 0.0193. Compared with DS-SLAM and DynaSLAM, our system improves real-time performance while ensuring accuracy and robustness. Meanwhile, the expected map with environmental semantic information is built, and the map redundancy caused by dynamic objects is successfully reduced. the test results in real scenes further demonstrate the effect of constructing static semantic maps and prove the effectiveness of our Algorithm.

scholarly journals Visual SLAM Based on Dynamic Object Removal

2020 ◽  
Author(s):  
Guoliang Liu
Keyword(s):  
Moving Objects ◽  
State Of The Art ◽  
Service Robot ◽  
Visual Slam ◽  
Dynamic Object ◽  
Dynamic Features ◽  
Dynamic Scenes ◽  
The Core ◽  
Localization And Mapping ◽  
Strong Hypothesis

Visual simultaneous localization and mapping (SLAM) is the core of intelligent robot navigation system. Many traditional SLAM algorithms assume that the scene is static. When a dynamic object appears in the environment, the accuracy of visual SLAM can degrade due to the interference of dynamic features of moving objects. This strong hypothesis limits the SLAM applications for service robot or driverless car intherealdynamicenvironment.Inthispaper,adynamicobject removal algorithm that combines object recognition and optical flow techniques is proposed in the visual SLAM framework for dynamic scenes. The experimental results show that our new method can detect moving object effectively and improve the SLAM performance compared to the state of the art methods.<br>

Sign in / Sign up

Export Citation Format

Share Document