RDS-SLAM: Real-Time Dynamic SLAM Using Semantic Segmentation Methods

As the techniques of autonomous driving become increasingly valued and universal, real-time semantic segmentation has become very popular and challenging in the field of deep learning and computer vision in recent years. However, in order to apply the deep learning model to edge devices accompanying sensors on vehicles, we need to design a structure that has the best trade-off between accuracy and inference time. In previous works, several methods sacrificed accuracy to obtain a faster inference time, while others aimed to find the best accuracy under the condition of real time. Nevertheless, the accuracies of previous real-time semantic segmentation methods still have a large gap compared to general semantic segmentation methods. As a result, we propose a network architecture based on a dual encoder and a self-attention mechanism. Compared with preceding works, we achieved a 78.6% mIoU with a speed of 39.4 FPS with a 1024 × 2048 resolution on a Cityscapes test submission.

Download Full-text

DUDMap: 3D RGB-D mapping for dense, unstructured, and dynamic environment

International Journal of Advanced Robotic Systems ◽

10.1177/17298814211016178 ◽

2021 ◽

Vol 18 (3) ◽

pp. 172988142110161

Author(s):

Özgür Hastürk ◽

Aydan M Erkmen

Keyword(s):

Real Time ◽

Dynamic Environment ◽

Semantic Segmentation ◽

Small Region ◽

Dynamic Environments ◽

Reconstruction Error ◽

Localization And Mapping ◽

Segmentation Methods ◽

High Dynamic ◽

Better Than

Simultaneous localization and mapping (SLAM) problem has been extensively studied by researchers in the field of robotics, however, conventional approaches in mapping assume a static environment. The static assumption is valid only in a small region, and it limits the application of visual SLAM in dynamic environments. The recently proposed state-of-the-art SLAM solutions for dynamic environments use different semantic segmentation methods such as mask R-CNN and SegNet; however, these frameworks are based on a sparse mapping framework (ORBSLAM). In addition, segmentation process increases the computational power, which makes these SLAM algorithms unsuitable for real-time mapping. Therefore, there is no effective dense RGB-D SLAM method for real-world unstructured and dynamic environments. In this study, we propose a novel real-time dense SLAM method for dynamic environments, where 3D reconstruction error is manipulated for identification of static and dynamic classes having generalized Gaussian distribution. Our proposed approach requires neither explicit object tracking nor object classifier, which makes it robust to any type of moving object and suitable for real-time mapping. Our method eliminates the repeated views and uses consistent data that enhance the performance of volumetric fusion. For completeness, we compare our proposed method using different types of high dynamic dataset, which are publicly available, to demonstrate the versatility and robustness of our approach. Experiments show that its tracking performance is better than other dense and dynamic SLAM approaches.

Download Full-text

MBFFNet: Multi-Branch Feature Fusion Network for Colonoscopy

Frontiers in Bioengineering and Biotechnology ◽

10.3389/fbioe.2021.696251 ◽

2021 ◽

Vol 9 ◽

Author(s):

Houcheng Su ◽

Bin Lin ◽

Xiaoshuang Huang ◽

Jiao Li ◽

Kailin Jiang ◽

...

Keyword(s):

Real Time ◽

Feature Fusion ◽

Rapid Development ◽

Semantic Segmentation ◽

Medical Image Segmentation ◽

Feature Maps ◽

Improve Model ◽

Segmentation Methods ◽

Rectal Polyps ◽

Good Potential

Colonoscopy is currently one of the main methods for the detection of rectal polyps, rectal cancer, and other diseases. With the rapid development of computer vision, deep learning–based semantic segmentation methods can be applied to the detection of medical lesions. However, it is challenging for current methods to detect polyps with high accuracy and real-time performance. To solve this problem, we propose a multi-branch feature fusion network (MBFFNet), which is an accurate real-time segmentation method for detecting colonoscopy. First, we use UNet as the basis of our model architecture and adopt stepwise sampling with channel multiplication to integrate features, which decreases the number of flops caused by stacking channels in UNet. Second, to improve model accuracy, we extract features from multiple layers and resize feature maps to the same size in different ways, such as up-sampling and pooling, to supplement information lost in multiplication-based up-sampling. Based on mIOU and Dice loss with cross entropy (CE), we conduct experiments in both CPU and GPU environments to verify the effectiveness of our model. The experimental results show that our proposed MBFFNet is superior to the selected baselines in terms of accuracy, model size, and flops. mIOU, F score, and Dice loss with CE reached 0.8952, 0.9450, and 0.1602, respectively, which were better than those of UNet, UNet++, and other networks. Compared with UNet, the flop count decreased by 73.2%, and the number of participants also decreased. The actual segmentation effect of MBFFNet is only lower than that of PraNet, the number of parameters is 78.27% of that of PraNet, and the flop count is 0.23% that of PraNet. In addition, experiments on other types of medical tasks show that MBFFNet has good potential for general application in medical image segmentation.

Download Full-text

A REAL-TIME SEMANTIC SEGMENTATION ALGORITHM FOR AERIAL IMAGERY

Computer Optics ◽

10.18287/2412-6179-2018-42-1-141-148 ◽

2018 ◽

Vol 42 (1) ◽

pp. 141-148 ◽

Cited By ~ 8

Author(s):

Y. B. Blokhinov ◽

V. A. Gorbachev ◽

Y. O. Rakutin ◽

D. A. Nikitin

Keyword(s):

Real Time ◽

Semantic Segmentation ◽

Time Algorithm ◽

Original Method ◽

Aerial Imagery ◽

Effective Algorithm ◽

Aerial Photos ◽

Segmentation Methods ◽

And Performance ◽

Segmentation Of Images

We propose a novel effective algorithm for real-time semantic segmentation of images that has the best accuracy in its class. Based on a comparative analysis of preliminary segmentation methods, methods for calculating attributes from image segments, as well as various algorithms of machine learning, the most effective methods in terms of their accuracy and performance are identified. Based on the research results, a modular near real-time algorithm of semantic segmentation is constructed. Training and testing is performed on the ISPRS Vaihingen collection of aerial photos of the visible and IR ranges, to which a pixel map of the terrain heights is attached. An original method for obtaining a normalized nDSM for the original DSM is proposed.

Download Full-text

Control Model Constructing on Real-Time Dynamic Reversible Lane in Condition of Connected Vehicle

CICTP 2020 ◽

10.1061/9780784483053.067 ◽

2020 ◽

Author(s):

Lina Mao ◽

Wenquan Li ◽

Pengsen Hu ◽

Guiliang Zhou ◽

Huiting Zhang ◽

...

Keyword(s):

Real Time ◽

Control Model ◽

Connected Vehicle ◽

Time Dynamic

Download Full-text

Use of Real-Time Dynamic Data to Generate Fracture Network Models

10.2523/11492-ms ◽

2007 ◽

Cited By ~ 1

Author(s):

Ammal Fannoush Al-Anazi ◽

Tayfun Babadagli

Keyword(s):

Real Time ◽

Network Models ◽

Fracture Network ◽

Dynamic Data ◽

Time Dynamic

Download Full-text

Development of environment design support mixed reality system capable of environment estimation using deep learning

Impact ◽

10.21820/23987073.2020.2.9 ◽

2020 ◽

Vol 2020 (2) ◽

pp. 9-11

Author(s):

Tomohiro Fukuda

Keyword(s):

Deep Learning ◽

Real Time ◽

Computer Games ◽

Construction Projects ◽

Mixed Reality ◽

Semantic Segmentation ◽

Environment Design ◽

Aviation Training ◽

Architecture And Design ◽

World Environment

Mixed reality (MR) is rapidly becoming a vital tool, not just in gaming, but also in education, medicine, construction and environmental management. The term refers to systems in which computer-generated content is superimposed over objects in a real-world environment across one or more sensory modalities. Although most of us have heard of the use of MR in computer games, it also has applications in military and aviation training, as well as tourism, healthcare and more. In addition, it has the potential for use in architecture and design, where buildings can be superimposed in existing locations to render 3D generations of plans. However, one major challenge that remains in MR development is the issue of real-time occlusion. This refers to hiding 3D virtual objects behind real articles. Dr Tomohiro Fukuda, who is based at the Division of Sustainable Energy and Environmental Engineering, Graduate School of Engineering at Osaka University in Japan, is an expert in this field. Researchers, led by Dr Tomohiro Fukuda, are tackling the issue of occlusion in MR. They are currently developing a MR system that realises real-time occlusion by harnessing deep learning to achieve an outdoor landscape design simulation using a semantic segmentation technique. This methodology can be used to automatically estimate the visual environment prior to and after construction projects.

Download Full-text