A Real-Time Semantic Segmentation Algorithm Based on Improved Lightweight Network

The method of Semantic Background Subtraction (SBS), which combines semantic segmentation and background subtraction, has recently emerged for the task of segmenting moving objects in video sequences. While SBS has been shown to improve background subtraction, a major difficulty is that it combines two streams generated at different frame rates. This results in SBS operating at the slowest frame rate of the two streams, usually being the one of the semantic segmentation algorithm. We present a method, referred to as “Asynchronous Semantic Background Subtraction” (ASBS), able to combine a semantic segmentation algorithm with any background subtraction algorithm asynchronously. It achieves performances close to that of SBS while operating at the fastest possible frame rate, being the one of the background subtraction algorithm. Our method consists in analyzing the temporal evolution of pixel features to possibly replicate the decisions previously enforced by semantics when no semantic information is computed. We showcase ASBS with several background subtraction algorithms and also add a feedback mechanism that feeds the background model of the background subtraction algorithm to upgrade its updating strategy and, consequently, enhance the decision. Experiments show that we systematically improve the performance, even when the semantic stream has a much slower frame rate than the frame rate of the background subtraction algorithm. In addition, we establish that, with the help of ASBS, a real-time background subtraction algorithm, such as ViBe, stays real time and competes with some of the best non-real-time unsupervised background subtraction algorithms such as SuBSENSE.

Download Full-text

A Study on Semantic Segmentation for Autonomous Vehicles

10.14210/cotb.v11n1.p059-061 ◽

2020 ◽

Author(s):

Vinícius Almeida dos Santos ◽

Rodrigo Lyra ◽

Thiago Felski Pereira

Keyword(s):

Image Processing ◽

Real Time ◽

Autonomous Vehicles ◽

Processing Time ◽

Semantic Segmentation ◽

Time Constraint ◽

Segmentation Algorithm ◽

Digital Cameras ◽

Important Challenge

Autonomous vehicles are already a reality, and there are still severalchallenges to overcome. One important challenge for the adoptionof these vehicles is perceiving its surroundings. This necessity ofperception can be fulfilled by digital cameras. When working withdigital image processing, the quality will be limited by real-timeconstraints. As several works indicate, this real-time constraint forautonomous vehicles is at most 100ms per frame. Also, by improvingthe processing time, the chances of accidents involving autonomousvehicles may be decreased. This paper analyses the advantages anddrawbacks of semantic segmentation and also presents a study toimplement perception for autonomous vehicles by accelerating asemantic segmentation algorithm, also used by other works on thefield. To accelerate the algorithm, spacial parallelism will be used.

Download Full-text

Development of environment design support mixed reality system capable of environment estimation using deep learning

Impact ◽

10.21820/23987073.2020.2.9 ◽

2020 ◽

Vol 2020 (2) ◽

pp. 9-11

Author(s):

Tomohiro Fukuda

Keyword(s):

Deep Learning ◽

Real Time ◽

Computer Games ◽

Construction Projects ◽

Mixed Reality ◽

Semantic Segmentation ◽

Environment Design ◽

Aviation Training ◽

Architecture And Design ◽

World Environment

Mixed reality (MR) is rapidly becoming a vital tool, not just in gaming, but also in education, medicine, construction and environmental management. The term refers to systems in which computer-generated content is superimposed over objects in a real-world environment across one or more sensory modalities. Although most of us have heard of the use of MR in computer games, it also has applications in military and aviation training, as well as tourism, healthcare and more. In addition, it has the potential for use in architecture and design, where buildings can be superimposed in existing locations to render 3D generations of plans. However, one major challenge that remains in MR development is the issue of real-time occlusion. This refers to hiding 3D virtual objects behind real articles. Dr Tomohiro Fukuda, who is based at the Division of Sustainable Energy and Environmental Engineering, Graduate School of Engineering at Osaka University in Japan, is an expert in this field. Researchers, led by Dr Tomohiro Fukuda, are tackling the issue of occlusion in MR. They are currently developing a MR system that realises real-time occlusion by harnessing deep learning to achieve an outdoor landscape design simulation using a semantic segmentation technique. This methodology can be used to automatically estimate the visual environment prior to and after construction projects.

Download Full-text

A lightweight network with attention decoder for real-time semantic segmentation

The Visual Computer ◽

10.1007/s00371-021-02115-4 ◽

2021 ◽

Author(s):

Kang Wang ◽

Jinfu Yang ◽

Shuai Yuan ◽

Mingai Li

Keyword(s):

Real Time ◽

Semantic Segmentation

Download Full-text

3D Semantic Segmentation Algorithm for Indoor Scenes based on Long-term Memory

2020 15th IEEE Conference on Industrial Electronics and Applications (ICIEA) ◽

10.1109/iciea48937.2020.9248387 ◽

2020 ◽

Author(s):

Ziyang Liu ◽

Weihai Chen ◽

Jianhua Wang ◽

Xingming Wu ◽

Haosong Yue ◽

...

Keyword(s):

Semantic Segmentation ◽

Segmentation Algorithm ◽

Long Term Memory ◽

Term Memory ◽

Indoor Scenes

Download Full-text

Real-time 2D–3D door detection and state classification on a low-power device

SN Applied Sciences ◽

10.1007/s42452-021-04588-3 ◽

2021 ◽

Vol 3 (5) ◽

Author(s):

João Gaspar Ramôa ◽

Vasco Lopes ◽

Luís A. Alexandre ◽

S. Mogo

Keyword(s):

Low Power ◽

Real Time ◽

Object Classification ◽

Semantic Segmentation ◽

Detection Algorithm ◽

Power Device ◽

Indoor Environments ◽

State Classification ◽

Segmentation Algorithms ◽

Indoor Spaces

AbstractIn this paper, we propose three methods for door state classification with the goal to improve robot navigation in indoor spaces. These methods were also developed to be used in other areas and applications since they are not limited to door detection as other related works are. Our methods work offline, in low-powered computers as the Jetson Nano, in real-time with the ability to differentiate between open, closed and semi-open doors. We use the 3D object classification, PointNet, real-time semantic segmentation algorithms such as, FastFCN, FC-HarDNet, SegNet and BiSeNet, the object detection algorithm, DetectNet and 2D object classification networks, AlexNet and GoogleNet. We built a 3D and RGB door dataset with images from several indoor environments using a 3D Realsense camera D435. This dataset is freely available online. All methods are analysed taking into account their accuracy and the speed of the algorithm in a low powered computer. We conclude that it is possible to have a door classification algorithm running in real-time on a low-power device.

Download Full-text