Fusion of sequential LiDAR measurements for semantic segmentation of multi-layer grid maps

2021 ◽  
Vol 0 (0) ◽  
Author(s):  
Frank Bieder ◽  
Sascha Wirges ◽  
Sven Richter ◽  
Christoph Stiller
Keyword(s):  
Autonomous Vehicles ◽  
Autonomous Vehicle ◽  
Ground Truth ◽  
Semantic Segmentation ◽  
Critical Systems ◽  
Lidar Measurements ◽  
Sequential Information ◽  
Semantic Scene ◽  
Time Critical ◽  
Top View

Abstract In this work, we improve the semantic segmentation of multi-layer top-view grid maps in the context of LiDAR-based perception for autonomous vehicles. To achieve this goal, we fuse sequential information from multiple consecutive LiDAR measurements with respect to the driven trajectory of an autonomous vehicle. By doing so, we enrich the multi-layer grid maps which are subsequently used as the input of a neural network. Our approach can be used for LiDAR-only 360 ° surround view semantic scene segmentation while being suitable for real-time critical systems. We evaluate the benefit of fusing sequential information using a dense semantic ground truth and discuss the effect on different classes.

Vision-Based Navigation of Autonomous Vehicles in Roadway Environments with Unexpected Hazards

2019 ◽  
Vol 2673 (12) ◽  
pp. 494-507 ◽  
Author(s):  
Mhafuzul Islam ◽  
Mashrur Chowdhury ◽  
Hongda Li ◽  
Hongxin Hu
Keyword(s):  
Object Detection ◽  
Autonomous Vehicles ◽  
Autonomous Vehicle ◽  
Semantic Segmentation ◽  
Steering Wheel ◽  
Potential Hazard ◽  
Driving System ◽  
Hazardous Object ◽  
Vision Based Navigation ◽  
Navigational System

Vision-based navigation of autonomous vehicles primarily depends on the deep neural network (DNN) based systems in which the controller obtains input from sensors/detectors, such as cameras, and produces a vehicle control output, such as a steering wheel angle to navigate the vehicle safely in a roadway traffic environment. Typically, these DNN-based systems in the autonomous vehicle are trained through supervised learning; however, recent studies show that a trained DNN-based system can be compromised by perturbation or adverse inputs. Similarly, this perturbation can be introduced into the DNN-based systems of autonomous vehicles by unexpected roadway hazards, such as debris or roadblocks. In this study, we first introduce a hazardous roadway environment that can compromise the DNN-based navigational system of an autonomous vehicle, and produce an incorrect steering wheel angle, which could cause crashes resulting in fatality or injury. Then, we develop a DNN-based autonomous vehicle driving system using object detection and semantic segmentation to mitigate the adverse effect of this type of hazard, which helps the autonomous vehicle to navigate safely around such hazards. We find that our developed DNN-based autonomous vehicle driving system, including hazardous object detection and semantic segmentation, improves the navigational ability of an autonomous vehicle to avoid a potential hazard by 21% compared with the traditional DNN-based autonomous vehicle driving system.

scholarly journals Road Segmentation using Semantic Segmentation Networks for ADAS

2019 ◽  
Vol 8 (11) ◽  
pp. 1740-1743
Keyword(s):  
Autonomous Vehicles ◽  
Autonomous Vehicle ◽  
Warning System ◽  
Semantic Segmentation ◽  
Boundary Line ◽  
Segmentation Result ◽  
Safe Driving ◽  
The Road ◽  
Straight Line ◽  
Road Segmentation

In this paper, we propose a method to automatically segment the road area from the input road images to support safe driving of autonomous vehicles. In the proposed method, the semantic segmentation network (SSN) is trained by using the deep learning method and the road area is segmented by utilizing the SSN. The SSN uses the weights initialized from the VGC-16 network to create the SegNet network. In order to fast the learning time and to obtain results, the class is simplified and learned so that it can be divided into two classes as the road area and the non-road area in the trained SegNet CNN network. In order to improve the accuracy of the road segmentation result, the boundary line of the road region with the straight-line component is detected through the Hough transform and the result is shown by dividing the accurate road region by combining with the segmentation result of the SSN. The proposed method can be applied to safe driving support by autonomously driving the autonomous vehicle by automatically classifying the road area during operation and applying it to the road area departure warning system

scholarly journals Birds Eye View Look-Up Table Estimation with Semantic Segmentation

2021 ◽  
Vol 11 (17) ◽  
pp. 8047
Author(s):  
Dongkyu Lee ◽  
Wee Peng Tay ◽  
Seok-Cheol Kee
Keyword(s):  
Pose Estimation ◽  
Low Cost ◽  
Autonomous Vehicle ◽  
Synthetic Data ◽  
Ground Truth ◽  
Image Plane ◽  
Semantic Segmentation ◽  
Single Camera ◽  
Look Up Table ◽  
Camera Pose

In this work, a study was carried out to estimate a look-up table (LUT) that converts a camera image plane to a birds eye view (BEV) plane using a single camera. The traditional camera pose estimation fields require high costs in researching and manufacturing autonomous vehicles for the future and may require pre-configured infra. This paper proposes an autonomous vehicle driving camera calibration system that is low cost and utilizes low infra. A network that outputs an image in the form of an LUT that converts the image into a BEV by estimating the camera pose under urban road driving conditions using a single camera was studied. We propose a network that predicts human-like poses from a single image. We collected synthetic data using a simulator, made BEV and LUT as ground truth, and utilized the proposed network and ground truth to train pose estimation function. In the progress, it predicts the pose by deciphering the semantic segmentation feature and increases its performance by attaching a layer that handles the overall direction of the network. The network outputs camera angle (roll/pitch/yaw) on the 3D coordinate system so that the user can monitor learning. Since the network's output is a LUT, there is no need for additional calculation, and real-time performance is improved.

scholarly journals SEMANTIC SCENE UNDERSTANDING FOR THE AUTONOMOUS PLATFORM

2020 ◽  
Vol XLIII-B2-2020 ◽  
pp. 637-644
Author(s):  
B. Vishnyakov ◽  
Y. Blokhinov ◽  
I. Sgibnev ◽  
V. Sheverdin ◽  
A. Sorokin ◽  
...  
Keyword(s):  
Data Collection ◽  
Autonomous Vehicles ◽  
Three Dimensional ◽  
Scene Understanding ◽  
Semantic Segmentation ◽  
Lidar Data ◽  
Point Cloud Segmentation ◽  
Detection Point ◽  
Sensor Platform ◽  
Semantic Scene

Abstract. In this paper we describe a new multi-sensor platform for data collection and algorithm testing. We propose a couple of methods for solution of semantic scene understanding problem for land autonomous vehicles. We describe our approaches for automatic camera and LiDAR calibration; three-dimensional scene reconstruction and odometry calculation; semantic segmentation that provides obstacle recognition and underlying surface classification; object detection; point cloud segmentation. Also, we describe our virtual simulation complex based on Unreal Engine, that can be used for both data collection and algorithm testing. We collected a large database of field and virtual data: more than 1,000,000 real images with corresponding LiDAR data and more than 3,500,000 simulated images with corresponding LiDAR data. All proposed methods were implemented and tested on our autonomous platform; accuracy estimates were obtained on the collected database.

A SYSTEM FOR DETECTING AND RECOGNIZING MOVING BIOLOGICAL OBJECTS FOR UNMANNED VEHICLES BASED ON INTELLIGENT EDGE COMPUTING

2021 ◽  
Author(s):  
М.А. АЛЬ-СВЕЙТИ ◽  
А.С. МУТХАННА ◽  
А.С. БОРОДИН ◽  
А.Е. КУЧЕРЯВЫЙ
Keyword(s):  
Autonomous Vehicles ◽  
Intelligent Transportation System ◽  
Autonomous Vehicle ◽  
Unmanned Vehicles ◽  
Mesh Network ◽  
New Approach ◽  
Remote Areas ◽  
The Road ◽  
Biological Objects ◽  
Time Critical

Обсуждается возможность применения бортовых платформ с целью поддержки наземных сетей для использования ресурсов автономных транспортных средств как части критичных к задержкам приложений. Бортовые платформы могут повысить безопасность поездок транспортных средств, доставляя на них своевременную информацию об окружающей обстановке даже в отдаленных районах земного шара. Обсуждаются требования и потенциальные решения для поддержки инфраструктуры автономных транспортных средств как части интеллектуальной транспортной системы. Предлагается использовать вдоль дороги энергоэффективные сенсоры, которые могут объединяться друг с другом в Mesh-сети. Кроме того, предлагается новый подход к обнаружению активности биологических объектов на обочине дороги, основанный на технологиях искусственного интеллекта. The article discusses the possibility of using onboard platforms to support the terrestrial networks for autonomous vehicles resources as a part of delay-critical applications. Onboard platforms can improve the safety of vehicle rides by delivering time-critical information about the environment to the vehicles, even in remote areas of the world. In this paper, we discuss requirements and potential solutions for supporting the autonomous vehicle infrastructure, as a part of an intelligent transportation system. It is proposed to use energy-efficient sensors along the road, which can connect with each other in a Mesh network. In addition, a new approach for the detection of biological objects activity on the roadside, based on artificial intelligence technologies is suggested.

scholarly journals Curve Path Detection in Autonomous Vehicle using Deep Learning

2018 ◽  
Author(s):  
Balasriram Kodi ◽  
Manimozhi M
Keyword(s):  
Deep Learning ◽  
Autonomous Vehicles ◽  
Autonomous Vehicle ◽  
Ground Truth ◽  
Autonomous Driving ◽  
Detection Algorithm ◽  
Lane Detection ◽  
Point To Point ◽  
And Control ◽  
Mapping Point

In the field of autonomous vehicles, lane detection and control plays an important role. In autonomous driving the vehicle has to follow the path to avoid the collision. A deep learning technique is used to detect the curved path in autonomous vehicles. In this paper a customized lane detection algorithm was implemented to detect the curvature of the lane. A ground truth labelling tool box for deep learning is used to detect the curved path in autonomous vehicle. By mapping point to point in each frame 80-90% computing efficiency and accuracy is achieved in detecting path.

scholarly journals INFLUENCE OF DOMAIN SHIFT FACTORS ON DEEP SEGMENTATION OF THE DRIVABLE PATH OF AN AUTONOMOUS VEHICLE

2018 ◽  
Vol XLII-2 ◽  
pp. 141-148
Author(s):  
R. P. A. Bormans ◽  
R. C. Lindenbergh ◽  
F. Karimi Nejadasl
Keyword(s):  
Autonomous Vehicles ◽  
Domain Adaptation ◽  
Autonomous Vehicle ◽  
Semantic Segmentation ◽  
Negative Influence ◽  
Target Domain ◽  
Real World Data ◽  
Horizon Line ◽  
Rgb Images ◽  
Weakly Supervised

One of the biggest challenges for an autonomous vehicle (and hence the WEpod) is to see the world as humans would see it. This understanding is the base for a successful and reliable future of autonomous vehicles. Real-world data and semantic segmentation generally are used to achieve full understanding of its surroundings. However, deploying a pretrained segmentation network to a new, previously unseen domain will not attain similar performance as it would on the domain where it is trained on due to the differences between the domains. Although research is done concerning the mitigation of this domain shift, the factors that cause these differences are not yet fully explored. We filled this gap with the investigation of several factors. A base network was created by a two-step finetuning procedure on a convolutional neural network (SegNet) which is pretrained on CityScapes (a dataset for semantic segmentation). The first tuning step is based on RobotCar (road scenery dataset recorded in Oxford, UK) while afterwards this network is fine-tuned for a second time but now on the KITTI (road scenery dataset recorded in Germany) dataset. With this base, experiments are used to obtain the importance of factors such as horizon line, colour and training order for a successful domain adaptation. In this case the domain adaptation is from the KITTI and RobotCar domain to the WEpod domain. For evaluation, groundtruth labels are created in a weakly-supervised setting. Negative influence was obtained for training on greyscale images instead of RGB images. This resulted in drops of IoU values up to 23.9 % for WEpod test images. The training order is a main contributor for domain adaptation with an increase in IoU of 4.7 %. This shows that the target domain (WEpod) is more closely related to RobotCar than to KITTI.

The human-like trajectory planning for autonomous vehicles based on optimal control in a test track environment

2021 ◽  
pp. 095440702110090
Author(s):  
Xing Xu ◽  
Minglei Li ◽  
Feng Wang ◽  
Ju Xie ◽  
Xiaohan Wu ◽  
...  
Keyword(s):  
Optimal Control ◽  
Autonomous Vehicles ◽  
Optimal Trajectory ◽  
Control Method ◽  
Autonomous Vehicle ◽  
Trajectory Data ◽  
Test Track ◽  
Optimal Control Method ◽  
The Future ◽  
On The Road

A human-like trajectory could give a safe and comfortable feeling for the occupants in an autonomous vehicle especially in corners. The research of this paper focuses on planning a human-like trajectory along a section road on a test track using optimal control method that could reflect natural driving behaviour considering the sense of natural and comfortable for the passengers, which could improve the acceptability of driverless vehicles in the future. A mass point vehicle dynamic model is modelled in the curvilinear coordinate system, then an optimal trajectory is generated by using an optimal control method. The optimal control problem is formulated and then solved by using the Matlab tool GPOPS-II. Trials are carried out on a test track, and the tested data are collected and processed, then the trajectory data in different corners are obtained. Different TLCs calculations are derived and applied to different track sections. After that, the human driver’s trajectories and the optimal line are compared to see the correlation using TLC methods. The results show that the optimal trajectory shows a similar trend with human’s trajectories to some extent when driving through a corner although it is not so perfectly aligned with the tested trajectories, which could conform with people’s driving intuition and improve the occupants’ comfort when driving in a corner. This could improve the acceptability of AVs in the automotive market in the future. The driver tends to move to the outside of the lane gradually after passing the apex when driving in corners on the road with hard-lines on both sides.

Sign in / Sign up

Export Citation Format

Share Document