Visual Semantic Landmark-Based Robust Mapping and Localization for Autonomous Indoor Parking

Autonomous parking in an indoor parking lot without human intervention is one of the most demanded and challenging tasks of autonomous driving systems. The key to this task is precise real-time indoor localization. However, state-of-the-art low-level visual feature-based simultaneous localization and mapping systems (VSLAM) suffer in monotonous or texture-less scenes and under poor illumination or dynamic conditions. Additionally, low-level feature-based mapping results are hard for human beings to use directly. In this paper, we propose a semantic landmark-based robust VSLAM for real-time localization of autonomous vehicles in indoor parking lots. The parking slots are extracted as meaningful landmarks and enriched with confidence levels. We then propose a robust optimization framework to solve the aliasing problem of semantic landmarks by dynamically eliminating suboptimal constraints in the pose graph and correcting erroneous parking slots associations. As a result, a semantic map of the parking lot, which can be used by both autonomous driving systems and human beings, is established automatically and robustly. We evaluated the real-time localization performance using multiple autonomous vehicles, and an repeatability of 0.3 m track tracing was achieved at a 10 kph of autonomous driving.

Download Full-text

Research on NDT-based Positioning for Autonomous Driving

E3S Web of Conferences ◽

10.1051/e3sconf/202125702055 ◽

2021 ◽

Vol 257 ◽

pp. 02055

Author(s):

Sijia Liu ◽

Jie Luo ◽

Jinmin Hu ◽

Haoru Luo ◽

Yu Liang

Keyword(s):

Real Time ◽

Electromagnetic Interference ◽

Autonomous Vehicles ◽

Autonomous Navigation ◽

Point Cloud ◽

Autonomous Driving ◽

Mapping Method ◽

Signal Loss ◽

Positioning Accuracy ◽

Localization And Mapping

Autonomous driving technology is one of the currently popular technologies, while positioning is the basic problem of autonomous navigation of autonomous vehicles. GPS is widely used as a relatively mature solution in the outdoor open road environment. However, GPS signals will be greatly affected in a complex environment with obstruction and electromagnetic interference, even signal loss may occur if serious, which has a great impact on the accuracy, stability and reliability of positioning. For the time being, L4 and most L3 autonomous driving modules still provide registration and positioning based on the high-precision map constructed. Based on this, this paper elaborates on the reconstruction of the experimental scene environment, using the SLAM (simultaneous localization and mapping) method to construct a highprecision point cloud map. On the constructed prior map, the 3D laser point cloud NDT matching method is used for real-time positioning, which is tested and verified on the “JAC Electric Vehicle” platform. The experimental results show that this algorithm has high positioning accuracy and its real-time performance meets the requirements, which can replace GPS signals to complete the positioning of autonomous vehicles when there is no GPS signal or the GPS signal is weak, and provide positioning accuracy meeting the requirements.

Download Full-text

On the Performance of One-Stage and Two-Stage Object Detectors in Autonomous Vehicles Using Camera Data

Remote Sensing ◽

10.3390/rs13010089 ◽

2020 ◽

Vol 13 (1) ◽

pp. 89

Author(s):

Manuel Carranza-García ◽

Jesús Torres-Mateo ◽

Pedro Lara-Benítez ◽

Jorge García-Gutiérrez

Keyword(s):

Deep Learning ◽

Real Time ◽

Autonomous Vehicles ◽

Remote Sensing Data ◽

Autonomous Driving ◽

Two Stage ◽

Detection Systems ◽

One Stage ◽

Time Requirements ◽

Speed Accuracy

Object detection using remote sensing data is a key task of the perception systems of self-driving vehicles. While many generic deep learning architectures have been proposed for this problem, there is little guidance on their suitability when using them in a particular scenario such as autonomous driving. In this work, we aim to assess the performance of existing 2D detection systems on a multi-class problem (vehicles, pedestrians, and cyclists) with images obtained from the on-board camera sensors of a car. We evaluate several one-stage (RetinaNet, FCOS, and YOLOv3) and two-stage (Faster R-CNN) deep learning meta-architectures under different image resolutions and feature extractors (ResNet, ResNeXt, Res2Net, DarkNet, and MobileNet). These models are trained using transfer learning and compared in terms of both precision and efficiency, with special attention to the real-time requirements of this context. For the experimental study, we use the Waymo Open Dataset, which is the largest existing benchmark. Despite the rising popularity of one-stage detectors, our findings show that two-stage detectors still provide the most robust performance. Faster R-CNN models outperform one-stage detectors in accuracy, being also more reliable in the detection of minority classes. Faster R-CNN Res2Net-101 achieves the best speed/accuracy tradeoff but needs lower resolution images to reach real-time speed. Furthermore, the anchor-free FCOS detector is a slightly faster alternative to RetinaNet, with similar precision and lower memory usage.

Download Full-text

A Novel Feature-Based Approach for Indoor Monocular SLAM

Electronics ◽

10.3390/electronics7110305 ◽

2018 ◽

Vol 7 (11) ◽

pp. 305 ◽

Cited By ~ 3

Author(s):

Seyyed Hoseini ◽

Peyman Kabiri

Keyword(s):

Camera Tracking ◽

Translation Error ◽

Natural Feature ◽

Localization And Mapping ◽

Monocular Slam ◽

Challenging Tasks ◽

Camera Pose ◽

Quality Video ◽

Feature Based ◽

Chessboard Pattern

Camera tracking and the construction of a robust and accurate map in unknown environments are still challenging tasks in computer vision and robotic applications. Visual Simultaneous Localization and Mapping (SLAM) along with Augmented Reality (AR) are two important applications, and their performance is entirely dependent on the accuracy of the camera tracking routine. This paper presents a novel feature-based approach for the monocular SLAM problem using a hand-held camera in room-sized workspaces with a maximum scene depth of 4–5 m. In the core of the proposed method, there is a Particle Filter (PF) responsible for the estimation of extrinsic parameters of the camera. In addition, contrary to key-frame based methods, the proposed system tracks the camera frame by frame and constructs a robust and accurate map incrementally. Moreover, the proposed algorithm initially constructs a metric sparse map. To this end, a chessboard pattern with a known cell size has been placed in front of the camera for a few frames. This enables the algorithm to accurately compute the pose of the camera and therefore, the depth of the primary detected natural feature points are easily calculated. Afterwards, camera pose estimation for each new incoming frame is carried out in a framework that is merely working with a set of visible natural landmarks. Moreover, to recover the depth of the newly detected landmarks, a delayed approach based on linear triangulation is used. The proposed method is applied to a realworld VGA quality video (640 × 480 pixels) where the translation error of the camera pose is less than 2 cm on average and the orientation error is less than 3 degrees, which indicates the effectiveness and accuracy of the developed algorithm.

Download Full-text

Automated Testing of Ultrawideband Positioning for Autonomous Driving

Journal of Robotics ◽

10.1155/2020/9345360 ◽

2020 ◽

Vol 2020 ◽

pp. 1-15

Author(s):

Benjamin Vedder ◽

Bo Joel Svensson ◽

Jonny Vinter ◽

Magnus Jonsson

Keyword(s):

Fault Tolerance ◽

Real Time ◽

Autonomous Vehicles ◽

Fault Injection ◽

Autonomous Driving ◽

Automated Testing ◽

Test Cases ◽

Positioning System ◽

Novel Approach ◽

Driving Model

Autonomous vehicles need accurate and dependable positioning, and these systems need to be tested extensively. We have evaluated positioning based on ultrawideband (UWB) ranging with our self-driving model car using a highly automated approach. Random drivable trajectories were generated, while the UWB position was compared against the Real-Time Kinematic Satellite Navigation (RTK-SN) positioning system which our model car also is equipped with. Fault injection was used to study the fault tolerance of the UWB positioning system. Addressed challenges are automatically generating test cases for real-time hardware, restoring the state between tests, and maintaining safety by preventing collisions. We were able to automatically generate and carry out hundreds of experiments on the model car in real time and rerun them consistently with and without fault injection enabled. Thereby, we demonstrate one novel approach to perform automated testing on complex real-time hardware.

Download Full-text

Design and Implementation of Edge-Fog-Cloud System through HD Map Generation from LiDAR Data of Autonomous Vehicles

Electronics ◽

10.3390/electronics9122084 ◽

2020 ◽

Vol 9 (12) ◽

pp. 2084

Author(s):

Junwon Lee ◽

Kieun Lee ◽

Aelee Yoo ◽

Changjoo Moon

Keyword(s):

Real Time ◽

Autonomous Vehicles ◽

Fog Computing ◽

Point Clouds ◽

Lidar Data ◽

High Definition ◽

Localization And Mapping ◽

Cloud Server ◽

Map Generation ◽

Self Driving Cars

Self-driving cars, autonomous vehicles (AVs), and connected cars combine the Internet of Things (IoT) and automobile technologies, thus contributing to the development of society. However, processing the big data generated by AVs is a challenge due to overloading issues. Additionally, near real-time/real-time IoT services play a significant role in vehicle safety. Therefore, the architecture of an IoT system that collects and processes data, and provides services for vehicle driving, is an important consideration. In this study, we propose a fog computing server model that generates a high-definition (HD) map using light detection and ranging (LiDAR) data generated from an AV. The driving vehicle edge node transmits the LiDAR point cloud information to the fog server through a wireless network. The fog server generates an HD map by applying the Normal Distribution Transform-Simultaneous Localization and Mapping(NDT-SLAM) algorithm to the point clouds transmitted from the multiple edge nodes. Subsequently, the coordinate information of the HD map generated in the sensor frame is converted to the coordinate information of the global frame and transmitted to the cloud server. Then, the cloud server creates an HD map by integrating the collected point clouds using coordinate information.

Download Full-text

Hololens Used for Precise Position Tracking of the Third Party Devices - Autonomous Vehicles

Communications - Scientific letters of the University of Zilina ◽

10.26552/com.c.2019.2.18-23 ◽

2019 ◽

Vol 21 (2) ◽

pp. 18-23

Author(s):

Jindrich Cyrus ◽

David Krcmarik ◽

Reza Moezzi ◽

Jan Koci ◽

Michal Petru

Keyword(s):

Autonomous Vehicles ◽

Complex Space ◽

Autonomous Driving ◽

Real Space ◽

Third Party ◽

Location Information ◽

Position Tracking ◽

Precise Position ◽

The Third ◽

Localization And Mapping

A completely new area of HoloLens usage is proposed. The Hololens is an augmented reality device, which provides the high precision location information. Such an information is normally used to accurately position holograms within the real space with respect to the viewer (user of HoloLens). The information is precise enough to use it for reporting the position for the purpose of autonomous driving. Several experiments have been executed in vast areas (20 m x 40 m) in order to find out the potential error coming from vibrations or other effects when moving the HoloLens. The results show that the technology can be used for spaces, which are previously known by the system - pre-scan of the space is needed. The big advantage of the system is its readiness for indoor positioning applications with no additional infrastructure needed, simultaneous localization and mapping, complex space mapping and reached precision. The disadvantage is mainly the costs.

Download Full-text

Feature-Based Real-Time Visual SLAM Using Kinect

Advanced Materials Research ◽

10.4028/www.scientific.net/amr.989-994.2651 ◽

2014 ◽

Vol 989-994 ◽

pp. 2651-2654

Author(s):

Yan Song ◽

Bo He

Keyword(s):

Real Time ◽

3D Models ◽

Image Features ◽

Microsoft Kinect ◽

Depth Information ◽

Visual Slam ◽

Depth Data ◽

Localization And Mapping ◽

Speed Up ◽

Feature Based

In this paper, a novel feature-based real-time visual Simultaneous localization and mapping (SLAM) system is proposed. This system generates colored 3-D reconstruction models and 3-D estimated trajectory using a Kinect style camera. Microsoft Kinect, a low priced 3-D camera, is the only sensor we use in our experiment. Kinect style sensors give RGB-D (red-green-blue depth) data which contains 2D image and per-pixel depth information. ORB (Oriented FAST and Rotated BRIEF) is the algorithm used to extract image features for speed up the whole system. Our system can be used to generate 3-D detailed reconstruction models. Furthermore, an estimated 3D trajectory of the sensor is given in this paper. The results of the experiments demonstrate that our system performs robustly and effectively in both getting detailed 3D models and mapping camera trajectory.

Download Full-text

Processor-in-the-Loop Architecture Design and Experimental Validation for an Autonomous Racing Vehicle

Applied Sciences ◽

10.3390/app11167225 ◽

2021 ◽

Vol 11 (16) ◽

pp. 7225

Author(s):

Eugenio Tramacere ◽

Sara Luciani ◽

Stefano Feraco ◽

Angelo Bonfitto ◽

Nicola Amati

Keyword(s):

Real Time ◽

Personal Computer ◽

Trajectory Planning ◽

Autonomous Vehicles ◽

High Performance ◽

Experimental Validation ◽

Low Cost ◽

Autonomous Driving ◽

Obstacle Detection ◽

System Level

Self-driving vehicles have experienced an increase in research interest in the last decades. Nevertheless, fully autonomous vehicles are still far from being a common means of transport. This paper presents the design and experimental validation of a processor-in-the-loop (PIL) architecture for an autonomous sports car. The considered vehicle is an all-wheel drive full-electric single-seater prototype. The retained PIL architecture includes all the modules required for autonomous driving at system level: environment perception, trajectory planning, and control. Specifically, the perception pipeline exploits obstacle detection algorithms based on Artificial Intelligence (AI), and the trajectory planning is based on a modified Rapidly-exploring Random Tree (RRT) algorithm based on Dubins curves, while the vehicle is controlled via a Model Predictive Control (MPC) strategy. The considered PIL layout is implemented firstly using a low-cost card-sized computer for fast code verification purposes. Furthermore, the proposed PIL architecture is compared in terms of performance to an alternative PIL using high-performance real-time target computing machine. Both PIL architectures exploit User Datagram Protocol (UDP) protocol to properly communicate with a personal computer. The latter PIL architecture is validated in real-time using experimental data. Moreover, they are also validated with respect to the general autonomous pipeline that runs in parallel on the personal computer during numerical simulation.

Download Full-text

Deep Learning Based Data Fusion for Sensor Fault Diagnosis and Tolerance in Autonomous Vehicles

10.21203/rs.3.rs-122424/v1 ◽

2020 ◽

Author(s):

Huihui Pan ◽

Weichao Sun ◽

Qiming Sun ◽

Huijun Gao

Keyword(s):

Fault Diagnosis ◽

Data Fusion ◽

Real Time ◽

Autonomous Vehicles ◽

Autonomous Driving ◽

Environmental Perception ◽

Sensor Data ◽

Time Data ◽

Multiple Sensors ◽

Perception System

Abstract Environmental perception is one of the key technologies to realize autonomous vehicles. Autonomous vehicles are often equipped with multiple sensors to form a multi-source environmental perception system. Those sensors are very sensitive to light or background conditions, which will introduce a variety of global and local fault signals that bring great safety risks to autonomous driving system during long-term running. In this paper, a real-time data fusion network with fault diagnosis and fault tolerance mechanism is designed. By introducing prior features to realize the lightweight of the backbone network, the features of the input data can be extracted in real time accurately. Through the temporal and spatial correlation between sensor data, the sensor redundancy is utilized to diagnose the local and global condence of sensor data in real time, eliminate the fault data, and ensure the accuracy and reliability of data fusion. Experiments show that the network achieves the state-of-the-art results in speed and accuracy, and can accurately detect the location of the target when some sensors are out of focus or out of order.

Download Full-text

Special Issue on Probabilistic Robotics and SLAM

Journal of Robotics and Mechatronics ◽

10.20965/jrm.2019.p0179 ◽

2019 ◽

Vol 31 (2) ◽

pp. 179-179

Author(s):

Keigo Watanabe ◽

Shoichi Maeyama ◽

Tetsuo Tomizawa ◽

Ryuichi Ueda ◽

Masahiro Tomono

Keyword(s):

Mobile Robots ◽

Measurement Errors ◽

Mobile Robotics ◽

Indoor Localization ◽

Autonomous Driving ◽

Special Issue ◽

Practical Applications ◽

Localization And Mapping ◽

Downhill Simplex ◽

Probabilistic Robotics

Intelligent mobile robots need self-localization, map generation, and the ability to explore unknown environments autonomously. Probabilistic processing can be applied to overcome the problems of movement uncertainties and measurement errors. Probabilistic robotics and simultaneous localization and mapping (SLAM) technologies are therefore strongly related, and they have been the focus of many studies. As more and more practical applications are found for intelligent mobile robots, such as for autonomous driving and cleaning, the applicability of these techniques has been increasing. In this special issue, we provide a wide variety of very interesting papers ranging from studies and developments in applied SLAM technologies to fundamental theories for SLAM. There are five academic papers, one each on the following topics: first visit navigation, controls for following rescue clues, indoor localization using magnetic field maps, a new solution for self-localization using downhill simplex method, and object detection for long-term map management through image-based learning. In addition, in the next number, there will be a review paper by Tsukuba University’s Prof. Tsubouchi, who is famous for the Tsukuba Challenge and research related to mobile robotics. We editors hope this special issue will help readers to develop mobile robots and use SLAM technologies and probabilistic approaches to produce successful applications.

Download Full-text