scholarly journals An Audification and Visualization System (AVS) of an Autonomous Vehicle for Blind and Deaf People Based on Deep Learning

Sensors ◽  
2019 ◽  
Vol 19 (22) ◽  
pp. 5035 ◽  
Author(s):  
Son ◽  
Jeong ◽  
Lee
Keyword(s):  
Deep Learning ◽  
Autonomous Vehicles ◽  
Autonomous Vehicle ◽  
Sensor Data ◽  
Computational Time ◽  
Deaf People ◽  
Instrument Cluster ◽  
Text Data ◽  
Visualization System ◽  
Cloud Server

When blind and deaf people are passengers in fully autonomous vehicles, an intuitive and accurate visualization screen should be provided for the deaf, and an audification system with speech-to-text (STT) and text-to-speech (TTS) functions should be provided for the blind. However, these systems cannot know the fault self-diagnosis information and the instrument cluster information that indicates the current state of the vehicle when driving. This paper proposes an audification and visualization system (AVS) of an autonomous vehicle for blind and deaf people based on deep learning to solve this problem. The AVS consists of three modules. The data collection and management module (DCMM) stores and manages the data collected from the vehicle. The audification conversion module (ACM) has a speech-to-text submodule (STS) that recognizes a user’s speech and converts it to text data, and a text-to-wave submodule (TWS) that converts text data to voice. The data visualization module (DVM) visualizes the collected sensor data, fault self-diagnosis data, etc., and places the visualized data according to the size of the vehicle’s display. The experiment shows that the time taken to adjust visualization graphic components in on-board diagnostics (OBD) was approximately 2.5 times faster than the time taken in a cloud server. In addition, the overall computational time of the AVS system was approximately 2 ms faster than the existing instrument cluster. Therefore, because the AVS proposed in this paper can enable blind and deaf people to select only what they want to hear and see, it reduces the overload of transmission and greatly increases the safety of the vehicle. If the AVS is introduced in a real vehicle, it can prevent accidents for disabled and other passengers in advance.

scholarly journals Imitating a Safe Human Driver Behaviour in Roundabouts Through Deep Learning

2020 ◽  
Vol 19 (1) ◽  
pp. 85-88
Author(s):  
A. S. J. Cervera ◽  
F. J. Alonso ◽  
F. S. García ◽  
A. D. Alvarez
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Autonomous Vehicles ◽  
Autonomous Vehicle ◽  
Driver Behaviour ◽  
Safety Issues ◽  
Current State ◽  
Surrounding Environment ◽  
Human Driver ◽  
State Of The Environment

Roundabouts provide safe and fast circulation as well as many environmental advantages, but drivers adopting unsafe behaviours while circulating through them may cause safety issues, provoking accidents. In this paper we propose a way of training an autonomous vehicle in order to behave in a human and safe way when entering a roundabout. By placing a number of cameras in our vehicle and processing their video feeds through a series of algorithms, including Machine Learning, we can build a representation of the state of the surrounding environment. Then, we use another set of Deep Learning algorithms to analyze the data and determine the safest way of circulating through a roundabout given the current state of the environment, including nearby vehicles with their estimated positions, speeds and accelerations. By watching multiple attempts of a human entering a roundabout with both safe and unsafe behaviours, our second set of algorithms can learn to mimic the human’s good attempts and act in the same way as him, which is key to a safe implementation of autonomous vehicles. This work details the series of steps that we took, from building the representation of our environment to acting according to it in order to attain safe entry into single lane roundabouts.

scholarly journals Computer vision based obstacle detection and target tracking for autonomous vehicles

2021 ◽  
Vol 336 ◽  
pp. 07004
Author(s):  
Ruoyu Fang ◽  
Cheng Cai
Keyword(s):  
Neural Network ◽  
Computer Vision ◽  
Deep Learning ◽  
Target Tracking ◽  
Real Time ◽  
Autonomous Vehicles ◽  
Autonomous Vehicle ◽  
Obstacle Detection ◽  
Pid Algorithm ◽  
Deep Learning Neural Network

Obstacle detection and target tracking are two major issues for intelligent autonomous vehicles. This paper proposes a new scheme to achieve target tracking and real-time obstacle detection of obstacles based on computer vision. ResNet-18 deep learning neural network is utilized for obstacle detection and Yolo-v3 deep learning neural network is employed for real-time target tracking. These two trained models can be deployed on an autonomous vehicle equipped with an NVIDIA Jetson Nano motherboard. The autonomous vehicle moves to avoid obstacles and follow tracked targets by camera. Adjusting the steering and movement of the autonomous vehicle according to the PID algorithm during the movement, therefore, will help the proposed vehicle achieve stable and precise tracking.

Comprehensive Overview of Neural Networks and Its Applications in Autonomous Vehicles

2019 ◽  
pp. 159-173
Author(s):  
Jay Rodge ◽  
Swati Jaiswal
Keyword(s):  
Neural Network ◽  
Artificial Intelligence ◽  
Neural Networks ◽  
Deep Learning ◽  
Autonomous Vehicles ◽  
Autonomous Vehicle ◽  
Learning Algorithms ◽  
Activation Functions ◽  
Comprehensive Overview ◽  
Unit Component

Deep learning and Artificial intelligence (AI) have been trending these days due to the capability and state-of-the-art results that they provide. They have replaced some highly skilled professionals with neural network-powered AI, also known as deep learning algorithms. Deep learning majorly works on neural networks. This chapter discusses about the working of a neuron, which is a unit component of neural network. There are numerous techniques that can be incorporated while designing a neural network, such as activation functions, training, etc. to improve its features, which will be explained in detail. It has some challenges such as overfitting, which are difficult to neglect but can be overcome using proper techniques and steps that have been discussed. The chapter will help the academician, researchers, and practitioners to further investigate the associated area of deep learning and its applications in the autonomous vehicle industry.

scholarly journals A Connected Autonomous Vehicle Testbed: Capabilities, Experimental Processes and Lessons Learned

Automation ◽  
2020 ◽  
Vol 1 (1) ◽  
pp. 17-32
Author(s):  
Thomas Kent ◽  
Anthony Pipe ◽  
Arthur Richards ◽  
Jim Hutchinson ◽  
Wolfgang Schuster
Keyword(s):  
Autonomous Vehicles ◽  
Autonomous Vehicle ◽  
Lessons Learned ◽  
Urban Setting ◽  
Sensor Data ◽  
Stable Set ◽  
Research And Innovation ◽  
Legal Frameworks ◽  
High Level ◽  
Key Steps

VENTURER was one of the first three UK government funded research and innovation projects on Connected Autonomous Vehicles (CAVs) and was conducted predominantly in the South West region of the country. A series of increasingly complex scenarios conducted in an urban setting were used to: (i) evaluate the technology created as a part of the project; (ii) systematically assess participant responses to CAVs and; (iii) inform the development of potential insurance models and legal frameworks. Developing this understanding contributed key steps towards facilitating the deployment of CAVs on UK roads. This paper aims to describe the VENTURER Project trials, their objectives and detail some of the key technologies used. Importantly we aim to introduce some informative challenges that were overcame and the subsequent project and technological lessons learned in a hope to help others plan and execute future CAV research. The project successfully integrated several technologies crucial to CAV development. These included, a Decision Making System using behaviour trees to make high level decisions; A pilot-control system to smoothly and comfortably turn plans into throttle and steering actuation; Sensing and perception systems to make sense of raw sensor data; Inter-CAV Wireless communication capable of demonstrating vehicle-to-vehicle communication of potential hazards. The closely coupled technology integration, testing and participant-focused trial schedule led to a greatly improved understanding of the engineering and societal barriers that CAV development faces. From a behavioural standpoint the importance of reliability and repeatability far outweighs a need for novel trajectories, while the sensor-to-perception capabilities are critical, the process of verification and validation is extremely time consuming. Additionally, the added capabilities that can be leveraged from inter-CAV communications shows the potential for improved road safety that could result. Importantly, to effectively conduct human factors experiments in the CAV sector under consistent and repeatable conditions, one needs to define a scripted and stable set of scenarios that uses reliable equipment and a controllable environmental setting. This requirement can often be at odds with making significant technology developments, and if both are part of a project’s goals then they may need to be separated from each other.

Grid-based Environment Estimation for Local Autonomous Vehicle Navigation

2016 ◽  
Author(s):  
Georg Tanzmeister
Keyword(s):  
Autonomous Vehicles ◽  
Autonomous Vehicle ◽  
Local Environment ◽  
Sensor Data ◽  
Test Vehicle ◽  
Safety Critical ◽  
Environment Model ◽  
Reliable Model ◽  
Autonomous Vehicle Navigation ◽  
Grid Based

This dissertation is focused on the environment model for automated vehicles. A reliable model of the local environment available in real-time is a prerequisite to enable almost any useful ­activity performed by a robot, such as planning motions to fulfill tasks. It is particularly important in safety critical applications, such as for autonomous vehicles in regular traffic. In this thesis, novel concepts for local mapping, tracking, the detection of principal moving directions, cost evaluations in motion planning, and road course estimation have been developed. An object- and sensor-independent grid representation forms the basis of all presented methods enabling a generic and robust estimation of the environment. All approaches have been evaluated with sensor data from real road scenarios, and their performance has been experimentally demonstrated with a test vehicle. ...

scholarly journals Trustworthiness of Dynamic Moving Sensors for Secure Mobile Edge Computing

Computers ◽  
2018 ◽  
Vol 7 (4) ◽  
pp. 63 ◽  
Author(s):  
John Yoon
Keyword(s):  
Deep Learning ◽  
Data Analytics ◽  
Radical Change ◽  
Sensor Data ◽  
Learning Approach ◽  
Meta Data ◽  
Research Areas ◽  
Cloud Server ◽  
Active Research ◽  
Target Environment

Wireless sensor network is an emerging technology, and the collaboration of wireless sensors becomes one of the active research areas for utilizing sensor data. Various sensors collaborate to recognize the changes of a target environment, to identify, if any radical change occurs. For the accuracy improvement, the calibration of sensors has been discussed, and sensor data analytics are becoming popular in research and development. However, they are not satisfactorily efficient for the situations where sensor devices are dynamically moving, abruptly appearing, or disappearing. If the abrupt appearance of sensors is a zero-day attack, and the disappearance of sensors is an ill-functioning comrade, then sensor data analytics of untrusted sensors will result in an indecisive artifact. The predefined sensor requirements or meta-data-based sensor verification is not adaptive to identify dynamically moving sensors. This paper describes a deep-learning approach to verify the trustworthiness of sensors by considering the sensor data only. The proposed verification on sensors can be done without having to use meta-data about sensors or to request consultation from a cloud server. The contribution of this paper includes (1) quality preservation of sensor data for mining analytics. The sensor data are trained to identify their characteristics of outliers: whether they are attack outliers, or outlier-like abrupt changes in environments; and (2) authenticity verification of dynamically moving sensors, which was possible. Previous unknown sensors are also identified by deep-learning approach.

An End-to-End Fully Automatic Bay Parking Approach for Autonomous Vehicles

2018 ◽  
Author(s):  
Rui Li ◽  
Weitian Wang ◽  
Yi Chen ◽  
Srivatsan Srinivasan ◽  
Venkat N. Krovi
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Autonomous Vehicles ◽  
Autonomous Vehicle ◽  
Human Vision ◽  
Automated Driving ◽  
Learning Framework ◽  
Research Platform ◽  
Automatic Parking ◽  
Fully Automatic

Fully automatic parking (FAP) is a key step towards the age of autonomous vehicle. Motivated by the contribution of human vision to human parking, in this paper, we propose a computer vision based FAP method for the autonomous vehicles. Based on the input images from a rear camera on the vehicle, a convolutional neural network (CNN) is trained to automatically output the steering and velocity commands for the vehicle controlling. The CNN is trained by Caffe deep learning framework. A 1/10th autonomous vehicle research platform (1/10-SAVRP), which configured with a vehicle controller unit, an automated driving processor, and a rear camera, is used for demonstrating the parking maneuver. The experimental results suggested that the proposed approach enabled the vehicle to gain the ability of parking independently without human input in different driving settings.

A Sensor Simulation Framework for Training and Testing Robots and Autonomous Vehicles

2021 ◽  
Vol 1 (2) ◽  
Author(s):  
Asher Elmquist ◽  
Radu Serban ◽  
Dan Negrut
Keyword(s):  
Computer Simulation ◽  
Autonomous Vehicles ◽  
Autonomous Vehicle ◽  
Sensor Data ◽  
Simulation Framework ◽  
Unstructured Environments ◽  
Physics Simulation ◽  
Virtual Sensing ◽  
Simulation Task ◽  
Reduced Scale

Abstract Computer simulation can be a useful tool when designing robots expected to operate independently in unstructured environments. In this context, one needs to simulate the dynamics of the robot’s mechanical system, the environment in which the robot operates, and the sensors which facilitate the robot’s perception of the environment. Herein, we focus on the sensing simulation task by presenting a virtual sensing framework built alongside an open-source, multi-physics simulation platform called Chrono. This framework supports camera, lidar, GPS, and IMU simulation. We discuss their modeling as well as the noise and distortion implemented to increase the realism of the synthetic sensor data. We close with two examples that show the sensing simulation framework at work: one pertains to a reduced scale autonomous vehicle and the second is related to a vehicle driven in a digital replica of a Madison neighborhood.

MPC-Based Control of Autonomous Vehicles With Localized Path Planning for Obstacle Avoidance Under Uncertainties

2019 ◽  
Author(s):  
Sai Rajeev Devaragudi ◽  
Bo Chen
Keyword(s):  
Path Planning ◽  
Obstacle Avoidance ◽  
Autonomous Vehicles ◽  
Autonomous Vehicle ◽  
Autonomous Driving ◽  
Obstacle Detection ◽  
Sensor Data ◽  
Constant Time ◽  
Longitudinal Motion ◽  
Driver Assistance Systems

Abstract This paper presents a Model Predictive Control (MPC) approach for longitudinal and lateral control of autonomous vehicles with a real-time local path planning algorithm. A heuristic graph search method (A* algorithm) combined with piecewise Bezier curve generation is implemented for obstacle avoidance in autonomous driving applications. Constant time headway control is implemented for a longitudinal motion to track lead vehicles and maintain a constant time gap. MPC is used to control the steering angle and the tractive force of the autonomous vehicle. Furthermore, a new method of developing Advanced Driver Assistance Systems (ADAS) algorithms and vehicle controllers using Model-In-the-Loop (MIL) testing is explored with the use of PreScan®. With PreScan®, various traffic scenarios are modeled and the sensor data are simulated by using physics-based sensor models, which are fed to the controller for data processing and motion planning. Obstacle detection and collision avoidance are demonstrated using the presented MPC controller.

Generalized Stable User Matching for Autonomous Vehicle Co-Ownership Programs

2020 ◽  
Vol 12 (2-3) ◽  
pp. 61-79 ◽  
Author(s):  
Anpeng Zhang ◽  
Jee Eun Kang ◽  
Changhyun Kwon
Keyword(s):  
Autonomous Vehicles ◽  
Ad Hoc ◽  
Autonomous Vehicle ◽  
Stable Matching ◽  
Computational Time ◽  
Combinatorial Properties ◽  
Fractional Ownership ◽  
Shared Mobility ◽  
New Form

We investigate a new form of car-sharing system that can be introduced in the market for autonomous vehicles called fractional ownership or co-ownership. Although dynamic ride sharing provides ad hoc shared mobility services without any long-term commitment, we consider co-ownership programs with which users can still “own” a car with committed usage and ownership. We assume that an autonomous vehicle is shared by a group of users, which is only accessible by the group. We use stable matching to help users find an appropriate group with which to share an autonomous vehicle and present a generalized stable matching model that allows flexible sizes of groups as well as various alternative objectives. We also present a heuristic algorithm to improve computational time owing to the combinatorial properties of the problem.

Sign in / Sign up

Export Citation Format

Share Document