scholarly journals An end-to-end approach to autonomous vehicle control using deep learning

2021 ◽  
Vol 13 (3) ◽  
pp. 32-41
Author(s):  
Gustavo Antonio Magera Novello ◽  
Henrique Yda Yamamoto ◽  
Eduardo Lobo Lustosa Cabral
Keyword(s):  
Neural Network ◽  
Autonomous Vehicle ◽  
Recurrent Network ◽  
Vehicle Control ◽  
Training Data ◽  
Steering Wheel ◽  
Accelerator Pedal ◽  
Convolutional Network ◽  
Human Driver ◽  
End To End

The objective of this work is to develop an autonomous vehicle controller inside Grand Theft Auto V game, used as a simulation environment. It is used an end-to-end approach, in which the model maps directly the inputs from the image of a car hood camera and a sequence of speed values to three driving commands: steering wheel angle, accelerator pedal pressure and brake pedal pressure. The developed model is composed of a convolutional neural network and a recurring neural network. The convolutional network processes the images and the recurrent network processes the speed data. The model learns from data generated by a human driver´s commands. Two interfaces are developed: one for collecting in-game training data and another to verify the performance of the model for the autonomous vehicle control. The results show that the model after training is capable to drive the vehicle as well as a human driver. This proves that a combination of a convolutional network with a recurrent network, using an end-to-end approach, is capable of obtaining a good driving performance even using only images and speed velocity as sensory data.

scholarly journals Evaluation of Power Insulator Detection Efficiency with the Use of Limited Training Dataset

2020 ◽  
Vol 10 (6) ◽  
pp. 2104
Author(s):  
Michał Tomaszewski ◽  
Paweł Michalski ◽  
Jakub Osuchowski
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Object Detection ◽  
Convolutional Neural Network ◽  
Deep Neural Networks ◽  
Detection Efficiency ◽  
Training Data ◽  
Training Dataset ◽  
Training Set ◽  
Convolutional Network

This article presents an analysis of the effectiveness of object detection in digital images with the application of a limited quantity of input. The possibility of using a limited set of learning data was achieved by developing a detailed scenario of the task, which strictly defined the conditions of detector operation in the considered case of a convolutional neural network. The described solution utilizes known architectures of deep neural networks in the process of learning and object detection. The article presents comparisons of results from detecting the most popular deep neural networks while maintaining a limited training set composed of a specific number of selected images from diagnostic video. The analyzed input material was recorded during an inspection flight conducted along high-voltage lines. The object detector was built for a power insulator. The main contribution of the presented papier is the evidence that a limited training set (in our case, just 60 training frames) could be used for object detection, assuming an outdoor scenario with low variability of environmental conditions. The decision of which network will generate the best result for such a limited training set is not a trivial task. Conducted research suggests that the deep neural networks will achieve different levels of effectiveness depending on the amount of training data. The most beneficial results were obtained for two convolutional neural networks: the faster region-convolutional neural network (faster R-CNN) and the region-based fully convolutional network (R-FCN). Faster R-CNN reached the highest AP (average precision) at a level of 0.8 for 60 frames. The R-FCN model gained a worse AP result; however, it can be noted that the relationship between the number of input samples and the obtained results has a significantly lower influence than in the case of other CNN models, which, in the authors’ assessment, is a desired feature in the case of a limited training set.

scholarly journals Improving the Learning of Self-driving Vehicles Based on Real Driving Behavior using Deep Neural Network Techniques

2020 ◽  
Author(s):  
Nayere Zaghari ◽  
Mahmood Fathy ◽  
Seyed Mahdi Jameii ◽  
Mohammad Sabokrou ◽  
Mohammad Shahverdy
Keyword(s):  
Neural Network ◽  
Deep Neural Network ◽  
Autonomous Vehicle ◽  
Driving Behavior ◽  
Obstacle Detection ◽  
Convolutional Network ◽  
The Real ◽  
Cloning Method ◽  
Vehicle Acceleration ◽  
Performance Results

Considering the significant advancements in autonomous vehicle technology, research in this field is of interest to researchers. To drive vehicles autonomously, controlling steer angle, gas hatch, and brakes need to be learned. The behavioral cloning method is used to imitate humans’ driving behavior. We created a dataset of driving in different routes and conditions and using the designed model, the output used for controlling the vehicle is obtained. In this paper, the Learning of Self-driving Vehicles Based on Real Driving Behavior Using Deep Neural Network Techniques (LSV-DNN) is proposed. We designed a convolutional network which uses the real driving data obtained through the vehicle’s camera and computer. The response of the driver is during driving is recorded in different situations and by converting the real driver’s driving video to images and transferring the data to an excel file, obstacle detection is carried out with the best accuracy and speed using the Yolo algorithm version 3. This way, the network learns the response of the driver to obstacles in different locations and the network is trained with the Yolo algorithm version 3 and the output of obstacle detection. Then, it outputs the steer angle and amount of brake, gas, and vehicle acceleration. The LSV-DNN is evaluated here via extensive simulations carried out in Python and TensorFlow environment. We evaluated the network error using the loss function. By comparing other methods which were conducted on the simulator’s data, we obtained good performance results for the designed network on the data from KITTI benchmark, the data collected using a private vehicle, and the data we collected.

Improving the Learning of Self-driving Vehicles Based on Real Driving Behavior using Deep Neural Network Techniques

2020 ◽  
Author(s):  
Nayere Zaghari ◽  
Mahmood Fathy ◽  
Seyed Mahdi Jameii ◽  
Mohammad Sabokrou ◽  
Mohammad Shahverdy
Keyword(s):  
Neural Network ◽  
Deep Neural Network ◽  
Autonomous Vehicle ◽  
Driving Behavior ◽  
Obstacle Detection ◽  
Convolutional Network ◽  
The Real ◽  
Cloning Method ◽  
Vehicle Acceleration ◽  
Performance Results

Considering the significant advancements in autonomous vehicle technology, research in this field is of interest to researchers. To drive vehicles autonomously, controlling steer angle, gas hatch, and brakes need to be learned. The behavioral cloning method is used to imitate humans’ driving behavior. We created a dataset of driving in different routes and conditions and using the designed model, the output used for controlling the vehicle is obtained. In this paper, the Learning of Self-driving Vehicles Based on Real Driving Behavior Using Deep Neural Network Techniques (LSV-DNN) is proposed. We designed a convolutional network which uses the real driving data obtained through the vehicle’s camera and computer. The response of the driver is during driving is recorded in different situations and by converting the real driver’s driving video to images and transferring the data to an excel file, obstacle detection is carried out with the best accuracy and speed using the Yolo algorithm version 3. This way, the network learns the response of the driver to obstacles in different locations and the network is trained with the Yolo algorithm version 3 and the output of obstacle detection. Then, it outputs the steer angle and amount of brake, gas, and vehicle acceleration. The LSV-DNN is evaluated here via extensive simulations carried out in Python and TensorFlow environment. We evaluated the network error using the loss function. By comparing other methods which were conducted on the simulator’s data, we obtained good performance results for the designed network on the data from KITTI benchmark, the data collected using a private vehicle, and the data we collected.

Autonomous Vehicle Control Using a Deep Neural Network and Jetson Nano

2020 ◽  
Author(s):  
Rocco Febbo ◽  
Brendan Flood ◽  
Julian Halloy ◽  
Patrick Lau ◽  
Kwai Wong ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Neural Network ◽  
Autonomous Vehicle ◽  
Vehicle Control

scholarly journals Fault Detection Based on Fully Convolutional Networks (FCN)

2021 ◽  
Vol 9 (3) ◽  
pp. 259
Author(s):  
Jizhong Wu ◽  
Bo Liu ◽  
Hao Zhang ◽  
Shumei He ◽  
Qianqian Yang
Keyword(s):  
Fault Detection ◽  
Seismic Data ◽  
Binary Image ◽  
Training Data ◽  
Real Field ◽  
Convolutional Network ◽  
Efficient Manner ◽  
Data Set ◽  
Seismic Image ◽  
End To End

It is of great significance to detect faults correctly in continental sandstone reservoirs in the east of China to understand the distribution of remaining structural reservoirs for more efficient development operation. However, the majority of the faults is characterized by small displacements and unclear components, which makes it hard to recognize them in seismic data via traditional methods. We consider fault detection as an end-to-end binary image-segmentation problem of labeling a 3D seismic image with ones as faults and zeros elsewhere. Thus, we developed a fully convolutional network (FCN) based method to fault segmentation and used the synthetic seismic data to generate an accurate and sufficient training data set. The architecture of FCN is a modified version of the VGGNet (A convolutional neural network was named by Visual Geometry Group). Transforming fully connected layers into convolution layers enables a classification net to create a heatmap. Adding the deconvolution layers produces an efficient network for end-to-end dense learning. Herein, we took advantage of the fact that a fault binary image is highly biased with mostly zeros but only very limited ones on the faults. A balanced crossentropy loss function was defined to adjust the imbalance for optimizing the parameters of our FCN model. Ultimately, the FCN model was applied on real field data to propose that our FCN model can outperform conventional methods in fault predictions from seismic images in a more accurate and efficient manner.

scholarly journals Improving the Learning of Self-driving Vehicles Based on Real Driving Behavior using Deep Neural Network Techniques

2020 ◽  
Author(s):  
Nayereh Zaghari ◽  
Mahmood Fathy ◽  
Seyed Mahdi Jameii ◽  
Mohammad Sabokrou ◽  
Mohammad Shahverdy
Keyword(s):  
Neural Network ◽  
Deep Neural Network ◽  
Autonomous Vehicle ◽  
Driving Behavior ◽  
Obstacle Detection ◽  
Convolutional Network ◽  
The Real ◽  
Cloning Method ◽  
Vehicle Acceleration ◽  
Performance Results

Considering the significant advancements in autonomous vehicle technology, research in this field is of interest to researchers. To drive vehicles autonomously, controlling steer angle, gas hatch, and brakes need to be learned. The behavioral cloning method is used to imitate humans’ driving behavior. We created a dataset of driving in different routes and conditions and using the designed model, the output used for controlling the vehicle is obtained. In this paper, the Learning of Self-driving Vehicles Based on Real Driving Behavior Using Deep Neural Network Techniques (LSV-DNN) is proposed. We designed a convolutional network which uses the real driving data obtained through the vehicle’s camera and computer. The response of the driver is during driving is recorded in different situations and by converting the real driver’s driving video to images and transferring the data to an excel file, obstacle detection is carried out with the best accuracy and speed using the Yolo algorithm version 3. This way, the network learns the response of the driver to obstacles in different locations and the network is trained with the Yolo algorithm version 3 and the output of obstacle detection. Then, it outputs the steer angle and amount of brake, gas, and vehicle acceleration. The LSV-DNN is evaluated here via extensive simulations carried out in Python and TensorFlow environment. We evaluated the network error using the loss function. By comparing other methods which were conducted on the simulator’s data, we obtained good performance results for the designed network on the data from KITTI benchmark, the data collected using a private vehicle, and the data we collected.

scholarly journals Gated Recurrent Attention for Multi-Style Speech Synthesis

2020 ◽  
Vol 10 (15) ◽  
pp. 5325
Author(s):  
Sung Jun Cheon ◽  
Joun Yeop Lee ◽  
Byoung Jin Choi ◽  
Hyeonseung Lee ◽  
Nam Soo Kim
Keyword(s):  
Neural Network ◽  
Speech Synthesis ◽  
Attention Mechanism ◽  
Training Data ◽  
Attention Model ◽  
Synthesis Techniques ◽  
Listening Tests ◽  
End To End ◽  
And Control

End-to-end neural network-based speech synthesis techniques have been developed to represent and synthesize speech in various prosodic style. Although the end-to-end techniques enable the transfer of a style with a single vector of style representation, it has been reported that the speaker similarity observed from the speech synthesized with unseen speaker-style is low. One of the reasons for this problem is that the attention mechanism in the end-to-end model is overfitted to the training data. To learn and synthesize voices of various styles, an attention mechanism that can preserve longer-term context and control the context is required. In this paper, we propose a novel attention model which employs gates to control the recurrences in the attention. To verify the proposed attention’s style modeling capability, perceptual listening tests were conducted. The experiments show that the proposed attention outperforms the location-sensitive attention in both similarity and naturalness.

scholarly journals Training configuration analysis of a convolutional neural network object tracker for night surveillance application

2020 ◽  
Vol 9 (2) ◽  
pp. 282
Author(s):  
Zulaikha Kadim ◽  
Mohd Asyraf Zulkifley ◽  
Nor Azwan Mohamed Kamari
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Training Data ◽  
Sequence Length ◽  
Convolutional Network ◽  
Low Contrast ◽  
Surveillance Videos ◽  
Lighting Conditions ◽  
Object Appearance ◽  
Surveillance Application

<span lang="EN-US">Automated surveillance during the night is important as it is the period when crimes usually happened. By providing continuous monitoring, coupled with a real-time alert system, appropriate action can be taken immediately if a crime is detected. However, low lighting conditions during the night can degrade the quality of surveillance videos, where the captured images will have low contrast and less discriminative features. Consequently, these factors contribute to the problem of bad appearance representation of the object of interest in the tracking algorithm. Thus, a convolutional neural network-based object tracker for night surveillance is proposed by exploiting the deep feature strength in representing object features spatially and semantically. The proposed convolutional network consists of six layers that consist of three convolutional neural networks (CNN) and three fully connected (FC) layers. The network will be trained by using a binary classifier approach of objects and its background classes, which is updated on a fixed interval so that it fully encapsulates the changes in object appearance as it moves in the scene.  The algorithm has been tested with different sets of training data configurations to find the best optimum ones with regards to VOT2015 evaluation protocols, tested on 14-night surveillance videos. The results show that the configuration of a total of 250 training samples with a sample ratio of 4:1 between positive and negative data delivers the best performance for the sequence length of [1,550]. It can be inferred that more information on the object is required compared to the background, where the background might be homogeneous due to low lighting conditions. In conclusion, this algorithm is suitable to be implemented for night surveillance application.</span>

Autonomous Vehicle Control System as a Mobile Robot by Artificial Neural Network

2017 ◽  
Vol 6 (3) ◽  
pp. 200
Author(s):  
Nazmul Haque ◽  
Md Hasnat Riaz
Keyword(s):  
Neural Network ◽  
Artificial Neural Network ◽  
Control System ◽  
Motion Control ◽  
Neural Control ◽  
Mobile Robotics ◽  
Autonomous Vehicle ◽  
Vehicle Control ◽  
Network Methods ◽  
Artificial Neural

<p><span>In this paper we have presented the artificial neural network controlled car in mobile robotics and intelligent car systems. The motion control architecture of the robot is presented with an importance on the support and directing units. This uses neural network methods and the values fundamental its design is drawn. A robust neural control system using a model of the process is also developed.</span></p>

scholarly journals Explore User Behaviour in Semi-autonomous Driving

2019 ◽  
Vol 1 (1) ◽  
pp. 3871-3880
Author(s):  
Yuan Shi ◽  
Jeyhoon Maskani ◽  
Giandomenico Caruso ◽  
Monica Bordegoni
Keyword(s):  
Autonomous Vehicle ◽  
Autonomous Driving ◽  
Lateral Position ◽  
Steering Wheel ◽  
Physiological Indicators ◽  
The Road ◽  
Road Map ◽  
Human Driver ◽  
And Performance ◽  
Physiological Sensors

AbstractThe control shifting between a human driver and a semi-autonomous vehicle is one of the most critical scenarios in the road-map of autonomous vehicle development. This paper proposes a methodology to study driver's behaviour in semi-autonomous driving with physiological-sensors-integrated driving simulators. A virtual scenario simulating take-over tasks has been implemented. The behavioural profile of the driver has been defined analysing key metrics collected by the simulator namely lateral position, steering wheel angle, throttle time, brake time, speed, and the take-over time. In addition, heart rate and skin conductance changes have been considered as physiological indicators to assess cognitive workload and reactivity. The methodology has been applied in an experimental study which results are crucial for taking insights on users’ behaviour. Results show that individual different driving styles and performance are able to be distinguished by calculating and elaborating the data collected by the system. This research provides potential directions for establishing a method to characterize a driver's behaviour in a semi-autonomous vehicle.

Sign in / Sign up

Export Citation Format

Share Document