Deep Traffic Light Perception with Spatiotemporal Analysis for Autonomous Driving

2020 ◽  
Author(s):  
Lixiao Yang ◽  
Xiaonian Wang ◽  
Jun Wang
Keyword(s):  
Autonomous Driving ◽  
Spatiotemporal Analysis ◽  
Light Perception ◽  
Traffic Light

scholarly journals ANALYSIS OF SPATIO-TEMPORAL TRAFFIC PATTERNS BASED ON PEDESTRIAN TRAJECTORIES

2016 ◽  
Vol XLI-B2 ◽  
pp. 497-503
Author(s):  
S. Busch ◽  
T. Schindler ◽  
T. Klinger ◽  
C. Brenner
Keyword(s):  
Public Transportation ◽  
Autonomous Driving ◽  
Video Data ◽  
Observation Data ◽  
Traffic Patterns ◽  
Traffic Light ◽  
The Road ◽  
Embedded Graph ◽  
Observed Behaviour ◽  
Spatio Temporal

For driver assistance and autonomous driving systems, it is essential to predict the behaviour of other traffic participants. Usually, standard filter approaches are used to this end, however, in many cases, these are not sufficient. For example, pedestrians are able to change their speed or direction instantly. Also, there may be not enough observation data to determine the state of an object reliably, e.g. in case of occlusions. In those cases, it is very useful if a prior model exists, which suggests certain outcomes. For example, it is useful to know that pedestrians are usually crossing the road at a certain location and at certain times. This information can then be stored in a map which then can be used as a prior in scene analysis, or in practical terms to reduce the speed of a vehicle in advance in order to minimize critical situations. In this paper, we present an approach to derive such a spatio-temporal map automatically from the observed behaviour of traffic participants in everyday traffic situations. In our experiments, we use one stationary camera to observe a complex junction, where cars, public transportation and pedestrians interact. We concentrate on the pedestrians trajectories to map traffic patterns. In the first step, we extract trajectory segments from the video data. These segments are then clustered in order to derive a spatial model of the scene, in terms of a spatially embedded graph. In the second step, we analyse the temporal patterns of pedestrian movement on this graph. We are able to derive traffic light sequences as well as the timetables of nearby public transportation. To evaluate our approach, we used a 4 hour video sequence. We show that we are able to derive traffic light sequences as well as time tables of nearby public transportation.

scholarly journals Design and Optimization of a Deep Neural Network Architecture for Traffic Light Detection

2020 ◽  
Vol 77 ◽  
pp. 01002
Author(s):  
Tomohide Fukuchi ◽  
Mark Ogbodo Ikechukwu ◽  
Abderazek Ben Abdallah
Keyword(s):  
Neural Network ◽  
Network Architecture ◽  
Deep Neural Network ◽  
Autonomous Driving ◽  
Research Trend ◽  
Traffic Light ◽  
Light Detection ◽  
Driving System ◽  
Design And Optimization ◽  
Autonomous Driving System

Autonomous Driving has recently become a research trend and efficient autonomous driving system is difficult to achieve due to safety concerns, Applying traffic light recognition to autonomous driving system is one of the factors to prevent accidents that occur as a result of traffic light violation. To realize safe autonomous driving system, we propose in this work a design and optimization of a traffic light detection system based on deep neural network. We designed a lightweight convolution neural network with parameters less than 10000 and implemented in software. We achieved 98.3% inference accuracy with 2.5 fps response time. Also we optimized the input image pixel values with normalization and optimized convolution layer with pipeline on FPGA with 5% resource consumption.

scholarly journals Improving Performance of the PRYSTINE Traffic Sign Classification by Using Perturbation-based Explainability Approach

2021 ◽  
Author(s):  
Ivars Namatēvs ◽  
Kaspars Sudars ◽  
Kaspars Ozols
Keyword(s):  
Autonomous Driving ◽  
Traffic Light ◽  
Traffic Sign ◽  
High Stakes ◽  
Box Models ◽  
Classifier Performance ◽  
The Status ◽  
Post Hoc ◽  
Fully Connected ◽  
Kernel Compression

Model understanding is critical in many domains, particularly those involved in high-stakes decisions, i.e., medicine, criminal justice, and autonomous driving. Explainable AI (XAI) methods are essential for working with black-box models such as Convolutional Neural Networks. This paper evaluates the traffic sign classifier of Deep Neural Network (DNN) from the Programmable Systems for Intelligence in Automobiles (PRYSTINE) project for explainability. The results of explanations were further used for the CNN PRYSTINE classifier vague kernels` compression. After all, the precision of the classifier was evaluated in different pruning scenarios. The proposed classifier performance methodology was realised by creating the original traffic sign and traffic light classification and explanation code. First, the status of the kernels of the network was evaluated for explainability. For this task, the post-hoc, local, meaningful perturbation-based forward explainable method was integrated into the model to evaluate each kernel status of the network. This method enabled distinguishing high and low-impact kernels in the CNN. Second, the vague kernels of the classifier of the last layer before the fully connected layer were excluded by withdrawing them from the network. Third, the network's precision was evaluated in different kernel compression levels. It is shown that by using the XAI approach for network kernel compression, the pruning of 5% of kernels leads only to a 1% loss in traffic sign and traffic light classification precision. The proposed methodology is crucial where execution time and processing capacity prevail.

scholarly journals DETECTION AND LOCALIZATION OF TRAFFIC LIGHTS USING YOLOV3 AND STEREO VISION

2020 ◽  
Vol XLIII-B2-2020 ◽  
pp. 1247-1252
Author(s):  
W. Omar ◽  
I. Lee ◽  
G. Lee ◽  
K. M. Park
Keyword(s):  
Image Processing ◽  
Autonomous Driving ◽  
Distance Measurement ◽  
Stereo Image ◽  
Detection Accuracy ◽  
Disparity Map ◽  
Traffic Light ◽  
Stereo Camera ◽  
Traffic Lights ◽  
Time Operation

Abstract. This paper focus on traffic light distance measurement using stereo camera which is a very important and challenging task in image processing domain, where it is used in several systems such as Driving Safety Support Systems (DSSS), autonomous driving and traffic mobility. In this paper, we propose an integrated traffic light distance measurement system for self-driving based on stereo image processing. Therefore, an algorithm to spatially locate the detected traffic light is required in order to make these detections useful. In this paper, an algorithm to detect, classify the traffic light colours and spatially locate traffic light are integrated. Detection and colours classification are made simultaneously via YOLOv3, using RGB images. 3D traffic light localization is achieved by estimating the distance from the vehicle to the traffic light, by looking at detector 2D bounding boxes and the disparity map generated by stereo camera. Moreover, Gaussian YOLOv3 weights based on KITTI and Berkeley datasets has been replaced with the COCO dataset. Therefore, a detection algorithm that can cope with mislocalizations is required in autonomous driving applications. This paper proposes an integrated method for improving the detection accuracy and traffic lights colours classification while supporting a real-time operation by modelling the bounding box (bbox) of YOLOv3. The obtained results show fair results within 20 meters away from the sensor, while misdetection and classification appeared in further distance.

scholarly journals ANALYSIS OF SPATIO-TEMPORAL TRAFFIC PATTERNS BASED ON PEDESTRIAN TRAJECTORIES

2016 ◽  
Vol XLI-B2 ◽  
pp. 497-503 ◽  
Author(s):  
S. Busch ◽  
T. Schindler ◽  
T. Klinger ◽  
C. Brenner
Keyword(s):  
Public Transportation ◽  
Autonomous Driving ◽  
Video Data ◽  
Observation Data ◽  
Traffic Patterns ◽  
Traffic Light ◽  
The Road ◽  
Embedded Graph ◽  
Observed Behaviour ◽  
Spatio Temporal

For driver assistance and autonomous driving systems, it is essential to predict the behaviour of other traffic participants. Usually, standard filter approaches are used to this end, however, in many cases, these are not sufficient. For example, pedestrians are able to change their speed or direction instantly. Also, there may be not enough observation data to determine the state of an object reliably, e.g. in case of occlusions. In those cases, it is very useful if a prior model exists, which suggests certain outcomes. For example, it is useful to know that pedestrians are usually crossing the road at a certain location and at certain times. This information can then be stored in a map which then can be used as a prior in scene analysis, or in practical terms to reduce the speed of a vehicle in advance in order to minimize critical situations. In this paper, we present an approach to derive such a spatio-temporal map automatically from the observed behaviour of traffic participants in everyday traffic situations. In our experiments, we use one stationary camera to observe a complex junction, where cars, public transportation and pedestrians interact. We concentrate on the pedestrians trajectories to map traffic patterns. In the first step, we extract trajectory segments from the video data. These segments are then clustered in order to derive a spatial model of the scene, in terms of a spatially embedded graph. In the second step, we analyse the temporal patterns of pedestrian movement on this graph. We are able to derive traffic light sequences as well as the timetables of nearby public transportation. To evaluate our approach, we used a 4 hour video sequence. We show that we are able to derive traffic light sequences as well as time tables of nearby public transportation.

Sign in / Sign up

Export Citation Format

Share Document