scholarly journals Road Artery Traffic Light Optimization with Use of the Reinforcement Learning

2014 ◽  
Vol 26 (2) ◽  
pp. 101-108 ◽  
Author(s):  
Rok Marsetič ◽  
Darja Šemrov ◽  
Marijan Žura
Keyword(s):  
Real Time ◽  
Learning Algorithm ◽  
Heuristic Method ◽  
Model Parameters ◽  
Traffic Light ◽  
Q Learning ◽  
Traffic Lights ◽  
Algorithm Comparison ◽  
Actuated Signal ◽  
The Impact

The basic principle of optimal traffic control is the appropriate real-time response to dynamic traffic flow changes. Signal plan efficiency depends on a large number of input parameters. An actuated signal system can adjust very well to traffic conditions, but cannot fully adjust to stochastic traffic volume oscillation. Due to the complexity of the problem analytical methods are not applicable for use in real time, therefore the purpose of this paper is to introduce heuristic method suitable for traffic light optimization in real time. With the evolution of artificial intelligence new possibilities for solving complex problems have been introduced. The goal of this paper is to demonstrate that the use of the Q learning algorithm for traffic lights optimization is suitable. The Q learning algorithm was verified on a road artery with three intersections. For estimation of the effectiveness and efficiency of the proposed algorithm comparison with an actuated signal plan was carried out. The results (average delay per vehicle and the number of vehicles that left road network) show that Q learning algorithm outperforms the actuated signal controllers. The proposed algorithm converges to the minimal delay per vehicle regardless of the stochastic nature of traffic. In this research the impact of the model parameters (learning rate, exploration rate, influence of communication between agents and reward type) on algorithm effectiveness were analysed as well.

scholarly journals V2V Routing in VANET Based on Fuzzy Logic and Reinforcement Learning

2021 ◽  
Vol 16 (1) ◽  
Author(s):  
Wanli Zhang ◽  
Xiaoying Yang ◽  
Qixiang Song ◽  
Liang Zhao
Keyword(s):  
Fuzzy Logic ◽  
Real Time ◽  
Vehicular Ad Hoc Networks ◽  
Ad Hoc ◽  
Routing Algorithm ◽  
Cluster Head ◽  
Convergence Speed ◽  
Q Learning ◽  
On The Road ◽  
The Impact

To ensure the transmission quality of real-time communications on the road, the research of routing protocol is crucial to improve effectiveness of data transmission in Vehicular Ad Hoc Networks (VANETs). The existing work Q-Learning based routing algorithm, QLAODV, is studied and its problems, including slow convergence speed and low accuracy, are found. Hence, we propose a new routing algorithm FLHQRP by considering the characteristics of real-time communication in VANETs in the paper. The virtual grid is introduced to divide the vehicle network into clusters. The node’s centrality and mobility, and bandwidth efficiency are processed by the Fuzzy Logic system to select the most suitable cluster head (CH) with the stable communication links in the cluster. A new heuristic function is also proposed in FLHQRP algorithm. It takes cluster as the environment state of heuristic Q-learning, by considering the delay to guide the forwarding process of the CH. This can speed up the learning convergence, and reduce the impact of node density on the convergence speed and accuracy of Q-learning. The problem of QLAODV is solved in the proposed algorithm since the experimental results show that FLHQRP has many advantages on delivery rate, end-to-end delay, and average hops in different network scenarios.

scholarly journals Does Personalized Nutrition Increase Weight Loss Self-Efficacy?

2020 ◽  
Vol 4 (Supplement_2) ◽  
pp. 1310-1310
Author(s):  
Lu Hu ◽  
Chan Wang ◽  
Huilin Li ◽  
Margaret Curran ◽  
Collin J Popp ◽  
...  
Keyword(s):  
Weight Loss ◽  
Real Time ◽  
Self Efficacy ◽  
Learning Algorithm ◽  
Mobile App ◽  
Small Sample ◽  
Dietary Counseling ◽  
Macronutrient Distribution ◽  
The Impact

Abstract Objectives We examined whether a diet personalized to reduce postprandial glycemic response (PPGR) to foods increases weight loss self-efficacy. Methods The Personal Diet Study is an ongoing clinical trial that aims to compare two weight loss diets: a one-size-fits-all, calorie-restricted, low-fat diet (Standardized) versus a diet having the same calorie restriction but utilizing a machine learning algorithm to predict and reduce PPGR (Personalized). Both groups receive the same behavioral counseling to enhance weight loss self-efficacy. Both groups self-monitor dietary intake using a mobile app, with Standardized receiving real-time feedback on calories and macronutrient distribution, and Personalized receiving real time feedback on calories, macronutrient distribution, and predicted PPGR. We examined changes in self-efficacy between baseline and 3 mos, using the 20-item Weight Efficacy Lifestyle questionnaire (WEL). Linear mixed models were used to analyze differences, adjusting for age, gender, and race. Results The analyses included the first 75 participants to complete 3-mos assessments (41 Personalized and 34 Standardized). The majority of the participants were white (69.3%), female (61.3%), with a mean age of 61.7 years (SD = 9.9) and BMI of 33.4 kg/m2 (SD = 4.8). At baseline, WEL scores were similar between the 2 groups [Standardized WEL: 118.8 (SD = 27.6); Personalized WEL: 124.9 (SD = 29.5), P = 0.47]. At 3 mos, the WEL score was significantly improved in both groups [16.0 (SD = 4.1) in the Standardized group (P < 0.001) and 7.4 (SD = 3.7) in the Personalized group (P = 0.048)], but the between group difference was not significant (P = 0.12). Conclusions Personalized feedback on predicted PPGRs does not appear to enhance weight loss self-efficacy at 3 mos. The lack of significance may be related to the short follow-up period in these preliminary analyses, the small sample accrued to date, or the fact that WEL is designed to assess confidence in various situations (e.g., depressed, anxious) that may not be impacted by personalization. These analyses will be replicated with a larger sample using data obtained through the 6-mos follow-up. New self-efficacy measures may be required to assess the impact of personalized dietary counseling. Funding Sources This research was supported by the American Heart Association.

scholarly journals Comment on Can assimilation of crowdsourced data in hydrological modelling improve flood prediction? by Mazzoleni et al. (2017)

2017 ◽  
Author(s):  
Daniele P. Viero
Keyword(s):  
Real Time ◽  
Data Uncertainty ◽  
Model Parameters ◽  
Design Strategies ◽  
Actual System ◽  
Recent Contribution ◽  
Flood Prediction ◽  
Irregular Frequency ◽  
Crowdsourced Data ◽  
The Impact

Abstract. In their recent contribution, Mazzoleni et al. (2017) investigated the integration of crowdsourced data (CSD) in hydrological models to improve the accuracy of real-time flood forecast. They showed that assimilation of CSD improves the overall model performance in all the considered case studies. The impact of irregular frequency of available crowdsourced data, and that of data uncertainty, were also deeply assessed. However, it has to be remarked that, in their work, the Authors used synthetic (i.e., not actually measured) crowdsourced data, because actual crowdsourced data were not available at the moment of the study. This point, briefly mentioned by the authors, deserves further discussion. In most real-world applications, rainfall-runoff models are calibrated using data from traditional sensors. Typically, CSD are collected at different locations, where semi-distributed models are not calibrated. In a context of equifinality and of poor identifiability of model parameters, the model internal states can hardly mimic the actual system states away from calibration points, thus reducing the chances of success in assimilating real (i.e., not synthetic) CSD. Additional criteria are given that are useful for the a-priori evaluation of crowdsourced data for real-time flood forecasting and, hopefully, to plan apt design strategies for both model calibration and collection of crowdsourced data.

scholarly journals Proposed SMART Traffic Control Signal in Brunei Darussalam

2015 ◽  
Vol 15 (2) ◽  
pp. 277
Author(s):  
Bibi Rawiyah Mulung ◽  
Andino Maseleno
Keyword(s):  
Real Time ◽  
Traffic Control ◽  
Traffic Signal ◽  
Control Signal ◽  
Time Signal ◽  
Signal Control ◽  
Traffic Light ◽  
Brunei Darussalam ◽  
Actuated Signal ◽  
Smart Traffic

This paper presents proposed SMART (Systematic Monitoring of Arterial Road Traffic Signals) traffic control signal in Brunei Darussalam. Traffic congestion due to stops and delays at traffic light signals has much been complained about in Brunei Darussalam as well as across the world during the recent years. There are primarily two types of traffic signal controls in Brunei Darussalam. The most common one is the fixed or pre-timed signal operation traffic light and the other one is the actuated signal operation traffic light. Although the actuated signal control is more efficient than the fixed or pre-fixed signal control in the sense that it provides fewer stops and delays to traffic on the major arteries, the best option for Brunei Darussalam would be to introduce smart traffic control signal. This type of traffic signal uses artificial intelligence to take the appropriate action by adjusting the times in real time to minimise the delay in the intersection while also coordinating with intersections in the neighbourhood. SMART Signal simultaneously collects event-based high-resolution traffic data from multiple intersections and generates real-time signal performance measures, including arterial travel time, number of stops, queue length, intersection delay, and level of service. In Brunei Darussalam, where we have numerous intersections where several arterial roads are linked to one another, The SMART signal traffic control method should be implemented.

Research on Control Method of Expressway Off-Ramp Based on Q-Learning Algorithm and Extension Control

2012 ◽  
Vol 433-440 ◽  
pp. 6033-6037
Author(s):  
Xiao Ming Liu ◽  
Xiu Ying Wang
Keyword(s):  
Queue Length ◽  
Control Method ◽  
Learning Algorithm ◽  
Control Strategies ◽  
Traffic Light ◽  
Q Learning ◽  
Reward Function ◽  
Movement Characteristics ◽  
Queue Lengths ◽  
Simulation Results

The movement characteristics of traffic flow nearby have the important influence on the main line. The control method of expressway off-ramp based on Q-learning and extension control is established by analyzing parameters of off-ramp and auxiliary road. First, the basic description of Q-learning algorithm and extension control is given and analyzed necessarily. Then reward function is gained through the extension control theory to judge the state of traffic light. Simulation results show that compared to the queue lengths of off-ramp and auxiliary road, control method based on Q-learning algorithm and extension control greatly reduced queue length of off-ramp, which demonstrates the feasibility of control strategies.

Jamming-Resilient Wideband Cognitive Radios with Multi-Agent Reinforcement Learning

2018 ◽  
Vol 10 (3) ◽  
pp. 1-23 ◽  
Author(s):  
Mohamed A. Aref ◽  
Sudharman K. Jayaweera
Keyword(s):  
Learning Algorithm ◽  
Cognitive Radios ◽  
System Model ◽  
Interference Avoidance ◽  
Q Learning ◽  
Selection Policy ◽  
Cognitive Framework ◽  
Multi Agent ◽  
Simulation Results ◽  
The Impact

This article presents a design of a wideband autonomous cognitive radio (WACR) for anti-jamming and interference-avoidance. The proposed system model allows multiple WACRs to simultaneously operate over the same spectrum range producing a multi-agent environment. The objective of each radio is to predict and evade a dynamic jammer signal as well as avoiding transmissions of other WACRs. The proposed cognitive framework is made of two operations: sensing and transmission. Each operation is helped by its own learning algorithm based on Q-learning, but both will be experiencing the same RF environment. The simulation results indicate that the proposed cognitive anti-jamming technique has low computational complexity and significantly outperforms non-cognitive sub-band selection policy while being sufficiently robust against the impact of sensing errors.

EXPRESS: Choosing the Light Meal: Real-time Aggregation of Calorie Information Reduces Meal Calories

2021 ◽  
pp. 002224372110223
Author(s):  
Eric M. VanEpps ◽  
Andras Molnar ◽  
Julie S. Downs ◽  
George Loewenstein
Keyword(s):  
Real Time ◽  
Red Light ◽  
Decision Makers ◽  
Caloric Content ◽  
Dynamic Feedback ◽  
Traffic Light ◽  
Real Time Traffic ◽  
Time Aggregation ◽  
Intentions To Return ◽  
The Impact

Numeric labeling of calories on restaurant menus has been implemented widely, but scientific studies have generally not found substantial effects on calories ordered. The present research tests the impact of a feedback format that is more targeted at how consumers select and revise their meals: real-time aggregation of calorie content to provide dynamic feedback about meal calories via a traffic light label. Because these labels intuitively signal when a meal shifts from healthy to unhealthy (via the change from green to a yellow or red light), they prompt decision makers to course correct in real time, before they finalize their choice. Results from five pre-registered experiments ( N = 11,900) show that providing real-time traffic light feedback about the total caloric content of a meal reduces calories in orders, even compared to similar aggregated feedback in numeric format. Patterns of ordering reveal this effect to be driven by people revising high-calorie orders more frequently, leading them to choose fewer and lower-calorie items. Consumers also like traffic light aggregation, indicating greater satisfaction with their order and greater intentions to return to restaurants that use them. The authors discuss how dynamic feedback using intuitive signals could yield benefits in contexts beyond food choice.

scholarly journals Traffic Light Controller using Image Processing

2021 ◽  
Vol 12 (2) ◽  
pp. 405-411 ◽  
Author(s):  
Lakshmanan M, Et. al.
Keyword(s):  
Image Processing ◽  
Edge Detection ◽  
Real Time ◽  
Traffic Congestion ◽  
Traffic Signals ◽  
Daily Basis ◽  
Traffic Density ◽  
Traffic Light ◽  
Traffic Lights ◽  
Traffic Systems

Traffic congestion at junctions is a serious issue on a daily basis. The prevailing traffic light controllers are unable to manage the different traffic flows. Most of the current systems operate on a timing mechanism that changes the signal after a particular interval of time. This may cause frustration and result in motorist's time waste. Traffic congestion is a major problem in the currently existing systems. Delays, safety, parking, and environmental problems are the main issues of current traffic systems that emit smoke and contribute to increasing Global Warming. Sensor-based systems reduce the waiting time and maximize the total number of vehicles that can cross an intersection. Our proposed system can control the traffic lights based on image processing without the need for traffic police. This can reduce congestion, delay, road accidents, need for manpower. Under image processing, we use sub techniques like RGB to Gray conversion, Image resizing, Image Enhancement, Edge detection, Image matching, and Timing allocation. A real-time image is captured for every 1 second. After edge detection procedure for both reference and real-time images, these images are compared using SURF Algorithm. Then the amount of traffic is detected and the details are stored in the server. Arduino is used for a traffic signal in the hardware part. It consists of a Wi-Fi module. The micro-controller used in the system Arduino. Four cameras are placed on respective roads and these cameras are used to capture images to analyze traffic density. Then the traffic signals are decided according to the density of traffic. Our technique can be effective to combat traffic on Indian Roads. A lot of time can be saved by deploying this system and also it conserves a lot of resources as well as the economy

scholarly journals Frequency Maps as Expert Instructions to Lessen Data Dependency on Real-time Traffic Light Recognition

2020 ◽  
Author(s):  
Hendrik Macedo ◽  
Thiago Almeida ◽  
Leonardo Matos ◽  
Bruno Prado
Keyword(s):  
Real Time ◽  
Autonomous Vehicles ◽  
Machine Learning Algorithms ◽  
Support Vector ◽  
Background Suppression ◽  
Processing Unit ◽  
Traffic Light ◽  
Traffic Lights ◽  
Real Time Traffic ◽  
Computational Resources

Research on Traffic Light Recognition (TLR) has grown in recent years, primarily driven by the growing interest in autonomous vehicles development. Machine Learning algorithms have been widely used to that purpose. Mainstream approaches, however, require large amount of data to properly work, and as a consequence, a lot of computational resources. In this paper we propose the use of Expert Instruction (IE) as a mechanism to reduce the amount of data required to provide accurate ML models for TLR. Given an image of the exterior scene taken from the inside of the vehicle, we stand the hypothesis that the picture of a traffic light is more likely to appear in the central and upper regions of the image. Frequency Maps of traffic light location were thus constructed to confirm this hypothesis. The frequency maps are the result of a manual effort of human experts in annotating each image with the coordinates of the region where the traffic light appears. Results show that EI increased the accuracy obtained by the classification algorithm in two different image datasets by at least 15%. Evaluation rates achieved by the inclusion of EI were also higher in further experiments, including traffic light detection followed by classification by the trained algorithm. The inclusion of EI in the PCANet achieved a precision of 83% and recall of 73% against 75.3% and 51.1%, respectively, of its counterpart. We finally presents a prototype of a TLR Device with that expert model embedded to assist drivers. The TLR uses a smartphone as a camera and processing unit. To show the feasibility of the apparatus, a dataset was obtained in real time usage and tested in an Adaptive Background Suppression Filter (AdaBSF) and Support Vector Machines (SVMs) algorithm to detect and recognize traffic lights. Results show precision of 100% and recall of 65%.

scholarly journals Deep Reinforcement Learning Model to Mitigate Congestion in Real-Time Traffic Light Networks

2021 ◽  
Vol 6 (10) ◽  
pp. 138
Author(s):  
Fábio de Souza Pereira Borges ◽  
Adelayda Pallavicini Fonseca ◽  
Reinaldo Crispiniano Garcia
Keyword(s):  
Reinforcement Learning ◽  
Real Time ◽  
Traffic Control ◽  
High Volume ◽  
Urban Traffic ◽  
Traffic Data ◽  
Traffic Light ◽  
Urban Network ◽  
Traffic Demand ◽  
Traffic Lights

Urban traffic congestion has a significant detrimental impact on the environment, public health and the economy, with at a high cost to society worldwide. Moreover, it is not possible to continually modify urban road infrastructure in order to mitigate increasing traffic demand. Therefore, it is important to develop traffic control models that can handle high-volume traffic data and synchronize traffic lights in an urban network in real time, without interfering with other initiatives. Within this context, this study proposes a model, based on deep reinforcement learning, for synchronizing the traffic signals of an urban traffic network composed of two intersections. The calibration of this model, including training of its neural network, was performed using real traffic data collected at the approach to each intersection. The results achieved through simulations were very promising, yielding significant improvements in indicators measured in relation to the pre-existing conditions in the network. The model was able to deal with a broad spectrum of traffic flows and, in peak demand periods, reduced delays and queue lengths by more than 28% and 42%, respectively.

Sign in / Sign up

Export Citation Format

Share Document