scholarly journals Solving Channel Allocation by Reinforcement Learning in Cognitive Enabled Vehicular Ad Hoc Networks

2021 ◽  
Author(s):  
Yunfan Su
Keyword(s):  
Dynamic Programming ◽  
Reinforcement Learning ◽  
Optimal Policy ◽  
Ad Hoc ◽  
Transition Probabilities ◽  
Channel Allocation ◽  
Dynamic Programming Method ◽  
Learning Method ◽  
Time Intervals ◽  
Model Free

Vehicular ad hoc network (VANET) is a promising technique that improves traffic safety and transportation efficiency and provides a comfortable driving experience. However, due to the rapid growth of applications that demand channel resources, efficient channel allocation schemes are required to utilize the performance of the vehicular networks. In this thesis, two Reinforcement learning (RL)-based channel allocation methods are proposed for a cognitive enabled VANET environment to maximize a long-term average system reward. First, we present a model-based dynamic programming method, which requires the calculations of the transition probabilities and time intervals between decision epochs. After obtaining the transition probabilities and time intervals, a relative value iteration (RVI) algorithm is used to find the asymptotically optimal policy. Then, we propose a model-free reinforcement learning method, in which we employ an agent to interact with the environment iteratively and learn from the feedback to approximate the optimal policy. Simulation results show that our reinforcement learning method can acquire a similar performance to that of the dynamic programming while both outperform the greedy method.

scholarly journals Solving Channel Allocation by Reinforcement Learning in Cognitive Enabled Vehicular Ad Hoc Networks

2021 ◽  
Author(s):  
Yunfan Su
Keyword(s):  
Dynamic Programming ◽  
Reinforcement Learning ◽  
Optimal Policy ◽  
Ad Hoc ◽  
Transition Probabilities ◽  
Channel Allocation ◽  
Dynamic Programming Method ◽  
Learning Method ◽  
Time Intervals ◽  
Model Free

Vehicular ad hoc network (VANET) is a promising technique that improves traffic safety and transportation efficiency and provides a comfortable driving experience. However, due to the rapid growth of applications that demand channel resources, efficient channel allocation schemes are required to utilize the performance of the vehicular networks. In this thesis, two Reinforcement learning (RL)-based channel allocation methods are proposed for a cognitive enabled VANET environment to maximize a long-term average system reward. First, we present a model-based dynamic programming method, which requires the calculations of the transition probabilities and time intervals between decision epochs. After obtaining the transition probabilities and time intervals, a relative value iteration (RVI) algorithm is used to find the asymptotically optimal policy. Then, we propose a model-free reinforcement learning method, in which we employ an agent to interact with the environment iteratively and learn from the feedback to approximate the optimal policy. Simulation results show that our reinforcement learning method can acquire a similar performance to that of the dynamic programming while both outperform the greedy method.

Energy Optimization of Solar Micro-Grid Using Multi Agent Reinforcement Learning

2015 ◽  
Vol 787 ◽  
pp. 843-847
Author(s):  
Leo Raju ◽  
R.S. Milton ◽  
S. Sakthiyanandan
Keyword(s):  
Reinforcement Learning ◽  
Energy Savings ◽  
Learning Method ◽  
Solar Pv ◽  
Q Learning ◽  
Pv Systems ◽  
Model Free ◽  
Individual Unit ◽  
Multi Agent ◽  
Micro Grid

In this paper, two solar Photovoltaic (PV) systems are considered; one in the department with capacity of 100 kW and the other in the hostel with capacity of 200 kW. Each one has battery and load. The capital cost and energy savings by conventional methods are compared and it is proved that the energy dependency from grid is reduced in solar micro-grid element, operating in distributed environment. In the smart grid frame work, the grid energy consumption is further reduced by optimal scheduling of the battery, using Reinforcement Learning. Individual unit optimization is done by a model free reinforcement learning method, called Q-Learning and it is compared with distributed operations of solar micro-grid using a Multi Agent Reinforcement Learning method, called Joint Q-Learning. The energy planning is designed according to the prediction of solar PV energy production and observed load pattern of department and the hostel. A simulation model was developed using Python programming.

Prioritized Sweeping Reinforcement Learning Based Routing for MANETs

2017 ◽  
Vol 5 (2) ◽  
pp. 383 ◽  
Author(s):  
Rahul M Desai ◽  
B P Patil
Keyword(s):  
Reinforcement Learning ◽  
Shortest Path ◽  
Ad Hoc ◽  
Mobile Ad Hoc Network ◽  
Network Routing ◽  
Delivery Ratio ◽  
Learning Method ◽  
Shortest Path Routing ◽  
Mobile Ad Hoc ◽  
Prioritized Sweeping

<p class="Default">In this paper, prioritized sweeping confidence based dual reinforcement learning based adaptive network routing is investigated. Shortest Path routing is always not suitable for any wireless mobile network as in high traffic conditions, shortest path will always select the shortest path which is in terms of number of hops, between source and destination thus generating more congestion. In prioritized sweeping reinforcement learning method, optimization is carried out over confidence based dual reinforcement routing on mobile ad hoc network and path is selected based on the actual traffic present on the network at real time. Thus they guarantee the least delivery time to reach the packets to the destination. Analysis is done on 50 Nodes Mobile ad hoc networks with random mobility. Various performance parameters such as Interval and number of nodes are used for judging the network. Packet delivery ratio, dropping ratio and delay shows optimum results using the prioritized sweeping reinforcement learning method.</p>

scholarly journals Autonomous PEV Charging Scheduling Using Deep-Q Network and Dyna-Q Reinforcement Learning

2021 ◽  
Author(s):  
Fan Wang
Keyword(s):  
Reinforcement Learning ◽  
Transition Probabilities ◽  
Real Life ◽  
Future Price ◽  
Traffic Condition ◽  
Data Set ◽  
User Behaviour ◽  
Model Free ◽  
Behaviour Data

This paper proposes a demand response method that aims to reduce the long-term charging cost of a plug-in electric vehicle (PEV) while overcoming obstacles such as the stochastic nature of the user’s driving be- haviour, traffic condition, energy usage, and energy price. The problem is formulated as a Markov Decision Process (MDP) with unknown transition probabilities and solved using deep reinforcement learning (RL) techniques. Existing methods using machine learning either requires initial user behaviour data, or converges far too slowly. This method does not require any initial data on the PEV owner’s driving behaviour and shows improvement on learning speed. A combination of both model-based and model-free learning called Dyna-Q algorithm is utilized. Every time a real experience is obtained, the model is updated and the RL agent will learn from both real data set and “imagined” experience from the model. Due to the vast amount of state space, a table-look up method is impractical and a value approximation method using deep neural networks is employed for estimating the long-term expected reward of all state-action pairs. An average of historical price is used to predict future price. Three different user behaviour without any initial PEV owner behaviour data are simulated. A purely model-free DQN method is shown to run out of battery during trips very often, and is impractical for real life charging scenarios. Simulation results demonstrate the effectiveness of the proposed approach and its ability to reach an optimal policy quicker while avoiding state of charge (SOC) depleting during trips when compared to existing PEV charging schemes for all three different users profiles.

Sign in / Sign up

Export Citation Format

Share Document