Congestion Control in Charging Stations Allocation with Q-Learning

Navigation systems can help in allocating public charging stations to electric vehicles (EVs) with the aim of minimizing EVs’ charging time by integrating sufficient data. However, the existing systems only consider their travel time and transform the allocation as a routing problem. In this paper, we involve the queuing time in stations as one part of EVs’ charging time, and another part is the travel time on roads. Roads and stations are easily congested resources, and we constructed a joint-resource congestion game to describe the interaction between vehicles and resources. With a finite number of vehicles and resources, there exists a Nash equilibrium. To realize a self-adaptive allocation work, we applied the Q-learning algorithm on systems, defining sets of states and actions in our constructed environment. After being allocated one by one, vehicles concurrently requesting to be charged will be processed properly. We collected urban road network data from Chongqing city and conducted experiments. The results illustrate the proposed method can be used to solve the problem, and its convergence performance was better than the genetic algorithm. The road capacity and the number of EVs affected the initial of Q-value, and not the convergence trends.

Download Full-text

Vehicle Navigation System based on Pollution Metric Analysis with Q-Learning Algorithm

Journal of Artificial Intelligence and Capsule Networks - September 2019 ◽

10.36548/jaicn.2021.4.005 ◽

2021 ◽

Vol 3 (4) ◽

pp. 330-340

Author(s):

B. Vivekanandam ◽

Balaganesh

Keyword(s):

Road Traffic ◽

Learning Algorithm ◽

Prediction Algorithm ◽

Pollution Level ◽

Navigation Systems ◽

Q Learning ◽

The Road ◽

Navigation Path ◽

Path Distance ◽

Quality Learning

The navigation systems available in the present scenario takes into account the path distance for their estimations. In some advanced navigation systems, the road traffic analysis is also considered in the algorithm for their predictions. The proposed work estimates a navigation path with respect to the present pollution level on the roadways. The work suggests an alternate path to avoid additional vehicles to enter the same road which is already impacted by air pollution. A Q-learning (Quality learning) prediction algorithm is trained in the proposed work with a self-made dataset for the estimations. The experimental work presented in the paper explores the accuracy and computational speed of the developed algorithm in comparison to the traditional algorithms.

Download Full-text

Identification of the Best Charging Time of Electric Vehicles in Fast Charging Stations Connected to Smart Grid Based on Q-Learning

2018 Electrical Power Distribution Conference (EPDC) ◽

10.1109/epdc.2018.8536301 ◽

2018 ◽

Cited By ~ 3

Author(s):

Morteza Rahimi Shaarbaf ◽

Mohsen Ghayeni

Keyword(s):

Smart Grid ◽

Electric Vehicles ◽

Q Learning ◽

Charging Time ◽

Fast Charging ◽

Charging Stations ◽

Grid Based

Download Full-text

System architecture to select the charging station by optimizing the travel time considering the destination of electric vehicle drivers in smart cities

Bulletin of Electrical Engineering and Informatics ◽

10.11591/eei.v9i1.1564 ◽

2020 ◽

Vol 9 (1) ◽

pp. 273-283

Author(s):

Ibrahim El-fedany ◽

Driss Kiouach ◽

Rachid Alaoui

Keyword(s):

Electric Vehicle ◽

Electric Vehicles ◽

Smart Cities ◽

Time Of Arrival ◽

Advance Reservation ◽

The Road ◽

Charging Station ◽

Charging Time ◽

On The Road ◽

Charging Stations

The main limitations of electric vehicles are the limited scope of the battery and their relatively long charging times. This may cause discomfort to drivers of electric vehicles due to a long waiting period at the service of the charging station, during their trips. In this paper, we suggest a model system based on argorithms, allowing the management of charging plans of electric vehicles to travel on the road to their destination in order to minimize the duration of the drivers' journey. The proposed system decision to select the charging station, during advance reservation of electric vehicles, take into account the time of arrival of electric vehicles at charging stations, the expected charging time at charging stations, the local status of the charging stations in real time, and the amount of energy sufficient for the electric vehicle to arrive at the selected charging station. Furthermore, the system periodically updates the electric vehicule reservations to adjust their recharge plans, when they reach their selected earlier station compared to other vehicules requesting new reservations, or they may not arrive as they were forecast, due to traffic jams on the road or certain reluctance on the part of the driver.

Download Full-text

Electric Vehicle Routing Problem with Charging Time and Variable Travel Time

Mathematical Problems in Engineering ◽

10.1155/2017/5098183 ◽

2017 ◽

Vol 2017 ◽

pp. 1-13 ◽

Cited By ~ 15

Author(s):

Sai Shao ◽

Wei Guan ◽

Bin Ran ◽

Zhengbing He ◽

Jun Bi

Keyword(s):

Travel Time ◽

Vehicle Routing ◽

Electric Vehicle ◽

Vehicle Routing Problem ◽

Model Performance ◽

Dijkstra Algorithm ◽

Routing Problem ◽

Charging Time ◽

Battery Power ◽

Operational Issues

An electric vehicle routing problem with charging time and variable travel time is developed to address some operational issues such as range limitation and charging demand. The model is solved by using genetic algorithm to obtain the routes, the vehicle departure time at the depot, and the charging plan. Meanwhile, a dynamic Dijkstra algorithm is applied to find the shortest path between any two adjacent nodes along the routes. To prevent the depletion of all battery power and ensure safe operation in transit, electric vehicles with insufficient battery power can be repeatedly recharged at charging stations. The fluctuations in travel time are implemented to reflect a dynamic traffic environment. In conclusion, a large and realistic case study with a road network in the Beijing urban area is conducted to evaluate the model performance and the solution technology and analyze the results.

Download Full-text

Reinforcement Learning Technique in Multiple Motorway Access Control Strategy Design

PROMET - Traffic&Transportation ◽

10.7307/ptt.v22i2.170 ◽

2012 ◽

Vol 22 (2) ◽

pp. 117-123 ◽

Cited By ~ 3

Author(s):

Kostandina Veljanovska ◽

Kristi M. Bombol ◽

Tomaž Maher

Keyword(s):

Artificial Intelligence ◽

Reinforcement Learning ◽

Access Control ◽

Travel Time ◽

Control Strategy ◽

Learning Algorithm ◽

Traffic Flows ◽

Traffic Demand ◽

Q Learning ◽

Total Travel Time

An appropriately designed motorway access control can decrease the total travel time spent in the system up to 30% and consequently increase the merging operations safety. To date, implemented traffic responsive motorway access control systems have been of local or regulatory type and not truly adaptive in the real sense of the meaning. Hence, traffic flow can be influenced positively by numerous intelligent transportation system (ITS) techniques. In this paper a contemporary approach is presented. It considers the design philosophy of an optimal and adaptive closed-loop multiple motorway access control strategy. The methodology proposed uses the artificial intelligence technique - known as reinforcement learning (RL) with multiple agents, and applies the Q-learning algorithm. One segment of the motorway network with three lanes in each direction and three motorway entries was designed. The detectors and traffic signals were placed at the entries (ramps). Traffic flows and traffic occupancy on the main line as well as the traffic demand on the motorway entries were taken as input model variables. The output variables referred to the travel speed on the corridor, the total travel time, and the total stop time. VISSIM micro-simulator and direct programming of the simulator functions were used in order to implement the RL technique. The peak hour was chosen for the time of simulation. The model was tested in two phases. Its effectiveness was compared to ALINEA. It was observed that the proposed strategy was capable of responding both to dynamic sensory inputs from the environment and to dynamically changing environment. The model of the environment and supervision were not required. The control policy changed as response to the inherent system characteristic changes. It was confirmed that the strategy was truly adaptive and real-time responsive to the traffic demand on the corridor. KEY WORDS: motorway access, traffic flows, control, strategy, artificial intelligence, Q-Learning, simulation

Download Full-text