scholarly journals Modeling and Optimization of Multiaction Dynamic Dispatching Problem for Shared Autonomous Electric Vehicles

2021 ◽  
Vol 2021 ◽  
pp. 1-19
Author(s):  
Ning Wang ◽  
Jiahui Guo
Keyword(s):  
Combinatorial Optimization ◽  
Electric Vehicles ◽  
Large Scale ◽  
Learning Algorithm ◽  
Optimization Theory ◽  
Single Step ◽  
Markov Decision ◽  
Dynamic Dispatching ◽  
Autonomous Mobility ◽  
Autonomous Electric Vehicles

The fusion of electricity, automation, and sharing is forming a new Autonomous Mobility-on-Demand (AMoD) system in current urban transportation, in which the Shared Autonomous Electric Vehicles (SAEVs) are a fleet to execute delivery, parking, recharging, and repositioning tasks automatically. To model the decision-making process of AMoD system and optimize multiaction dynamic dispatching of SAEVs over a long horizon, the dispatching problem of SAEVs is modeled according to Markov Decision Process (MDP) at first. Then two optimization models from short-sighted view and farsighted view based on combinatorial optimization theory are built, respectively. The former focuses on the instant and single-step reward, while the latter aims at the accumulative and multistep return. After that, the Kuhn–Munkres algorithm is set as the baseline method to solve the first model to achieve optimal multiaction allocation instructions for SAEVs, and the combination of deep Q-learning algorithm and Kuhn–Munkres algorithm is designed to solve the second model to realize the global optimization. Finally, a toy example, a macrosimulation of 1 month, and a microsimulation of 6 hours based on actual historical operation data are conducted. Results show that (1) the Kuhn–Munkres algorithm ensures the computational effectiveness in the large-scale real-time application of the AMoD system; (2) the second optimization model considering long-term return can decrease average user waiting time and achieve a 2.78% increase in total revenue compared with the first model; (3) and integrating combinatorial optimization theory with reinforcement learning theory is a perfect package for solving the multiaction dynamic dispatching problem of SAEVs.

A Multi-Step Reinforcement Learning Algorithm

2010 ◽  
Vol 44-47 ◽  
pp. 3611-3615 ◽  
Author(s):  
Zhi Cong Zhang ◽  
Kai Shun Hu ◽  
Hui Yu Huang ◽  
Shuai Li ◽  
Shao Yong Zhao
Keyword(s):  
Reinforcement Learning ◽  
Markov Decision Process ◽  
Decision Process ◽  
Large Scale ◽  
Learning Algorithm ◽  
Machine Learning Method ◽  
Learning Method ◽  
K Value ◽  
Markov Decision ◽  
Action Value

Reinforcement learning (RL) is a state or action value based machine learning method which approximately solves large-scale Markov Decision Process (MDP) or Semi-Markov Decision Process (SMDP). A multi-step RL algorithm called Sarsa(,k) is proposed, which is a compromised variation of Sarsa and Sarsa(). It is equivalent to Sarsa if k is 1 and is equivalent to Sarsa() if k is infinite. Sarsa(,k) adjust its performance by setting k value. Two forms of Sarsa(,k), forward view Sarsa(,k) and backward view Sarsa(,k), are constructed and proved equivalent in off-line updating.

scholarly journals Feature Reinforcement Learning: Part II. Structured MDPs

2021 ◽  
Vol 12 (1) ◽  
pp. 71-86
Author(s):  
Marcus Hutter
Keyword(s):  
Large Scale ◽  
Learning Algorithm ◽  
Building Blocks ◽  
Dynamic Bayesian Networks ◽  
The Novel ◽  
Learning Agents ◽  
Cost Criterion ◽  
Markov Decision ◽  
Primary Contribution ◽  
Real World Problems

Abstract The Feature Markov Decision Processes ( MDPs) model developed in Part I (Hutter, 2009b) is well-suited for learning agents in general environments. Nevertheless, unstructured (Φ)MDPs are limited to relatively simple environments. Structured MDPs like Dynamic Bayesian Networks (DBNs) are used for large-scale real-world problems. In this article I extend ΦMDP to ΦDBN. The primary contribution is to derive a cost criterion that allows to automatically extract the most relevant features from the environment, leading to the “best” DBN representation. I discuss all building blocks required for a complete general learning algorithm, and compare the novel ΦDBN model to the prevalent POMDP approach.

scholarly journals The Synergies of Shared Autonomous Electric Vehicles with Renewable Energy in a Virtual Power Plant and Microgrid

Energies ◽  
2018 ◽  
Vol 11 (8) ◽  
pp. 2016 ◽  
Author(s):  
Riccardo Iacobucci ◽  
Benjamin McLellan ◽  
Tetsuo Tezuka
Keyword(s):  
Renewable Energy ◽  
Power Plant ◽  
Electric Vehicles ◽  
Large Scale ◽  
Renewable Energy Sources ◽  
Transport Sector ◽  
Virtual Power ◽  
Virtual Power Plant ◽  
Vehicle Charging ◽  
Autonomous Electric Vehicles

The introduction of shared autonomous electric vehicles (SAEVs), expected within the next decade, can transform the car into a service, accelerate electrification of the transport sector, and allow for large scale control of electric vehicle charging. In this work, we investigate the potential for this system to provide aggregated storage when combined with intermittent renewable energy sources. We develop a simulation methodology for the optimization of vehicle charging in the context of a virtual power plant or microgrid, with and without grid connection or distributed dispatchable generators. The model considers aggregate storage availability from vehicles based on transport patterns taking into account the necessary vehicle redistribution. We investigate the case of a grid-connected VPP with rooftop solar and the case of a isolated microgrid with solar, wind, and dispatchable generation. We conduct a comprehensive sensitivity analysis to study the effect of several parameters on the results for both cases.

scholarly journals Characterization of the Compressive Load on a Lithium-Ion Battery for Electric Vehicle Application

Machines ◽  
2021 ◽  
Vol 9 (4) ◽  
pp. 71
Author(s):  
Seyed Saeed Madani ◽  
Erik Schaltz ◽  
Søren Knudsen Kær
Keyword(s):  
Electrochemical Impedance Spectroscopy ◽  
Impedance Spectroscopy ◽  
Lithium Ion Batteries ◽  
Lithium Ion Battery ◽  
Electric Vehicles ◽  
Large Scale ◽  
Electrochemical Impedance ◽  
Applied Pressure ◽  
Lithium Ion ◽  
Battery Cells

Lithium-ion batteries are being implemented in different large-scale applications, including aerospace and electric vehicles. For these utilizations, it is essential to improve battery cells with a great life cycle because a battery substitute is costly. For their implementation in real applications, lithium-ion battery cells undergo extension during the course of discharging and charging. To avoid disconnection among battery pack ingredients and deformity during cycling, compacting force is exerted to battery packs in electric vehicles. This research used a mechanical design feature that can address these issues. This investigation exhibits a comprehensive description of the experimental setup that can be used for battery testing under pressure to consider lithium-ion batteries’ safety, which could be employed in electrified transportation. Besides, this investigation strives to demonstrate how exterior force affects a lithium-ion battery cell’s performance and behavior corresponding to static exterior force by monitoring the applied pressure at the dissimilar state of charge. Electrochemical impedance spectroscopy was used as the primary technique for this research. It was concluded that the profiles of the achieved spectrums from the experiments seem entirely dissimilar in comparison with the cases without external pressure. By employing electrochemical impedance spectroscopy, it was noticed that the pure ohmic resistance, which is related to ion transport resistance of the separator, could substantially result in the corresponding resistance increase.

scholarly journals Temporal concatenation for Markov decision processes

2021 ◽  
pp. 1-28
Author(s):  
Ruiyang Song ◽  
Kuang Xu
Keyword(s):  
Markov Decision Processes ◽  
Large Scale ◽  
Optimal Solution ◽  
Upper Bounds ◽  
Black Box ◽  
Decision Processes ◽  
Optimal Solutions ◽  
Wide Range ◽  
Markov Decision ◽  
Speed Up

We propose and analyze a temporal concatenation heuristic for solving large-scale finite-horizon Markov decision processes (MDP), which divides the MDP into smaller sub-problems along the time horizon and generates an overall solution by simply concatenating the optimal solutions from these sub-problems. As a “black box” architecture, temporal concatenation works with a wide range of existing MDP algorithms. Our main results characterize the regret of temporal concatenation compared to the optimal solution. We provide upper bounds for general MDP instances, as well as a family of MDP instances in which the upper bounds are shown to be tight. Together, our results demonstrate temporal concatenation's potential of substantial speed-up at the expense of some performance degradation.

A Scalable Redefined Stochastic Blockmodel

2021 ◽  
Vol 15 (3) ◽  
pp. 1-28
Author(s):  
Xueyan Liu ◽  
Bo Yang ◽  
Hechang Chen ◽  
Katarzyna Musial ◽  
Hongxu Chen ◽  
...  
Keyword(s):  
Large Scale ◽  
Network Science ◽  
Learning Algorithm ◽  
State Of The Art ◽  
Real World Data ◽  
Computational Overhead ◽  
Stochastic Blockmodel ◽  
Np Hard Problem ◽  
Large Scale Networks ◽  
The Cost

Stochastic blockmodel (SBM) is a widely used statistical network representation model, with good interpretability, expressiveness, generalization, and flexibility, which has become prevalent and important in the field of network science over the last years. However, learning an optimal SBM for a given network is an NP-hard problem. This results in significant limitations when it comes to applications of SBMs in large-scale networks, because of the significant computational overhead of existing SBM models, as well as their learning methods. Reducing the cost of SBM learning and making it scalable for handling large-scale networks, while maintaining the good theoretical properties of SBM, remains an unresolved problem. In this work, we address this challenging task from a novel perspective of model redefinition. We propose a novel redefined SBM with Poisson distribution and its block-wise learning algorithm that can efficiently analyse large-scale networks. Extensive validation conducted on both artificial and real-world data shows that our proposed method significantly outperforms the state-of-the-art methods in terms of a reasonable trade-off between accuracy and scalability. 1

scholarly journals Investigation of the Impact of Large-Scale Integration of Electric Vehicles for a Swedish Distribution Network

Energies ◽  
2019 ◽  
Vol 12 (24) ◽  
pp. 4717 ◽  
Author(s):  
Sylvester Johansson ◽  
Jonas Persson ◽  
Stavros Lazarou ◽  
Andreas Theocharis
Keyword(s):  
Electric Vehicles ◽  
Power Distribution ◽  
Large Scale ◽  
Electrical Power ◽  
Distribution Network ◽  
Transport Sector ◽  
Smart Charging ◽  
Large Scale Integration ◽  
Scale Integration ◽  
The Impact

Social considerations for a sustainable future lead to market demands for electromobility. Hence, electrical power distribution operators are concerned about the real ongoing problem of the electrification of the transport sector. In this regard, the paper aims to investigate the large-scale integration of electric vehicles in a Swedish distribution network. To this end, the integration pattern is taken into consideration as appears in the literature for other countries and applies to the Swedish culture. Moreover, different charging power levels including smart charging techniques are examined for several percentages of electric vehicles penetration. Industrial simulation tools proven for their accuracy are used for the study. The results indicate that the grid can manage about 50% electric vehicles penetration at its current capacity. This percentage decreases when higher charging power levels apply, while the transformers appear overloaded in many cases. The investigation of alternatives to increase the grid’s capabilities reveal that smart techniques are comparable to the conventional re-dimension of the grid. At present, the increased integration of electric vehicles is manageable by implementing a combination of smart gird and upgrade investments in comparison to technically expensive alternatives based on grid digitalization and algorithms that need to be further confirmed for their reliability for power sharing and energy management.

Sign in / Sign up

Export Citation Format

Share Document