A Monotone Approximate Dynamic Programming Approach for the Stochastic Scheduling, Allocation, and Inventory Replenishment Problem: Applications to Drone and Electric Vehicle Battery Swap Stations

There is a growing interest in using electric vehicles (EVs) and drones for many applications. However, battery-oriented issues, including range anxiety and battery degradation, impede adoption. Battery swap stations are one alternative to reduce these concerns that allow the swap of depleted for full batteries in minutes. We consider the problem of deriving actions at a battery swap station when explicitly considering the uncertain arrival of swap demand, battery degradation, and replacement. We model the operations at a battery swap station using a finite horizon Markov decision process model for the stochastic scheduling, allocation, and inventory replenishment problem (SAIRP), which determines when and how many batteries are charged, discharged, and replaced over time. We present theoretical proofs for the monotonicity of the value function and monotone structure of an optimal policy for special SAIRP cases. Because of the curses of dimensionality, we develop a new monotone approximate dynamic programming (ADP) method, which intelligently initializes a value function approximation using regression. In computational tests, we demonstrate the superior performance of the new regression-based monotone ADP method compared with exact methods and other monotone ADP methods. Furthermore, with the tests, we deduce policy insights for drone swap stations.

Real-time Eco-Driving Control in Electrified Connected and Autonomous Vehicles Using Approximate Dynamic Programming

Connected and Automated Vehicles (CAVs), particularly those with a hybrid electric powertrain, have the potential to significantly improve vehicle energy savings in real-world driving conditions. In particular, the Eco-Driving problem seeks to design optimal speed and power usage profiles based on available information from connectivity and advanced mapping features to minimize the fuel consumption over an itinerary. This paper presents a hierarchical multi-layer Model Predictive Control (MPC) approach for improving the fuel economy of a 48V mild-hybrid powertrain in a connected vehicle environment. Approximate Dynamic Programming (ADP) is used to solve the Receding Horizon Optimal Control Problem (RHOCP), where the terminal cost for the RHOCP is approximated as the base-policy obtained from the long-term optimization. The controller was tested virtually (with deterministic and Monte Carlo simulation) across multiple real-world routes, demonstrating energy savings of more than 20%. The controller was then deployed on a test vehicle equipped with a rapid prototyping embedded controller. In-vehicle testing confirm the energy savings obtained in simulation and demonstrate the real-time ability of the controller.