A deep reinforcement learning-based approach for the home delivery and installation routing problem

2021 ◽  
pp. 108362
Author(s):  
Huaxin Qiu ◽  
Sutong Wang ◽  
Yunqiang Yin ◽  
Dujuan Wang ◽  
Yanzhang Wang
Keyword(s):  
Reinforcement Learning ◽  
Home Delivery ◽  
Routing Problem

scholarly journals A Deep Reinforcement Learning Approach for Global Routing

2019 ◽  
Vol 142 (6) ◽  
Author(s):  
Haiguang Liao ◽  
Wentai Zhang ◽  
Xuliang Dong ◽  
Barnabas Poczos ◽  
Kenji Shimada ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Integrated Circuits ◽  
Printed Circuit Boards ◽  
Greedy Algorithms ◽  
Global Routing ◽  
Hydraulic Systems ◽  
Routing Problem ◽  
Future Data ◽  
Circuit Components ◽  
Optimization Mechanism

Abstract Global routing has been a historically challenging problem in the electronic circuit design, where the challenge is to connect a large and arbitrary number of circuit components with wires without violating the design rules for the printed circuit boards or integrated circuits. Similar routing problems also exist in the design of complex hydraulic systems, pipe systems, and logistic networks. Existing solutions typically consist of greedy algorithms and hard-coded heuristics. As such, existing approaches suffer from a lack of model flexibility and usually fail to solve sub-problems conjointly. As an alternative approach, this work presents a deep reinforcement learning method for solving the global routing problem in a simulated environment. At the heart of the proposed method is deep reinforcement learning that enables an agent to produce a policy for routing based on the variety of problems, and it is presented with leveraging the conjoint optimization mechanism of deep reinforcement learning. Conjoint optimization mechanism is explained and demonstrated in detail; the best network structure and the parameters of the learned model are explored. Based on the fine-tuned model, routing solutions and rewards are presented and analyzed. The results indicate that the approach can outperform the benchmark method of a sequential A* method, suggesting a promising potential for deep reinforcement learning for global routing and other routing or path planning problems in general. Another major contribution of this work is the development of a global routing problem sets generator with the ability to generate parameterized global routing problem sets with different size and constraints, enabling evaluation of different routing algorithms and the generation of training datasets for future data-driven routing approaches.

scholarly journals A variable neighborhood search algorithm with reinforcement learning for a real-life periodic vehicle routing problem with time windows and open routes

2020 ◽  
Vol 54 (5) ◽  
pp. 1467-1494
Author(s):  
Binhui Chen ◽  
Rong Qu ◽  
Ruibin Bai ◽  
Wasakorn Laesanklang
Keyword(s):  
Reinforcement Learning ◽  
Large Scale ◽  
Search Algorithm ◽  
Real Life ◽  
Planning Horizon ◽  
Neighborhood Search ◽  
Variable Neighbourhood Search ◽  
Container Transportation ◽  
Solution Quality ◽  
Routing Problem

This paper studies a real-life container transportation problem with a wide planning horizon divided into multiple shifts. The trucks in this problem do not return to depot after every single shift but at the end of every two shifts. The mathematical model of the problem is first established, but it is unrealistic to solve this large scale problem with exact search methods. Thus, a Variable Neighbourhood Search algorithm with Reinforcement Learning (VNS-RLS) is thus developed. An urgency level-based insertion heuristic is proposed to construct the initial solution. Reinforcement learning is then used to guide the search in the local search improvement phase. Our study shows that the Sampling scheme in single solution-based algorithms does not significantly improve the solution quality but can greatly reduce the rate of infeasible solutions explored during the search. Compared to the exact search and the state-of-the-art algorithms, the proposed VNS-RLS produces promising results.

scholarly journals Routing of Electric Vehicles With Intermediary Charging Stations: A Reinforcement Learning Approach

2021 ◽  
Vol 4 ◽  
Author(s):  
Marina Dorokhova ◽  
Christophe Ballif ◽  
Nicolas Wyrsch
Keyword(s):  
Reinforcement Learning ◽  
Electric Vehicles ◽  
Mathematical Formulation ◽  
Route Planning ◽  
Learning Approach ◽  
Training Procedure ◽  
Routing Problem ◽  
Policy Model ◽  
Model Free ◽  
Charging Stations

In the past few years, the importance of electric mobility has increased in response to growing concerns about climate change. However, limited cruising range and sparse charging infrastructure could restrain a massive deployment of electric vehicles (EVs). To mitigate the problem, the need for optimal route planning algorithms emerged. In this paper, we propose a mathematical formulation of the EV-specific routing problem in a graph-theoretical context, which incorporates the ability of EVs to recuperate energy. Furthermore, we consider a possibility to recharge on the way using intermediary charging stations. As a possible solution method, we present an off-policy model-free reinforcement learning approach that aims to generate energy feasible paths for EV from source to target. The algorithm was implemented and tested on a case study of a road network in Switzerland. The training procedure requires low computing and memory demands and is suitable for online applications. The results achieved demonstrate the algorithm’s capability to take recharging decisions and produce desired energy feasible paths.

scholarly journals Routing algorithms as tools for integrating social distancing with emergency evacuation

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Yi-Lin Tsai ◽  
Chetanya Rastogi ◽  
Peter K. Kitanidis ◽  
Christopher B. Field
Keyword(s):  
Reinforcement Learning ◽  
Emergency Evacuation ◽  
Decision Makers ◽  
Emergency Vehicle ◽  
Hurricane Evacuation ◽  
Routing Problem ◽  
Social Distancing ◽  
Sweep Algorithm ◽  
Time Required ◽  
Vehicle Capacity

AbstractOne of the lessons from the COVID-19 pandemic is the importance of social distancing, even in challenging circumstances such as pre-hurricane evacuation. To explore the implications of integrating social distancing with evacuation operations, we describe this evacuation process as a Capacitated Vehicle Routing Problem (CVRP) and solve it using a DNN (Deep Neural Network)-based solution (Deep Reinforcement Learning) and a non-DNN solution (Sweep Algorithm). A central question is whether Deep Reinforcement Learning provides sufficient extra routing efficiency to accommodate increased social distancing in a time-constrained evacuation operation. We found that, in comparison to the Sweep Algorithm, Deep Reinforcement Learning can provide decision-makers with more efficient routing. However, the evacuation time saved by Deep Reinforcement Learning does not come close to compensating for the extra time required for social distancing, and its advantage disappears as the emergency vehicle capacity approaches the number of people per household.

Evaluation of Distance-Based and Cordon-Based Urban Freight Road Pricing in E-Commerce Environment with Multiagent Model

2012 ◽  
Vol 2269 (1) ◽  
pp. 127-134 ◽  
Author(s):  
Joel S. E. Teo ◽  
Eiichi Taniguchi ◽  
Ali Gul Qureshi
Keyword(s):  
Time Windows ◽  
Road Pricing ◽  
Home Delivery ◽  
Routing Problem ◽  
Urban Freight ◽  
Q Learning ◽  
Pollution Levels ◽  
Goods And Services ◽  
Implementation Policy ◽  
The City

E-commerce is gradually changing the way shoppers acquire goods and services. Shoppers seek ways to purchase goods easily through the Internet, and shippers or producers offer cheap ways to deliver goods to their customers through the services of carriers for home delivery. A theoretical model was established to evaluate city logistics schemes for multiple stakeholders before implementation. Policy measures to manage truck operations in the city and keep pollution levels at a minimum were evaluated. Cordon-based freight road pricing was found to provide better pollution reduction compared with distance-based pricing, but cordon-based pricing had less impact on areas outside a city. The problem was solved with a modeling approach for multiagent systems that used a vehicle routing problem with time windows, freight electronic marketplaces, and Q-learning.

LIRP optimization of cold chain logistics in satellite warehouse mode of supermarket chains

2021 ◽  
pp. 1-15
Author(s):  
Bo Shu ◽  
Fanghua Pei ◽  
Kaifu Zheng ◽  
Mengxia Yu
Keyword(s):  
Time Windows ◽  
Home Delivery ◽  
Cold Chain ◽  
Inventory Routing ◽  
Logistics System ◽  
Routing Problem ◽  
Penalty Cost ◽  
Location Allocation ◽  
Soft Time Windows ◽  
Fresh Products

Aiming at the problem of high cost in cold chain logistics of fresh products home-delivery in supermarket chain in the new retail era, the paper constructs the model of Location Inventory Routing Problem (LIRP) optimization in Satellite Warehouse mode in view of customer satisfaction with the broken line soft time windows. The model minimizes the total cost of the cold chain logistics system of supermarket chain through the location allocation, inventory optimization, the determination of distribution service relationship between Satellite Warehouse and customer, and the constraint of time penalty cost. Then, the paper designed an improved ant colony optimization to solve the LIRP model of supermarket chain. Finally, the simulation in MATLAB verifies and analyzes the validity of the model and algorithm. Therefore, LIRP optimization model in Satellite Warehouse mode can effectively improve the operational efficiency of fresh products home-delivery in the supermarket chain and thus reduce the logistics cost.

Sign in / Sign up

Export Citation Format

Share Document