A Reinforcement Learning-Based Framework for Solving Physical Design Routing Problem in the Absence of Large Test Sets

2019 ◽  
Author(s):  
Upma Gandhi ◽  
Ismail Bustany ◽  
William Swartz ◽  
Laleh Behjat
Keyword(s):  
Reinforcement Learning ◽  
Physical Design ◽  
Routing Problem ◽  
Test Sets

scholarly journals Attention Routing: Track-Assignment Detailed Routing Using Attention-Based Reinforcement Learning

2020 ◽  
Author(s):  
Haiguang Liao ◽  
Qingyi Dong ◽  
Xuliang Dong ◽  
Wentai Zhang ◽  
Wangyang Zhang ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Integrated Circuits ◽  
Analog Circuits ◽  
Routing Algorithm ◽  
Physical Design ◽  
Advanced Technology ◽  
Design Rule ◽  
Solution Quality ◽  
Routing Problem ◽  
Complex Design

Abstract In the physical design of integrated circuits, global and detailed routing are critical stages involving the determination of the interconnected paths of each net on a circuit while satisfying the design constraints. Existing actual routers as well as routability predictors either have to resort to expensive approaches that lead to high computational times, or use heuristics that do not generalize well. Even though new, learning-based routing methods have been proposed to address this need, requirements on labelled data and difficulties in addressing complex design rule constraints have limited their adoption in advanced technology node physical design problems. In this work, we propose a new router — attention router, which is the first attempt to solve the track-assignment detailed routing problem by applying reinforcement learning. Complex design rule constraints are encoded into the routing algorithm and an attention-model-based REINFORCE algorithm is applied to solve the most critical step in routing: sequencing device pairs to be routed. The attention router and its baseline genetic router are applied to solve different commercial advanced technologies analog circuits problem sets. The attention router demonstrates generalization ability to unseen problems and is also able to achieve more than 100× acceleration over the genetic router without severely compromising the routing solution quality. Increasing the number of training problems greatly improves the performance of attention router. We also discover a similarity between the attention router and the baseline genetic router in terms of positive correlations in cost and routing patterns, which demonstrate the attention router’s ability to be utilized not only as a detailed router but also as a predictor for routability and congestion.

scholarly journals A Deep Reinforcement Learning Approach for Global Routing

2019 ◽  
Vol 142 (6) ◽  
Author(s):  
Haiguang Liao ◽  
Wentai Zhang ◽  
Xuliang Dong ◽  
Barnabas Poczos ◽  
Kenji Shimada ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Integrated Circuits ◽  
Printed Circuit Boards ◽  
Greedy Algorithms ◽  
Global Routing ◽  
Hydraulic Systems ◽  
Routing Problem ◽  
Future Data ◽  
Circuit Components ◽  
Optimization Mechanism

Abstract Global routing has been a historically challenging problem in the electronic circuit design, where the challenge is to connect a large and arbitrary number of circuit components with wires without violating the design rules for the printed circuit boards or integrated circuits. Similar routing problems also exist in the design of complex hydraulic systems, pipe systems, and logistic networks. Existing solutions typically consist of greedy algorithms and hard-coded heuristics. As such, existing approaches suffer from a lack of model flexibility and usually fail to solve sub-problems conjointly. As an alternative approach, this work presents a deep reinforcement learning method for solving the global routing problem in a simulated environment. At the heart of the proposed method is deep reinforcement learning that enables an agent to produce a policy for routing based on the variety of problems, and it is presented with leveraging the conjoint optimization mechanism of deep reinforcement learning. Conjoint optimization mechanism is explained and demonstrated in detail; the best network structure and the parameters of the learned model are explored. Based on the fine-tuned model, routing solutions and rewards are presented and analyzed. The results indicate that the approach can outperform the benchmark method of a sequential A* method, suggesting a promising potential for deep reinforcement learning for global routing and other routing or path planning problems in general. Another major contribution of this work is the development of a global routing problem sets generator with the ability to generate parameterized global routing problem sets with different size and constraints, enabling evaluation of different routing algorithms and the generation of training datasets for future data-driven routing approaches.

scholarly journals A variable neighborhood search algorithm with reinforcement learning for a real-life periodic vehicle routing problem with time windows and open routes

2020 ◽  
Vol 54 (5) ◽  
pp. 1467-1494
Author(s):  
Binhui Chen ◽  
Rong Qu ◽  
Ruibin Bai ◽  
Wasakorn Laesanklang
Keyword(s):  
Reinforcement Learning ◽  
Large Scale ◽  
Search Algorithm ◽  
Real Life ◽  
Planning Horizon ◽  
Neighborhood Search ◽  
Variable Neighbourhood Search ◽  
Container Transportation ◽  
Solution Quality ◽  
Routing Problem

This paper studies a real-life container transportation problem with a wide planning horizon divided into multiple shifts. The trucks in this problem do not return to depot after every single shift but at the end of every two shifts. The mathematical model of the problem is first established, but it is unrealistic to solve this large scale problem with exact search methods. Thus, a Variable Neighbourhood Search algorithm with Reinforcement Learning (VNS-RLS) is thus developed. An urgency level-based insertion heuristic is proposed to construct the initial solution. Reinforcement learning is then used to guide the search in the local search improvement phase. Our study shows that the Sampling scheme in single solution-based algorithms does not significantly improve the solution quality but can greatly reduce the rate of infeasible solutions explored during the search. Compared to the exact search and the state-of-the-art algorithms, the proposed VNS-RLS produces promising results.

scholarly journals Routing of Electric Vehicles With Intermediary Charging Stations: A Reinforcement Learning Approach

2021 ◽  
Vol 4 ◽  
Author(s):  
Marina Dorokhova ◽  
Christophe Ballif ◽  
Nicolas Wyrsch
Keyword(s):  
Reinforcement Learning ◽  
Electric Vehicles ◽  
Mathematical Formulation ◽  
Route Planning ◽  
Learning Approach ◽  
Training Procedure ◽  
Routing Problem ◽  
Policy Model ◽  
Model Free ◽  
Charging Stations

In the past few years, the importance of electric mobility has increased in response to growing concerns about climate change. However, limited cruising range and sparse charging infrastructure could restrain a massive deployment of electric vehicles (EVs). To mitigate the problem, the need for optimal route planning algorithms emerged. In this paper, we propose a mathematical formulation of the EV-specific routing problem in a graph-theoretical context, which incorporates the ability of EVs to recuperate energy. Furthermore, we consider a possibility to recharge on the way using intermediary charging stations. As a possible solution method, we present an off-policy model-free reinforcement learning approach that aims to generate energy feasible paths for EV from source to target. The algorithm was implemented and tested on a case study of a road network in Switzerland. The training procedure requires low computing and memory demands and is suitable for online applications. The results achieved demonstrate the algorithm’s capability to take recharging decisions and produce desired energy feasible paths.

scholarly journals Routing algorithms as tools for integrating social distancing with emergency evacuation

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Yi-Lin Tsai ◽  
Chetanya Rastogi ◽  
Peter K. Kitanidis ◽  
Christopher B. Field
Keyword(s):  
Reinforcement Learning ◽  
Emergency Evacuation ◽  
Decision Makers ◽  
Emergency Vehicle ◽  
Hurricane Evacuation ◽  
Routing Problem ◽  
Social Distancing ◽  
Sweep Algorithm ◽  
Time Required ◽  
Vehicle Capacity

AbstractOne of the lessons from the COVID-19 pandemic is the importance of social distancing, even in challenging circumstances such as pre-hurricane evacuation. To explore the implications of integrating social distancing with evacuation operations, we describe this evacuation process as a Capacitated Vehicle Routing Problem (CVRP) and solve it using a DNN (Deep Neural Network)-based solution (Deep Reinforcement Learning) and a non-DNN solution (Sweep Algorithm). A central question is whether Deep Reinforcement Learning provides sufficient extra routing efficiency to accommodate increased social distancing in a time-constrained evacuation operation. We found that, in comparison to the Sweep Algorithm, Deep Reinforcement Learning can provide decision-makers with more efficient routing. However, the evacuation time saved by Deep Reinforcement Learning does not come close to compensating for the extra time required for social distancing, and its advantage disappears as the emergency vehicle capacity approaches the number of people per household.

Sign in / Sign up

Export Citation Format

Share Document