scholarly journals Subgradients of value functions in parametric dynamic programming

2009 ◽  
Vol 193 (1) ◽  
pp. 12-22 ◽  
Author(s):  
B.T. Kien ◽  
Y.C. Liou ◽  
N.-C. Wong ◽  
J.-C. Yao
Keyword(s):  
Dynamic Programming ◽  
Value Functions

APPROXIMATE DYNAMIC PROGRAMMING TECHNIQUES FOR SKILL-BASED ROUTING IN CALL CENTERS

2012 ◽  
Vol 26 (4) ◽  
pp. 581-591 ◽  
Author(s):  
D. Roubos ◽  
S. Bhulai
Keyword(s):  
Dynamic Programming ◽  
Call Center ◽  
Approximate Dynamic Programming ◽  
Call Centers ◽  
Problem Instance ◽  
Value Functions ◽  
Skill Sets ◽  
Programming Techniques ◽  
Optimal Dynamic ◽  
Routing Policies

We consider the problem of dynamic multi-skill routing in call centers. Calls from different customer classes are offered to the call center according to a Poisson process. The agents are grouped into pools according to their heterogeneous skill sets that determine the calls that they can handle. Each pool of agents serves calls with independent exponentially distributed service times. Arriving calls that cannot be served directly are placed in a buffer that is dedicated to the customer class. We obtain nearly optimal dynamic routing policies that are scalable with the problem instance and can be computed online. The algorithm is based on approximate dynamic programming techniques. In particular, we perform one-step policy improvement using a polynomial approximation to relative value functions. We compare the performance of this method with decomposition techniques. Numerical experiments demonstrate that our method outperforms leading routing policies and has close to optimal performance.

scholarly journals Symbolic Dynamic Programming for Continuous State MDPs with Linear Program Transitions

2021 ◽  
Author(s):  
Jihwan Jeong ◽  
Parth Jaggi ◽  
Scott Sanner
Keyword(s):  
Dynamic Programming ◽  
Closed Form ◽  
Piecewise Linear ◽  
Linear Program ◽  
Symbolic Dynamic ◽  
State Transitions ◽  
Traffic Signal Control ◽  
Value Functions ◽  
Continuous State ◽  
Solution Methods

Recent advances in symbolic dynamic programming (SDP) have significantly broadened the class of MDPs for which exact closed-form value functions can be derived. However, no existing solution methods can solve complex discrete and continuous state MDPs where a linear program determines state transitions --- transitions that are often required in problems with underlying constrained flow dynamics arising in problems ranging from traffic signal control to telecommunications bandwidth planning. In this paper, we present a novel SDP solution method for MDPs with LP transitions and continuous piecewise linear dynamics by introducing a novel, fully symbolic argmax operator. On three diverse domains, we show the first automated exact closed-form SDP solution to these challenging problems and the significant advantages of our SDP approach over discretized approximations.

scholarly journals The Dynamic Programming Method of Stochastic Differential Game for Functional Forward-Backward Stochastic System

2013 ◽  
Vol 2013 ◽  
pp. 1-14 ◽  
Author(s):  
Shaolin Ji ◽  
Chuanfeng Sun ◽  
Qingmeng Wei
Keyword(s):  
Dynamic Programming ◽  
Differential Game ◽  
Backward Stochastic Differential Equation ◽  
Transformation Method ◽  
Dynamic Programming Method ◽  
Stochastic Differential Game ◽  
Value Functions ◽  
Girsanov Transformation ◽  
Path Dependent ◽  
Hamilton Jacobi Bellman

This paper is devoted to a stochastic differential game (SDG) of decoupled functional forward-backward stochastic differential equation (FBSDE). For our SDG, the associated upper and lower value functions of the SDG are defined through the solution of controlled functional backward stochastic differential equations (BSDEs). Applying the Girsanov transformation method introduced by Buckdahn and Li (2008), the upper and the lower value functions are shown to be deterministic. We also generalize the Hamilton-Jacobi-Bellman-Isaacs (HJBI) equations to the path-dependent ones. By establishing the dynamic programming principal (DPP), we derive that the upper and the lower value functions are the viscosity solutions of the corresponding upper and the lower path-dependent HJBI equations, respectively.

scholarly journals Gradient-Bounded Dynamic Programming with Submodular and Concave extensible Value Functions

2020 ◽  
Vol 53 (2) ◽  
pp. 6825-6830
Author(s):  
Denis Lebedev ◽  
Paul Goulart ◽  
Kostas Margellos
Keyword(s):  
Dynamic Programming ◽  
Value Functions

scholarly journals A NONLINEAR PROGRAMMING METHOD FOR DYNAMIC PROGRAMMING

2016 ◽  
Vol 21 (2) ◽  
pp. 336-361 ◽  
Author(s):  
Yongyang Cai ◽  
Kenneth L. Judd ◽  
Thomas S. Lontzek ◽  
Valentina Michelangeli ◽  
Che-Lin Su
Keyword(s):  
Dynamic Programming ◽  
Nonlinear Programming ◽  
Approximation Theory ◽  
Infinite Horizon ◽  
Numerical Results ◽  
Programming Method ◽  
Value Functions ◽  
Programming Formulation ◽  
Linear Approach ◽  
Nonlinear Programming Method

A nonlinear programming formulation is introduced to solve infinite-horizon dynamic programming problems. This extends the linear approach to dynamic programming by using ideas from approximation theory to approximate value functions. Our numerical results show that this nonlinear programming is efficient and accurate, and avoids inefficient discretization.

scholarly journals An Annuitization Problem in the Tax-Deferred Annuity Model

2021 ◽  
Vol 2021 ◽  
pp. 1-12
Author(s):  
Yanan Li
Keyword(s):  
Dynamic Programming ◽  
Value Function ◽  
Hjb Equation ◽  
Time Dependent ◽  
Dynamic Programming Principle ◽  
Value Functions ◽  
Deferred Annuity ◽  
Consumption Strategies ◽  
The Value Function ◽  
Dependent Mortality

This paper examines the optimal annuitization, investment, and consumption strategies of an individual facing a time-dependent mortality rate in the tax-deferred annuity model and considers both the case when the rate of buying annuities is unrestricted and the case when it is restricted. At the beginning, by using the dynamic programming principle, we obtain the corresponding HJB equation. Since the existence of the tax and the time-dependence of the value function make the corresponding HJB equation hard to solve, firstly, we analyze the problem in a simpler case and use some numerical methods to get the solution and some of its useful properties. Then, by using the obtained properties and Kuhn–Tucker conditions, we discuss the problem in general cases and get the value functions and the optimal annuitization strategies, respectively.

Sign in / Sign up

Export Citation Format

Share Document