Computing optimal (s, S) policies in inventory models with continuous demands

1985 ◽  
Vol 17 (2) ◽  
pp. 424-442 ◽  
Author(s):  
A. Federgruen ◽  
P. Zipkin
Keyword(s):  
Markov Decision Process ◽  
Decision Process ◽  
Inventory Models ◽  
Cost Criterion ◽  
Continuous State Space ◽  
Continuous State ◽  
Phase Type ◽  
Phase Type Distributions ◽  
Markov Decision ◽  
Continuous Demand

Special algorithms have been developed to compute an optimal (s, S) policy for an inventory model with discrete demand and under standard assumptions (stationary data, a well-behaved one-period cost function, full backlogging and the average cost criterion). We present here an iterative algorithm for continuous demand distributions which avoids any form of prior discretization. The method can be viewed as a modified form of policy iteration applied to a Markov decision process with continuous state space. For phase-type distributions, the calculations can be done in closed form.

Computing optimal (s, S) policies in inventory models with continuous demands

1985 ◽  
Vol 17 (02) ◽  
pp. 424-442 ◽  
Author(s):  
A. Federgruen ◽  
P. Zipkin
Keyword(s):  
Markov Decision Process ◽  
Decision Process ◽  
Inventory Models ◽  
Cost Criterion ◽  
Continuous State Space ◽  
Continuous State ◽  
Phase Type ◽  
Phase Type Distributions ◽  
Markov Decision ◽  
Continuous Demand

Special algorithms have been developed to compute an optimal (s, S) policy for an inventory model with discrete demand and under standard assumptions (stationary data, a well-behaved one-period cost function, full backlogging and the average cost criterion). We present here an iterative algorithm for continuous demand distributions which avoids any form of prior discretization. The method can be viewed as a modified form of policy iteration applied to a Markov decision process with continuous state space. For phase-type distributions, the calculations can be done in closed form.

Construction of Semi-Markov Decision Process Models of Continuous State Space Environments Using Growing Cell Structures and Multiagentk-Certainty Exploration Method

2009 ◽  
Vol 13 (6) ◽  
pp. 608-614
Author(s):  
Takeshi Tateyama ◽  
◽  
Seiichi Kawata ◽  
Yoshiki Shimomura ◽  
◽  
...  
Keyword(s):  
State Space ◽  
Markov Decision Process ◽  
Decision Process ◽  
Discrete State ◽  
Cell Structures ◽  
Growing Cell ◽  
Continuous State Space ◽  
Continuous State ◽  
Markov Decision ◽  
Growing Cell Structures

k-certainty exploration method, an efficient reinforcement learning algorithm, is not applied to environments whose state space is continuous because continuous state space must be changed to discrete state space. Our purpose is to construct discrete semi-Markov decision process (SMDP) models of such environments using growing cell structures to autonomously divide continuous state space then usingk-certainty exploration method to construct SMDP models. Multiagentk-certainty exploration method is then used to improve exploration efficiency. Mobile robot simulation demonstrated our proposal's usefulness and efficiency.

Sign in / Sign up

Export Citation Format

Share Document