Approximation solution and suboptimality for discounted semi-markov decision problems with countable state space

Optimization ◽  
2004 ◽  
Vol 53 (4) ◽  
pp. 339-353 ◽  
Author(s):  
D. Hudak ◽  
V. Nollau
Keyword(s):  
State Space ◽  
Decision Problems ◽  
Approximation Solution ◽  
Countable State Space ◽  
Markov Decision Problems ◽  
Countable State ◽  
Markov Decision

Constrained Discounted Markov Decision Chains

1991 ◽  
Vol 5 (4) ◽  
pp. 463-475 ◽  
Author(s):  
Linn I. Sennott
Keyword(s):  
State Space ◽  
Discrete Time ◽  
Queueing Systems ◽  
Operating Cost ◽  
Countable State Space ◽  
Countable State ◽  
Markov Decision ◽  
Holding Cost

A Markov decision chain with countable state space incurs two types of costs: an operating cost and a holding cost. The objective is to minimize the expected discounted operating cost, subject to a constraint on the expected discounted holding cost. The existence of an optimal randomized simple policy is proved. This is a policy that randomizes between two stationary policies, that differ in at most one state. Several examples from the control of discrete time queueing systems are discussed.

A general markov decision method II: Applications

1977 ◽  
Vol 9 (2) ◽  
pp. 316-335 ◽  
Author(s):  
G. De Leve ◽  
A. Federgruen ◽  
H. C. Tijms
Keyword(s):  
State Space ◽  
Wide Class ◽  
Average Cost ◽  
Preceding Paper ◽  
Decision Problems ◽  
New Approach ◽  
Decision Method ◽  
Markov Decision Problems ◽  
Markov Decision ◽  
Potential Customers

In a preceding paper [2] we have introduced a new approach for solving a wide class of Markov decision problems in which the state-space may be general and the system may be continuously controlled. The criterion is the average cost. This paper discusses two applications of this approach. The first application concerns a house-selling problem in which a constructor builds houses which may be sold at any stage of the construction and potential customers make offers depending on the stage of the construction. The second application considers an M/M/c queueing problem in which the number of operating servers can be controlled by turning servers on or off.

A general markov decision method II: Applications

1977 ◽  
Vol 9 (02) ◽  
pp. 316-335 ◽  
Author(s):  
G. De Leve ◽  
A. Federgruen ◽  
H. C. Tijms
Keyword(s):  
State Space ◽  
Wide Class ◽  
Average Cost ◽  
Preceding Paper ◽  
Decision Problems ◽  
New Approach ◽  
Decision Method ◽  
Markov Decision Problems ◽  
Markov Decision ◽  
Potential Customers

In a preceding paper [2] we have introduced a new approach for solving a wide class of Markov decision problems in which the state-space may be general and the system may be continuously controlled. The criterion is the average cost. This paper discusses two applications of this approach. The first application concerns a house-selling problem in which a constructor builds houses which may be sold at any stage of the construction and potential customers make offers depending on the stage of the construction. The second application considers an M/M/c queueing problem in which the number of operating servers can be controlled by turning servers on or off.

Sign in / Sign up

Export Citation Format

Share Document