Approximating general Markovian decision-problems by clustering their state- and action-spaces

Author(s):  
Willibald Doeringer
1980 ◽  
Vol 12 (1) ◽  
pp. 154-173 ◽  
Author(s):  
Gerhard Hübner

A stationary Markovian decision model is considered with general state and action spaces where the transition probabilities are weakened to be bounded transition measures (this is useful for many applications). New and improved bounds are given for the optimal value of stationary problems with a large planning horizon if either only a few steps of iteration are carried out or, in addition, a solution of the infinite-stage problem is known. Similar estimates are obtained for the quality of policies which are composed of nearly optimal decisions from the first few steps or from the infinite-stage solution.


1980 ◽  
Vol 12 (01) ◽  
pp. 154-173
Author(s):  
Gerhard Hübner

A stationary Markovian decision model is considered with general state and action spaces where the transition probabilities are weakened to be bounded transition measures (this is useful for many applications). New and improved bounds are given for the optimal value of stationary problems with a large planning horizon if either only a few steps of iteration are carried out or, in addition, a solution of the infinite-stage problem is known. Similar estimates are obtained for the quality of policies which are composed of nearly optimal decisions from the first few steps or from the infinite-stage solution.


2019 ◽  
Vol 56 (3) ◽  
pp. 810-829
Author(s):  
János Flesch ◽  
Dries Vermeulen ◽  
Anna Zseleva

AbstractWe consider decision problems with arbitrary action spaces, deterministic transitions, and infinite time horizon. In the usual setup when probability measures are countably additive, a general version of Kuhn’s theorem implies under fairly general conditions that for every mixed strategy of the decision maker there exists an equivalent behavior strategy. We examine to what extent this remains valid when probability measures are only assumed to be finitely additive. Under the classical approach of Dubins and Savage (2014), we prove the following statements: (1) If the action space is finite, every mixed strategy has an equivalent behavior strategy. (2) Even if the action space is infinite, at least one optimal mixed strategy has an equivalent behavior strategy. The approach by Dubins and Savage turns out to be essentially maximal: these two statements are no longer valid if we take any extension of their approach that considers all singleton plays.


2013 ◽  
Vol 46 (24) ◽  
pp. 159-164
Author(s):  
W.P. Marcos ◽  
M.V.M. Ferreira ◽  
A.P. Cortes ◽  
J.J.P.Z.S. Tavares

Sign in / Sign up

Export Citation Format

Share Document