Myopic Solutions of Homogeneous Sequential Decision Processes

2006 ◽  
Author(s):  
Matthew J. Sobel ◽  
Wei Wei
Keyword(s):  
Decision Processes ◽  
Sequential Decision

scholarly journals Enforcing Almost-Sure Reachability in POMDPs

2021 ◽  
pp. 602-625
Author(s):  
Sebastian Junges ◽  
Nils Jansen ◽  
Sanjit A. Seshia
Keyword(s):  
Markov Decision Processes ◽  
Empirical Evaluation ◽  
Decision Processes ◽  
Limited Information ◽  
Sequential Decision ◽  
Goal State ◽  
Learning Agent ◽  
Markov Decision ◽  
System Configurations ◽  
Partially Observable

AbstractPartially-Observable Markov Decision Processes (POMDPs) are a well-known stochastic model for sequential decision making under limited information. We consider the EXPTIME-hard problem of synthesising policies that almost-surely reach some goal state without ever visiting a bad state. In particular, we are interested in computing the winning region, that is, the set of system configurations from which a policy exists that satisfies the reachability specification. A direct application of such a winning region is the safe exploration of POMDPs by, for instance, restricting the behavior of a reinforcement learning agent to the region. We present two algorithms: A novel SAT-based iterative approach and a decision-diagram based alternative. The empirical evaluation demonstrates the feasibility and efficacy of the approaches.

Sequential decision processes with essential unobservables

1969 ◽  
Vol 1 (2) ◽  
pp. 271-287 ◽  
Author(s):  
P. Whittle
Keyword(s):  
Loss Function ◽  
Posterior Distribution ◽  
Decision Processes ◽  
Sequential Decision ◽  
Sufficient Statistics ◽  
Considerable Simplification

We consider sequential decision processes in which the posterior distribution of some unobservable must be included among the sufficient statistics. The technique of an earlier paper (Whittle (1964)) is applied to show that the residual loss function is necessarily homogeneous of degree one in this distribution, if certain conventions are adopted. This point leads to considerable simplification. Examples are given of the great variety of problems which can advantageously be formulated in this manner.

scholarly journals Three different operations research models for the same (s,S) policy

2001 ◽  
Vol 5 (1) ◽  
pp. 47-59
Author(s):  
Omar Ben-Ayed
Keyword(s):  
Dynamic Programming ◽  
Markov Chain ◽  
Operations Research ◽  
Decision Processes ◽  
Periodic Review ◽  
Inventory Problem ◽  
Sequential Decision ◽  
Review System ◽  
Numerical Example ◽  
Chain Dynamic

Operations Research techniques are usually presented as distinct models. Difficult as it may often be, achieving linkage between these models could reveal their interdependency and make them easier for the user to understand. In this article three different models, namely Markov Chain, Dynamic Programming, and Markov Sequential Decision Processes, are used to solve an inventory problem based on the periodic review system. We show how the three models converge to the same (s,S) policy and we provide a numerical example to illustrate such a convergence.

Sign in / Sign up

Export Citation Format

Share Document