Sequential decision processes with essential unobservables

1969 ◽  
Vol 1 (2) ◽  
pp. 271-287 ◽  
Author(s):  
P. Whittle
Keyword(s):  
Loss Function ◽  
Posterior Distribution ◽  
Decision Processes ◽  
Sequential Decision ◽  
Sufficient Statistics ◽  
Considerable Simplification

We consider sequential decision processes in which the posterior distribution of some unobservable must be included among the sufficient statistics. The technique of an earlier paper (Whittle (1964)) is applied to show that the residual loss function is necessarily homogeneous of degree one in this distribution, if certain conventions are adopted. This point leads to considerable simplification. Examples are given of the great variety of problems which can advantageously be formulated in this manner.

Sequential decision processes with essential unobservables

1969 ◽  
Vol 1 (02) ◽  
pp. 271-287 ◽  
Author(s):  
P. Whittle
Keyword(s):  
Loss Function ◽  
Posterior Distribution ◽  
Decision Processes ◽  
Sequential Decision ◽  
Sufficient Statistics ◽  
Considerable Simplification

We consider sequential decision processes in which the posterior distribution of some unobservable must be included among the sufficient statistics. The technique of an earlier paper (Whittle (1964)) is applied to show that the residual loss function is necessarily homogeneous of degree one in this distribution, if certain conventions are adopted. This point leads to considerable simplification. Examples are given of the great variety of problems which can advantageously be formulated in this manner.

scholarly journals Enforcing Almost-Sure Reachability in POMDPs

2021 ◽  
pp. 602-625
Author(s):  
Sebastian Junges ◽  
Nils Jansen ◽  
Sanjit A. Seshia
Keyword(s):  
Markov Decision Processes ◽  
Empirical Evaluation ◽  
Decision Processes ◽  
Limited Information ◽  
Sequential Decision ◽  
Goal State ◽  
Learning Agent ◽  
Markov Decision ◽  
System Configurations ◽  
Partially Observable

AbstractPartially-Observable Markov Decision Processes (POMDPs) are a well-known stochastic model for sequential decision making under limited information. We consider the EXPTIME-hard problem of synthesising policies that almost-surely reach some goal state without ever visiting a bad state. In particular, we are interested in computing the winning region, that is, the set of system configurations from which a policy exists that satisfies the reachability specification. A direct application of such a winning region is the safe exploration of POMDPs by, for instance, restricting the behavior of a reinforcement learning agent to the region. We present two algorithms: A novel SAT-based iterative approach and a decision-diagram based alternative. The empirical evaluation demonstrates the feasibility and efficacy of the approaches.

scholarly journals Bayes Estimators of Exponentiated Inverse Rayleigh Distribution using Lindleys Approximation

2021 ◽  
pp. 60-71
Author(s):  
Bashiru Omeiza Sule ◽  
Taiwo Mobolaji Adegoke ◽  
Kafayat Tolani Uthman
Keyword(s):  
Loss Function ◽  
Posterior Distribution ◽  
Real Life ◽  
Rayleigh Distribution ◽  
Loss Functions ◽  
Bayes Estimators ◽  
Shape Parameters ◽  
True Parameter ◽  
Squared Error Loss Function ◽  
Scale Parameters

In this paper, Bayes estimators of the unknown shape and scale parameters of the Exponentiated Inverse Rayleigh Distribution (EIRD) have been derived using both the frequentist and bayesian methods. The Bayes theorem was adopted to obtain the posterior distribution of the shape and scale parameters of an Exponentiated Inverse Rayleigh Distribution (EIRD) using both conjugate and non-conjugate prior distribution under different loss functions (such as Entropy Loss Function, Linex Loss Function and Scale Invariant Squared Error Loss Function). The posterior distribution derived for both shape and scale parameters are intractable and a Lindley approximation was adopted to obtain the parameters of interest. The loss function were employed to obtain the estimates for both scale and shape parameters with an assumption that the both scale and shape parameters are unknown and independent. Also the Bayes estimate for the simulated datasets and real life datasets were obtained. The Bayes estimates obtained under dierent loss functions are close to the true parameter value of the shape and scale parameters. The estimators are then compared in terms of their Mean Square Error (MSE) using R programming language. We deduce that the MSE reduces as the sample size (n) increases.

Sign in / Sign up

Export Citation Format

Share Document