Optimal Stopping Problem in A Finite State Partially Observable Markov Chain

1983 ◽  
Vol 4 (2) ◽  
pp. 159-176 ◽  
Author(s):  
Tōru Nakai
2020 ◽  
Vol 24 (4) ◽  
pp. 1083-1132
Author(s):  
Giorgia Callegaro ◽  
Claudia Ceci ◽  
Giorgio Ferrari

Abstract We consider a government that aims at reducing the debt-to-(gross domestic product) (GDP) ratio of a country. The government observes the level of the debt-to-GDP ratio and an indicator of the state of the economy, but does not directly observe the development of the underlying macroeconomic conditions. The government’s criterion is to minimise the sum of the total expected costs of holding debt and of debt reduction policies. We model this problem as a singular stochastic control problem under partial observation. The contribution of the paper is twofold. Firstly, we provide a general formulation of the model in which the level of the debt-to-GDP ratio and the value of the macroeconomic indicator evolve as a diffusion and a jump-diffusion, respectively, with coefficients depending on the regimes of the economy. The latter are described through a finite-state continuous-time Markov chain. We reduce the original problem via filtering techniques to an equivalent one with full information (the so-called separated problem), and we provide a general verification result in terms of a related optimal stopping problem under full information. Secondly, we specialise to a case study in which the economy faces only two regimes and the macroeconomic indicator has a suitable diffusive dynamics. In this setting, we provide the optimal debt reduction policy. This is given in terms of the continuous free boundary arising in an auxiliary fully two-dimensional optimal stopping problem.


1982 ◽  
Vol 19 (1) ◽  
pp. 72-81 ◽  
Author(s):  
George E. Monahan

The problem of optimal stopping in a Markov chain when there is imperfect state information is formulated as a partially observable Markov decision process. Properties of the optimal value function are developed. It is shown that under mild conditions the optimal policy is well structured. An efficient algorithm, which uses the structural information in the computation of the optimal policy, is presented.


1982 ◽  
Vol 19 (01) ◽  
pp. 72-81 ◽  
Author(s):  
George E. Monahan

The problem of optimal stopping in a Markov chain when there is imperfect state information is formulated as a partially observable Markov decision process. Properties of the optimal value function are developed. It is shown that under mild conditions the optimal policy is well structured. An efficient algorithm, which uses the structural information in the computation of the optimal policy, is presented.


Sign in / Sign up

Export Citation Format

Share Document