scholarly journals Discrete homing problems

2013 ◽  
Vol 23 (1) ◽  
pp. 5-18
Author(s):  
Mario Lefebvre ◽  
Moussa Kounta
Keyword(s):  
Optimal Control ◽  
Markov Chain ◽  
Markov Chains ◽  
Cost Function ◽  
Discrete Time ◽  
Value Function ◽  
The Cost ◽  
The Value Function

Abstract We consider the so-called homing problem for discrete-time Markov chains. The aim is to optimally control the Markov chain until it hits a given boundary. Depending on a parameter in the cost function, the optimizer either wants to maximize or minimize the time spent by the controlled process in the continuation region. Particular problems are considered and solved explicitly. Both the optimal control and the value function are obtained

An Optimal Control Problem in a Risk Model with Stochastic Premiums and Periodic Dividend Payments

2017 ◽  
Vol 34 (03) ◽  
pp. 1740013 ◽  
Author(s):  
Xixi Yang ◽  
Jiyang Tan ◽  
Hanjun Zhang ◽  
Ziqiang Li
Keyword(s):  
Optimal Control ◽  
Markov Chain ◽  
Value Function ◽  
Risk Model ◽  
Control Strategies ◽  
Random Variable ◽  
Time Interval ◽  
Positive Real ◽  
Dividend Payments ◽  
The Value Function

In this paper, a discrete-time risk model is considered. We assume that the premium received in each time interval is a positive real-valued random variable, and the sequence of premiums is a Markov chain. In any time interval the probability of a claim occurrence is related to the premium received in the corresponding period. We discuss control strategies for dividends paid periodically to the shareholders under two cases: absence and presence of ceiling restriction for dividend rates. We provide algorithms and some properties for the optimal control strategies by transforming the value function.

On the absorption probabilities and mean time for absorption for discrete Markov chains

2021 ◽  
Vol 0 (0) ◽  
Author(s):  
Nikolaos Halidias
Keyword(s):  
Markov Chain ◽  
Random Walk ◽  
Markov Chains ◽  
Generating Function ◽  
Discrete Time ◽  
Probability Generating Function ◽  
The Mean ◽  
Mean Time ◽  
Absorption Probabilities

Abstract In this note we study the probability and the mean time for absorption for discrete time Markov chains. In particular, we are interested in estimating the mean time for absorption when absorption is not certain and connect it with some other known results. Computing a suitable probability generating function, we are able to estimate the mean time for absorption when absorption is not certain giving some applications concerning the random walk. Furthermore, we investigate the probability for a Markov chain to reach a set A before reach B generalizing this result for a sequence of sets A 1 , A 2 , … , A k {A_{1},A_{2},\dots,A_{k}} .

scholarly journals OBLIGATION BLACKWELL GAMES AND P-AUTOMATA

2017 ◽  
Vol 82 (2) ◽  
pp. 420-452
Author(s):  
KRISHNENDU CHATTERJEE ◽  
NIR PITERMAN
Keyword(s):  
Markov Chains ◽  
Decision Problem ◽  
Value Function ◽  
Acceptance Condition ◽  
Parity Games ◽  
The Value Function

AbstractWe generalize winning conditions in two-player games by adding a structural acceptance condition called obligations. Obligations are orthogonal to the linear winning conditions that define whether a play is winning. Obligations are a declaration that player 0 can achieve a certain value from a configuration. If the obligation is met, the value of that configuration for player 0 is 1.We define the value in such games and show that obligation games are determined. For Markov chains with Borel objectives and obligations, and finite turn-based stochastic parity games with obligations we give an alternative and simpler characterization of the value function. Based on this simpler definition we show that the decision problem of winning finite turn-based stochastic parity games with obligations is in NP∩co-NP. We also show that obligation games provide a game framework for reasoning about p-automata.

scholarly journals A Counterexample to Temporal Differences Learning

1995 ◽  
Vol 7 (2) ◽  
pp. 270-279 ◽  
Author(s):  
Dimitri P. Bertsekas
Keyword(s):  
Neural Network ◽  
Markov Chain ◽  
Cost Function ◽  
Least Squares ◽  
Temporal Differences ◽  
Q Learning ◽  
Absorbing Markov Chain ◽  
Transition Costs ◽  
Compact Representations ◽  
The Cost

Sutton's TD(λ) method aims to provide a representation of the cost function in an absorbing Markov chain with transition costs. A simple example is given where the representation obtained depends on λ. For λ = 1 the representation is optimal with respect to a least-squares error criterion, but as λ decreases toward 0 the representation becomes progressively worse and, in some cases, very poor. The example suggests a need to understand better the circumstances under which TD(0) and Q-learning obtain satisfactory neural network-based compact representations of the cost function. A variation of TD(0) is also given, which performs better on the example.

scholarly journals Hamilton-Jacobi equations for optimal control on networks with entry or exit costs

2019 ◽  
Vol 25 ◽  
pp. 15 ◽  
Author(s):  
Manh Khang Dao
Keyword(s):  
Optimal Control ◽  
Partial Differential Equations ◽  
Recent Work ◽  
Comparison Principle ◽  
Value Function ◽  
Exit Costs ◽  
Theory Of Optimal Control ◽  
New Feature ◽  
Jacobi Equations ◽  
The Value Function

We consider an optimal control on networks in the spirit of the works of Achdou et al. [NoDEA Nonlinear Differ. Equ. Appl. 20 (2013) 413–445] and Imbert et al. [ESAIM: COCV 19 (2013) 129–166]. The main new feature is that there are entry (or exit) costs at the edges of the network leading to a possible discontinuous value function. We characterize the value function as the unique viscosity solution of a new Hamilton-Jacobi system. The uniqueness is a consequence of a comparison principle for which we give two different proofs, one with arguments from the theory of optimal control inspired by Achdou et al. [ESAIM: COCV 21 (2015) 876–899] and one based on partial differential equations techniques inspired by a recent work of Lions and Souganidis [Atti Accad. Naz. Lincei Rend. Lincei Mat. Appl. 27 (2016) 535–545].

Lumpability for non-irreducible finite markov chains

1984 ◽  
Vol 21 (03) ◽  
pp. 567-574 ◽  
Author(s):  
Atef M. Abdel-Moneim ◽  
Frederick W. Leysieffer
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
State Space ◽  
Discrete Time ◽  
The State ◽  
Discrete Time Markov Chain ◽  
Finite Markov Chains

Conditions under which a function of a finite, discrete-time Markov chain, X(t), is again Markov are given, when X(t) is not irreducible. These conditions are given in terms of an interrelationship between two partitions of the state space of X(t), the partition induced by the minimal essential classes of X(t) and the partition with respect to which lumping is to be considered.

scholarly journals Optimal capital injections and dividends with tax in a risk model in discrete time

2020 ◽  
Vol 10 (1) ◽  
pp. 235-259
Author(s):  
Katharina Bata ◽  
Hanspeter Schmidli
Keyword(s):  
Discrete Time ◽  
Optimal Strategy ◽  
Bellman Equation ◽  
Value Function ◽  
Risk Model ◽  
Barrier Type ◽  
Optimal Capital ◽  
Capital Injections ◽  
The Value Function ◽  
Special Case

AbstractWe consider a risk model in discrete time with dividends and capital injections. The goal is to maximise the value of a dividend strategy. We show that the optimal strategy is of barrier type. That is, all capital above a certain threshold is paid as dividend. A second problem adds tax to the dividends but an injection leads to an exemption from tax. We show that the value function fulfils a Bellman equation. As a special case, we consider the case of premia of size one. In this case we show that the optimal strategy is a two barrier strategy. That is, there is a barrier if a next dividend of size one can be paid without tax and a barrier if the next dividend of size one will be taxed. In both models, we illustrate the findings by de Finetti’s example.

Sign in / Sign up

Export Citation Format

Share Document