Stability of Markovian processes III: Foster–Lyapunov criteria for continuous-time processes

In Part I we developed stability concepts for discrete chains, together with Foster–Lyapunov criteria for them to hold. Part II was devoted to developing related stability concepts for continuous-time processes. In this paper we develop criteria for these forms of stability for continuous-parameter Markovian processes on general state spaces, based on Foster-Lyapunov inequalities for the extended generator. Such test function criteria are found for non-explosivity, non-evanescence, Harris recurrence, and positive Harris recurrence. These results are proved by systematic application of Dynkin's formula. We also strengthen known ergodic theorems, and especially exponential ergodic results, for continuous-time processes. In particular we are able to show that the test function approach provides a criterion for f-norm convergence, and bounding constants for such convergence in the exponential ergodic case. We apply the criteria to several specific processes, including linear stochastic systems under non-linear feedback, work-modulated queues, general release storage processes and risk processes.

Download Full-text

Stability of Markovian processes III: Foster–Lyapunov criteria for continuous-time processes

Advances in Applied Probability ◽

10.2307/1427522 ◽

1993 ◽

Vol 25 (3) ◽

pp. 518-548 ◽

Cited By ~ 337

Author(s):

Sean P. Meyn ◽

R. L. Tweedie

Keyword(s):

Continuous Time ◽

Stochastic Systems ◽

Linear Feedback ◽

Test Function ◽

Markovian Processes ◽

Harris Recurrence ◽

Lyapunov Inequalities ◽

Continuous Time Processes ◽

Risk Processes ◽

Stability Concepts

In Part I we developed stability concepts for discrete chains, together with Foster–Lyapunov criteria for them to hold. Part II was devoted to developing related stability concepts for continuous-time processes. In this paper we develop criteria for these forms of stability for continuous-parameter Markovian processes on general state spaces, based on Foster-Lyapunov inequalities for the extended generator.Such test function criteria are found for non-explosivity, non-evanescence, Harris recurrence, and positive Harris recurrence. These results are proved by systematic application of Dynkin's formula.We also strengthen known ergodic theorems, and especially exponential ergodic results, for continuous-time processes. In particular we are able to show that the test function approach provides a criterion for f-norm convergence, and bounding constants for such convergence in the exponential ergodic case.We apply the criteria to several specific processes, including linear stochastic systems under non-linear feedback, work-modulated queues, general release storage processes and risk processes.

Download Full-text

Stability of Markovian processes II: continuous-time processes and sampled chains

Advances in Applied Probability ◽

10.1017/s0001867800025520 ◽

1993 ◽

Vol 25 (03) ◽

pp. 487-517 ◽

Cited By ~ 47

Author(s):

Sean P. Meyn ◽

R. L. Tweedie

Keyword(s):

Discrete Time ◽

Operations Research ◽

Continuous Parameter ◽

Major Class ◽

State Dependent ◽

Markovian Processes ◽

Continuous Components ◽

Test Sets ◽

Continuous Time Processes ◽

Stability Concepts

In this paper we extend the results of Meyn and Tweedie (1992b) from discrete-time parameter to continuous-parameter Markovian processes Φ evolving on a topological space. We consider a number of stability concepts for such processes in terms of the topology of the space, and prove connections between these and standard probabilistic recurrence concepts. We show that these structural results hold for a major class of processes (processes with continuous components) in a manner analogous to discrete-time results, and that complex operations research models such as storage models with state-dependent release rules, or diffusion models such as those with hypoelliptic generators, have this property. Also analogous to discrete time, ‘petite sets', which are known to provide test sets for stability, are here also shown to provide conditions for continuous components to exist. New ergodic theorems for processes with irreducible and countably reducible skeleton chains are derived, and we show that when these conditions do not hold, then the process may be decomposed into an uncountable orbit of skeleton chains.

Download Full-text

Constrained regulation problem for continuous-time stochastic systems under state and control constraints

Journal of Vibration and Control ◽

10.1177/10775463211028075 ◽

2021 ◽

pp. 107754632110280

Author(s):

Xindong Si ◽

Hongli Yang

Keyword(s):

Continuous Time ◽

Stochastic Systems ◽

Invariant Set ◽

Point Of View ◽

Linear Feedback ◽

Stochastic Nonlinear Systems ◽

Computation Method ◽

Computational Point ◽

Linear Feedback Controller ◽

Optimization Methodology

Constrained regulation problem (CRP) for continuous-time stochastic systems is investigated in this article. New existence conditions of linear feedback control law for continuous-time stochastic systems under constraints are proposed. The computation method for solving constrained regulation problem of stochastic systems considered in this article is also presented. Continuous-time stochastic linear systems and stochastic nonlinear systems are focused on, respectively. First, the condition of polyhedral invariance for stochastic systems is established by using the theory of positive invariant set and the principle of comparison. Second, the asymptotic stability conditions in the sense of expectation for two types of stochastic systems are established. Finally, finding the linear feedback controller model and corresponding algorithm of constrained regulation problem for two types of stochastic systems are also proposed by using the obtained condition. The presented model of the stochastic constrained regulation problem in this article is formulated as a linear programming problem, which can be easily implemented from a computational point of view. Our approach establishes a connection between the stochastic constrained regulation problem and positively invariant set theory, as well as provides the possibility of using optimization methodology to find the solution of stochastic constrained regulation problem, which differs from other methods. Numerical examples illustrate the proposed method.

Download Full-text

Stability of Markovian processes II: continuous-time processes and sampled chains

Advances in Applied Probability ◽

10.2307/1427521 ◽

1993 ◽

Vol 25 (3) ◽

pp. 487-517 ◽

Cited By ~ 158

Author(s):

Sean P. Meyn ◽

R. L. Tweedie

Keyword(s):

Discrete Time ◽

Operations Research ◽

Continuous Parameter ◽

Major Class ◽

State Dependent ◽

Markovian Processes ◽

Continuous Components ◽

Test Sets ◽

Continuous Time Processes ◽

Stability Concepts

In this paper we extend the results of Meyn and Tweedie (1992b) from discrete-time parameter to continuous-parameter Markovian processes Φ evolving on a topological space.We consider a number of stability concepts for such processes in terms of the topology of the space, and prove connections between these and standard probabilistic recurrence concepts. We show that these structural results hold for a major class of processes (processes with continuous components) in a manner analogous to discrete-time results, and that complex operations research models such as storage models with state-dependent release rules, or diffusion models such as those with hypoelliptic generators, have this property. Also analogous to discrete time, ‘petite sets', which are known to provide test sets for stability, are here also shown to provide conditions for continuous components to exist.New ergodic theorems for processes with irreducible and countably reducible skeleton chains are derived, and we show that when these conditions do not hold, then the process may be decomposed into an uncountable orbit of skeleton chains.

Download Full-text