scholarly journals Discrete-Time Constrained Average Stochastic Games with Independent State Processes

Mathematics ◽  
2019 ◽  
Vol 7 (11) ◽  
pp. 1089
Author(s):  
Wenzhao Zhang
Keyword(s):  
Discrete Time ◽  
Stochastic Games ◽  
Transition Probability ◽  
Existence Condition ◽  
Linear Program ◽  
Global Minimizer ◽  
Independent State ◽  
Occupation Measures ◽  
Best Response ◽  
Mathematic Program

In this paper, we consider the discrete-time constrained average stochastic games with independent state processes. The state space of each player is denumerable and one-stage cost functions can be unbounded. In these game models, each player chooses an action each time which influences the transition probability of a Markov chain controlled only by this player. Moreover, each player needs to pay some costs which depend on the actions of all the players. First, we give an existence condition of stationary constrained Nash equilibria based on the technique of average occupation measures and the best response linear program. Then, combining the best response linear program and duality program, we present a non-convex mathematic program and prove that each stationary Nash equilibrium is a global minimizer of this mathematic program. Finally, a controlled wireless network is presented to illustrate our main results.

A stochastic games framework for verification and control of discrete time stochastic hybrid systems

Automatica ◽  
2013 ◽  
Vol 49 (9) ◽  
pp. 2665-2674 ◽  
Author(s):  
Jerry Ding ◽  
Maryam Kamgarpour ◽  
Sean Summers ◽  
Alessandro Abate ◽  
John Lygeros ◽  
...  
Keyword(s):  
Hybrid Systems ◽  
Discrete Time ◽  
Stochastic Games ◽  
Stochastic Hybrid Systems ◽  
And Control

scholarly journals Zero-sum constrained stochastic games with independent state processes

2005 ◽  
Vol 62 (3) ◽  
pp. 375-386 ◽  
Author(s):  
Eitan Altman ◽  
Konstantin Avrachenkov ◽  
Richard Marquez ◽  
Gregory Miller
Keyword(s):  
Stochastic Games ◽  
Independent State ◽  
Zero Sum

scholarly journals On N-person stochastic games by denumerable state space

1978 ◽  
Vol 10 (2) ◽  
pp. 452-471 ◽  
Author(s):  
A. Federgruen
Keyword(s):  
State Space ◽  
Stochastic Games ◽  
Transition Probability ◽  
Discount Factor ◽  
Average Return ◽  
Countable State ◽  
Transition Probability Matrices ◽  
Probability Matrices ◽  
Factor Α ◽  
Action Spaces

This paper considers non-cooperative N-person stochastic games with a countable state space and compact metric action spaces. We concentrate upon the average return per unit time criterion for which the existence of an equilibrium policy is established under a number of recurrency conditions with respect to the transition probability matrices associated with the stationary policies. These results are obtained by establishing the existence of total discounted return equilibrium policies, for each discount factor α ∈ [0, 1) and by showing that under each one of the aforementioned recurrency conditions, average return equilibrium policies appear as limit policies of sequences of discounted return equilibrium policies, with discount factor tending to one.Finally, we review and extend the results that are known for the case where both the state space and the action spaces are finite.

scholarly journals Frekuensi Hari Hujan Menurut Bulan di Kota Balikpapan dengan Rantai Markov Waktu Diskrit

2019 ◽  
Vol 1 (2) ◽  
pp. 5-10
Author(s):  
Muhammad Azka
Keyword(s):  
Markov Chain ◽  
Discrete Time ◽  
Transition Probability ◽  
Transition Probability Matrix ◽  
Transition Diagram ◽  
Discrete Time Markov Chain ◽  
Applied Method ◽  
Frequency Rate

The problem proposed in this research is about the amount rainy day per a month at Balikpapan city and discretetime markov chain. The purpose is finding the probability of rainy day with the frequency rate of rainy at the next month if given the frequency rate of rainy at the prior month. The applied method in this research is classifying the amount of rainy day be three frequency levels, those are, high, medium, and low. If a month, the amount of rainy day is less than 11 then the frequency rate for the month is classified low, if a month, the amount of rainy day between 10 and 20, then it is classified medium and if it is more than 20, then it is classified high. The result is discrete-time markov chain represented with the transition probability matrix, and the transition diagram.

scholarly journals Finite-Time Nonfragile Dissipative Control for Discrete-Time Neural Networks with Markovian Jumps and Mixed Time-Delays

Complexity ◽  
2019 ◽  
Vol 2019 ◽  
pp. 1-17 ◽  
Author(s):  
Ling Hou ◽  
Dongyan Chen ◽  
Chan He
Keyword(s):  
Neural Networks ◽  
Discrete Time ◽  
Finite Time ◽  
Sufficient Conditions ◽  
Existence Condition ◽  
Mode Switching ◽  
Linear Matrix ◽  
Mixed Delays ◽  
Time Varying Delay ◽  
Mixed Time Delays

This paper considers the stochastic finite-time dissipative (SFTD) control problem based on nonfragile controller for discrete-time neural networks (NNS) with Markovian jumps and mixed delays, in which the mode switching phenomenon, is described as Markov chain, and the mixed delays are composed of discrete time-varying delay and distributed delays. First, by selecting an appropriate Lyapunov-Krasovskii functional and applying stochastic analysis methods, some parameters-dependent sufficient conditions for solvability of stochastic finite-time boundedness are derived. Then, the main results are extended to SFTD control. Furthermore, existence condition of nonfragile controller is derived based on solution of linear matrix inequalities (LMIs). Finally, two numerical examples are employed to show the effectiveness of the obtained methods.

Discrete-time singularly perturbed Markov chains: aggregation, occupation measures, and switching diffusion limit

2003 ◽  
Vol 35 (2) ◽  
pp. 449-476 ◽  
Author(s):  
G. Yin ◽  
Q. Zhang ◽  
G. Badowski
Keyword(s):  
Markov Chains ◽  
Discrete Time ◽  
Continuous Time ◽  
Communication Systems ◽  
Large Scale ◽  
Optimization Problems ◽  
Singularly Perturbed ◽  
Diffusion Limit ◽  
Switching Diffusion ◽  
Occupation Measures

This work is devoted to asymptotic properties of singularly perturbed Markov chains in discrete time. The motivation stems from applications in discrete-time control and optimization problems, manufacturing and production planning, stochastic networks, and communication systems, in which finite-state Markov chains are used to model large-scale and complex systems. To reduce the complexity of the underlying system, the states in each recurrent class are aggregated into a single state. Although the aggregated process may not be Markovian, its continuous-time interpolation converges to a continuous-time Markov chain whose generator is a function determined by the invariant measures of the recurrent states. Sequences of occupation measures are defined. A mean square estimate on a sequence of unscaled occupation measures is obtained. Furthermore, it is proved that a suitably scaled sequence of occupation measures converges to a switching diffusion.

Countable-state-space Markov chains with two time scales and applications to queueing systems

2002 ◽  
Vol 34 (3) ◽  
pp. 662-688 ◽  
Author(s):  
G. Yin ◽  
Hanqin Zhang
Keyword(s):  
Markov Chains ◽  
Time Scale ◽  
Transition Probability ◽  
Queueing Systems ◽  
Fast Time ◽  
State Spaces ◽  
Occupation Measures ◽  
Countable State ◽  
Probability Matrices ◽  
Markov Modulated

Motivated by various applications in queueing systems, this work is devoted to continuous-time Markov chains with countable state spaces that involve both fast-time scale and slow-time scale with the aim of approximating the time-varying queueing systems by their quasistationary counterparts. Under smoothness conditions on the generators, asymptotic expansions of probability vectors and transition probability matrices are constructed. Uniform error bounds are obtained, and then sequences of occupation measures and their functionals are examined. Mean square error estimates of a sequence of occupation measures are obtained; a scaled sequence of functionals of occupation measures is shown to converge to a Gaussian process with zero mean. The representation of the variance of the limit process is also explicitly given. The results obtained are then applied to treat Mt/Mt/1 queues and Markov-modulated fluid buffer models.

Sign in / Sign up

Export Citation Format

Share Document