scholarly journals On N-person stochastic games by denumerable state space

1978 ◽  
Vol 10 (2) ◽  
pp. 452-471 ◽  
Author(s):  
A. Federgruen
Keyword(s):  
State Space ◽  
Stochastic Games ◽  
Transition Probability ◽  
Discount Factor ◽  
Average Return ◽  
Countable State ◽  
Transition Probability Matrices ◽  
Probability Matrices ◽  
Factor Α ◽  
Action Spaces

This paper considers non-cooperative N-person stochastic games with a countable state space and compact metric action spaces. We concentrate upon the average return per unit time criterion for which the existence of an equilibrium policy is established under a number of recurrency conditions with respect to the transition probability matrices associated with the stationary policies. These results are obtained by establishing the existence of total discounted return equilibrium policies, for each discount factor α ∈ [0, 1) and by showing that under each one of the aforementioned recurrency conditions, average return equilibrium policies appear as limit policies of sequences of discounted return equilibrium policies, with discount factor tending to one.Finally, we review and extend the results that are known for the case where both the state space and the action spaces are finite.

scholarly journals On N-person stochastic games by denumerable state space

1978 ◽  
Vol 10 (02) ◽  
pp. 452-471 ◽  
Author(s):  
A. Federgruen
Keyword(s):  
State Space ◽  
Stochastic Games ◽  
Transition Probability ◽  
Discount Factor ◽  
Average Return ◽  
Countable State ◽  
Transition Probability Matrices ◽  
Probability Matrices ◽  
Factor Α ◽  
Action Spaces

This paper considers non-cooperative N-person stochastic games with a countable state space and compact metric action spaces. We concentrate upon the average return per unit time criterion for which the existence of an equilibrium policy is established under a number of recurrency conditions with respect to the transition probability matrices associated with the stationary policies. These results are obtained by establishing the existence of total discounted return equilibrium policies, for each discount factor α ∈ [0, 1) and by showing that under each one of the aforementioned recurrency conditions, average return equilibrium policies appear as limit policies of sequences of discounted return equilibrium policies, with discount factor tending to one. Finally, we review and extend the results that are known for the case where both the state space and the action spaces are finite.

scholarly journals Characterization and simplification of optimal strategies in positive stochastic games

2018 ◽  
Vol 55 (3) ◽  
pp. 728-741 ◽  
Author(s):  
János Flesch ◽  
Arkadi Predtetchinski ◽  
William Sudderth
Keyword(s):  
State Space ◽  
Optimal Strategy ◽  
Stochastic Games ◽  
Optimal Strategies ◽  
Action Space ◽  
Positive Zero ◽  
Countable State ◽  
Zero Sum ◽  
Action Spaces

Abstract We consider positive zero-sum stochastic games with countable state and action spaces. For each player, we provide a characterization of those strategies that are optimal in every subgame. These characterizations are used to prove two simplification results. We show that if player 2 has an optimal strategy then he/she also has a stationary optimal strategy, and prove the same for player 1 under the assumption that the state space and player 2's action space are finite.

scholarly journals Infinitely divisible random transition probabilities with application to dependent Markov chains

1984 ◽  
Vol 25 (4) ◽  
pp. 463-472
Author(s):  
Peter L. Chesson
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
White Noise ◽  
Transition Probabilities ◽  
Transition Probability ◽  
Transition Rates ◽  
Infinitely Divisible ◽  
Random Transition ◽  
Transition Probability Matrices ◽  
Probability Matrices

AbstractRandom transition probability matrices with stationary independent factors define “white noise” environment processes for Markov chains. Two examples are considered in detail. Such environment processes can be used to construct several Markov chains which are dependent, have the same transition probabilities and are jointly a Markov chain. Transition rates for such processes are evaluated. These results have application to the study of animal movements.

Order and Homogeneity of Family Specific Brand-switching Processes

1966 ◽  
Vol 3 (1) ◽  
pp. 48-54 ◽  
Author(s):  
William F. Massy
Keyword(s):  
Markov Process ◽  
Markov Processes ◽  
Transition Probability ◽  
Empirical Work ◽  
Brand Choice ◽  
First Order ◽  
Brand Switching ◽  
Transition Probability Matrices ◽  
Probability Matrices ◽  
Crucial Assumption

Most empirical work on Markov processes for brand choice has been based on aggregative data. This article explores the validity of the crucial assumption that underlies such analyses, i.e., that all the families in the sample follow a Markov process with the same or similar transition probability matrices. The results show that there is a great deal of diversity among families’ switching processes, and that many of them are of zero rather than first order.

Transition probability matrices for correlated random walks

1980 ◽  
Vol 17 (01) ◽  
pp. 253-258 ◽  
Author(s):  
R. B. Nain ◽  
Kanwar Sen
Keyword(s):  
Random Walk ◽  
Random Walks ◽  
Difference Equations ◽  
Generating Functions ◽  
Transition Probability ◽  
Correlated Random Walk ◽  
One Dimensional ◽  
Correlated Random Walks ◽  
Transition Probability Matrices ◽  
Probability Matrices

For correlated random walks a method of transition probability matrices as an alternative to the much-used methods of probability generating functions and difference equations has been investigated in this paper. To illustrate the use of transition probability matrices for computing the various probabilities for correlated random walks, the transition probability matrices for restricted/unrestricted one-dimensional correlated random walk have been defined and used to obtain some of the probabilities.

scholarly journals 4525 Estimates of Dose Response using the Dixon Up-and-Down Method and BOIN study designs

2020 ◽  
Vol 4 (s1) ◽  
pp. 48-48
Author(s):  
Charles Gene Minard
Keyword(s):  
Decision Making ◽  
Standard Deviation ◽  
Effective Dose ◽  
Transition Probability ◽  
Experimental Unit ◽  
The Mean ◽  
Transition Probability Matrices ◽  
Probability Matrices ◽  
Study Designs ◽  
Dose Levels

OBJECTIVES/GOALS: The Dixon up-and-down method (U/D), original developed for testing explosives, is especially common in anesthesia research studies. The objective of this research is to compare the performance of the U/D method for obtaining and analyzing sensitivity data with that of the Bayesian Optimal Interval (BOIN) method. METHODS/STUDY POPULATION: A simulation study will compare the performance of the U/D method with the BOIN design. The two study designs offer alternative decision-making algorithms with respect to the dose at which the next experimental unit is treated. These alternative decisions may impact the precision of point estimates of the mean and standard deviation of the effective dose to elicit a response. Transition probability matrices are developed, and maximum likelihood estimates of the unknown parameters assessed for accuracy. For simulation, the effective dose is assumed to be randomly distributed with a known mean and standard deviation. Fixed dose levels are defined, and decisions for what level the next experimental unit should be treated at are defined by the Dixon up-and-down method and the BOIN design. For the U/D method, the stimulus is increased by one level in the absence of a response or decreased if a response occurs from an initial stimulus. A target toxicity probability of 0.50 is used to define the dose escalation or de-escalation rules for the application of the BOIN design. RESULTS/ANTICIPATED RESULTS: A feature of both methods is that the consecutive observations are concentrated about the mean value of the effective dose. However, the BOIN design tends to be more concentrated between these two dose levels. In the presence of severe adverse events, the BOIN design can choose to eliminate doses that are too toxic whereas the U/D design cannot eliminate any dose levels. Transition probability matrices are defined and parameters for the distribution of the effective dose are estimated using maximum likelihood estimation. Mean squared errors for the estimated mean and standard deviations compare the two study designs. DISCUSSION/SIGNIFICANCE OF IMPACT: The BOIN design offers an alternative method for decision-making compared with the U/D method. The BOIN design tends to concentrate dose levels about the mean more than the U/D. This may provide better estimates of the mean and standard deviation of the effective dose for eliciting a response in some circumstances.

scholarly journals The ENSO Transition Probabilities

2017 ◽  
Vol 30 (13) ◽  
pp. 4951-4964 ◽  
Author(s):  
G. Conti ◽  
A. Navarra ◽  
J. Tribbia
Keyword(s):  
General Circulation ◽  
Transition Probabilities ◽  
Transition Probability ◽  
General Circulation Models ◽  
Spring Predictability Barrier ◽  
Predictability Index ◽  
Sea Surface Temperature Anomalies ◽  
Transition Probability Matrices ◽  
Probability Matrices ◽  
Basic Prediction

ENSO is investigated here by considering it as a transition from different states. Transition probability matrices can be defined to describe the evolution of ENSO in this way. Sea surface temperature anomalies are classified into four categories, or states, and the probability to move from one state to another has been calculated for both observations and a simulation from a GCM. This could be useful for understanding and diagnosing general circulation models elucidating the mechanisms that govern ENSO in models. Furthermore, these matrices have been used to define a predictability index of ENSO based on the entropy concept introduced by Shannon. The index correctly identifies the emergence of the spring predictability barrier and the seasonal variations of the transition probabilities. The transition probability matrices could also be used to formulate a basic prediction model for ENSO that was tested here on a case study.

A multinomial model for transition probability matrices

1975 ◽  
Vol 12 (03) ◽  
pp. 498-506 ◽  
Author(s):  
G. G. S. Pegram
Keyword(s):  
Markov Chains ◽  
Conventional Method ◽  
Transition Probability ◽  
Multinomial Model ◽  
Arbitrary Size ◽  
Current Estimation ◽  
Transition Probability Matrices ◽  
Probability Matrices ◽  
Two Parameters

A model for the transition probability matrices (t.p.m.'s) of finite discrete Markov chains is suggested which may help those who wish to use a larger number of states than would seem reasonable with the data available in the current estimation situation. The model is especially useful in that a finite t.p.m. of arbitrary size can be specified by as few as two parameters. An example of the model's estimation and use is presented, showing it in a fair light in comparison with the conventional method of t.p.m. specification.

Sign in / Sign up

Export Citation Format

Share Document