Reconstruction of transition probabilities by maximum entropy in the mean

2002 ◽  
Author(s):  
Henryk Gzyl
1990 ◽  
Vol 27 (01) ◽  
pp. 134-145
Author(s):  
Matthias Fassbender

This paper establishes the existence of an optimal stationary strategy in a leavable Markov decision process with countable state space and undiscounted total reward criterion. Besides assumptions of boundedness and continuity, an assumption is imposed on the model which demands the continuity of the mean recurrence times on a subset of the stationary strategies, the so-called ‘good strategies'. For practical applications it is important that this assumption is implied by an assumption about the cost structure and the transition probabilities. In the last part we point out that our results in general cannot be deduced from related works on bias-optimality by Dekker and Hordijk, Wijngaard or Mann.


2020 ◽  
Author(s):  
Gabriel Rioux ◽  
Rustum Choksi ◽  
Tim Hoheisel ◽  
Pierre Marechal ◽  
Christopher Scarvelis

1991 ◽  
Vol 23 (04) ◽  
pp. 667-682
Author(s):  
J. M. McNamara ◽  
S. Merad ◽  
E. J. Collins

This paper considers a version of the Hawk–Dove game of Maynard Smith and Price (1973) in which animals compete for a sequence of food items. Actions may depend on an animal's energy reserves. Costs and transition probabilities under a given policy depend on the mean level of aggressiveness, p, of the rest of the population. We find the optimal policy for a single animal under an average cost criterion when ρ is constant over time. We then consider the whole interacting population when individual members follow the same stationary policy. It is shown that the mean aggressiveness, p, asymptotically approaches a limiting value in this population. We then consider the existence of evolutionarily stable strategies for the population. It is shown that such strategies always exist but may not be unique.


1987 ◽  
Vol 1 (3) ◽  
pp. 251-264 ◽  
Author(s):  
Sheldon M. Ross

In this paper we propose a new approach for estimating the transition probabilities and mean occupation times of continuous-time Markov chains. Our approach is to approximate the probability of being in a state (or the mean time already spent in a state) at time t by the probability of being in that state (or the mean time already spent in that state) at a random time that is gamma distributed with mean t.


1990 ◽  
Vol 27 (1) ◽  
pp. 134-145
Author(s):  
Matthias Fassbender

This paper establishes the existence of an optimal stationary strategy in a leavable Markov decision process with countable state space and undiscounted total reward criterion.Besides assumptions of boundedness and continuity, an assumption is imposed on the model which demands the continuity of the mean recurrence times on a subset of the stationary strategies, the so-called ‘good strategies'. For practical applications it is important that this assumption is implied by an assumption about the cost structure and the transition probabilities. In the last part we point out that our results in general cannot be deduced from related works on bias-optimality by Dekker and Hordijk, Wijngaard or Mann.


Hydrology ◽  
2020 ◽  
Vol 7 (1) ◽  
pp. 15
Author(s):  
Paolo Fabbri ◽  
Carlo Gaetan ◽  
Luca Sartore ◽  
Nico Dalla Libera

The reconstruction of hydro-stratigraphic units in subsoil (a general term indicating all the materials below ground level) plays an important role in the assessment of soil heterogeneity, which is a keystone in groundwater flow and transport modeling. A geostatistical approach appears to be a good way to reconstruct subsoil, and now other methods besides the classical indicator (co)kriging are available as alternative approximations of the conditional probabilities. Some of these techniques take specifically into account categorical variables as lithologies, but they are computationally prohibitive. Moreover, the stage before subsoil prediction/simulation can be very informative from a hydro-stratigraphic point of view, as the detailed transiogram analysis of this paper demonstrates. In this context, an application of the spMC package for the R software is presented by using a test site located within the Venetian alluvial plain (NE Italy). First, a detailed transiogram analysis was conducted, and then a maximum entropy approach, based on transition probabilities, named Markovian-type Categorical Prediction (MCP), was applied to approximate the posterior conditional probabilities. The study highlights some advantages of the presented approach in term of hydrogeological knowledge and computational efficiency. The spMC package couples transiogram analysis with a maximum entropy approach by taking advantage of High-Performance Computing (HPC) techniques. These characteristics make the spMC package useful for simulating hydro-stratigraphic units in subsoil, despite the use of a large number of lithologies (categories). The results obtained by spMC package suggest that this software should be considered a good candidate for simulating subsoil lithological distributions, especially of limited areas.


1988 ◽  
Vol 43 (1) ◽  
pp. 73-77
Author(s):  
G. L. Hofacker ◽  
R. D. Levine

Abstract A principle of evolution of highly complex systems is proposed. It is based on extremal properties of the information I (X, Y) characterizing two states X and Y with respect to each other, I(X, Y) = H(Y) -H(Y/X), where H(Y) is the entropy of state Y,H (Y/X) the entropy in state Y given the probability distribu­tion P(X) and transition probabilities P(Y/X).As I(X, Y) is maximal in P(Y) but minimal in P(Y/X), the extremal properties of I(X, Y) con­stitute a principle superior to the maximum entropy principle while containing the latter as a special case. The principle applies to complex systems evolving with time where fundamental equations are unknown or too difficult to solve. For the case of a system evolving from X to Y it is shown that the principle predicts a canonic distribution for a state Y with a fixed average energy .


Sign in / Sign up

Export Citation Format

Share Document