scholarly journals Breaking the Curse of Dimension in Multi-Marginal Kantorovich Optimal Transport on Finite State Spaces

2018 ◽  
Vol 50 (4) ◽  
pp. 3996-4019 ◽  
Author(s):  
Gero Friesecke ◽  
Daniela Vögler
Keyword(s):  
Optimal Transport ◽  
State Spaces ◽  
Curse Of Dimension ◽  
Finite State

The Computation of Average Optimal Policies in Denumerable State Markov Decision Chains

1997 ◽  
Vol 29 (01) ◽  
pp. 114-137
Author(s):  
Linn I. Sennott
Keyword(s):  
Discrete Time ◽  
Average Cost ◽  
Queueing Systems ◽  
State Spaces ◽  
Original Process ◽  
Optimal Policies ◽  
Finite State ◽  
Markov Decision ◽  
Optimal Average ◽  
Infinite State

This paper studies the expected average cost control problem for discrete-time Markov decision processes with denumerably infinite state spaces. A sequence of finite state space truncations is defined such that the average costs and average optimal policies in the sequence converge to the optimal average cost and an optimal policy in the original process. The theory is illustrated with several examples from the control of discrete-time queueing systems. Numerical results are discussed.

scholarly journals On the total reward variance for continuous-time Markov reward chains

2006 ◽  
Vol 43 (04) ◽  
pp. 1044-1052 ◽  
Author(s):  
Nico M. Van Dijk ◽  
Karel Sladký
Keyword(s):  
Growth Rate ◽  
Discrete Time ◽  
Continuous Time ◽  
Asymptotically Linear ◽  
State Spaces ◽  
Total Reward ◽  
Finite State ◽  
Markov Reward ◽  
Discrete Time Case

As an extension of the discrete-time case, this note investigates the variance of the total cumulative reward for continuous-time Markov reward chains with finite state spaces. The results correspond to discrete-time results. In particular, the variance growth rate is shown to be asymptotically linear in time. Expressions are provided to compute this growth rate.

scholarly journals On the limiting spectral density of random matrices filled with stochastic processes

2019 ◽  
Vol 27 (2) ◽  
pp. 89-105 ◽  
Author(s):  
Matthias Löwe ◽  
Kristina Schubert
Keyword(s):  
Stochastic Process ◽  
Spectral Density ◽  
Random Matrices ◽  
Markov Processes ◽  
Random Matrix ◽  
Matrix Theory ◽  
Gibbs Measures ◽  
State Spaces ◽  
The Matrix ◽  
Finite State

Abstract We discuss the limiting spectral density of real symmetric random matrices. In contrast to standard random matrix theory, the upper diagonal entries are not assumed to be independent, but we will fill them with the entries of a stochastic process. Under assumptions on this process which are satisfied, e.g., by stationary Markov chains on finite sets, by stationary Gibbs measures on finite state spaces, or by Gaussian Markov processes, we show that the limiting spectral distribution depends on the way the matrix is filled with the stochastic process. If the filling is in a certain way compatible with the symmetry condition on the matrix, the limiting law of the empirical eigenvalue distribution is the well-known semi-circle law. For other fillings we show that the semi-circle law cannot be the limiting spectral density.

scholarly journals A METHOD FOR INFERRING HIERARCHICAL DYNAMICS IN STOCHASTIC PROCESSES

2008 ◽  
Vol 11 (01) ◽  
pp. 1-16 ◽  
Author(s):  
OLOF GÖRNERUP ◽  
MARTIN NILSSON JACOBI
Keyword(s):  
Time Series ◽  
Stochastic Processes ◽  
A Priori ◽  
Main Idea ◽  
A Priori Knowledge ◽  
Finite State Automaton ◽  
State Spaces ◽  
Markovian Dynamics ◽  
Finite State ◽  
Operational Algorithm

Complex systems may often be characterized by their hierarchical dynamics. In this paper we present a method and an operational algorithm that automatically infer this property in a broad range of systems — discrete stochastic processes. The main idea is to systematically explore the set of projections from the state space of a process to smaller state spaces, and to determine which of the projections impose Markovian dynamics on the coarser level. These projections, which we call Markov projections, then constitute the hierarchical dynamics of the system. The algorithm operates on time series or other statistics, so a priori knowledge of the intrinsic workings of a system is not required in order to determine its hierarchical dynamics. We illustrate the method by applying it to two simple processes — a finite state automaton and an iterated map.

scholarly journals The morphological state space revisited: what do phylogenetic patterns in homoplasy tell us about the number of possible character states?

2015 ◽  
Vol 5 (6) ◽  
pp. 20150049 ◽  
Author(s):  
Jennifer F. Hoyal Cuthill
Keyword(s):  
State Space ◽  
Morphological Evolution ◽  
Character Evolution ◽  
Character State ◽  
State Spaces ◽  
Phylogenetic Constraints ◽  
Finite State ◽  
Finite States ◽  
Close Relatives ◽  
Morphological State

Biological variety and major evolutionary transitions suggest that the space of possible morphologies may have varied among lineages and through time. However, most models of phylogenetic character evolution assume that the potential state space is finite. Here, I explore what the morphological state space might be like, by analysing trends in homoplasy (repeated derivation of the same character state). Analyses of ten published character matrices are compared against computer simulations with different state space models: infinite states, finite states, ordered states and an ‘inertial' model, simulating phylogenetic constraints. Of these, only the infinite states model results in evolution without homoplasy, a prediction which is not generally met by real phylogenies. Many authors have interpreted the ubiquity of homoplasy as evidence that the number of evolutionary alternatives is finite. However, homoplasy is also predicted by phylogenetic constraints on the morphological distance that can be traversed between ancestor and descendent. Phylogenetic rarefaction (sub-sampling) shows that finite and inertial state spaces do produce contrasting trends in the distribution of homoplasy. Two clades show trends characteristic of phylogenetic inertia, with decreasing homoplasy (increasing consistency index) as we sub-sample more distantly related taxa. One clade shows increasing homoplasy, suggesting exhaustion of finite states. Different clades may, therefore, show different patterns of character evolution. However, when parsimony uninformative characters are excluded (which may occur without documentation in cladistic studies), it may no longer be possible to distinguish inertial and finite state spaces. Interestingly, inertial models predict that homoplasy should be clustered among comparatively close relatives (parallel evolution), whereas finite state models do not. If morphological evolution is often inertial in nature, then homoplasy (false homology) may primarily occur between close relatives, perhaps being replaced by functional analogy at higher taxonomic scales.

Large deviations and the Bayesian estimation of higher-order Markov transition functions

1996 ◽  
Vol 33 (1) ◽  
pp. 18-27 ◽  
Author(s):  
F. Papangelou
Keyword(s):  
Large Deviations ◽  
Bayesian Estimation ◽  
Transition Function ◽  
Higher Order ◽  
Positive Probability ◽  
State Spaces ◽  
Empirical Distributions ◽  
Transition Functions ◽  
Markov Transition ◽  
Finite State

In the Bayesian estimation of higher-order Markov transition functions on finite state spaces, a prior distribution may assign positive probability to arbitrarily high orders. If there are n observations available, we show (for natural priors) that, with probability one, as n → ∞ the Bayesian posterior distribution ‘discriminates accurately' for orders up to β log n, if β is smaller than an explicitly determined β0. This means that the ‘large deviations' of the posterior are controlled by the relative entropies of the true transition function with respect to all others, much as the large deviations of the empirical distributions are governed by their relative entropies with respect to the true transition function. An example shows that the result can fail even for orders β log n if β is large.

scholarly journals A comparative analysis of the successive lumping and the lattice path counting algorithms

2016 ◽  
Vol 53 (1) ◽  
pp. 106-120 ◽  
Author(s):  
Michael N. Katehakis ◽  
Laurens C. Smit ◽  
Floske M. Spieksma
Keyword(s):  
Lattice Path ◽  
Queueing Models ◽  
Rate Matrix ◽  
Steady State Distribution ◽  
State Spaces ◽  
Finite State ◽  
Counting Algorithms ◽  
Path Counting ◽  
Lattice Path Counting ◽  
Numerical Complexity

Abstract This paper provides a comparison of the successive lumping (SL) methodology developed in Katehakis et al. (2015) with the popular lattice path counting (Mohanty (1979)) in obtaining rate matrices for queueing models, satisfying the specific quasi birth and death structure as in Van Leeuwaarden et al. (2009) and Van Leeuwaarden and Winands (2006). The two methodologies are compared both in terms of applicability requirements and numerical complexity by analyzing their performance for the same classical queueing models considered in Van Leeuwaarden et al. (2009). The main findings are threefold. First, when both methods are applicable, the SL-based algorithms outperform the lattice path counting algorithm (LPCA). Second, there are important classes of problems (for example, models with (level) nonhomogenous rates or with finite state spaces) for which the SL methodology is applicable and for which the LPCA cannot be used. Third, another main advantage of SL algorithms over lattice path counting is that the former includes a method to compute the steady state distribution using this rate matrix.

Sign in / Sign up

Export Citation Format

Share Document