Large deviations and the Bayesian estimation of higher-order Markov transition functions

1996 ◽  
Vol 33 (1) ◽  
pp. 18-27 ◽  
Author(s):  
F. Papangelou
Keyword(s):  
Large Deviations ◽  
Bayesian Estimation ◽  
Transition Function ◽  
Higher Order ◽  
Positive Probability ◽  
State Spaces ◽  
Empirical Distributions ◽  
Transition Functions ◽  
Markov Transition ◽  
Finite State

In the Bayesian estimation of higher-order Markov transition functions on finite state spaces, a prior distribution may assign positive probability to arbitrarily high orders. If there are n observations available, we show (for natural priors) that, with probability one, as n → ∞ the Bayesian posterior distribution ‘discriminates accurately' for orders up to β log n, if β is smaller than an explicitly determined β0. This means that the ‘large deviations' of the posterior are controlled by the relative entropies of the true transition function with respect to all others, much as the large deviations of the empirical distributions are governed by their relative entropies with respect to the true transition function. An example shows that the result can fail even for orders β log n if β is large.

Large deviations and the Bayesian estimation of higher-order Markov transition functions

1996 ◽  
Vol 33 (01) ◽  
pp. 18-27
Author(s):  
F. Papangelou
Keyword(s):  
Large Deviations ◽  
Bayesian Estimation ◽  
Transition Function ◽  
Higher Order ◽  
Positive Probability ◽  
State Spaces ◽  
Empirical Distributions ◽  
Transition Functions ◽  
Markov Transition ◽  
Finite State

In the Bayesian estimation of higher-order Markov transition functions on finite state spaces, a prior distribution may assign positive probability to arbitrarily high orders. If there are n observations available, we show (for natural priors) that, with probability one, as n → ∞ the Bayesian posterior distribution ‘discriminates accurately' for orders up to β log n, if β is smaller than an explicitly determined β 0. This means that the ‘large deviations' of the posterior are controlled by the relative entropies of the true transition function with respect to all others, much as the large deviations of the empirical distributions are governed by their relative entropies with respect to the true transition function. An example shows that the result can fail even for orders β log n if β is large.

The Computation of Average Optimal Policies in Denumerable State Markov Decision Chains

1997 ◽  
Vol 29 (01) ◽  
pp. 114-137
Author(s):  
Linn I. Sennott
Keyword(s):  
Discrete Time ◽  
Average Cost ◽  
Queueing Systems ◽  
State Spaces ◽  
Original Process ◽  
Optimal Policies ◽  
Finite State ◽  
Markov Decision ◽  
Optimal Average ◽  
Infinite State

This paper studies the expected average cost control problem for discrete-time Markov decision processes with denumerably infinite state spaces. A sequence of finite state space truncations is defined such that the average costs and average optimal policies in the sequence converge to the optimal average cost and an optimal policy in the original process. The theory is illustrated with several examples from the control of discrete-time queueing systems. Numerical results are discussed.

Completion Grammars and Completion Automata Revisited

1977 ◽  
Vol 37 ◽  
pp. 19-40
Author(s):  
Luc Steels
Keyword(s):  
Language Processing ◽  
Word Order ◽  
Generative Capacity ◽  
New Class ◽  
Automatic Translation ◽  
Linguistic Model ◽  
Transition Functions ◽  
Finite State ◽  
Formal Properties

An extension of completion grammars is being introduced such that the model now deals with prefix, infix, postfix and post-infix word order patterns. It is shown that this extension does not affect the weak generative capacity of the system, which was known to be of type 2. Also the existing notion of a completion automaton is reworked, mainly to have the distinction in word order be reflected by the operations of the automaton rather than by the transition functions of the underlying finite state machine. In some recent publications (e.g. Steels (1975), Steels and Vermeir (1976), Steels (1976a&b» we have been dealing with a linguistic model known as compZetion grammars. These grammars were designed to cope with a functional viewpoint on language, this means they deal with case structures for language,expressions, instead of phrase structures as do the well-known Chomsky-type grammars. The model of completion grammars was developed in a context of research on language processing and automatic translation. In particular it reflects the current tendency to build semantics directed systems, rather than syntax directed ones. (See for a more detailed discussion on the distinction between the two Wilks (1975) and Winograd (1973). For the use of completion grammars in the design of semantics directed systems, we refer to Steels (1975;1976a&b). What will concern us in this paper is an extension of the model, and a study of the formal properties of these extended systems. Also we will introduce a new class of automata. The paper is organized as follows. First we extend the notion of a completion grammar, we give some intuitive explanations for the extension (1.1.), specify the basic definitions (1.2.) and study its weak generative capacity (1.3.). A second section deals with the automata. Again we start with intuitive explanations (2.1.), give the basic definitions and various examples (2.2.) and finally prove the relation between the grammars and the automata (2.3.).

scholarly journals Multi-Zone Unit for Recurrent Neural Networks

2020 ◽  
Vol 34 (04) ◽  
pp. 5150-5157
Author(s):  
Fandong Meng ◽  
Jinchao Zhang ◽  
Yang Liu ◽  
Jie Zhou
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Critical Role ◽  
Transition Function ◽  
Learning Problems ◽  
Arbitrary Length ◽  
Transition Functions ◽  
Multiple Datasets ◽  
Analysis Task ◽  
Hidden States

Recurrent neural networks (RNNs) have been widely used to deal with sequence learning problems. The input-dependent transition function, which folds new observations into hidden states to sequentially construct fixed-length representations of arbitrary-length sequences, plays a critical role in RNNs. Based on single space composition, transition functions in existing RNNs often have difficulty in capturing complicated long-range dependencies. In this paper, we introduce a new Multi-zone Unit (MZU) for RNNs. The key idea is to design a transition function that is capable of modeling multiple space composition. The MZU consists of three components: zone generation, zone composition, and zone aggregation. Experimental results on multiple datasets of the character-level language modeling task and the aspect-based sentiment analysis task demonstrate the superiority of the MZU.

scholarly journals On the total reward variance for continuous-time Markov reward chains

2006 ◽  
Vol 43 (04) ◽  
pp. 1044-1052 ◽  
Author(s):  
Nico M. Van Dijk ◽  
Karel Sladký
Keyword(s):  
Growth Rate ◽  
Discrete Time ◽  
Continuous Time ◽  
Asymptotically Linear ◽  
State Spaces ◽  
Total Reward ◽  
Finite State ◽  
Markov Reward ◽  
Discrete Time Case

As an extension of the discrete-time case, this note investigates the variance of the total cumulative reward for continuous-time Markov reward chains with finite state spaces. The results correspond to discrete-time results. In particular, the variance growth rate is shown to be asymptotically linear in time. Expressions are provided to compute this growth rate.

scholarly journals On the limiting spectral density of random matrices filled with stochastic processes

2019 ◽  
Vol 27 (2) ◽  
pp. 89-105 ◽  
Author(s):  
Matthias Löwe ◽  
Kristina Schubert
Keyword(s):  
Stochastic Process ◽  
Spectral Density ◽  
Random Matrices ◽  
Markov Processes ◽  
Random Matrix ◽  
Matrix Theory ◽  
Gibbs Measures ◽  
State Spaces ◽  
The Matrix ◽  
Finite State

Abstract We discuss the limiting spectral density of real symmetric random matrices. In contrast to standard random matrix theory, the upper diagonal entries are not assumed to be independent, but we will fill them with the entries of a stochastic process. Under assumptions on this process which are satisfied, e.g., by stationary Markov chains on finite sets, by stationary Gibbs measures on finite state spaces, or by Gaussian Markov processes, we show that the limiting spectral distribution depends on the way the matrix is filled with the stochastic process. If the filling is in a certain way compatible with the symmetry condition on the matrix, the limiting law of the empirical eigenvalue distribution is the well-known semi-circle law. For other fillings we show that the semi-circle law cannot be the limiting spectral density.

scholarly journals A METHOD FOR INFERRING HIERARCHICAL DYNAMICS IN STOCHASTIC PROCESSES

2008 ◽  
Vol 11 (01) ◽  
pp. 1-16 ◽  
Author(s):  
OLOF GÖRNERUP ◽  
MARTIN NILSSON JACOBI
Keyword(s):  
Time Series ◽  
Stochastic Processes ◽  
A Priori ◽  
Main Idea ◽  
A Priori Knowledge ◽  
Finite State Automaton ◽  
State Spaces ◽  
Markovian Dynamics ◽  
Finite State ◽  
Operational Algorithm

Complex systems may often be characterized by their hierarchical dynamics. In this paper we present a method and an operational algorithm that automatically infer this property in a broad range of systems — discrete stochastic processes. The main idea is to systematically explore the set of projections from the state space of a process to smaller state spaces, and to determine which of the projections impose Markovian dynamics on the coarser level. These projections, which we call Markov projections, then constitute the hierarchical dynamics of the system. The algorithm operates on time series or other statistics, so a priori knowledge of the intrinsic workings of a system is not required in order to determine its hierarchical dynamics. We illustrate the method by applying it to two simple processes — a finite state automaton and an iterated map.

Sign in / Sign up

Export Citation Format

Share Document