Large Deviations of Empirical Processes

AbstractWe study a model of active particles that perform a simple random walk and on top of that have a preferred direction determined by an internal state which is modelled by a stationary Markov process. First we calculate the limiting diffusion coefficient. Then we show that the ‘active part’ of the diffusion coefficient is in some sense maximal for reversible state processes. Further, we obtain a large deviations principle for the active particle in terms of the large deviations rate function of the empirical process corresponding to the state process. Again we show that the rate function and free energy function are (pointwise) optimal for reversible state processes. Finally, we show that in the case with two states, the Fourier–Laplace transform of the distribution, the moment generating function and the free energy function can be computed explicitly. Along the way we provide several examples.

Download Full-text

Weighted approximate Bayesian computation via Sanov’s theorem

Computational Statistics ◽

10.1007/s00180-021-01093-4 ◽

2021 ◽

Author(s):

Cecilia Viscardi ◽

Michele Boreale ◽

Fabio Corradi

Keyword(s):

Large Deviations ◽

Posterior Distribution ◽

Approximate Bayesian Computation ◽

Bayesian Computation ◽

Information Theoretic ◽

Discrete Random Variables ◽

Positive Weights ◽

Approximate Bayesian ◽

Information Theoretic Method ◽

Computational Resources

AbstractWe consider the problem of sample degeneracy in Approximate Bayesian Computation. It arises when proposed values of the parameters, once given as input to the generative model, rarely lead to simulations resembling the observed data and are hence discarded. Such “poor” parameter proposals do not contribute at all to the representation of the parameter’s posterior distribution. This leads to a very large number of required simulations and/or a waste of computational resources, as well as to distortions in the computed posterior distribution. To mitigate this problem, we propose an algorithm, referred to as the Large Deviations Weighted Approximate Bayesian Computation algorithm, where, via Sanov’s Theorem, strictly positive weights are computed for all proposed parameters, thus avoiding the rejection step altogether. In order to derive a computable asymptotic approximation from Sanov’s result, we adopt the information theoretic “method of types” formulation of the method of Large Deviations, thus restricting our attention to models for i.i.d. discrete random variables. Finally, we experimentally evaluate our method through a proof-of-concept implementation.

Download Full-text