Approximating quasi-stationary distributions with interacting reinforced random walks

We propose two numerical schemes for approximating quasi-stationary distributions (QSD) of finite state Markov chains with absorbing states. Both schemes are described in terms of interacting chains where the interaction is given in terms of the total time occupation measure of all particles in the system and has the impact of reinforcing transitions, in an appropriate fashion, to states where the collection of particles has spent more time. The schemes can be viewed as combining the key features of the two basic simulation-based methods for approximating QSD originating from the works of Fleming and Viot (1979) and Aldous, Flannery and Palacios (1998), respectively. The key difference between the two schemes studied here is that in the first method one starts with $a(n)$ particles at time $0$ and number of particles stays constant over time whereas in the second method we start with one particle and at most one particle is added at each time instant in such a manner that there are $a(n)$ particles at time $n$. We prove almost sure convergence to the unique QSD and establish Central Limit Theorems for the two schemes under the key assumption that $a(n)=o(n)$. Exploratory numerical results are presented to illustrate the performance.

Convergence theorem of Pettis integrable multivalued pramart

PurposeIn this work, the authors are interested in the notion of vector valued and set valued Pettis integrable pramarts. The notion of pramart is more general than that of martingale. Every martingale is a pramart, but the converse is not generally true.Design/methodology/approachIn this work, the authors present several properties and convergence theorems for Pettis integrable pramarts with convex weakly compact values in a separable Banach space.FindingsThe existence of the conditional expectation of Pettis integrable mutifunctions indexed by bounded stopping times is provided. The authors prove the almost sure convergence in Mosco and linear topologies of Pettis integrable pramarts with values in (cwk(E)) the family of convex weakly compact subsets of a separable Banach space.Originality/valueThe purpose of the present paper is to present new properties and various new convergence results for convex weakly compact valued Pettis integrable pramarts in Banach space.

The Rescaled Pólya Urn and the Wright–Fisher Process with Mutation

In recent papers the authors introduce, study and apply a variant of the Eggenberger–Pólya urn, called the “rescaled” Pólya urn, which, for a suitable choice of the model parameters, exhibits a reinforcement mechanism mainly based on the last observations, a random persistent fluctuation of the predictive mean and the almost sure convergence of the empirical mean to a deterministic limit. In this work, motivated by some empirical evidence, we show that the multidimensional Wright–Fisher diffusion with mutation can be obtained as a suitable limit of the predictive means associated to a family of rescaled Pólya urns.

Convergence in Lp Norm

This chapter looks in detail at proofs of the weak law of large numbers (convergence in probability) using the technique of establishing convergence in Lp‐norm. The extension to a proof of almost‐sure convergence is given, and then special results for martingale differences, mixingales, and approximable processes. These results are proved in array notation to allow general forms of heterogeneity.

Stochastic Convergence

The modes of convergence introduced in Chapter 12 are studied in detail. Conditions for almost‐sure convergence are derived via the Borel–Cantelli lemma. Convergence in probability is contrasted, and then a number of results for convergence of transformed series are given. Convergence in LP‐norm is introduced as a sufficient condition for convergence in probability. Examples are given, and the chapter concludes with a preliminary look at the laws of large numbers.

An approximate Taylor method for Stochastic Functional Differential Equations via polynomial condition

The subject of this paper is an analytic approximate method for a class of stochastic functional differential equations with coefficients that do not necessarily satisfy the Lipschitz condition nor linear growth condition but they satisfy some polynomial conditions. Also, equations from the observed class have unique solutions with bounded moments. Approximate equations are defined on partitions of the time interval and their drift and diffusion coefficients are Taylor approximations of the coefficients of the initial equation. Taylor approximations require Fréchet derivatives since the coefficients of the initial equation are functionals. The main results of this paper are the Lp and almost sure convergence of the sequence of the approximate solutions to the exact solution of the initial equation. An example that illustrates the theoretical results and contains the proof of the existence, uniqueness and moment boundedness of the approximate solution is displayed.

Inexact and Stochastic Generalized Conditional Gradient with Augmented Lagrangian and Proximal Step

In this paper we propose and analyze inexact and stochastic versions of the CGALP algorithm developed in [25], which we denote ICGALP , that allow for errors in the computation of several important quantities. In particular this allows one to compute some gradients, proximal terms, and/or linear minimization oracles in an inexact fashion that facilitates the practical application of the algorithm to computationally intensive settings, e.g., in high (or possibly infinite) dimensional Hilbert spaces commonly found in machine learning problems. The algorithm is able to solve composite minimization problems involving the sum of three convex proper lower-semicontinuous functions subject to an affine constraint of the form Ax = b for some bounded linear operator A. Only one of the functions in the objective is assumed to be differentiable, the other two are assumed to have an accessible proximal operator and a linear minimization oracle. As main results, we show convergence of the Lagrangian values (so-called convergence in the Bregman sense) and asymptotic feasibility of the affine constraint as well as strong convergence of the sequence of dual variables to a solution of the dual problem, in an almost sure sense. Almost sure convergence rates are given for the Lagrangian values and the feasibility gap for the ergodic primal variables. Rates in expectation are given for the Lagrangian values and the feasibility gap subsequentially in the pointwise sense. Numerical experiments verifying the predicted rates of convergence are shown as well.

Forward-reflected-backward method with variance reduction

We propose a variance reduced algorithm for solving monotone variational inequalities. Without assuming strong monotonicity, cocoercivity, or boundedness of the domain, we prove almost sure convergence of the iterates generated by the algorithm to a solution. In the monotone case, the ergodic average converges with the optimal O(1/k) rate of convergence. When strong monotonicity is assumed, the algorithm converges linearly, without requiring the knowledge of strong monotonicity constant. We finalize with extensions and applications of our results to monotone inclusions, a class of non-monotone variational inequalities and Bregman projections.