A Discrete Bouncy Particle Sampler

Abstract Most Markov chain Monte Carlo methods operate in discrete time and are reversible with respect to the target probability. Nevertheless, it is now understood that the use of nonreversible Markov chains can be beneficial in many contexts. In particular, the recently-proposed bouncy particle sampler leverages a continuous-time and nonreversible Markov process and empirically shows state-of-the-art performances when used to explore certain probability densities; however, its implementation typically requires the computation of local upper bounds on the gradient of the log target density. We present the discrete bouncy particle sampler, a general algorithm based upon a guided random walk, a partial refreshment of direction, and a delayed-rejection step. We show that the bouncy particle sampler can be understood as a scaling limit of a special case of our algorithm. In contrast to the bouncy particle sampler, implementing the discrete bouncy particle sampler only requires point-wise evaluation of the target density and its gradient. We propose extensions of the basic algorithm for situations when the exact gradient of the target density is not available. In a Gaussian setting, we establish a scaling limit for the radial process as dimension increases to infinity. We leverage this result to obtain the theoretical efficiency of the discrete bouncy particle sampler as a function of the partial-refreshment parameter, which leads to a simple and robust tuning criterion. A further analysis in a more general setting suggests that this tuning criterion applies more generally. Theoretical and empirical efficiency curves are then compared for different targets and algorithm variations.

Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence ◽

Improved Strong Worst-case Upper Bounds for MDP Planning

10.24963/ijcai.2017/248 ◽

2017 ◽

Author(s):

Anchit Gupta ◽

Shivaram Kalyanakrishnan

Keyword(s):

Upper Bound ◽

General Setting ◽

Upper Bounds ◽

Policy Iteration ◽

Sequential Decision ◽

Markov Decision Problem ◽

Worst Case ◽

Running Time ◽

Markov Decision ◽

The Markov Decision Problem (MDP) plays a central role in AI as an abstraction of sequential decision making. We contribute to the theoretical analysis of MDP PLANNING, which is the problem of computing an optimal policy for a given MDP. Specifically, we furnish improved STRONG WORST-CASE upper bounds on the running time of MDP planning. Strong bounds are those that depend only on the number of states n and the number of actions k in the specified MDP; they have no dependence on affiliated variables such as the discount factor and the number of bits needed to represent the MDP. Worst-case bounds apply to EVERY run of an algorithm; randomised algorithms can typically yield faster EXPECTED running times. While the special case of 2-action MDPs (that is, k = 2) has recently received some attention, bounds for general k have remained to be improved for several decades. Our contributions are to this general case. For k >= 3, the tightest strong upper bound shown to date for MDP planning belongs to a family of algorithms called Policy Iteration. This bound is only a polynomial improvement over a trivial bound of poly(n, k) k^{n} [Mansour and Singh, 1999]. In this paper, we generalise a contrasting algorithm called the Fibonacci Seesaw, and derive a bound of poly(n, k) k^{0.6834n}. The key construct we use is a template to map algorithms for the 2-action setting to the general setting. Interestingly, this idea can also be used to design Policy Iteration algorithms with a running time upper bound of poly(n, k) k^{0.7207n}. Both our results improve upon bounds that have stood for several decades.

Bounding the Size and Probability of Epidemics on Networks

Journal of Applied Probability ◽

10.1239/jap/1214950363 ◽

2008 ◽

Vol 45 (2) ◽

pp. 498-512 ◽

Cited By ~ 25

Author(s):

Joel C. Miller

Keyword(s):

Infectious Disease ◽

Lower Bounds ◽

Upper Bounds ◽

Marginal Probability ◽

Transmission Properties ◽

Disease Spreading ◽

We consider an infectious disease spreading along the edges of a network which may have significant clustering. The individuals in the population have heterogeneous infectiousness and/or susceptibility. We define the out-transmissibility of a node to be the marginal probability that it would infect a randomly chosen neighbor given its infectiousness and the distribution of susceptibility. For a given distribution of out-transmissibility, we find the conditions which give the upper (or lower) bounds on the size and probability of an epidemic, under weak assumptions on the transmission properties, but very general assumptions on the network. We find similar bounds for a given distribution of in-transmissibility (the marginal probability of being infected by a neighbor). We also find conditions giving global upper bounds on the size and probability. The distributions leading to these bounds are network independent. In the special case of networks with high girth (locally tree-like), we are able to prove stronger results. In general, the probability and size of epidemics are maximal when the population is homogeneous and minimal when the variance of in- or out-transmissibility is maximal.

Sliding-Window Thompson Sampling for Non-Stationary Settings

Journal of Artificial Intelligence Research ◽

10.1613/jair.1.11407 ◽

2020 ◽

Vol 68 ◽

pp. 311-364

Author(s):

Francesco Trovo ◽

Stefano Paladino ◽

Marcello Restelli ◽

Nicola Gatti

Keyword(s):

Real World ◽

State Of The Art ◽

Sliding Window ◽

Upper Bounds ◽

Decision Problems ◽

Sequential Decision ◽

Thompson Sampling ◽

The Past ◽

Real World Applications ◽

Window Approach

Multi-Armed Bandit (MAB) techniques have been successfully applied to many classes of sequential decision problems in the past decades. However, non-stationary settings -- very common in real-world applications -- received little attention so far, and theoretical guarantees on the regret are known only for some frequentist algorithms. In this paper, we propose an algorithm, namely Sliding-Window Thompson Sampling (SW-TS), for nonstationary stochastic MAB settings. Our algorithm is based on Thompson Sampling and exploits a sliding-window approach to tackle, in a unified fashion, two different forms of non-stationarity studied separately so far: abruptly changing and smoothly changing. In the former, the reward distributions are constant during sequences of rounds, and their change may be arbitrary and happen at unknown rounds, while, in the latter, the reward distributions smoothly evolve over rounds according to unknown dynamics. Under mild assumptions, we provide regret upper bounds on the dynamic pseudo-regret of SW-TS for the abruptly changing environment, for the smoothly changing one, and for the setting in which both the non-stationarity forms are present. Furthermore, we empirically show that SW-TS dramatically outperforms state-of-the-art algorithms even when the forms of non-stationarity are taken separately, as previously studied in the literature.

On Some Issues in Shakedown Analysis

Journal of Applied Mechanics ◽

10.1115/1.1379368 ◽

2001 ◽

Vol 68 (5) ◽

pp. 799-808 ◽

Cited By ~ 35

Author(s):

G. Maier

Keyword(s):

Heterogeneous Media ◽

Direct Methods ◽

Cost Effective ◽

Upper Bounds ◽

Saturated Porous Media ◽

Safety Factors ◽

Shakedown Analysis ◽

Time Stepping ◽

Kinematic Theorem ◽

Shakedown analysis, and its more classical special case of limit analysis, basically consists of “direct” (as distinct from time-stepping) methods apt to assess safety factors for variable repeated external actions and procedures which provide upper bounds on history-dependent quantities. The issues reviewed and briefly discussed herein are: some recent engineering-oriented and cost-effective methods resting on Koiter’s kinematic theorem and applied to periodic heterogeneous media; recent extensions (after the earlier ones to dynamics and creep) to another area characterized by time derivatives, namely poroplasticity of fluid-saturated porous media. Links with some classical or more consolidated direct methods are pointed out.

Polarization of Separating Invariants

Canadian Journal of Mathematics ◽

10.4153/cjm-2008-027-2 ◽

2008 ◽

Vol 60 (3) ◽

pp. 556-571 ◽

Cited By ~ 20

Author(s):

Jan Draisma ◽

Gregor Kemper ◽

David Wehlau

Keyword(s):

Finite Group ◽

Finite Groups ◽

Group Actions ◽

Upper Bounds ◽

Weyl’S Theorem ◽

Finite Group Actions ◽

Separating Invariants ◽

The Difference ◽

AbstractWe prove a characteristic free version of Weyl’s theorem on polarization. Our result is an exact analogue ofWeyl’s theorem, the difference being that our statement is about separating invariants rather than generating invariants. For the special case of finite group actions we introduce the concept of cheap polarization, and show that it is enough to take cheap polarizations of invariants of just one copy of a representation to obtain separating vector invariants for any number of copies. This leads to upper bounds on the number and degrees of separating vector invariants of finite groups.

Proceedings of The Royal Society A Mathematical Physical and Engineering Sciences ◽

Stability and memory-loss go hand-in-hand: three results in dynamics and computation

10.1098/rspa.2020.0563 ◽

2020 ◽

Vol 476 (2242) ◽

pp. 20200563

Author(s):

G. Manjunath

Keyword(s):

State Of The Art ◽

Memory Loss ◽

General Setting ◽

Biologically Inspired ◽

Driven Systems ◽

Hardware Implementations ◽

Internal States ◽

Definition Of ◽

Necessary And Sufficient ◽

Dedicated Hardware

The search for universal laws that help establish a relationship between dynamics and computation is driven by recent expansionist initiatives in biologically inspired computing. A general setting to understand both such dynamics and computation is a driven dynamical system that responds to a temporal input. Surprisingly, we find memory-loss a feature of driven systems to forget their internal states helps provide unambiguous answers to the following fundamental stability questions that have been unanswered for decades: what is necessary and sufficient so that slightly different inputs still lead to mostly similar responses? How does changing the driven system’s parameters affect stability? What is the mathematical definition of the edge-of-criticality? We anticipate our results to be timely in understanding and designing biologically inspired computers that are entering an era of dedicated hardware implementations for neuromorphic computing and state-of-the-art reservoir computing applications.

Towards understanding residual and dilated dense neural networks via convolutional sparse coding

National Science Review ◽

10.1093/nsr/nwaa159 ◽

2020 ◽

Author(s):

Zhiyang Zhang ◽

Shihua Zhang

Keyword(s):

Neural Network ◽

Sparse Coding ◽

State Of The Art ◽

Theoretical Interpretation ◽

Theoretical Understanding ◽

Dilated Convolution ◽

Thresholding Algorithm ◽

Propagation Rule ◽

Special Case ◽

Iterative Soft Thresholding

Abstract Convolutional neural network (CNN) and its variants have led to many state-of-the-art results in various fields. However, a clear theoretical understanding of such networks is still lacking. Recently, a multilayer convolutional sparse coding (ML-CSC) model has been proposed and proved to equal such simply stacked networks (plain networks). Here, we consider the initialization, the dictionary design and the number of iterations to be factors in each layer that greatly affect the performance of the ML-CSC model. Inspired by these considerations, we propose two novel multilayer models: the residual convolutional sparse coding (Res-CSC) model and the mixed-scale dense convolutional sparse coding (MSD-CSC) model. They are closely related to the residual neural network (ResNet) and the mixed-scale (dilated) dense neural network (MSDNet), respectively. Mathematically, we derive the skip connection in the ResNet as a special case of a new forward propagation rule for the ML-CSC model. We also find a theoretical interpretation of dilated convolution and dense connection in the MSDNet by analyzing the MSD-CSC model, which gives a clear mathematical understanding of each. We implement the iterative soft thresholding algorithm and its fast version to solve the Res-CSC and MSD-CSC models. The unfolding operation can be employed for further improvement. Finally, extensive numerical experiments and comparison with competing methods demonstrate their effectiveness.

ON THE QUASI-STATIONARY DISTRIBUTION OF SIS MODELS

Probability in the Engineering and Informational Sciences ◽

10.1017/s0269964816000188 ◽

2016 ◽

Vol 30 (4) ◽

pp. 622-639 ◽

Author(s):

Gaofeng Da ◽

Maochao Xu ◽

Shouhuai Xu

Keyword(s):

Stationary Distribution ◽

Upper Bound ◽

Hazard Rate ◽

State Of The Art ◽

Upper Bounds ◽

Hazard Rate Order ◽

Reversed Hazard Rate ◽

Novel Method ◽

Better Than ◽

Quasi Stationary Distribution

In this paper, we propose a novel method for constructing upper bounds of the quasi-stationary distribution of SIS processes. Using this method, we obtain an upper bound that is better than the state-of-the-art upper bound. Moreover, we prove that the fixed point map Φ [7] actually preserves the equilibrium reversed hazard rate order under a certain condition. This allows us to further improve the upper bound. Some numerical results are presented to illustrate the results.

On a Counting Theorem for Weakly Admissible Lattices

International Mathematics Research Notices ◽

10.1093/imrn/rnaa102 ◽

2020 ◽

Author(s):

Reynold Fregoli

Keyword(s):

Diophantine Approximation ◽

General Setting ◽

Upper Bounds ◽

Lattice Points ◽

Precise Estimate ◽

Linear Subspaces ◽

Minimal Structure ◽

Partition Method

Abstract We give a precise estimate for the number of lattice points in certain bounded subsets of $\mathbb{R}^{n}$ that involve “hyperbolic spikes” and occur naturally in multiplicative Diophantine approximation. We use Wilkie’s o-minimal structure $\mathbb{R}_{\exp }$ and expansions thereof to formulate our counting result in a general setting. We give two different applications of our counting result. The 1st one establishes nearly sharp upper bounds for sums of reciprocals of fractional parts and thereby sheds light on a question raised by Lê and Vaaler, extending previous work of Widmer and of the author. The 2nd application establishes new examples of linear subspaces of Khintchine type thereby refining a theorem by Huang and Liu. For the proof of our counting result, we develop a sophisticated partition method that is crucial for further upcoming work on sums of reciprocals of fractional parts over distorted boxes.

ROUTING BALANCED COMMUNICATIONS ON HAMILTON DECOMPOSABLE NETWORKS

Parallel Processing Letters ◽

10.1142/s0129626404001969 ◽

2004 ◽

Vol 14 (03n04) ◽

pp. 377-385 ◽

Cited By ~ 2

Author(s):

LADISLAV STACHO ◽

JOZEF ŠIRÁŇ ◽

SANMING ZHOU

Keyword(s):

Wave Length ◽

Upper Bounds ◽

Ring Network ◽

Special Case ◽

Better Than

In [10] the authors proved upper bounds for the arc-congestion and wave-length number of any permutation demand on a bidirected ring. In this note, we give generalizations of their results in two directions. The first one is that instead of considering only permutation demands we consider any balanced demand, and the second one is that instead of the ring network we consider any Hamilton decomposable network. Thus, we obtain upper bounds (which are best possible in general) for the arc-congestion and wavelength number of any balanced demand on a Hamilton decomposable network. As a special case, we obtain upper bounds on arc- and edge-forwarding indices of Hamilton decomposable networks that are in many cases better than the known ones.