Asymptotics of quasi-stationary distributions of small noise stochastic dynamical systems in unbounded domains

We consider a collection of Markov chains that model the evolution of multitype biological populations. The state space of the chains is the positive orthant, and the boundary of the orthant is the absorbing state for the Markov chain and represents the extinction states of different population types. We are interested in the long-term behavior of the Markov chain away from extinction, under a small noise scaling. Under this scaling, the trajectory of the Markov process over any compact interval converges in distribution to the solution of an ordinary differential equation (ODE) evolving in the positive orthant. We study the asymptotic behavior of the quasi-stationary distributions (QSD) in this scaling regime. Our main result shows that, under conditions, the limit points of the QSD are supported on the union of interior attractors of the flow determined by the ODE. We also give lower bounds on expected extinction times which scale exponentially with the system size. Results of this type when the deterministic dynamical system obtained under the scaling limit is given by a discrete-time evolution equation and the dynamics are essentially in a compact space (namely, the one-step map is a bounded function) have been studied by Faure and Schreiber (2014). Our results extend these to a setting of an unbounded state space and continuous-time dynamics. The proofs rely on uniform large deviation results for small noise stochastic dynamical systems and methods from the theory of continuous-time dynamical systems. In general, QSD for Markov chains with absorbing states and unbounded state spaces may not exist. We study one basic family of binomial-Poisson models in the positive orthant where one can use Lyapunov function methods to establish existence of QSD and also to argue the tightness of the QSD of the scaled sequence of Markov chains. The results from the first part are then used to characterize the support of limit points of this sequence of QSD.

Robust Assembly Assistance Using Informed Tree Search with Markov Chains

Manual work accounts for one of the largest workgroups in the European manufacturing sector, and improving the training capacity, quality, and speed brings significant competitive benefits to companies. In this context, this paper presents an informed tree search on top of a Markov chain that suggests possible next assembly steps as a key component of an innovative assembly training station for manual operations. The goal of the next step suggestions is to provide support to inexperienced workers or to assist experienced workers by providing choices for the next assembly step in an automated manner without the involvement of a human trainer on site. Data stemming from 179 experiment participants, 111 factory workers, and 68 students, were used to evaluate different prediction methods. From our analysis, Markov chains fail in new scenarios and, therefore, by using an informed tree search to predict the possible next assembly step in such situations, the prediction capability of the hybrid algorithm increases significantly while providing robust solutions to unseen scenarios. The proposed method proved to be the most efficient for next assembly step prediction among all the evaluated predictors and, thus, the most suitable method for an adaptive assembly support system such as for manual operations in industry.

Nonreversible Markov Chain Monte Carlo Algorithm for Efficient Generation of Self-Avoiding Walks

We introduce an efficient nonreversible Markov chain Monte Carlo algorithm to generate self-avoiding walks with a variable endpoint. In two dimensions, the new algorithm slightly outperforms the two-move nonreversible Berretti-Sokal algorithm introduced by H. Hu, X. Chen, and Y. Deng, while for three-dimensional walks, it is 3–5 times faster. The new algorithm introduces nonreversible Markov chains that obey global balance and allow for three types of elementary moves on the existing self-avoiding walk: shorten, extend or alter conformation without changing the length of the walk.

Methodology for the Assessment of Imprecise Multi-State System Availability

Most existing studies of a system’s availability in the presence of epistemic uncertainties assume that the system is binary. In this paper, a new methodology for the estimation of the availability of multi-state systems is developed, taking into consideration epistemic uncertainties. This paper formulates a combined approach, based on continuous Markov chains and interval contraction methods, to address the problem of computing the availability of multi-state systems with imprecise failure and repair rates. The interval constraint propagation method, which we refer to as the forward–backward propagation (FBP) contraction method, allows us to contract the probability intervals, keeping all the values that may be consistent with the set of constraints. This methodology is guaranteed, and several numerical examples of systems with complex architectures are studied.