Annealing evolutionary stochastic approximation Monte Carlo for global optimization

In this paper we establish the theory of weak convergence (toward a normal distribution) for both single-chain and population stochastic approximation Markov chain Monte Carlo (MCMC) algorithms (SAMCMC algorithms). Based on the theory, we give an explicit ratio of convergence rates for the population SAMCMC algorithm and the single-chain SAMCMC algorithm. Our results provide a theoretic guarantee that the population SAMCMC algorithms are asymptotically more efficient than the single-chain SAMCMC algorithms when the gain factor sequence decreases slower than O(1 / t), where t indexes the number of iterations. This is of interest for practical applications.

Download Full-text

Parallel MCMC methods for global optimization

Monte Carlo Methods and Applications ◽

10.1515/mcma-2019-2043 ◽

2019 ◽

Vol 25 (3) ◽

pp. 227-237

Author(s):

Lihao Zhang ◽

Zeyang Ye ◽

Yuefan Deng

Keyword(s):

Monte Carlo ◽

Global Optimization ◽

Statistical Theory ◽

Simulated Annealing ◽

Parallel Computers ◽

Mcmc Methods ◽

Benchmark Function ◽

Mcmc Method ◽

Validation And Verification ◽

Parallel Scheme

Abstract We introduce a parallel scheme for simulated annealing, a widely used Markov chain Monte Carlo (MCMC) method for optimization. Our method is constructed and analyzed under the classical framework of MCMC. The benchmark function for optimization is used for validation and verification of the parallel scheme. The experimental results, along with the proof based on statistical theory, provide us with insights into the mechanics of the parallelization of simulated annealing for high parallel efficiency or scalability for large parallel computers.

Download Full-text

Asymptotic Global Behavior for Stochastic Approximation and Diffusions with Slowly Decreasing Noise Effects: Global Minimization via Monte Carlo

SIAM Journal on Applied Mathematics ◽

10.1137/0147010 ◽

1987 ◽

Vol 47 (1) ◽

pp. 169-185 ◽

Cited By ~ 91

Author(s):

H. J. Kushner

Keyword(s):

Monte Carlo ◽

Stochastic Approximation ◽

Global Behavior ◽

Global Minimization ◽

Noise Effects

Download Full-text

Mixed Efficient Global Optimization for Time-Dependent Reliability Analysis

Journal of Mechanical Design ◽

10.1115/1.4029520 ◽

2015 ◽

Vol 137 (5) ◽

Cited By ~ 87

Author(s):

Zhen Hu ◽

Xiaoping Du

Keyword(s):

Monte Carlo Simulation ◽

Monte Carlo ◽

Global Optimization ◽

Reliability Analysis ◽

Surrogate Model ◽

Extreme Value ◽

Time Dependent ◽

Efficient Global Optimization ◽

Highly Nonlinear ◽

Adaptive Kriging

Time-dependent reliability analysis requires the use of the extreme value of a response. The extreme value function is usually highly nonlinear, and traditional reliability methods, such as the first order reliability method (FORM), may produce large errors. The solution to this problem is using a surrogate model of the extreme response. The objective of this work is to improve the efficiency of building such a surrogate model. A mixed efficient global optimization (m-EGO) method is proposed. Different from the current EGO method, which draws samples of random variables and time independently, the m-EGO method draws samples for the two types of samples simultaneously. The m-EGO method employs the adaptive Kriging–Monte Carlo simulation (AK–MCS) so that high accuracy is also achieved. Then, Monte Carlo simulation (MCS) is applied to calculate the time-dependent reliability based on the surrogate model. Good accuracy and efficiency of the m-EGO method are demonstrated by three examples.

Download Full-text