Efficient stochastic optimisation by unadjusted Langevin Monte Carlo

Stochastic approximation methods play a central role in maximum likelihood estimation problems involving intractable likelihood functions, such as marginal likelihoods arising in problems with missing or incomplete data, and in parametric empirical Bayesian estimation. Combined with Markov chain Monte Carlo algorithms, these stochastic optimisation methods have been successfully applied to a wide range of problems in science and industry. However, this strategy scales poorly to large problems because of methodological and theoretical difficulties related to using high-dimensional Markov chain Monte Carlo algorithms within a stochastic approximation scheme. This paper proposes to address these difficulties by using unadjusted Langevin algorithms to construct the stochastic approximation. This leads to a highly efficient stochastic optimisation methodology with favourable convergence properties that can be quantified explicitly and easily checked. The proposed methodology is demonstrated with three experiments, including a challenging application to statistical audio analysis and a sparse Bayesian logistic regression with random effects problem.

D6.3 Report on stochastic optimisation for simple problems

This report addresses the general matter of optimisation under uncertainties, following a previous report on stochastic sensitivities (deliverable 6.2). It describes several theoretical methods, as well their application into implementable algorithms. The specific case of the conditional value at risk chosen as risk measure, with its challenges, is prominently discussed. In particular, the issue of smoothness – or lack thereof – is addressed through several possible approaches. The whole report is written in the context of high-performance computing, with concern for parallelisation and cost-efficiency.

D6.4 Report on stochastic optimisation for unsteady problems

This report brings together methodological research on stochastic optimisation and work on benchmark and target applications of the ExaQute project, with a focus on unsteady problems. A practical, general method for the optimisation of the conditional value at risk is proposed. Three different optimisation problems are described: an oscillator problem selected as a suitable trial and illustration case; the shape optimisation of an airfoil, chosen as a benchmark application in the project; the shape optimisation of a tall building, which is the challenging target application set for ExaQUte. For each problem, the current developments and results are presented, the application of the proposed method is discussed, and the work to be done until the end of the project is laid out.