Investigation of determinism-related issues in the Sobol′ low-discrepancy sequence for producing sound global sensitivity analysis indices

A computationally efficient and robust sampling scheme can support a sensitivity analysis of models to discover their behaviour through Quasi Monte Carlo approximation. This is especially useful for complex models, as often occur in environmental domains when model runtime can be prohibitive. The Sobol' sequence is one of the most used quasi-random low-discrepancy sequences as it can explore the parameter space significantly more evenly than pseudo-random sequences. The built-in determinism of the Sobol' sequence assists in achieving this attractive property. However, the Sobol' sequence tends to deteriorate in the sense that the estimated errors are distributed inconsistently across model parameters as the dimensions of a model increase. By testing multiple Sobol' sequence implementations, it is clear that the deterministic nature of the Sobol' sequence occasionally introduces relatively large errors in sensitivity indices produced by well-known global sensitivity analysis methods, and that the errors do not diminish by averaging through multiple replications. Problematic sensitivity indices may mistakenly guide modellers to make type I and II errors in trying to identify sensitive parameters, and this will potentially impact model reduction attempts based on these sensitivity measurements. This work investigates the cause of the Sobol' sequence's determinism-related issues. References I. A. Antonov and V. M. Saleev. An economic method of computing LPτ-sequences. USSR Comput. Math. Math. Phys. 19.1 (1979), pp. 252–256. doi: 10.1016/0041-5553(79)90085-5 P. Bratley and B. L. Fox. Algorithm 659: Implementing Sobol’s quasirandom sequence generator. ACM Trans. Math. Soft. 14.1 (1988), pp. 88–100. doi: 10.1145/42288.214372 J. Feinberg and H. P. Langtangen. Chaospy: An open source tool for designing methods of uncertainty quantification. J. Comput. Sci. 11 (2015), pp. 46–57. doi: 10.1016/j.jocs.2015.08.008 on p. C90). S. Joe and F. Y. Kuo. Constructing Sobol sequences with better two-dimensional projections. SIAM J. Sci. Comput. 30.5 (2008), pp. 2635–2654. doi: 10.1137/070709359 S. Joe and F. Y. Kuo. Remark on algorithm 659: Implementing Sobol’s quasirandom sequence generator. ACM Trans. Math. Soft. 29.1 (2003), pp. 49–57. doi: 10.1145/641876.641879 W. J. Morokoff and R. E. Caflisch. Quasi-random sequences and their discrepancies. SIAM J. Sci. Comput. 15.6 (1994), pp. 1251–1279. doi: 10.1137/0915077 X. Sun, B. Croke, S. Roberts, and A. Jakeman. Comparing methods of randomizing Sobol’ sequences for improving uncertainty of metrics in variance-based global sensitivity estimation. Reliab. Eng. Sys. Safety 210 (2021), p. 107499. doi: 10.1016/j.ress.2021.107499 S. Tarantola, W. Becker, and D. Zeitz. A comparison of two sampling methods for global sensitivity analysis. Comput. Phys. Com. 183.5 (2012), pp. 1061–1072. doi: 10.1016/j.cpc.2011.12.015 S. Tezuka. Discrepancy between QMC and RQMC, II. Uniform Dist. Theory 6.1 (2011), pp. 57–64. url: https://pcwww.liv.ac.uk/~karpenk/JournalUDT/vol06/no1/5Tezuka11-1.pdf I. M. Sobol′. On the distribution of points in a cube and the approximate evaluation of integrals. USSR Comput. Math. Math. Phys. 7.4 (1967), pp. 86–112. doi: 10.1016/0041-5553(67)90144-9 I. M. Sobol′. Sensitivity estimates for nonlinear mathematical models. Math. Model. Comput. Exp 1.4 (1993), pp. 407–414.

Studying the Impact of Initialization for Population-Based Algorithms with Low-Discrepancy Sequences

To solve different kinds of optimization challenges, meta-heuristic algorithms have been extensively used. Population initialization plays a prominent role in meta-heuristic algorithms for the problem of optimization. These algorithms can affect convergence to identify a robust optimum solution. To investigate the effectiveness of diversity, many scholars have a focus on the reliability and quality of meta-heuristic algorithms for enhancement. To initialize the population in the search space, this dissertation proposes three new low discrepancy sequences for population initialization instead of uniform distribution called the WELL sequence, Knuth sequence, and Torus sequence. This paper also introduces a detailed survey of the different initialization methods of PSO and DE based on quasi-random sequence families such as the Sobol sequence, Halton sequence, and uniform random distribution. For well-known benchmark test problems and learning of artificial neural network, the proposed methods for PSO (TO-PSO, KN-PSO, and WE-PSO), BA (BA-TO, BA-WE, and BA-KN), and DE (DE-TO, DE-WE, and DE-KN) have been evaluated. The synthesis of our strategies demonstrates promising success over uniform random numbers using low discrepancy sequences. The experimental findings indicate that the initialization based on low discrepancy sequences is exceptionally stronger than the uniform random number. Furthermore, our work outlines the profound effects on convergence and heterogeneity of the proposed methodology. It is expected that a comparative simulation survey of the low discrepancy sequence would be beneficial for the investigator to analyze the meta-heuristic algorithms in detail.

Comparative Analysis of Low Discrepancy Sequence-Based Initialization Approaches Using Population-Based Algorithms for Solving the Global Optimization Problems

Metaheuristic algorithms have been widely used to solve diverse kinds of optimization problems. For an optimization problem, population initialization plays a significant role in metaheuristic algorithms. These algorithms can influence the convergence to find an efficient optimal solution. Mainly, for recognizing the importance of diversity, several researchers have worked on the performance for the improvement of metaheuristic algorithms. Population initialization is a vital factor in metaheuristic algorithms such as PSO and DE. Instead of applying the random distribution for the initialization of the population, quasirandom sequences are more useful for the improvement the diversity and convergence factors. This study presents three new low-discrepancy sequences named WELL sequence, Knuth sequence, and Torus sequence to initialize the population in the search space. This paper also gives a comprehensive survey of the various PSO and DE initialization approaches based on the family of quasirandom sequences such as Sobol sequence, Halton sequence, and uniform random distribution. The proposed methods for PSO (TO-PSO, KN-PSO, and WE-PSO) and DE (DE-TO, DE-WE, and DE-KN) have been examined for well-known benchmark test problems and training of the artificial neural network. The finding of our techniques shows promising performance using the family of low-discrepancy sequences over uniform random numbers. For a fair comparison, the approaches using low-discrepancy sequences for PSO and DE are compared with the other family of low-discrepancy sequences and uniform random number and depict the superior results. The experimental results show that the low-discrepancy sequences-based initialization performed exceptionally better than a uniform random number. Moreover, the outcome of our work presents a foresight on how the proposed technique profoundly impacts convergence and diversity. It is anticipated that this low-discrepancy sequence comparative simulation survey would be helpful for studying the metaheuristic algorithm in detail for the researcher.

Comparison of Different Bat Initialization Techniques for Global Optimization Problems

Bat algorithm (BA) is a population-based stochastic search technique that has been widely used to solve the diverse kind of optimization problems. Population initialization is the current ongoing research problem in evolutionary computing algorithms. Appropriate population initialization assists the algorithm to investigate the swarm search space effectively. BA faces premature convergence problem to find actual global optimization value. Low discrepancy sequences are slightly lesser random number than pseudo-random; however, they are more powerful for computational approaches. In this work, new population initialization approach Halton (BA-HA), Sobol (BA-SO), and Torus (BA-TO) are proposed, which helps bats to avoid from the premature convergence. The proposed approaches are examined on standard benchmark functions, and simulation results are compared with standard BA initialized with uniform distribution. The results depict that substantial enhancement can be attained in the performance of standard BA while varying the random numbers sequences to low discrepancy sequences.

Implementing de-biased estimators using mixed sequences

We describe an implementation of the de-biased estimator using mixed sequences; these are sequences obtained from pseudorandom and low-discrepancy sequences. We use this implementation to numerically solve some stochastic differential equations from computational finance. The mixed sequences, when combined with Brownian bridge or principal component analysis constructions, offer convergence rates significantly better than the Monte Carlo implementation.