scholarly journals Investigation of determinism-related issues in the Sobol′ low-discrepancy sequence for producing sound global sensitivity analysis indices

2021 ◽  
Vol 62 ◽  
pp. C84-C97
Author(s):  
Xifu Sun ◽  
Barry Croke ◽  
Stephen Roberts ◽  
Anthony Jakeman
Keyword(s):  
Sensitivity Analysis ◽  
Global Sensitivity Analysis ◽  
Model Parameters ◽  
Type I ◽  
Random Sequences ◽  
Global Sensitivity ◽  
Low Discrepancy Sequences ◽  
Sensitivity Indices ◽  
Sequence Generator ◽  
Math Phys

A computationally efficient and robust sampling scheme can support a sensitivity analysis of models to discover their behaviour through Quasi Monte Carlo approximation. This is especially useful for complex models, as often occur in environmental domains when model runtime can be prohibitive. The Sobol' sequence is one of the most used quasi-random low-discrepancy sequences as it can explore the parameter space significantly more evenly than pseudo-random sequences. The built-in determinism of the Sobol' sequence assists in achieving this attractive property. However, the Sobol' sequence tends to deteriorate in the sense that the estimated errors are distributed inconsistently across model parameters as the dimensions of a model increase. By testing multiple Sobol' sequence implementations, it is clear that the deterministic nature of the Sobol' sequence occasionally introduces relatively large errors in sensitivity indices produced by well-known global sensitivity analysis methods, and that the errors do not diminish by averaging through multiple replications. Problematic sensitivity indices may mistakenly guide modellers to make type I and II errors in trying to identify sensitive parameters, and this will potentially impact model reduction attempts based on these sensitivity measurements. This work investigates the cause of the Sobol' sequence's determinism-related issues. References I. A. Antonov and V. M. Saleev. An economic method of computing LPτ-sequences. USSR Comput. Math. Math. Phys. 19.1 (1979), pp. 252–256. doi: 10.1016/0041-5553(79)90085-5 P. Bratley and B. L. Fox. Algorithm 659: Implementing Sobol’s quasirandom sequence generator. ACM Trans. Math. Soft. 14.1 (1988), pp. 88–100. doi: 10.1145/42288.214372 J. Feinberg and H. P. Langtangen. Chaospy: An open source tool for designing methods of uncertainty quantification. J. Comput. Sci. 11 (2015), pp. 46–57. doi: 10.1016/j.jocs.2015.08.008 on p. C90). S. Joe and F. Y. Kuo. Constructing Sobol sequences with better two-dimensional projections. SIAM J. Sci. Comput. 30.5 (2008), pp. 2635–2654. doi: 10.1137/070709359 S. Joe and F. Y. Kuo. Remark on algorithm 659: Implementing Sobol’s quasirandom sequence generator. ACM Trans. Math. Soft. 29.1 (2003), pp. 49–57. doi: 10.1145/641876.641879 W. J. Morokoff and R. E. Caflisch. Quasi-random sequences and their discrepancies. SIAM J. Sci. Comput. 15.6 (1994), pp. 1251–1279. doi: 10.1137/0915077 X. Sun, B. Croke, S. Roberts, and A. Jakeman. Comparing methods of randomizing Sobol’ sequences for improving uncertainty of metrics in variance-based global sensitivity estimation. Reliab. Eng. Sys. Safety 210 (2021), p. 107499. doi: 10.1016/j.ress.2021.107499 S. Tarantola, W. Becker, and D. Zeitz. A comparison of two sampling methods for global sensitivity analysis. Comput. Phys. Com. 183.5 (2012), pp. 1061–1072. doi: 10.1016/j.cpc.2011.12.015 S. Tezuka. Discrepancy between QMC and RQMC, II. Uniform Dist. Theory 6.1 (2011), pp. 57–64. url: https://pcwww.liv.ac.uk/~karpenk/JournalUDT/vol06/no1/5Tezuka11-1.pdf I. M. Sobol′. On the distribution of points in a cube and the approximate evaluation of integrals. USSR Comput. Math. Math. Phys. 7.4 (1967), pp. 86–112. doi: 10.1016/0041-5553(67)90144-9 I. M. Sobol′. Sensitivity estimates for nonlinear mathematical models. Math. Model. Comput. Exp 1.4 (1993), pp. 407–414.

Comprehensive global sensitivity analysis of a repository model using different types of transformations and metamodeling techniques

2021 ◽  
Author(s):  
Sabine M. Spiessl ◽  
Dirk-A. Becker ◽  
Sergei Kucherenko
Keyword(s):  
Sensitivity Analysis ◽  
Global Sensitivity Analysis ◽  
Higher Order ◽  
Model Parameters ◽  
High Dimensional Model Representation ◽  
Model Output ◽  
Global Sensitivity ◽  
Sparse Polynomial ◽  
Sensitivity Indices ◽  
Highly Nonlinear

<p>Due to their highly nonlinear, non-monotonic or even discontinuous behavior, sensitivity analysis of final repository models can be a demanding task. Most of the output of repository models is typically distributed over several orders of magnitude and highly skewed. Many values of a probabilistic investigation are very low or even zero. Although this is desirable in view of repository safety it can distort the evidence of sensitivity analysis. For the safety assessment of the system, the highest values of outputs are mainly essential and if those are only a few, their dependence on specific parameters may appear insignificant. By applying a transformation, different model output values are differently weighed, according to their magnitude, in sensitivity analysis. Probabilistic methods of higher-order sensitivity analysis, applied on appropriately transformed model output values, provide a possibility for more robust identification of relevant parameters and their interactions. This type of sensitivity analysis is typically done by decomposing the total unconditional variance of the model output into partial variances corresponding to different terms in the ANOVA decomposition. From this, sensitivity indices of increasing order can be computed. The key indices used most often are the first-order index (SI1) and the total-order index (SIT). SI1 refers to the individual impact of one parameter on the model and SIT represents the total effect of one parameter on the output in interactions with all other parameters. The second-order sensitivity indices (SI2) describe the interactions between two model parameters.</p><p>In this work global sensitivity analysis has been performed with three different kinds of output transformations (log, shifted and Box-Cox transformation) and two metamodeling approaches, namely the Random-Sampling High Dimensional Model Representation (RS-HDMR) [1] and the Bayesian Sparse PCE (BSPCE) [2] approaches. Both approaches are implemented in the SobolGSA software [3, 4] which was used in this work. We analyzed the time-dependent output with two approaches for sensitivity analysis, i.e., the pointwise and generalized approaches. With the pointwise approach, the output at each time step is analyzed independently. The generalized approach considers averaged output contributions at all previous time steps in the analysis of the current step. Obtained results indicate that robustness can be improved by using appropriate transformations and choice of coefficients for the transformation and the metamodel.</p><p>[1] M. Zuniga, S. Kucherenko, N. Shah (2013). Metamodelling with independent and dependent inputs. Computer Physics Communications, 184 (6): 1570-1580.</p><p>[2] Q. Shao, A. Younes, M. Fahs, T.A. Mara (2017). Bayesian sparse polynomial chaos expansion for global sensitivity analysis. Computer Methods in Applied Mechanics and Engineering, 318: 474-496.</p><p>[3] S. M. Spiessl, S. Kucherenko, D.-A. Becker, O. Zaccheus (2018). Higher-order sensitivity analysis of a final repository model with discontinuous behaviour. Reliability Engineering and System Safety, doi: https://doi.org/10.1016/j.ress.2018.12.004.</p><p>[4] SobolGSA software (2021). User manual https://www.imperial.ac.uk/process-systems-engineering/research/free-software/sobolgsa-software/.</p>

Sensitivity analysis and the challenges posed by multiple approaches: a multifaceted mess

2020 ◽  
Author(s):  
Monica Riva ◽  
Aronne Dell'Oca ◽  
Alberto Guadagnini
Keyword(s):  
Sensitivity Analysis ◽  
Model Calibration ◽  
Global Sensitivity Analysis ◽  
System Model ◽  
Model Parameters ◽  
Model Output ◽  
Multiple Model ◽  
Total Uncertainty ◽  
Global Sensitivity ◽  
Sensitivity Indices

<p>Modern models of environmental and industrial systems have reached a relatively high level of complexity. The latter aspect could hamper an unambiguous understanding of the functioning of a model, i.e., how it drives relationships and dependencies among inputs and outputs of interest. Sensitivity Analysis tools can be employed to examine this issue.</p><p>Global sensitivity analysis (GSA) approaches rest on the evaluation of sensitivity across the entire support within which system model parameters are supposed to vary. In this broad context, it is important to note that the definition of a sensitivity metric must be linked to the nature of the question(s) the GSA is meant to address. These include, for example: (i) which are the most important model parameters with respect to given model output(s)?; (ii) could we set some parameter(s) (thus assisting model calibration) at prescribed value(s) without significantly affecting model results?; (iii) at which space/time locations can one expect the highest sensitivity of model output(s) to model parameters and/or knowledge of which parameter(s) could be most beneficial for model calibration?</p><p>The variance-based Sobol’ Indices (e.g., Sobol, 2001) represent one of the most widespread GSA metrics, quantifying the average reduction in the variance of a model output stemming from knowledge of the input. Amongst other techniques, Dell’Oca et al. [2017] proposed a moment-based GSA approach which enables one to quantify the influence of uncertain model parameters on the (statistical) moments of a target model output.</p><p>Here, we embed in these sensitivity indices the effect of uncertainties both in the system model conceptualization and in the ensuing model(s) parameters. The study is grounded on the observation that physical processes and natural systems within which they take place are complex, rendering target state variables amenable to multiple interpretations and mathematical descriptions. As such, predictions and uncertainty analyses based on a single model formulation can result in statistical bias and possible misrepresentation of the total uncertainty, thus justifying the assessment of multiple model system conceptualizations. We then introduce copula-based sensitivity metrics which allow characterizing the global (with respect to the input) value of the sensitivity and the degree of variability (across the whole range of the input values) of the sensitivity for each value that the prescribed model output can possibly undertake, as driven by a governing model. In this sense, such an approach to sensitivity is global with respect to model input(s) and local with respect to model output, thus enabling one to discriminate the relevance of an input across the entire range of values of the modeling goal of interest. The methodology is demonstrated in the context of flow and reactive transport scenarios.</p><p> </p><p><strong>References</strong></p><p>Sobol, I. M., 2001. Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates. Math. Comput. Sim., 55, 271-280.</p><p>Dell’Oca, A., Riva, M., Guadagnini, A., 2017. Moment-based metrics for global sensitivity analysis of hydrological systems. Hydr. Earth Syst. Sci., 21, 6219-6234.</p>

Probabilistic Analysis of a Metamorphic Mechanism Based on a Global Sensitivity Analysis: A Preliminary Study

2017 ◽  
Author(s):  
Sarah C. Baxter ◽  
Philip A. Voglewede
Keyword(s):  
Sensitivity Analysis ◽  
Probabilistic Analysis ◽  
Global Sensitivity Analysis ◽  
Kinematic Model ◽  
Mathematical Structure ◽  
Motion Path ◽  
Model Parameters ◽  
Global Sensitivity ◽  
Sensitivity Indices ◽  
The Impact

Mathematical modeling is an important part of the engineering design cycle. Most models require application specific input parameters that are established by calculation or experiment. The accuracy of model predictions depends on underlying model assumptions as well as how uncertainty in knowledge of the parameters is transmitted through the mathematical structure of the model. Knowledge about the relative impact of individual parameters can help establish priorities in developing/choosing specific parameters and provide insight into a range of parameters that produce ‘equally good’ designs. In this work Global Sensitivity Analysis (GSA) is examined as a technique that can contribute to this insight by developing Sensitivity Indices, a measure of the relative importance, for each parameter. The approach is illustrated on a kinematic model of a metamorphic 4-bar mechanism. The model parameters are the lengths of the four links. The results of this probabilistic analysis highlight the synergy that must exist between all four link lengths to create a design that can follow the desired motion path. The impact of individual link lengths, however, rises and falls depending on where the mechanism is along its motion path.

scholarly journals Hierarchical Sensitivity Analysis for Large Scale Process-based Hydrological Modeling with Application in an Amazonian Watershed

2019 ◽  
Author(s):  
Haifan Liu ◽  
Heng Dai ◽  
Jie Niu ◽  
Bill X. Hu ◽  
Han Qiu ◽  
...  
Keyword(s):  
Sensitivity Analysis ◽  
Large Scale ◽  
Overland Flow ◽  
Global Sensitivity Analysis ◽  
Hydrologic Model ◽  
Model Parameters ◽  
Hydrological Models ◽  
Global Sensitivity ◽  
Uncertainty Sources ◽  
Sensitivity Indices

Abstract. Sensitivity analysis is an effective tool for identifying important uncertainty sources and improving model calibration and predictions, especially for integrated systems with heterogeneous parameter inputs and complex processes coevolution. In this work, an advanced hierarchical global sensitivity analysis framework, which integrates a hierarchical uncertainty framework and a variance-based global sensitivity analysis, was implemented to quantitatively analyze several uncertainties of a three-dimensional, process-based hydrologic model (PAWS). The uncertainty sources considered include model parameters, model structures (with/without overland flow module), and climate forcing. We apply the approach in a ~ 9000 km2 Amazon catchment modeled at 1 km resolution to provide a demonstration of multiple uncertainty source quantification using a large-scale process-based hydrologic model. The sensitivity indices are assessed based on three important hydrologic outputs: evapotranspiration (ET), ground evaporation (EG), and groundwater contribution to streamflow (QG). It is found that, in general, model parameters (especially those within the streamside model grid cells) are the most important uncertainty contributor for all sensitivity indices. In addition, the overland flow module significantly contributes to model predictive uncertainty. These results can assist model calibration and provide modelers a better understanding of the general sources of uncertainty in predictions of complex hydrological systems in Amazonia. We demonstrated a pilot example for comprehensive global sensitivity analysis of large-scale complex hydrological models in this research. The hierarchical sensitivity analysis methodology used is mathematically rigorous and can be applied to a wide range of large-scale hydrological models with various sources of uncertainty.

scholarly journals Global Sensitivity Analysis of a Homogenized Constrained Mixture Model of Arterial Growth and Remodeling

2021 ◽  
Author(s):  
Sebastian Brandstaeter ◽  
Sebastian L. Fuchs ◽  
Jonas Biehler ◽  
Roland C. Aydin ◽  
Wolfgang A. Wall ◽  
...  
Keyword(s):  
Sensitivity Analysis ◽  
Global Sensitivity Analysis ◽  
Model Parameters ◽  
Computational Studies ◽  
Growth And Remodeling ◽  
Global Sensitivity ◽  
Output Variability ◽  
Constrained Mixture ◽  
The One ◽  
Influential Parameters

AbstractGrowth and remodeling in arterial tissue have attracted considerable attention over the last decade. Mathematical models have been proposed, and computational studies with these have helped to understand the role of the different model parameters. So far it remains, however, poorly understood how much of the model output variability can be attributed to the individual input parameters and their interactions. To clarify this, we propose herein a global sensitivity analysis, based on Sobol indices, for a homogenized constrained mixture model of aortic growth and remodeling. In two representative examples, we found that 54–80% of the long term output variability resulted from only three model parameters. In our study, the two most influential parameters were the one characterizing the ability of the tissue to increase collagen production under increased stress and the one characterizing the collagen half-life time. The third most influential parameter was the one characterizing the strain-stiffening of collagen under large deformation. Our results suggest that in future computational studies it may - at least in scenarios similar to the ones studied herein - suffice to use population average values for the other parameters. Moreover, our results suggest that developing methods to measure the said three most influential parameters may be an important step towards reliable patient-specific predictions of the enlargement of abdominal aortic aneurysms in clinical practice.

scholarly journals A Global Sensitivity Analysis Framework for Hybrid Simulation with Stochastic Substructures

2021 ◽  
Vol 7 ◽  
Author(s):  
Nikolaos Tsokanas ◽  
Xujia Zhu ◽  
Giuseppe Abbiati ◽  
Stefano Marelli ◽  
Bruno Sudret ◽  
...  
Keyword(s):  
Sensitivity Analysis ◽  
Hybrid Model ◽  
Surrogate Model ◽  
Global Sensitivity Analysis ◽  
Hybrid Simulation ◽  
Hybrid Models ◽  
Analysis Framework ◽  
Global Sensitivity ◽  
Sensitivity Indices ◽  
Realistic Situation

Hybrid simulation is an experimental method used to investigate the dynamic response of a reference prototype structure by decomposing it to physically-tested and numerically-simulated substructures. The latter substructures interact with each other in a real-time feedback loop and their coupling forms the hybrid model. In this study, we extend our previous work on metamodel-based sensitivity analysis of deterministic hybrid models to the practically more relevant case of stochastic hybrid models. The aim is to cover a more realistic situation where the physical substructure response is not deterministic, as nominally identical specimens are, in practice, never actually identical. A generalized lambda surrogate model recently developed by some of the authors is proposed to surrogate the hybrid model response, and Sobol’ sensitivity indices are computed for substructure quantity of interest response quantiles. Normally, several repetitions of every single sample of the inputs parameters would be required to replicate the response of a stochastic hybrid model. In this regard, a great advantage of the proposed framework is that the generalized lambda surrogate model does not require repeated evaluations of the same sample. The effectiveness of the proposed hybrid simulation global sensitivity analysis framework is demonstrated using an experiment.

scholarly journals A Global Sensitivity Analysis Framework for Hybrid Simulation

2021 ◽  
Author(s):  
Giuseppe Abbiati ◽  
Stefano Marelli ◽  
Nikolaos Tsokanas ◽  
Bruno Sudret ◽  
Bozidar Stojadinovic
Keyword(s):  
Sensitivity Analysis ◽  
Hybrid Model ◽  
Polynomial Chaos ◽  
Global Sensitivity Analysis ◽  
Hybrid Simulation ◽  
Polynomial Chaos Expansion ◽  
Chaos Expansion ◽  
Analysis Framework ◽  
Global Sensitivity ◽  
Sensitivity Indices

Hybrid Simulation is a dynamic response simulation paradigm that merges physical experiments and computational models into a hybrid model. In earthquake engineering, it is used to investigate the response of structures to earthquake excitation. In the context of response to extreme loads, the structure, its boundary conditions, damping, and the ground motion excitation itself are all subjected to large parameter variability. However, in current seismic response testing practice, Hybrid Simulation campaigns rely on a few prototype structures with fixed parameters subjected to one or two ground motions of different intensity. While this approach effectively reveals structural weaknesses, it does not reveal the sensitivity of structure's response. This thus far missing information could support the planning of further experiments as well as drive modeling choices in subsequent analysis and evaluation phases of the structural design process.This paper describes a Global Sensitivity Analysis framework for Hybrid Simulation. This framework, based on Sobol' sensitivity indices, is used to quantify the sensitivity of the response of a structure tested using the Hybrid Simulation approach due to the variability of the prototype structure and the excitation parameters. Polynomial Chaos Expansion is used to surrogate the hybrid model response. Thereafter, Sobol' sensitivity indices are obtained as a by-product of polynomial coefficients, entailing a reduced number of Hybrid Simulations compared to a crude Monte Carlo approach. An experimental verification example highlights the excellent performance of Polynomial Chaos Expansion surrogates in terms of stable estimates of Sobol' sensitivity indices in the presence of noise caused by random experimental errors.

Nonintrusive Global Sensitivity Analysis for Linear Systems With Process Noise

2019 ◽  
Vol 14 (2) ◽  
Author(s):  
Souransu Nandi ◽  
Tarunraj Singh
Keyword(s):  
Sensitivity Analysis ◽  
Linear Systems ◽  
Global Sensitivity Analysis ◽  
Benchmark Problems ◽  
Model Parameters ◽  
Parameter Uncertainties ◽  
Sobol Indices ◽  
Global Sensitivity ◽  
Time Invariant ◽  
The Impact

The focus of this paper is on the global sensitivity analysis (GSA) of linear systems with time-invariant model parameter uncertainties and driven by stochastic inputs. The Sobol' indices of the evolving mean and variance estimates of states are used to assess the impact of the time-invariant uncertain model parameters and the statistics of the stochastic input on the uncertainty of the output. Numerical results on two benchmark problems help illustrate that it is conceivable that parameters, which are not so significant in contributing to the uncertainty of the mean, can be extremely significant in contributing to the uncertainty of the variances. The paper uses a polynomial chaos (PC) approach to synthesize a surrogate probabilistic model of the stochastic system after using Lagrange interpolation polynomials (LIPs) as PC bases. The Sobol' indices are then directly evaluated from the PC coefficients. Although this concept is not new, a novel interpretation of stochastic collocation-based PC and intrusive PC is presented where they are shown to represent identical probabilistic models when the system under consideration is linear. This result now permits treating linear models as black boxes to develop intrusive PC surrogates.

scholarly journals Sampling behavioral model parameters for ensemble-based sensitivity analysis using Gaussian process emulation and active subspaces

2020 ◽  
Vol 34 (11) ◽  
pp. 1813-1830
Author(s):  
Daniel Erdal ◽  
Sinan Xiao ◽  
Wolfgang Nowak ◽  
Olaf A. Cirpka
Keyword(s):  
Sensitivity Analysis ◽  
Gaussian Process ◽  
Parameter Space ◽  
Global Sensitivity Analysis ◽  
Model Parameters ◽  
Large Sample Size ◽  
Global Sensitivity ◽  
Environmental Models ◽  
Additional Costs ◽  
Active Subspace

Abstract Ensemble-based uncertainty quantification and global sensitivity analysis of environmental models requires generating large ensembles of parameter-sets. This can already be difficult when analyzing moderately complex models based on partial differential equations because many parameter combinations cause an implausible model behavior even though the individual parameters are within plausible ranges. In this work, we apply Gaussian Process Emulators (GPE) as surrogate models in a sampling scheme. In an active-training phase of the surrogate model, we target the behavioral boundary of the parameter space before sampling this behavioral part of the parameter space more evenly by passive sampling. Active learning increases the subsequent sampling efficiency, but its additional costs pay off only for a sufficiently large sample size. We exemplify our idea with a catchment-scale subsurface flow model with uncertain material properties, boundary conditions, and geometric descriptors of the geological structure. We then perform a global-sensitivity analysis of the resulting behavioral dataset using the active-subspace method, which requires approximating the local sensitivities of the target quantity with respect to all parameters at all sampled locations in parameter space. The Gaussian Process Emulator implicitly provides an analytical expression for this gradient, thus improving the accuracy of the active-subspace construction. When applying the GPE-based preselection, 70–90% of the samples were confirmed to be behavioral by running the full model, whereas only 0.5% of the samples were behavioral in standard Monte-Carlo sampling without preselection. The GPE method also provided local sensitivities at minimal additional costs.

Sign in / Sign up

Export Citation Format

Share Document