SENSITIVITY ANALYSIS OF NONLINEAR MODELS TO PARAMETER PERTURBATIONS FOR SMALL SIZE ENSEMBLES OF MODEL OUTPUTS

2011 ◽  
Vol 21 (12) ◽  
pp. 3589-3609 ◽  
Author(s):  
L. M. IVANOV ◽  
R. T. TOKMAKIAN
Keyword(s):  
Sensitivity Analysis ◽  
Degrees Of Freedom ◽  
Nonlinear Models ◽  
A Priori ◽  
Distribution Functions ◽  
Cumulative Distribution ◽  
Model Parameters ◽  
Model Sensitivity ◽  
Climate System Model ◽  
Sensitivity Indices

A new technique for nonlinear sensitivity analysis of geophysical models for small size ensembles of model outputs has been developed. Such an analysis utilizes the following metrics: (a) Sobol–Saltelli sensitivity indices and cumulative distribution functions if perturbations of model parameters are random, and (b) a Hartley-like measure if perturbations of model parameters are nonrandom and parametrized through fuzzy sets. The indices and the Hartley-like measure allow for ranging model parameters along their significance to the model output. Our calculations demonstrate that accurate estimates of the sensitivity indices are possible even if an ensemble of random perturbations contains considerably less than 100 members. Some calculations were successfully provided for random ensembles with 20–30 members only but, in general 50–100 member ensembles are required to get robust and significant estimations of model sensitivity. The fuzzy set concept allows for robust estimations for small size nonrandom ensembles of model outputs (50–100 members) and accounts for additional a priori information on model sensitivity coming from different sources. The Lorenz 63 model (a few degrees of freedom) and the ocean component (POP) of the Community Climate System Model (CCSM3) (several thousand degrees of freedom) are used to illustrate the sensitivity analysis based on this approach.

scholarly journals Practical Experience of Sensitivity Analysis: Comparing Six Methods, on Three Hydrological Models, with Three Performance Criteria

Water ◽  
2019 ◽  
Vol 11 (5) ◽  
pp. 1062 ◽  
Author(s):  
Anqi Wang ◽  
Dimitri P. Solomatine
Keyword(s):  
Sensitivity Analysis ◽  
Practical Experience ◽  
Distribution Functions ◽  
Ease Of Use ◽  
Cumulative Distribution ◽  
Performance Criteria ◽  
Hydrological Models ◽  
Sensitivity Indices ◽  
Kolmogorov Smirnov

Currently, practically no modeling study is expected to be carried out without some form of Sensitivity Analysis (SA). At the same time, there is a large number of various methods and it is not always easy for practitioners to choose one. The aim of this paper is to briefly review main classes of SA methods, and to present the results of the practical comparative analysis of applying them. Six different global SA methods: Sobol, eFAST (extended Fourier Amplitude Sensitivity Test), Morris, LH-OAT, RSA (Regionalized Sensitivity Analysis), and PAWN are tested on three conceptual rainfall-runoff models with varying complexity: (GR4J, Hymod, and HBV) applied to the case study of Bagmati basin (Nepal). The methods are compared with respect to effectiveness, efficiency, and convergence. A practical framework of selecting and using the SA methods is presented. The result shows that, first of all, all the six SA methods are effective. Morris and LH-OAT methods are the most efficient methods in computing SI and ranking. eFAST performs better than Sobol, and thus it can be seen as its viable alternative for Sobol. PAWN and RSA methods have issues of instability, which we think are due to the ways Cumulative Distribution Functions (CDFs) are built, and using Kolmogorov–Smirnov statistics to compute Sensitivity Indices. All the methods require sufficient number of runs to reach convergence. Difference in efficiency of different methods is an inevitable consequence of the differences in the underlying principles. For SA of hydrological models, it is recommended to apply the presented practical framework assuming the use of several methods, and to explicitly take into account the constraints of effectiveness, efficiency (including convergence), ease of use, and availability of software.

Comprehensive global sensitivity analysis of a repository model using different types of transformations and metamodeling techniques

2021 ◽  
Author(s):  
Sabine M. Spiessl ◽  
Dirk-A. Becker ◽  
Sergei Kucherenko
Keyword(s):  
Sensitivity Analysis ◽  
Global Sensitivity Analysis ◽  
Higher Order ◽  
Model Parameters ◽  
High Dimensional Model Representation ◽  
Model Output ◽  
Global Sensitivity ◽  
Sparse Polynomial ◽  
Sensitivity Indices ◽  
Highly Nonlinear

<p>Due to their highly nonlinear, non-monotonic or even discontinuous behavior, sensitivity analysis of final repository models can be a demanding task. Most of the output of repository models is typically distributed over several orders of magnitude and highly skewed. Many values of a probabilistic investigation are very low or even zero. Although this is desirable in view of repository safety it can distort the evidence of sensitivity analysis. For the safety assessment of the system, the highest values of outputs are mainly essential and if those are only a few, their dependence on specific parameters may appear insignificant. By applying a transformation, different model output values are differently weighed, according to their magnitude, in sensitivity analysis. Probabilistic methods of higher-order sensitivity analysis, applied on appropriately transformed model output values, provide a possibility for more robust identification of relevant parameters and their interactions. This type of sensitivity analysis is typically done by decomposing the total unconditional variance of the model output into partial variances corresponding to different terms in the ANOVA decomposition. From this, sensitivity indices of increasing order can be computed. The key indices used most often are the first-order index (SI1) and the total-order index (SIT). SI1 refers to the individual impact of one parameter on the model and SIT represents the total effect of one parameter on the output in interactions with all other parameters. The second-order sensitivity indices (SI2) describe the interactions between two model parameters.</p><p>In this work global sensitivity analysis has been performed with three different kinds of output transformations (log, shifted and Box-Cox transformation) and two metamodeling approaches, namely the Random-Sampling High Dimensional Model Representation (RS-HDMR) [1] and the Bayesian Sparse PCE (BSPCE) [2] approaches. Both approaches are implemented in the SobolGSA software [3, 4] which was used in this work. We analyzed the time-dependent output with two approaches for sensitivity analysis, i.e., the pointwise and generalized approaches. With the pointwise approach, the output at each time step is analyzed independently. The generalized approach considers averaged output contributions at all previous time steps in the analysis of the current step. Obtained results indicate that robustness can be improved by using appropriate transformations and choice of coefficients for the transformation and the metamodel.</p><p>[1] M. Zuniga, S. Kucherenko, N. Shah (2013). Metamodelling with independent and dependent inputs. Computer Physics Communications, 184 (6): 1570-1580.</p><p>[2] Q. Shao, A. Younes, M. Fahs, T.A. Mara (2017). Bayesian sparse polynomial chaos expansion for global sensitivity analysis. Computer Methods in Applied Mechanics and Engineering, 318: 474-496.</p><p>[3] S. M. Spiessl, S. Kucherenko, D.-A. Becker, O. Zaccheus (2018). Higher-order sensitivity analysis of a final repository model with discontinuous behaviour. Reliability Engineering and System Safety, doi: https://doi.org/10.1016/j.ress.2018.12.004.</p><p>[4] SobolGSA software (2021). User manual https://www.imperial.ac.uk/process-systems-engineering/research/free-software/sobolgsa-software/.</p>

Abstract WP291: Stochastic Methods Can Resolve the Dilemma of Emergency Stroke Transport

Stroke ◽  
2020 ◽  
Vol 51 (Suppl_1) ◽  
Author(s):  
Daniel A Paydarfar ◽  
David Paydarfar ◽  
Peter J Mucha ◽  
Joshua Chang
Keyword(s):  
Census Data ◽  
Infarct Volume ◽  
Statistical Significance ◽  
Distribution Functions ◽  
Cumulative Distribution ◽  
Practical Significance ◽  
Stochastic Methods ◽  
Model Parameters ◽  
Collateral Flow ◽  
Acute Stroke Care

Introduction: Drip and Ship (DNS) and Mothership (MS) are well-known emergency transport strategies in acute stroke care, but the criteria for choosing between the two is widely debated. Existing models define time-dependent outcomes but cannot resolve this debate with statistical significance because the independent variables are deterministic. We propose a novel stochastic framework that quantifies statistical significance between DNS and MS in a network of primary and comprehensive stroke centers. Methods: We represented the physiology of ischemic core growth as a stochastic first-order differential equation, enabling infarct volume at time of reperfusion to be calculated and mapped to 90-day mRS. Using Texas as a case study, we configured the state’s stroke network within 15,811 geographic blocks as defined by census data. For each block, we ran Monte Carlo simulations to generate Beta distributions of large- and small-vessel infarct volumes, which were then translated into cumulative distribution functions of mRS. A two-sample Kolmogorov-Smirnov test for significance, and Cohen’s d effect size statistic for practical significance were computed between each DNS and MS pair. Stable effect sizes were assured by sampling > 5,000 total infarct volumes for each block. All model parameters were established from large cohort studies or trials. Results: Of the 13,113 blocks where the primary stroke center is the closest hospital from origin, DNS produces significantly better stroke outcomes than MS in 79.0% (0.3% SEM; P < 0.05; 0.2 < d < 0.5). For the subset of patients with large-vessel strokes, MS produces significantly better outcomes in 44.6% of blocks (1.3% SEM; P < 0.05; 0.4 < d < 0.85). Conclusion: Stochastic methods enable the use of clinically relevant metrics for comparative significance of DNS and MS in a geographic region. This formalism, which has not been incorporated in previous models, can be further generalized beyond stochastic infarct volumes if sufficiently large datasets become available. For example, the kinetic growth model can integrate the statistical distributions of times (pre-hospital and hospital) leading up to intervention, and patient attributes that affect outcomes, such as the degree of collateral flow and comorbidities.

scholarly journals Clustering a Global Field of Atmospheric Profiles by Mixture Decomposition of Copulas

2005 ◽  
Vol 22 (10) ◽  
pp. 1445-1459 ◽  
Author(s):  
Mathieu Vrac ◽  
Alain Chédin ◽  
Edwin Diday
Keyword(s):  
A Priori ◽  
Distribution Functions ◽  
Cumulative Distribution ◽  
Specific Humidity ◽  
Vertical Profiles ◽  
Copula Functions ◽  
Mixture Density ◽  
Temperature And Humidity ◽  
Priori Information ◽  
Humidity Profiles

Abstract This work focuses on the clustering of a large dataset of atmospheric vertical profiles of temperature and humidity in order to model a priori information for the problem of retrieving atmospheric variables from satellite observations. Here, each profile is described by cumulative distribution functions (cdfs) of temperature and specific humidity. The method presented here is based on an extension of the mixture density problem to this kind of data. This method allows dependencies between and among temperature and moisture to be taken into account, through copula functions, which are particular distribution functions, linking a (joint) multivariate distribution with its (marginal) univariate distributions. After a presentation of vertical profiles of temperature and humidity and the method used to transform them into cdfs, the clustering method is detailed and then applied to provide a partition into seven clusters based, first, on the temperature profiles only; second, on the humidity profiles only; and, third, on both the temperature and humidity profiles. The clusters are statistically described and explained in terms of airmass types, with reference to meteorological maps. To test the robustness and the relevance of the method for a larger number of clusters, a partition into 18 classes is established, where it is shown that even the smallest clusters are significant. Finally, comparisons with more classical efficient clustering or model-based methods are presented, and the advantages of the approach are discussed.

Sensitivity analysis and the challenges posed by multiple approaches: a multifaceted mess

2020 ◽  
Author(s):  
Monica Riva ◽  
Aronne Dell'Oca ◽  
Alberto Guadagnini
Keyword(s):  
Sensitivity Analysis ◽  
Model Calibration ◽  
Global Sensitivity Analysis ◽  
System Model ◽  
Model Parameters ◽  
Model Output ◽  
Multiple Model ◽  
Total Uncertainty ◽  
Global Sensitivity ◽  
Sensitivity Indices

&lt;p&gt;Modern models of environmental and industrial systems have reached a relatively high level of complexity. The latter aspect could hamper an unambiguous understanding of the functioning of a model, i.e., how it drives relationships and dependencies among inputs and outputs of interest. Sensitivity Analysis tools can be employed to examine this issue.&lt;/p&gt;&lt;p&gt;Global sensitivity analysis (GSA) approaches rest on the evaluation of sensitivity across the entire support within which system model parameters are supposed to vary. In this broad context, it is important to note that the definition of a sensitivity metric must be linked to the nature of the question(s) the GSA is meant to address. These include, for example: (i) which are the most important model parameters with respect to given model output(s)?; (ii) could we set some parameter(s) (thus assisting model calibration) at prescribed value(s) without significantly affecting model results?; (iii) at which space/time locations can one expect the highest sensitivity of model output(s) to model parameters and/or knowledge of which parameter(s) could be most beneficial for model calibration?&lt;/p&gt;&lt;p&gt;The variance-based Sobol&amp;#8217; Indices (e.g., Sobol, 2001) represent one of the most widespread GSA metrics, quantifying the average reduction in the variance of a model output stemming from knowledge of the input. Amongst other techniques, Dell&amp;#8217;Oca et al. [2017] proposed a moment-based GSA approach which enables one to quantify the influence of uncertain model parameters on the (statistical) moments of a target model output.&lt;/p&gt;&lt;p&gt;Here, we embed in these sensitivity indices the effect of uncertainties both in the system model conceptualization and in the ensuing model(s) parameters. The study is grounded on the observation that physical processes and natural systems within which they take place are complex, rendering target state variables amenable to multiple interpretations and mathematical descriptions. As such, predictions and uncertainty analyses based on a single model formulation can result in statistical bias and possible misrepresentation of the total uncertainty, thus justifying the assessment of multiple model system conceptualizations. We then introduce copula-based sensitivity metrics which allow characterizing the global (with respect to the input) value of the sensitivity and the degree of variability (across the whole range of the input values) of the sensitivity for each value that the prescribed model output can possibly undertake, as driven by a governing model. In this sense, such an approach to sensitivity is global with respect to model input(s) and local with respect to model output, thus enabling one to discriminate the relevance of an input across the entire range of values of the modeling goal of interest. The methodology is demonstrated in the context of flow and reactive transport scenarios.&lt;/p&gt;&lt;p&gt;&amp;#160;&lt;/p&gt;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;&lt;p&gt;Sobol, I. M., 2001. Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates. Math. Comput. Sim., 55, 271-280.&lt;/p&gt;&lt;p&gt;Dell&amp;#8217;Oca, A., Riva, M., Guadagnini, A., 2017. Moment-based metrics for global sensitivity analysis of hydrological systems. Hydr. Earth Syst. Sci., 21, 6219-6234.&lt;/p&gt;

Optimal experiment design for nonlinear models subject to large prior uncertainties

1987 ◽  
Vol 253 (3) ◽  
pp. R530-R534 ◽  
Author(s):  
E. Walter ◽  
L. Pronzato
Keyword(s):  
Optimal Design ◽  
Parameter Uncertainty ◽  
Fisher Information Matrix ◽  
Nonlinear Models ◽  
A Priori ◽  
Information Matrix ◽  
Experiment Design ◽  
Model Parameters ◽  
Optimal Experiment ◽  
Classical Experiment

Classical experiment design generally yields an experiment that depends on the value of the parameters to be estimated, which are, of course, unknown. Assuming that the model parameters belong to a population with known statistics, we propose to take the a priori parameter uncertainty into account by optimizing the mathematical expectation of a functional of the Fisher information matrix. This optimization is performed with a stochastic approximation algorithm that makes robust experiment design almost as simple as classical D-optimal design. The resulting methodology is applied to the choice of measurement times for multiexponential models.

scholarly journals Sensitivity analysis of a galaxy formation model

2019 ◽  
Vol 493 (2) ◽  
pp. 1827-1841
Author(s):  
Piotr Oleśkiewicz ◽  
Carlton M Baugh
Keyword(s):  
Sensitivity Analysis ◽  
Galaxy Formation ◽  
Luminosity Function ◽  
Sampling Technique ◽  
Model Parameters ◽  
Formation Model ◽  
Multiple Parameters ◽  
Sensitivity Indices ◽  
The Galaxy ◽  
Galaxy Population

ABSTRACT We present the first application of a variance-based sensitivity analysis (SA) to a model that aims to predict the evolution and properties of the whole galaxy population. SA is a well-established technique in other quantitative sciences, but is a relatively novel tool for the evaluation of astrophysical models. We perform a multiparameter exploration of the GALFORM semi-analytic galaxy formation model, to compute how sensitive the present-day K-band luminosity function is to varying different model parameters. The parameter space is scanned using a low-discrepancy sampling technique proposed by Saltelli. We first demonstrate the usefulness of the SA approach by varying just two model parameters, one that controls supernova feedback and the other the heating of gas by active galactic nucleus. The SA analysis matches our physical intuition regarding how these parameters affect the predictions for different parts of the galaxy luminosity function. We then use SA to compute Sobol’ sensitivity indices varying seven model parameters, connecting the variance in the model output to the variance in the input parameters. The sensitivity is computed in luminosity bins, allowing us to probe the origin of the model predictions in detail. We discover that the SA correctly identifies the least important and most important parameters. Moreover, the SA also captures the combined responses of varying multiple parameters at the same time. Our study marks a much needed step away from the traditional 'one-at-a-time' parameter variation often used in this area and improves the transparency of multiparameter models of galaxy formation.

Probabilistic Analysis of a Metamorphic Mechanism Based on a Global Sensitivity Analysis: A Preliminary Study

2017 ◽  
Author(s):  
Sarah C. Baxter ◽  
Philip A. Voglewede
Keyword(s):  
Sensitivity Analysis ◽  
Probabilistic Analysis ◽  
Global Sensitivity Analysis ◽  
Kinematic Model ◽  
Mathematical Structure ◽  
Motion Path ◽  
Model Parameters ◽  
Global Sensitivity ◽  
Sensitivity Indices ◽  
The Impact

Mathematical modeling is an important part of the engineering design cycle. Most models require application specific input parameters that are established by calculation or experiment. The accuracy of model predictions depends on underlying model assumptions as well as how uncertainty in knowledge of the parameters is transmitted through the mathematical structure of the model. Knowledge about the relative impact of individual parameters can help establish priorities in developing/choosing specific parameters and provide insight into a range of parameters that produce ‘equally good’ designs. In this work Global Sensitivity Analysis (GSA) is examined as a technique that can contribute to this insight by developing Sensitivity Indices, a measure of the relative importance, for each parameter. The approach is illustrated on a kinematic model of a metamorphic 4-bar mechanism. The model parameters are the lengths of the four links. The results of this probabilistic analysis highlight the synergy that must exist between all four link lengths to create a design that can follow the desired motion path. The impact of individual link lengths, however, rises and falls depending on where the mechanism is along its motion path.

scholarly journals Towards a conceptual approach to predetermining long-return-period avalanche run-out distances

2004 ◽  
Vol 50 (169) ◽  
pp. 268-278 ◽  
Author(s):  
Maurice Meunier ◽  
Christophe Ancey
Keyword(s):  
Distribution Function ◽  
Field Data ◽  
Cumulative Distribution Function ◽  
Extreme Event ◽  
A Priori ◽  
Rheological Behaviour ◽  
Cumulative Distribution ◽  
Model Parameters ◽  
Conceptual Approach ◽  
Deterministic Models

AbstractInvestigating snow avalanches using a purely statistical approach raises several issues. First, even in the heavily populated areas of the Alps, there are few data on avalanche motion or extension. Second, most of the field data are related to the point of furthest reach in the avalanche path (run-out distance or altitude). As data of this kind are tightly dependent on the avalanche path profile, it is a priori not permissible to extrapolate the cumulative distribution function fitted to these data without severe restrictions or further assumptions. Using deterministic models is also problematic, as these are not really physically based models. For instance, they do not include all the phenomena occurring in the avalanche movement, and the rheological behaviour of the snow is not known. Consequently, it is not easy to predetermine extreme-event extensions. Here, in order to overcome this problem, we propose to use a conceptual approach. First, using an avalanche-dynamics numerical model, we fitted the model parameters (friction coefficients and the volume of snow involved in the avalanches) to the field data. Then, using these parameters as random variables, we adjusted appropriate statistical distributions. The last steps involved simulating a large number of (fictitious) avalanches using the Monte Carlo approach. Thus, the cumulative distribution function of the run-out distance can be computed over a much broader range than was initially possible with the historical data. In this paper, we develop the proposed method through a complete case study, comparing two different models.

Sign in / Sign up

Export Citation Format

Share Document