Model Calibration and Issues Related to Validation, Sensitivity Analysis, Post-audit, Uncertainty Evaluation and Assessment of Prediction Data Needs

Groundwater ◽  
2007 ◽  
pp. 237-282 ◽  
Author(s):  
Claire R. Tiedeman ◽  
Mary C. Hill
2020 ◽  
Author(s):  
Urmila Agrawal ◽  
Pavel Etingov ◽  
Renke Huang

<pre>High quality generator dynamic models are critical to reliable and accurate power systems studies and planning. With the availability of PMU measurements, measurement-based approach for model validation has gained significant prominence. Currently, the model validation results are analyzed by visually comparing real--world PMU measurements with the model-based response measurements, and parameter adjustments rely mostly on engineering experience. This paper proposes advanced performance metrics to systematically quantify the generator dynamic model validation results by separately taking into consideration slow governor response and comparatively fast oscillatory response. The performance metric for governor response is based on the step response characteristics of a system and the metric for oscillatory response is based on the response of generator to each system mode calculated using modal analysis. The proposed metrics in this paper is aimed at providing critical information to help with the selection of parameters to be tuned for model calibration by performing enhanced sensitivity analysis, and also help with rule-based model calibration. Results obtained using both simulated and real-world measurements validate the effectiveness of the proposed performance metrics and sensitivity analysis for model validation and calibration.</pre>


2012 ◽  
Vol 134 (8) ◽  
Author(s):  
Dorin Drignei ◽  
Zissimos P. Mourelatos

Computer, or simulation, models are ubiquitous in science and engineering. Two research topics in building computer models, generally treated separately, are sensitivity analysis and computer model calibration. In sensitivity analysis, one quantifies the effect of each input factor on outputs, whereas in calibration, one finds the values of input factors that provide the best match to a set of test data. In this article, we show a connection between these two seemingly separate concepts for problems with transient signals. We use global sensitivity analysis for computer models with transient signals to screen out inactive input factors, thus making the calibration algorithm numerically more stable. We show that the computer model does not vary with respect to parameters having zero total sensitivity indices, indicating that such parameters are impossible to calibrate and must be screened out. Because the computer model can be computationally intensive, we construct a fast statistical surrogate of the computer model which is used for both sensitivity analysis and computer model calibration. We illustrate our approach with both a simple example and an automotive application involving a road load data acquisition (RLDA) computer model.


Author(s):  
Rasool Khosravanian ◽  
Bernt Sigve Aadnøy

Abstract The requirement of uncertainty analysis has shifted the transformation of sensitivity analysis from the deterministic area to the stochastic area.Geomechanical wellbore integrity problems during drilling operation can occur due to wellbore shear failure or tensile failure. To guarantee wellbore integrity, breakout and fracture geomechanical analysis is essential to estimate the Safe Mud Weight Window (SMWW). Wellbore stability problems causes many challenges in a drilling operation, such as pipe sticking, wellbore collapse, fluid loss and poor cement jobs. A drilling engineer must minimize the risk of these problems, however, there is a considerable uncertainty of different parameters such as geomechanical rock properties of drilled formation, and, data and parameters gathering are often incomplete. This uncertainty of main parameters have impact on the resulting SMWW.This paper perform an uncertainty evaluation of wellbore stability and its effect on the optimum interval of SMWW. The SMWW Uncertainty Evaluation of Wellbore Stability assessment for two failure criteria are compared, Mohr-Coulomb and Modified Lade criterion. We apply Monte Carlo simulations to investigate the uncertainty of the models and we do a sensitivity analysis and confidence level analysis. The paper will show the advantage of including uncertainty evaluation when determining the optimum SMWW window, as opposed to classical deterministic analysis. A case study is presented to draw a perfect understanding of the foundation of the MCS approach with practical and good results. It confirmed the capability of the proposed approach in solving such a strong-nonlinear, complex real problem.


2020 ◽  
Author(s):  
Monica Riva ◽  
Aronne Dell'Oca ◽  
Alberto Guadagnini

&lt;p&gt;Modern models of environmental and industrial systems have reached a relatively high level of complexity. The latter aspect could hamper an unambiguous understanding of the functioning of a model, i.e., how it drives relationships and dependencies among inputs and outputs of interest. Sensitivity Analysis tools can be employed to examine this issue.&lt;/p&gt;&lt;p&gt;Global sensitivity analysis (GSA) approaches rest on the evaluation of sensitivity across the entire support within which system model parameters are supposed to vary. In this broad context, it is important to note that the definition of a sensitivity metric must be linked to the nature of the question(s) the GSA is meant to address. These include, for example: (i) which are the most important model parameters with respect to given model output(s)?; (ii) could we set some parameter(s) (thus assisting model calibration) at prescribed value(s) without significantly affecting model results?; (iii) at which space/time locations can one expect the highest sensitivity of model output(s) to model parameters and/or knowledge of which parameter(s) could be most beneficial for model calibration?&lt;/p&gt;&lt;p&gt;The variance-based Sobol&amp;#8217; Indices (e.g., Sobol, 2001) represent one of the most widespread GSA metrics, quantifying the average reduction in the variance of a model output stemming from knowledge of the input. Amongst other techniques, Dell&amp;#8217;Oca et al. [2017] proposed a moment-based GSA approach which enables one to quantify the influence of uncertain model parameters on the (statistical) moments of a target model output.&lt;/p&gt;&lt;p&gt;Here, we embed in these sensitivity indices the effect of uncertainties both in the system model conceptualization and in the ensuing model(s) parameters. The study is grounded on the observation that physical processes and natural systems within which they take place are complex, rendering target state variables amenable to multiple interpretations and mathematical descriptions. As such, predictions and uncertainty analyses based on a single model formulation can result in statistical bias and possible misrepresentation of the total uncertainty, thus justifying the assessment of multiple model system conceptualizations. We then introduce copula-based sensitivity metrics which allow characterizing the global (with respect to the input) value of the sensitivity and the degree of variability (across the whole range of the input values) of the sensitivity for each value that the prescribed model output can possibly undertake, as driven by a governing model. In this sense, such an approach to sensitivity is global with respect to model input(s) and local with respect to model output, thus enabling one to discriminate the relevance of an input across the entire range of values of the modeling goal of interest. The methodology is demonstrated in the context of flow and reactive transport scenarios.&lt;/p&gt;&lt;p&gt;&amp;#160;&lt;/p&gt;&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;&lt;/p&gt;&lt;p&gt;Sobol, I. M., 2001. Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates. Math. Comput. Sim., 55, 271-280.&lt;/p&gt;&lt;p&gt;Dell&amp;#8217;Oca, A., Riva, M., Guadagnini, A., 2017. Moment-based metrics for global sensitivity analysis of hydrological systems. Hydr. Earth Syst. Sci., 21, 6219-6234.&lt;/p&gt;


Sign in / Sign up

Export Citation Format

Share Document