Improving Identifiability in Model Calibration Using Multiple Responses

2012 ◽  
Vol 134 (10) ◽  
Author(s):  
Paul D. Arendt ◽  
Daniel W. Apley ◽  
Wei Chen ◽  
David Lamb ◽  
David Gorsich

In physics-based engineering modeling, the two primary sources of model uncertainty, which account for the differences between computer models and physical experiments, are parameter uncertainty and model discrepancy. Distinguishing the effects of the two sources of uncertainty can be challenging. For situations in which identifiability cannot be achieved using only a single response, we propose to improve identifiability by using multiple responses that share a mutual dependence on a common set of calibration parameters. To that end, we extend the single response modular Bayesian approach for calculating posterior distributions of the calibration parameters and the discrepancy function to multiple responses. Using an engineering example, we demonstrate that including multiple responses can improve identifiability (as measured by posterior standard deviations) by an amount that ranges from minimal to substantial, depending on the characteristics of the specific responses that are combined.

Author(s):  
Zhen Jiang ◽  
Wei Chen ◽  
Daniel W. Apley

In physics-based engineering modeling and uncertainty quantification, distinguishing the effects of two main sources of uncertainty — calibration parameter uncertainty and model discrepancy — is challenging. Previous research has shown that identifiability can sometimes be improved by experimentally measuring multiple responses of the system that share a mutual dependence on a common set of calibration parameters. In this paper, we address the issue of how to select the most appropriate subset of responses to measure experimentally, to best enhance identifiability. We propose a preposterior analysis approach that, prior to conducting the physical experiments but after conducting computer simulations, can predict the degree of identifiability that will result using different subsets of responses to measure experimentally. We quantify identifiability via the posterior covariance of the calibration parameters, and predict it via the preposterior covariance from a modular Bayesian Monte Carlo analysis of a multi-response Gaussian process model. The proposed method is applied to a simply supported beam example to select two out of six responses to best improve identifiability. The estimated preposterior covariance is compared to the actual posterior covariance to demonstrate the effectiveness of the method.


2017 ◽  
Vol 47 (3) ◽  
pp. 681-713 ◽  
Author(s):  
Frank van Berkum ◽  
Katrien Antonio ◽  
Michel Vellekoop

AbstractInsurance companies and pension funds must value liabilities using mortality rates that are appropriate for their portfolio. These can only be estimated in a reliable way from a sufficiently large historical dataset for such portfolios, which is often not available. We overcome this problem by introducing a model to estimate portfolio-specific mortality simultaneously with population mortality. By using a Bayesian framework, we automatically generate the appropriate weighting for the limited statistical information in a given portfolio and the more extensive information that is available for the whole population. This allows us to separate parameter uncertainty from uncertainty due to the randomness in individual deaths for a given realization of mortality rates. When we apply our method to a dataset of assured lives in England and Wales, we find that different prior specifications for the portfolio-specific factors lead to significantly different posterior distributions for hazard rates. However, in short-term predictive distributions for future numbers of deaths, individual mortality risk turns out to be more important than parameter uncertainty in the portfolio-specific factors, both for large and for small portfolios.


2015 ◽  
Vol 47 (2) ◽  
pp. 239-259 ◽  
Author(s):  
Teklu T. Hailegeorgis ◽  
Knut Alfredsen

Identification of distributed precipitation–runoff models for hourly runoff simulation based on transfer of full parameters (FP) and partial parameters (PP) are lacking for boreal mid-Norway. We evaluated storage–discharge relationships based model (Kirchmod), the Basic-Grid-Model (BGM) and a simplified Hydrologiska Byråns Vattenbalansavdelning (HBV) model for multi-basins (26 catchments). A regional calibration objective function, which uses all streamflow records in the region, was used to optimize local calibration parameters for each catchment and regional parameters yielding maximum regional weighted average (MRWA) performance measures (PM). Based on regional median Nash–Sutcliffe efficiency (NSE) and NSEln (for log-transformed series) for the calibration and validation periods, the Kirchmod model performed better than the others. Parsimony of the Kirchmod model provided less parameter uncertainty for the FP case but did not guarantee parameter identifiability. Tradeoffs between parsimony and performance were observed despite advantages of parsimony to reduce parameter correlations for the PP, which requires preliminary sensitivity analysis to identify which parameters to transfer. There are potential advantages of using the MRWA method for parameter transfer in space. However, temporal validation indicated marked deterioration of the PM. The tradeoffs between parameter transfers in space and time substantiate both spatial and temporal validation of the regional calibration methodology.


2012 ◽  
Vol 9 (1) ◽  
pp. 879-926 ◽  
Author(s):  
M. Migliavacca ◽  
O. Sonnentag ◽  
T. F. Keenan ◽  
A. Cescatti ◽  
J. O'Keefe ◽  
...  

Abstract. Phenology, the timing of recurring life cycle events, controls numerous land surface feedbacks to the climate systems through the regulation of exchanges of carbon, water and energy between the biosphere and atmosphere. Land surface models, however, are known to have systematic errors in the simulation of spring phenology, which potentially could propagate to uncertainty in modeled responses to future climate change. Here, we analyzed the Harvard Forest phenology record to investigate and characterize the sources of uncertainty in phenological forecasts and the subsequent impacts on model forecasts of carbon and water cycling in the future. Using a model-data fusion approach, we combined information from 20 yr of phenological observations of 11 North American woody species with 12 phenological models of different complexity to predict leaf bud-burst. The evaluation of different phenological models indicated support for spring warming models with photoperiod limitations and, though to a lesser extent, to chilling models based on the alternating model structure. We assessed three different sources of uncertainty in phenological forecasts: parameter uncertainty, model uncertainty, and driver uncertainty. The latter was characterized running the models to 2099 using 2 different IPCC climate scenarios (A1fi vs. B1, i.e. high CO2 emissions vs. low CO2 emissions scenario). Parameter uncertainty was the smallest (average 95% CI: 2.4 day century−1 for scenario B1 and 4.5 day century−1 for A1fi), whereas driver uncertainty was the largest (up to 8.4 day century−1 in the simulated trends). The uncertainty related to model structure is also large and the predicted bud-burst trends as well as the shape of the smoothed projections varied somewhat among models (±7.7 day century−1 for A1fi, ±3.6 day century−1 for B1). The forecast sensitivity of bud-burst to temperature (i.e. days bud-burst advanced per degree of warming) varied between 2.2 day °C−1 and 5.2 day °C−1 depending on model structure. We quantified the impact of uncertainties in bud-burst forecasts on simulated carbon and water fluxes using a process-based terrestrial biosphere model. Uncertainty in phenology model structure led to uncertainty in the description of the seasonality of processes, which accumulated to uncertainty in annual model estimates of gross primary productivity (GPP) and evapotranspiration (ET) of 9.6% and 2.9% respectively. A sensitivity analysis shows that a variation of ±10 days in bud-burst dates led to a variation of ±5.0% for annual GPP and about ±2.0% for ET. For phenology models, differences among future climate scenarios represent the largest source of uncertainty, followed by uncertainties related to model structure, and finally, uncertainties related to model parameterization. The uncertainties we have quantified will affect the description of the seasonality of processes and in particular the simulation of carbon uptake by forest ecosystems, with a larger impact of uncertainties related to phenology model structure, followed by uncertainties related to phenological model parameterization.


Water ◽  
2019 ◽  
Vol 11 (11) ◽  
pp. 2417 ◽  
Author(s):  
Yiheng Xiang ◽  
Lu Li ◽  
Jie Chen ◽  
Chong-Yu Xu ◽  
Jun Xia ◽  
...  

The impacts of climate change on water resources in snow- and glacier-dominated basins are of great importance for water resource management. The Snowmelt Runoff Model (SRM) was developed to simulate and predict daily streamflow for high mountain basins where snowmelt runoff is a major contributor. However, there are many sources of uncertainty when using an SRM for hydrological simulations, such as low-quality input data, imperfect model structure and model parameters, and uncertainty from climate scenarios. Among these, the identification of model parameters is considered to be one of the major sources of uncertainty. This study evaluates the parameter uncertainty for SRM simulation based on different calibration strategies, as well as its impact on future hydrological projections in a data-scarce deglaciating river basin. The generalized likelihood uncertainty estimation (GLUE) method implemented by Monte Carlo sampling was used to estimate the model uncertainty arising from parameters calibrated by means of different strategies. Future snowmelt runoff projections under climate change impacts in the middle of the century and their uncertainty were assessed using average annual hydrographs, annual discharge and flow duration curves as the evaluation criteria. The results show that: (1) the strategy with a division of one or two sub-period(s) in a hydrological year is more appropriate for SRM calibration, and is also more rational for hydrological climate change impact assessment; (2) the multi-year calibration strategy is also more stable; and (3) the future runoff projection contains a large amount of uncertainty, among which parameter uncertainty plays a significant role. The projections also indicate that the onset of snowmelt runoff is likely to shift earlier in the year, and the discharge over the snowmelt season is projected to increase. Overall, this study emphasizes the importance of considering the parameter uncertainty of time-varying hydrological processes in hydrological modelling and climate change impact assessment.


2018 ◽  
Vol 7 (3.18) ◽  
pp. 4
Author(s):  
Donald Stephen ◽  
Shahren Ahmad Zaidi Adruce

When utilizing single-response questions for a survey, researchers often overlook the possibility that an item can have a smorgasbord of viable answers. It results in the loss of information as it forces the respondents to select a best-of-fit option. A multiple-responses question allows the respondent to select any number of answers from a set of preformatted options. The ability to capture a flexible number of responses allows collectively exhaustive concepts to manifest for deductive verification. This paper explores the practical use of Cochran’s Q test and pairwise McNemar test to examine the proportion of responses derived from the results of Multiple Responses Analysis (MRA). This includes Cochran’s Q operation on MRA data table using a simulated data set. Cochran’s Q test detects if there is a difference in the proportion of multiple concepts. In the case of a significant result, it would require a post hoc analysis to pinpoint the exact difference in pairwise proportions. This pairwise difference can be detected by utilizing pairwise McNemar test with Bonferroni Correction. This paper serves as a reference for researchers and practitioners who need to examine the proportion of collectively exhaustive concepts collected from a multiple responses item.  


2012 ◽  
Vol 9 (6) ◽  
pp. 2063-2083 ◽  
Author(s):  
M. Migliavacca ◽  
O. Sonnentag ◽  
T. F. Keenan ◽  
A. Cescatti ◽  
J. O'Keefe ◽  
...  

Abstract. Phenology, the timing of recurring life cycle events, controls numerous land surface feedbacks to the climate system through the regulation of exchanges of carbon, water and energy between the biosphere and atmosphere. Terrestrial biosphere models, however, are known to have systematic errors in the simulation of spring phenology, which potentially could propagate to uncertainty in modeled responses to future climate change. Here, we used the Harvard Forest phenology record to investigate and characterize sources of uncertainty in predicting phenology, and the subsequent impacts on model forecasts of carbon and water cycling. Using a model-data fusion approach, we combined information from 20 yr of phenological observations of 11 North American woody species, with 12 leaf bud-burst models that varied in complexity. Akaike's Information Criterion indicated support for spring warming models with photoperiod limitations and, to a lesser extent, models that included chilling requirements. We assessed three different sources of uncertainty in phenological forecasts: parameter uncertainty, model uncertainty, and driver uncertainty. The latter was characterized running the models to 2099 using 2 different IPCC climate scenarios (A1fi vs. B1, i.e. high CO2 emissions vs. low CO2 emissions scenario). Parameter uncertainty was the smallest (average 95% Confidence Interval – CI: 2.4 days century−1 for scenario B1 and 4.5 days century−1 for A1fi), whereas driver uncertainty was the largest (up to 8.4 days century−1 in the simulated trends). The uncertainty related to model structure is also large and the predicted bud-burst trends as well as the shape of the smoothed projections varied among models (±7.7 days century−1 for A1fi, ±3.6 days century−1 for B1). The forecast sensitivity of bud-burst to temperature (i.e. days bud-burst advanced per degree of warming) varied between 2.2 days °C−1 and 5.2 days °C−1 depending on model structure. We quantified the impact of uncertainties in bud-burst forecasts on simulated photosynthetic CO2 uptake and evapotranspiration (ET) using a process-based terrestrial biosphere model. Uncertainty in phenology model structure led to uncertainty in the description of forest seasonality, which accumulated to uncertainty in annual model estimates of gross primary productivity (GPP) and ET of 9.6% and 2.9%, respectively. A sensitivity analysis shows that a variation of ±10 days in bud-burst dates led to a variation of ±5.0% for annual GPP and about ±2.0% for ET. For phenology models, differences among future climate scenarios (i.e. driver) represent the largest source of uncertainty, followed by uncertainties related to model structure, and finally, related to model parameterization. The uncertainties we have quantified will affect the description of the seasonality of ecosystem processes and in particular the simulation of carbon uptake by forest ecosystems, with a larger impact of uncertainties related to phenology model structure, followed by uncertainties related to phenological model parameterization.


Sign in / Sign up

Export Citation Format

Share Document