RAT (Robustness Assessment Test): a straightforward evaluation of hydrological model robustness to a changing climate

2021 ◽  
Author(s):  
Vazken Andréassian ◽  
Léonard Santos ◽  
Torben Sonnenborg ◽  
Alban de Lavenne ◽  
Göran Lindström ◽  
...  
Keyword(s):  
Climate Change ◽  
Hydrological Model ◽  
Hydrological Models ◽  
Climate Variables ◽  
Data Set ◽  
Hydrological Characteristics ◽  
Changing Climate ◽  
Split Sample ◽  
Sample Test ◽  
Model Robustness

<p>Hydrological models are increasingly used under evolving climatic conditions. They should thus be evaluated regarding their temporal transferability (application in different time periods) and extrapolation capacity (application beyond the range of known past conditions). In theory, parameters of hydrological models are independent of climate. In practice, however, many published studies based on the Split-Sample Test (Klemeš, 1986), have shown that model performances decrease systematically when it is used out of its calibration period. The RAT test proposed here aims at evaluating model robustness to a changing climate by assessing potential undesirable dependencies of hydrological model performances to climate variables. The test compares, over a long data period, the annual value of several climate variables (temperature, precipitation and aridity index) and the bias of the model over each year. If a significant relation exists between the climatic variable and the bias, the model is not considered to be robust to climate change on the catchment. The test has been compared to the Generalized Split-Sample Test (Coron et al., 2012) and showed similar results.</p><p>Here, we report on a large scale application of the test for three hydrological models with different level of complexity (GR6J, HYPE, MIKE-SHE) on a data set of 352 catchments in Denmark, France and Sweden. The results show that the test behaves differently given the evaluated variable (be temperature, precipitation or aridity) and the hydrological characteristics of each catchment. They also show that, although of different level of complexity, the robustness of the three models is similar on the overall data set. However, they are not robust on the same catchments and, then, are not sensitive to the same hydrological characteristics. This example highlights the applicability of the RAT test regardless of the model set-up and calibration procedure and its ability to provide a first evaluation of the model robustness to climate change.</p><p> </p><p><strong>References</strong></p><p>Coron, L., V. Andréassian, C. Perrin, J. Lerat, J. Vaze, M. Bourqui, and F. Hendrickx, 2012. Crash testing hydrological models in contrasted climate conditions: An experiment on 216 Australian catchments, Water Resour. Res., 48, W05552, doi:10.1029/2011WR011721</p><p>Klemeš, V., 1986. Operational testing of hydrological simulation models, Hydrol. Sci. J., 31, 13–24, doi:10.1080/02626668609491024</p><p> </p>

scholarly journals Technical Note – RAT: a Robustness Assessment Test for calibrated and uncalibrated hydrological models

2021 ◽  
Author(s):  
Pierre Nicolle ◽  
Vazken Andréassian ◽  
Paul Royer-Gaspard ◽  
Charles Perrin ◽  
Guillaume Thirel ◽  
...  
Keyword(s):  
Climate Change ◽  
Climate Change Impact ◽  
Hydrological Model ◽  
Technical Note ◽  
New Method ◽  
Hydrological Models ◽  
Split Sample ◽  
Sample Test ◽  
Change Impact ◽  
Robustness Assessment

Abstract. In this note, we present RAT, a new method to assess the robustness of hydrological models. RAT can be seen as an alternative to the classical split-sample test widely used in hydrology. And because the RAT method does not require multiple calibrations, we suggest that it can be applied even to uncalibrated models. The RAT method can be used to determine whether a hydrological model is "safe" for being used for climate change impact studies.

scholarly journals The transferability of hydrological models under nonstationary climatic conditions

2012 ◽  
Vol 16 (4) ◽  
pp. 1239-1254 ◽  
Author(s):  
C. Z. Li ◽  
L. Zhang ◽  
H. Wang ◽  
Y. Q. Zhang ◽  
F. L. Yu ◽  
...  
Keyword(s):  
Monte Carlo Simulation ◽  
Monte Carlo ◽  
Hydrological Model ◽  
Climate Scenario ◽  
Climatic Conditions ◽  
Performance Criteria ◽  
Future Climate Change ◽  
Hydrological Models ◽  
Split Sample ◽  
Sample Test

Abstract. This paper investigates issues involved in calibrating hydrological models against observed data when the aim of the modelling is to predict future runoff under different climatic conditions. To achieve this objective, we tested two hydrological models, DWBM and SIMHYD, using data from 30 unimpaired catchments in Australia which had at least 60 yr of daily precipitation, potential evapotranspiration (PET), and streamflow data. Nash-Sutcliffe efficiency (NSE), modified index of agreement (d1) and water balance error (WBE) were used as performance criteria. We used a differential split-sample test to split up the data into 120 sub-periods and 4 different climatic sub-periods in order to assess how well the calibrated model could be transferred different periods. For each catchment, the models were calibrated for one sub-period and validated on the other three. Monte Carlo simulation was used to explore parameter stability compared to historic climatic variability. The chi-square test was used to measure the relationship between the distribution of the parameters and hydroclimatic variability. The results showed that the performance of the two hydrological models differed and depended on the model calibration. We found that if a hydrological model is set up to simulate runoff for a wet climate scenario then it should be calibrated on a wet segment of the historic record, and similarly a dry segment should be used for a dry climate scenario. The Monte Carlo simulation provides an effective and pragmatic approach to explore uncertainty and equifinality in hydrological model parameters. Some parameters of the hydrological models are shown to be significantly more sensitive to the choice of calibration periods. Our findings support the idea that when using conceptual hydrological models to assess future climate change impacts, a differential split-sample test and Monte Carlo simulation should be used to quantify uncertainties due to parameter instability and non-uniqueness.

scholarly journals The transferability of hydrological models under nonstationary climatic conditions

2011 ◽  
Vol 8 (5) ◽  
pp. 8701-8736 ◽  
Author(s):  
C. Z. Li ◽  
L. Zhang ◽  
H. Wang ◽  
Y. Q. Zhang ◽  
F. L. Yu ◽  
...  
Keyword(s):  
Monte Carlo Simulation ◽  
Monte Carlo ◽  
Hydrological Model ◽  
Climate Scenario ◽  
Climatic Conditions ◽  
Performance Criteria ◽  
Future Climate Change ◽  
Hydrological Models ◽  
Split Sample ◽  
Sample Test

Abstract. This paper investigates issues involved in calibrating hydrological models against observed data when the aim of the modelling is to predict future runoff under different climatic conditions. To achieve this objective, we tested two hydrological models, DWBM and SIMHYD, using data from 30 unimpaired catchments in Australia which had at least 60 years of daily precipitation, potential evapotranspiration (PET), and streamflow data. Nash-Sutcliffe efficiency (NSE) and absolute percentage water balance error (WBE) were used as performance criteria. We used a differential split-sample test to split up the data into 120 sub-periods and 4 different climatic sub-periods in order to assess how well the calibrated model could be transferred different periods. For each catchment, the models were calibrated for one sub-period and validated on the other three. Monte Carlo simulation was used to explore parameter stability compared to historic climatic variability. The chi-square test was used to measure the relationship between the distribution of the parameters and hydroclimatic variability. The results showed that the performance of the two hydrological models differed and depended on the model calibration. We found that if a hydrological model is set up to simulate runoff for a wet climate scenario then it should be calibrated on a wet segment of the historic record, and similarly a dry segment should be used for a dry climate scenario. The Monte Carlo simulation provides an effective and pragmatic approach to explore uncertainty and equifinality in hydrological model parameters. Some parameters of the hydrological models are shown to be significantly more sensitive to the choice of calibration periods. Our findings support the idea that when using conceptual hydrological models to assess future climate change impacts, a differential split-sample test and Monte Carlo simulation can reduce uncertainties due to parameter instability and non-uniqueness.

scholarly journals Technical note: PMR – a proxy metric to assess hydrological model robustness in a changing climate

2021 ◽  
Author(s):  
Paul Royer-Gaspard ◽  
Vazken Andréassian ◽  
Guillaume Thirel
Keyword(s):  
Hydrological Model ◽  
Evaluation Studies ◽  
Computational Cost ◽  
Climatic Conditions ◽  
Technical Note ◽  
Performance Metric ◽  
Reliable Assessment ◽  
Sample Test ◽  
The Impact ◽  
Model Robustness

Abstract. The ability of hydrological models to perform in climatic conditions different from those encountered in calibration is crucial to ensure a reliable assessment of the impact of climate change in water management sectors. However, most evaluation studies based on the Differential Split-Sample Test (DSST) endorsed the consensus that rainfall-runoff models lack climatic robustness. Models typically exhibit substantial errors on streamflow volumes applied under climatologically different conditions. In this technical note, we propose a new performance metric to evaluate model robustness without applying the DSST and which performs with a single hydrological model calibration. The Proxy for Model Robustness (PMR) is based on the systematic computation of model error on sliding sub-periods of the whole streamflow time series. We demonstrate that the metric shows patterns similar to those obtained with the DSST for a conceptual model on a set of 377 French catchments. An analysis of sensitivity to the length of the sub-periods shows that this length influences the values of the PMR and its adequation with DSST biases. We recommend a range of a few years for the choice of sub-period lengths, although this should be context-dependent. Our work makes it possible to evaluate the temporal transferability of any hydrological model, including uncalibrated models, at a very low computational cost.

Using machine learning to downscale simulations of climate change induced changes to the shallow groundwater table

2021 ◽  
Author(s):  
Raphael Schneider ◽  
Hans Jørgen Henriksen ◽  
Julian Koch ◽  
Lars Troldborg ◽  
Simon Stisen
Keyword(s):  
Climate Change ◽  
Climate Models ◽  
Hydrological Model ◽  
Shallow Groundwater ◽  
Groundwater Table ◽  
Training Data ◽  
Computational Time ◽  
Hydrological Models ◽  
Shallow Groundwater Table ◽  
Induced Changes

<p>The DK-model (https://vandmodel.dk/in-english) is a national water resource model, covering all of Denmark. Its core is a distributed, integrated surface-subsurface hydrological model in 500m horizontal resolution. With recent efforts, a version at a higher resolution of 100m was created. The higher resolution was, amongst others, desired by end-users and to better represent surface and surface-near phenomena such as the location of the uppermost groundwater table. Being presently located close to the surface across substantial parts of the country and partly expected to rise, the groundwater table and its future development due to climate change is of great interest. A rising groundwater table is associated with potential risks for infrastructure, agriculture and ecosystems. However, the 25-fold jump in resolution of the hydrological model also increases the computational effort. Hence, it was deemed unfeasible to run the 100m resolution hydrological model nation-wide with an ensemble of climate models to evaluate climate change impact. The full ensemble run could only be performed with the 500m version of the model. To still produce the desired outputs at 100m resolution, a downscaling method was applied as described in the following.</p><p>Five selected subcatchment models covering around 9% of Denmark were run with five selected climate models at 100m resolution (using less than 3% of the computational time for hydrological models compared to a national, full ensemble run at 100m). Using the simulated changes at 100m resolution from those models as training data, combined with a set of covariates including the simulated changes in 500m resolution, Random Forest (RF) algorithms were trained to downscale simulated changes from 500m to 100m.</p><p>Generalizing the trained RF algorithms, Denmark-wide maps of expected climate change induced changes to the shallow groundwater table at 100m resolution were modelled. To verify the downscaling results, amongst others, the RF algorithms were successfully validated against results from a sixth hydrological subcatchment model at 100m resolution not used in training the algorithms.</p><p>The experience gained also opens for various other applications of similar algorithms where computational limitations inhibit running distributed hydrological models at fine resolutions: The results suggest the potential to downscale other model outputs that are desired at fine resolutions.</p>

Are large scale models useful? A case of nested model domains for assessing current and future stream runoff and sediments

2021 ◽  
Author(s):  
Alena Bartosova ◽  
Berit Arheimer ◽  
Alban de Lavenne ◽  
René Capell ◽  
Johan Strömqvist
Keyword(s):  
Climate Change ◽  
Spatial Patterns ◽  
Large Scale ◽  
Aridity Index ◽  
Global Climate Models ◽  
Hydrological Models ◽  
Nested Model ◽  
Nested Models ◽  
Changing Climate ◽  
Scale Models

<p>Continental and global dynamic hydrological models have emerged recently as tools for e.g. flood forecasting, large-scale climate impact analyses, and estimation of time-dynamic water fluxes into sea basins. One such tool is a dynamic process-based rainfall-runoff and water quality model Hydrological Predictions for Environment (HYPE). We present and compare historical simulations of runoff, soil moisture, aridity, and sediment concentrations for three nested model domains using global, continental (Europe), and national (Sweden) catchment-based HYPE applications. Future impacts on hydrological variables from changing climate were then assessed using the global and continental HYPE applications with ensembles based on 3 CMIP5 global climate models (GCMs).</p><p>Simulated historical sediment concentrations varied considerably among the nested models in spatial patterns while runoff values were more similar. Regardless of the variation, the global model was able to provide information on climate change impacts comparable to those from the continental and national models for hydrological indicators. Output variables that were calibrated, e.g. runoff, were shown to result in more reliable and consistent projected changes among the different model scales than derived variables such as the actual aridity index. The comparison was carried out for ensemble averages as well as individual GCMs to illustrate the variability and the need for robust assessments.</p><p>Global hydrological models are shown to be valuable tools for e.g. first screenings of climate change effects and detection of spatial patterns and can be useful to provide information on current and future hydrological states at various domains. The challenge is (1) in deciding when we should use the large-scale models and (2) in interpreting the results, considering the uncertainty of the model results and quality of data especially at the global scale. Comparison across nested domains demonstrates the significance of scale which needs to be considered when interpreting the impacts alongside with model performance.</p><p>Bartosova et al, 2021: Large-scale hydrological and sediment modeling in nested domains under current and changing climate. Accepted to Special Issue Journal of Hydraulic Engineering.</p>

scholarly journals Implication of calibration period on modelling climate change impact on future runoff

2015 ◽  
Vol 371 ◽  
pp. 3-6 ◽  
Author(s):  
F. H. S. Chiew ◽  
H. Zheng ◽  
J. Vaze
Keyword(s):  
Climate Change ◽  
Climate Change Impact ◽  
Hydrological Model ◽  
Historical Data ◽  
Hydrological Models ◽  
Calibration Period ◽  
Calibration Dataset ◽  
Change Impact ◽  
Hydrologic Responses ◽  
Data Period

Abstract. This paper explores the consideration and implication of calibration period on the modelled climate change impact on future runoff. The results show that modelled runoff and hydrologic responses can be influenced by the choice of historical data period used to calibrate and develop the hydrological model. Modelling approaches that do not take this into account may therefore underestimate the range and uncertainty in future runoff projections. Nevertheless, the uncertainty associated with the choice of hydrological models and consideration of calibration dataset for modelling climate change impact on runoff is likely to be small compared to the uncertainty in the future rainfall projections.

scholarly journals A comparative analysis of projected impacts of climate change on river runoff from global and catchment-scale hydrological models

2010 ◽  
Vol 7 (5) ◽  
pp. 7191-7229 ◽  
Author(s):  
S. N. Gosling ◽  
R. G. Taylor ◽  
N. W. Arnell ◽  
M. C. Todd
Keyword(s):  
Climate Change ◽  
Hydrological Model ◽  
Climate Model ◽  
Annual Runoff ◽  
Structural Uncertainty ◽  
Hydrological Models ◽  
Catchment Scale ◽  
Monthly Runoff ◽  
Global Mean ◽  
Impacts Of Climate Change

Abstract. We present a comparative analysis of projected impacts of climate change on river runoff from two types of distributed hydrological model, a global hydrological model (GHM) and catchment-scale hydrological models (CHM). Analyses are conducted for six catchments that are global in coverage and feature strong contrasts in spatial scale as well as climatic and developmental conditions. These include the Liard (Canada), Mekong (SE Asia), Okavango (SW Africa), Rio Grande (Brazil), Xiangxi (China) and Harper's Brook (UK). A single GHM (Mac-PDM.09) is applied to all catchments whilst different CHMs are applied for each catchment. The CHMs include SLURP v. 12.2 (Liard), SLURP v. 12.7 (Mekong), Pitman (Okavango), MGB-IPH (Rio Grande), AV-SWAT-X 2005 (Xiangxi) and Cat-PDM (Harper's Brook). Simulations of mean annual runoff, mean monthly runoff and high (Q5) and low (Q95) monthly runoff under baseline (1961–1990) and climate change scenarios are presented. We compare the simulated runoff response of each hydrological model to (1) prescribed increases in global-mean air temperature of 1.0, 2.0, 3.0, 4.0, 5.0 and 6.0 °C relative to baseline from the UKMO HadCM3 Global Climate Model (GCM) to explore response to different amounts of climate forcing, and (2) a prescribed increase in global-mean air temperature of 2.0 °C relative to baseline for seven GCMs to explore response to climate model structural uncertainty. We find that the differences in projected changes of mean annual runoff between the two types of hydrological model can be substantial for a given GCM, and they are generally larger for indicators of high and low monthly runoff. However, they are relatively small in comparison to the range of projections across the seven GCMs. Hence, for the six catchments and seven GCMs we considered, climate model structural uncertainty is greater than the uncertainty associated with the type of hydrological model applied. Moreover, shifts in the seasonal cycle of runoff with climate change are represented similarly by both hydrological models, although for some catchments the monthly timing of high and low flows differs. This implies that for studies that seek to quantify and assess the role of climate model uncertainty on catchment-scale runoff, it may be equally as feasible to apply a GHM as it is to apply a CHM, especially when climate modelling uncertainty across the range of available GCMs is as large as it currently is. Whilst the GHM is able to represent the broad climate change signal that is represented by the CHMs, we find however, that for some catchments there are differences between GHMs and CHMs in mean annual runoff due to differences in potential evapotranspiration estimation methods, in the representation of the seasonality of runoff, and in the magnitude of changes in extreme (Q5, Q95) monthly runoff, all of which have implications for future water management issues.

scholarly journals High-resolution Med-CORDEX regional climate model simulations for hydrological impact studies: a first evaluation in Morocco

2013 ◽  
Vol 10 (5) ◽  
pp. 5687-5737 ◽  
Author(s):  
Y. Tramblay ◽  
D. Ruelland ◽  
S. Somot ◽  
R. Bouaicha ◽  
E. Servat
Keyword(s):  
Climate Change ◽  
High Resolution ◽  
Regional Climate Model ◽  
Bias Correction ◽  
Hydrological Model ◽  
Climate Model ◽  
Regional Climate ◽  
Impact Studies ◽  
Hydrological Impact ◽  
Split Sample

Abstract. In the framework of the international CORDEX program, new regional climate model (RCM) simulations at high spatial resolutions are becoming available for the Mediterranean region (Med-CORDEX initiative). This study provides the first evaluation for hydrological impact studies of these high-resolution simulations. Different approaches are compared to analyze the climate change impacts on the hydrology of a catchment located in North Morocco, using a high-resolution RCM (ALADIN-Climate) from the Med-CORDEX initiative at two different spatial resolutions (50 km and 12 km) and for two different Radiative Concentration Pathway scenarios (RCP4.5 and RCP8.5). The main issues addressed in the present study are: (i) what is the impact of increased RCM resolution on present-climate hydrological simulations and on future projections? (ii) Are the bias-correction of the RCM model and the parameters of the hydrological model stationary and transferable to different climatic conditions? (iii) What is the climate and hydrological change signal based on the new Radiative Concentration Pathways scenarios (RCP4.5 and RCP8.5)? Results indicate that high resolution simulations at 12 km better reproduce the seasonal patterns, the seasonal distributions and the extreme events of precipitation. The parameters of the hydrological model, calibrated to reproduce runoff at the monthly time step over the 1984–2010 period, do not show a strong variability between dry and wet calibration periods in a differential split-sample test. However the bias correction of precipitation by quantile-matching does not give satisfactory results in validation using the same differential split-sample testing method. Therefore a quantile-perturbation method that does not rely on any stationarity assumption and produces ensembles of perturbed series of precipitation was introduced. The climate change signal under scenarios 4.5 and 8.5 indicates a decrease of respectively −30% to −57% in surface runoff for the mid-term (2041–2062), when for the same period the projections for precipitation are ranging between −15% and −19% and for temperature between +1.28°C and +1.87°C.

Sign in / Sign up

Export Citation Format

Share Document