Objective functions used as performance metrics for hydrological models: state-of-the-art and critical analysis

ABSTRACT Hydrological models (HMs) can be applied for different purposes, and a key step is model calibration using objective functions (OF) to quantify the agreement between observed and calculated discharges. Fully understanding the OF is important to properly take advantage of model calibration and interpret the results. This study evaluates 36 OF proposed in the literature, considering two watersheds of different hydrological regimes. Daily simulated streamflow time-series, using a distributed hydrological model (MGB-IPH), and ten daily streamflow synthetic time-series, generated from the observed and calculated streamflows, were used in the analysis of each watershed. These synthetic data were used to evaluate how does each metric evaluate hypothetical cases that present isolated very well known error behaviors. Despite of all NSE-derived (Nash-Sutcliffe efficiency) metrics that use the square of the residuals in their formulation have shown higher sensitivity to errors in high flows, the ones that use daily and monthly averages of flow rates in absolute terms were more stringent than the others to assess HMs performance. Low flow errors were better evaluated by metrics that use the flow logarithm. The constant presence of zero flow rates deteriorate them significantly, with the exception of the metrics TRMSE (Transformed root mean square error) did not demonstrate this problem. An observed limitation of the formulations of some metrics was that the errors of overestimation or underestimation are compensated. Our results reassert that each metric should be interpreted specifically thinking about the aspects it has been proposed for, and simultaneously taking into account a set of metrics would lead to a broader evaluation of HM ability (e.g. multiobjective model evaluation). We recommend that the use of synthetic time series as those proposed in this work could be useful as an auxiliary step towards better understanding the evaluation of a calibrated hydrological model for each study case, taking into account model capabilities and observed hydrologic regime characteristics.

Download Full-text

Streamflow characteristics from modeled runoff time series – importance of calibration criteria selection

Hydrology and Earth System Sciences ◽

10.5194/hess-21-5443-2017 ◽

2017 ◽

Vol 21 (11) ◽

pp. 5443-5457 ◽

Cited By ~ 17

Author(s):

Sandra Pool ◽

Marc J. P. Vis ◽

Rodney R. Knight ◽

Jan Seibert

Keyword(s):

Time Series ◽

Model Calibration ◽

Performance Metrics ◽

Low Flow ◽

Flow Conditions ◽

Ungauged Catchments ◽

Calibration Process ◽

Model Efficiency ◽

Runoff Model ◽

Calibration Approach

Abstract. Ecologically relevant streamflow characteristics (SFCs) of ungauged catchments are often estimated from simulated runoff of hydrologic models that were originally calibrated on gauged catchments. However, SFC estimates of the gauged donor catchments and subsequently the ungauged catchments can be substantially uncertain when models are calibrated using traditional approaches based on optimization of statistical performance metrics (e.g., Nash–Sutcliffe model efficiency). An improved calibration strategy for gauged catchments is therefore crucial to help reduce the uncertainties of estimated SFCs for ungauged catchments. The aim of this study was to improve SFC estimates from modeled runoff time series in gauged catchments by explicitly including one or several SFCs in the calibration process. Different types of objective functions were defined consisting of the Nash–Sutcliffe model efficiency, single SFCs, or combinations thereof. We calibrated a bucket-type runoff model (HBV – Hydrologiska Byråns Vattenavdelning – model) for 25 catchments in the Tennessee River basin and evaluated the proposed calibration approach on 13 ecologically relevant SFCs representing major flow regime components and different flow conditions. While the model generally tended to underestimate the tested SFCs related to mean and high-flow conditions, SFCs related to low flow were generally overestimated. The highest estimation accuracies were achieved by a SFC-specific model calibration. Estimates of SFCs not included in the calibration process were of similar quality when comparing a multi-SFC calibration approach to a traditional model efficiency calibration. For practical applications, this implies that SFCs should preferably be estimated from targeted runoff model calibration, and modeled estimates need to be carefully interpreted.

Download Full-text

A new fractal-theory-based criterion for hydrological model calibration

10.5194/hess-2020-543 ◽

2020 ◽

Author(s):

Zhixu Bai ◽

Yao Wu ◽

Di Ma ◽

Yue-Ping Xu

Keyword(s):

Time Series ◽

Model Calibration ◽

Hydrological Model ◽

Fractal Theory ◽

Fractal Dimensions ◽

Hydrological Models ◽

Classical Criterion ◽

Potential Benefits ◽

New Criterion ◽

Squared Residuals

Abstract. Fractality has been found in many areas and has been used to describe the internal features of time series. But is it possible to use fractal theory to improve the performance of hydrological models? This study aims to investigate the potential benefits of applying fractal theory in model calibration. A new criterion named ratio of fractal dimensions (RD) is defined as the ratio of fractal dimensions of simulated and observed streamflow series. To combine the advantages of fractal theory with classical criteria based on squared residuals, a multi-objective calibration strategy is designed. The selected classical criterion is Nash-Sutcliffe efficiency (E). The E-RD strategy is tested in three study cases with different climate and geography. The results of experiment reveal that, from most aspects, introducing RD into model calibration makes the simulation of streamflow components more reasonable. Besides, in calibration, only little decrease of E occurs when pursuing better RD. We therefore recommend choosing the best E among the parameter sets whose RD is around 1.

Download Full-text

Seasonal streamflow forecasting by conditioning climatology with precipitation indices

10.5194/hess-2016-285 ◽

2016 ◽

Author(s):

Louise Crochemore ◽

Maria-Helena Ramos ◽

Florian Pappenberger ◽

Charles Perrin

Keyword(s):

Time Series ◽

Hydrological Model ◽

Historical Data ◽

Standardized Precipitation Index ◽

Low Flow ◽

Drought Risk ◽

Hydrological Models ◽

Streamflow Forecasting ◽

Streamflow Forecasts ◽

Seasonal Streamflow

Abstract. Many fields such as drought risk assessment or reservoir management can benefit from long-range streamflow forecasts. The simplest way to make probabilistic streamflow forecasts can be to use historical streamflow time series, if available. Another approach is to use ensemble climate scenarios as input to a hydrological model. Climatology (i.e. time series of climate conditions recorded over a long time period) has long been used in long-range streamflow forecasting. However, in the last decade, the use of general circulation model (GCM) outputs as input to hydrological models has developed. While precipitation climatology and historical streamflows offer reliable ensembles, forecasts based on GCM outputs can offer sharper ensembles, partly due to the initialisation of GCMs and hydrological models on current conditions. This study proposes to condition historical data based on GCM precipitation forecasts to get the most out of both data sources and improve seasonal streamflow forecasting in France. Four conditioning statistics based on ECMWF System 4 forecasts of cumulative precipitation and of the Standardized Precipitation Index (SPI) were used to select traces within historical streamflows and historical precipitations. The four conditioned precipitation scenarios were used as input to the GR6J hydrological model to obtain eight conditioned streamflow forecast scenarios. These streamflow scenarios were compared to three references: an ensemble based on historical streamflows, the widespread Ensemble Streamflow Prediction (ESP) ensemble, and System 4 precipitation forecasts. These ensembles were evaluated based on their sharpness, reliability and overall performance. An overall comparison of forecast ensembles showed that conditioning past observations based on the three-month Standardized Precipitation Index (SPI3) improved the sharpness of ensembles based on historical data, while maintaining a good reliability. An evaluation of forecast ensembles for low-flow forecasting showed that the SPI3-conditioned ensembles provided reliable forecasts of low-flow duration and deficit volume based on the 80th exceedance percentile. Last, drought risk forecasting was illustrated for the 2003 drought.

Download Full-text

Stepwise improvement of hydrological models using satellite-based evaporation and total water storage estimations

10.5194/egusphere-egu21-796 ◽

2021 ◽

Author(s):

Markus Hrachowitz ◽

Petra Hulsman ◽

Hubert Savenije

Keyword(s):

Model Calibration ◽

Hydrological Model ◽

Model Development ◽

Water Storage ◽

Spatiotemporal Variability ◽

Model Structure ◽

Hydrological Models ◽

Total Water ◽

Spatiotemporal Model ◽

Multiple Variables

Hydrological models are often calibrated with respect to flow observations at the basin outlet. As a result, flow predictions may seem reliable but this is not necessarily the case for the spatiotemporal variability of system-internal processes, especially in large river basins. Satellite observations contain valuable information not only for poorly gauged basins with limited ground observations and spatiotemporal model calibration, but also for stepwise model development. This study explored the value of satellite observations to improve our understanding of hydrological processes through stepwise model structure adaption and to calibrate models both temporally and spatially. More specifically, satellite-based evaporation and total water storage anomaly observations were used to diagnose model deficiencies and to subsequently improve the hydrological model structure and the selection of feasible parameter sets. A distributed, process based hydrological model was developed for the Luangwa river basin in Zambia and calibrated with respect to discharge as benchmark. This model was modified stepwise by testing five alternative hypotheses related to the process of upwelling groundwater in wetlands, which was assumed to be negligible in the benchmark model, and the spatial discretization of the groundwater reservoir. Each model hypothesis was calibrated with respect to 1) discharge and 2) multiple variables simultaneously including discharge and the spatiotemporal variability in the evaporation and total water storage anomalies. The benchmark model calibrated with respect to discharge reproduced this variable well, as also the basin-averaged evaporation and total water storage anomalies. However, the evaporation in wetland dominated areas and the spatial variability in the evaporation and total water storage anomalies were poorly modelled. The model improved the most when introducing upwelling groundwater flow from a distributed groundwater reservoir and calibrating it with respect to multiple variables simultaneously. This study showed satellite-based evaporation and total water storage anomaly observations provide valuable information for improved understanding of hydrological processes through stepwise model development and spatiotemporal model calibration.

Download Full-text

How good are hydrological models for gap-filling streamflow data?

Hydrology and Earth System Sciences ◽

10.5194/hess-22-4593-2018 ◽

2018 ◽

Vol 22 (8) ◽

pp. 4593-4604 ◽

Cited By ~ 8

Author(s):

Yongqiang Zhang ◽

David Post

Keyword(s):

Model Calibration ◽

Hydrological Model ◽

Hydrological Models ◽

Hydrological Response ◽

Gap Filling ◽

Annual Streamflow ◽

Response Variable ◽

Benchmark Data ◽

Streamflow Data ◽

Streamflow Trend

Abstract. Gap-filling streamflow data is a critical step for most hydrological studies, such as streamflow trend, flood, and drought analysis and hydrological response variable estimates and predictions. However, there is a lack of quantitative evaluation of the gap-filled data accuracy in most hydrological studies. Here we show that when the missing data rate is less than 10 %, the gap-filled streamflow data obtained using calibrated hydrological models perform almost the same as the benchmark data (less than 1 % missing) when estimating annual trends for 217 unregulated catchments widely spread across Australia. Furthermore, the relative streamflow trend bias caused by the gap filling is not very large in very dry catchments where the hydrological model calibration is normally poor. Our results clearly demonstrate that the gap filling using hydrological modelling has little impact on the estimation of annual streamflow and its trends.

Download Full-text

Fuzzy committees of specialized rainfall-runoff models: further enhancements and tests

Hydrology and Earth System Sciences ◽

10.5194/hess-17-4441-2013 ◽

2013 ◽

Vol 17 (11) ◽

pp. 4441-4451 ◽

Cited By ~ 6

Author(s):

N. Kayastha ◽

J. Ye ◽

F. Fenicia ◽

V. Kuzmin ◽

D. P. Solomatine

Keyword(s):

Hydrological Model ◽

Weighting Function ◽

Hydrological Models ◽

Membership Functions ◽

Rainfall Runoff ◽

Objective Functions ◽

Hydrological Regimes

Abstract. Often a single hydrological model cannot capture the details of a complex rainfall–runoff relationship, and a possibility here is building specialized models to be responsible for a particular aspect of this relationship and combining them to form a committee model. This study extends earlier work of using fuzzy committees to combine hydrological models calibrated for different hydrological regimes – by considering the suitability of the different weighting function for objective functions and different class of membership functions used to combine the specialized models and compare them with the single optimal models.

Download Full-text

Hydrological model calibration for derived flood frequency analysis using stochastic rainfall and probability distributions of peak flows

Hydrology and Earth System Sciences ◽

10.5194/hess-18-353-2014 ◽

2014 ◽

Vol 18 (1) ◽

pp. 353-365 ◽

Cited By ~ 35

Author(s):

U. Haberlandt ◽

I. Radtke

Keyword(s):

Time Series ◽

Frequency Analysis ◽

Model Calibration ◽

Hydrological Model ◽

Probability Distributions ◽

Flood Frequency ◽

Rainfall Data ◽

Flood Frequency Analysis ◽

Peak Flows ◽

Hourly Rainfall

Abstract. Derived flood frequency analysis allows the estimation of design floods with hydrological modeling for poorly observed basins considering change and taking into account flood protection measures. There are several possible choices regarding precipitation input, discharge output and consequently the calibration of the model. The objective of this study is to compare different calibration strategies for a hydrological model considering various types of rainfall input and runoff output data sets and to propose the most suitable approach. Event based and continuous, observed hourly rainfall data as well as disaggregated daily rainfall and stochastically generated hourly rainfall data are used as input for the model. As output, short hourly and longer daily continuous flow time series as well as probability distributions of annual maximum peak flow series are employed. The performance of the strategies is evaluated using the obtained different model parameter sets for continuous simulation of discharge in an independent validation period and by comparing the model derived flood frequency distributions with the observed one. The investigations are carried out for three mesoscale catchments in northern Germany with the hydrological model HEC-HMS (Hydrologic Engineering Center's Hydrologic Modeling System). The results show that (I) the same type of precipitation input data should be used for calibration and application of the hydrological model, (II) a model calibrated using a small sample of extreme values works quite well for the simulation of continuous time series with moderate length but not vice versa, and (III) the best performance with small uncertainty is obtained when stochastic precipitation data and the observed probability distribution of peak flows are used for model calibration. This outcome suggests to calibrate a hydrological model directly on probability distributions of observed peak flows using stochastic rainfall as input if its purpose is the application for derived flood frequency analysis.

Download Full-text

Preprocessing of hydrological models’ input in eWaterCycle with ESMValTool

10.5194/egusphere-egu21-6051 ◽

2021 ◽

Author(s):

Fakhereh Alidoost ◽

Jerom Aerts ◽

Bouwe Andela ◽

Jaro Camphuijsen ◽

Nick van De Giesen ◽

...

Keyword(s):

Programming Languages ◽

Hydrological Model ◽

Performance Metrics ◽

Climate Model ◽

Meteorological Data ◽

Climate Science ◽

Hydrological Models ◽

Earth System ◽

Evaluation Tool ◽

Spatial Coverage

Hydrological models exhibit great complexity and diversity in the exact methodologies applied, competing for hypotheses of hydrologic behaviour, technology stacks, and programming languages used in those models. The preprocessing of forcing (meteorological) data is often performed by various sets of scripts that may or may not be included with model source codes, making it hard to reproduce results. Moreover, forcing data can be retrieved from a wide variety of forcing products with discrepant variable names and frequencies, spatial and temporal resolutions, and spatial coverage. Even though there is an infinite amount of preprocessing scripts for different models, these preprocessing scripts use only a limited set of operations, mainly re-gridding, temporal and spatial manipulations, variable derivation, and unit conversion. Also, these exact same preprocessing functions are used in analysis and evaluation of output from Earth system models in climate science.Within the context of the eWaterCycle II project (https://www.ewatercycle.org/), a common preprocessing system has been created for hydrological modelling based on ESMValTool (Earth System Model Evaluation Tool). ESMValTool is a community-driven diagnostic and performance metrics tool that supports a broad range of preprocessing functions. Using a YAML script called a recipe, instructions are provided to ESMValTool: the datasets which need to be analyzed, the preprocessors that need to be applied, and the model-specific analysis (i.e. diagnostic script) which need to be run on data. ESMValTool is modular and flexible so all preprocessing functions can also be used directly in a Python script and additional analyses can easily be added.The current preprocessing pipeline of the eWaterCycle using ESMValTool consists of hydrological model-specific scripts and supports ERA5 and ERA-Interim data provided by the ECMWF (European Centre for Medium-Range Weather Forecasts), as well as CMIP5 and CMIP6 climate model data. The pipeline starts with the downloading and CMORization (Climate Model Output Rewriter) of input data. Then a recipe is prepared to find the data and run the preprocessors. When ESMValTool runs a recipe, it produces preprocessed data that can be passed as input to a hydrological model. It will also store provenance and citation information to ensure transparency and reproducibility. This leads to less time spent on building custom preprocessing, more reproducible and comparable hydrological science.In this presentation, we will give an overview of the current preprocessing pipeline of the eWaterCycle, outline ESMValTool preprocessing functions, and introduce available hydrological recipes and diagnostic scripts for the PCRGLOB, WFLOW, HYPE, MARRMOT and LISFLOOD models.

Download Full-text

Multi-objective calibration of a distributed eco-hydrological model using several remotely sensed information

10.5194/egusphere-egu2020-18880 ◽

2020 ◽

Author(s):

Félix Francés ◽

Carlos Echeverría ◽

Maria Gonzalez-Sanchis ◽

Fernando Rivas

Keyword(s):

Remote Sensing ◽

Model Calibration ◽

Hydrological Model ◽

Model Performance ◽

Temporal Information ◽

Temporal Data ◽

Hydrological Models ◽

Validation Period ◽

Spatio Temporal ◽

Calibration Approach

Calibration of eco-hydrological models is difficult to carry on, even more if observed data sets are scarce. It is known that calibration using traditional trial-and-error approach depends strongly of the knowledge and the subjectivity of the hydrologist, and automatic calibration has a strong dependency of the objective-function and the initial values established to initialize the process.The traditional calibration approach mainly focuses on the temporal variation of the discharge at the catchment outlet point, representing an integrated catchment response and provides thus only limited insight on the lumped behaviour of the catchment. It has been long demonstrated the limited capabilities of such an approach when models are validated at interior points of a river basin. The development of distributed eco-hydrological models and the burst of spatio-temporal data provided by remote sensing appear as key alternative to overcome those limitations. Indeed, remote sensing imagery provides not only temporal information but also valuable information on spatial patterns, which can facilitate a spatial-pattern-oriented model calibration.However, there is still a lack of how to effectively handle spatio-temporal data when included in model calibration and how to evaluate the accuracy of the simulated spatial patterns. Moreover, it is still unclear whether including spatio-temporal data improves model performance in face to an unavoidable more complex and time-demanding calibration procedure. To elucidate in this sense, we performed three different multiobjective calibration configurations: (1) including only temporal information of discharges at the catchment outlet (2) including both temporal and spatio-temporal information and (3) only including spatio-temporal information. In the three approaches, we calibrated the same distributed eco-hydrological model (TETIS) in the same study area: Carraixet Basin, and used the same multi-objective algorithm: MOSCEM-UA. The spatio-temporal information obtained from satellite has been the surface soil moisture (from SMOS-BEC) and the leaf area index (from MODIS).Even though the performance of the first calibration approach (only temporal information included) was slightly better than the others, all calibration approaches provided satisfactory and similar results within the calibration period. To put these results into test, we also validated the model performance by using historical data that was not used to calibrate the model (validation period). Within the validation period, the second calibration approach obtained better performance than the others, pointing out the higher reliability of the obtained parameter values when including spatio-temporal data (in this case, in combination with temporal data) in the model calibration. It is also reliable to mention that the approaches considering only spatio-temporal information provided interesting results in terms of discharges, considering that this variable was not used at all for calibration purposes.

Download Full-text

Performance of HEC-HMS and SWAT to simulate streamflow in the sub-humid tropical Hemavathi catchment

Journal of Water and Climate Change ◽

10.2166/wcc.2021.072 ◽

2021 ◽

Author(s):

N. C. Sanjay Shekar ◽

D. C. Vinay

Keyword(s):

Model Calibration ◽

Assessment Tool ◽

Swat Model ◽

Model Performance ◽

Curve Number ◽

Coefficient Of Determination ◽

Hydrological Models ◽

Daily Streamflow ◽

Compensation Factor ◽

Elevation Model

Abstract The present study was conducted to examine the accuracy and applicability of the hydrological models Soil and Water Assessment Tool (SWAT) and Hydrologic Engineering Center (HEC)- Hydrologic Modeling System (HMS) to simulate streamflows. Models combined with the ArcGIS interface have been used for hydrological study in the humid tropical Hemavathi catchment (5,427 square kilometer). The critical focus of the streamflow analysis was to determine the efficiency of the models when the models were calibrated and optimized using observed flows in the simulation of streamflows. Daily weather gauge stations data were used as inputs for the models from 2014–2020 period. Other data inputs required to run the models included land use/land cover (LU/LC) classes resulting from remote sensing satellite imagery, soil map and digital elevation model (DEM). For evaluating the model performance and calibration, daily stream discharge from the catchment outlet data were used. For the SWAT model calibration, available water holding capacity by soil (SOL_AWC), curve number (CN) and soil evaporation compensation factor (ESCO) are identified as the sensitive parameters. Initial abstraction (Ia) and lag time (Tlag) are the significant parameters identified for the HEC-HMS model calibration. The models were subsequently adjusted by autocalibration for 2014–2017 to minimize the variations in simulated and observed streamflow values at the catchment outlet (Akkihebbal). The hydrological models were validated for the 2018–2020 period by using the calibrated models. For evaluating the simulating daily streamflows during calibration and validation phases, performances of the models were conducted by using the Nash-Sutcliffe model efficiency (NSE) and coefficient of determination (R2). The SWAT model yielded high R2 and NSE values of 0.85 and 0.82 for daily streamflow comparisons for the catchment outlet at the validation time, suggesting that the SWAT model showed relatively good results than the HEC-HMS model. Also, under modified LU/LC and ungauged streamflow conditions, the calibrated models can be later used to simulate streamflows for future predictions. Overall, the SWAT model seems to have done well in streamflow analysis capably for hydrological studies.

Download Full-text