Towards Open and FAIR Hydrological Modelling with eWaterCycle

2021 ◽  
Author(s):  
Niels Drost ◽  
Jerom P.M. Aerts ◽  
Fakhereh Alidoost ◽  
Bouwe Andela ◽  
Jaro Camphuijsen ◽  
...  
Keyword(s):  
Ad Hoc ◽  
Hydrological Model ◽  
Climate Model ◽  
Hydrological Modelling ◽  
Analysis Tool ◽  
Hydrological Models ◽  
Software Environment ◽  
Processing Pipeline ◽  
Basic Model ◽  
Input Processing

<p>The eWaterCycle platform (https://www.ewatercycle.org/) is a fully Open Source system designed explicitly to advance the state of Open and FAIR Hydrological modelling. While working with Hydrologists to create a fully Open and FAIR comparison study, we noticed that many ad-hoc tools and scripts are used to create input (forcing, parameters) for a hydrological model from the source datasets such as climate reanalysis and land-use data. To make this part of the modelling process better reproducible and more transparent we have created a common forcing input processing pipeline based on an existing climate model analysis tool: ESMValTool (https://www.esmvaltool.org/). </p><p>Using ESMValTool, the eWaterCycle platform can perform commonly required preprocessing steps such as cropping, re-gridding, and variable derivation in a standardized manner. If needed, it also allows for custom steps for a hydrological model. Our pre-processing pipeline directly supports commonly used datasets such as ERA-5, ERA-Interim, and CMIP climate model data, and creates ready-to-run forcing data for a number of Hydrological models.</p><p>Besides creating forcing data, the eWaterCycle platform allows scientists to run Hydrological models in a standardized way using Jupyter notebooks, wrapping the models inside a container environment, and interfacing to these using BMI, the Basic Model Interface (https://bmi.readthedocs.io/). The container environment (based on Docker) stores the entire software stack, including the operating system and libraries, in such a way that a model run can be reproduced using an identical software environment on any other computer.</p><p>The reproducible processing of forcing and a reproducible software environment are important steps towards our goal of fully reproducible, Open, and FAIR Hydrological modelling. Ultimately, we hope to make it possible to fully reproduce a hydrological model experiment from data pre-processing to analysis, using only a few clicks.</p>

Preprocessing of hydrological models’ input in eWaterCycle with ESMValTool

2021 ◽  
Author(s):  
Fakhereh Alidoost ◽  
Jerom Aerts ◽  
Bouwe Andela ◽  
Jaro Camphuijsen ◽  
Nick van De Giesen ◽  
...  
Keyword(s):  
Programming Languages ◽  
Hydrological Model ◽  
Performance Metrics ◽  
Climate Model ◽  
Meteorological Data ◽  
Climate Science ◽  
Hydrological Models ◽  
Earth System ◽  
Evaluation Tool ◽  
Spatial Coverage

<p>Hydrological models exhibit great complexity and diversity in the exact methodologies applied, competing for hypotheses of hydrologic behaviour, technology stacks, and programming languages used in those models. The preprocessing of forcing (meteorological) data is often performed by various sets of scripts that may or may not be included with model source codes, making it hard to reproduce results. Moreover, forcing data can be retrieved from a wide variety of forcing products with discrepant variable names and frequencies, spatial and temporal resolutions, and spatial coverage. Even though there is an infinite amount of preprocessing scripts for different models, these preprocessing scripts use only a limited set of operations, mainly re-gridding, temporal and spatial manipulations, variable derivation, and unit conversion. Also, these exact same preprocessing functions are used in analysis and evaluation of output from Earth system models in climate science.</p><p>Within the context of the eWaterCycle II project (https://www.ewatercycle.org/), a common preprocessing system has been created for hydrological modelling based on ESMValTool (Earth System Model Evaluation Tool). ESMValTool is a community-driven diagnostic and performance metrics tool that supports a broad range of preprocessing functions. Using a YAML script called a recipe, instructions are provided to ESMValTool: the datasets which need to be analyzed, the preprocessors that need to be applied, and the model-specific analysis (i.e. diagnostic script) which need to be run on data. ESMValTool is modular and flexible so all preprocessing functions can also be used directly in a Python script and additional analyses can easily be added.</p><p>The current preprocessing pipeline of the eWaterCycle using ESMValTool consists of hydrological model-specific scripts and supports ERA5 and ERA-Interim data provided by the ECMWF (European Centre for Medium-Range Weather Forecasts), as well as CMIP5 and CMIP6 climate model data. The pipeline starts with the downloading and CMORization (Climate Model Output Rewriter) of input data. Then a recipe is prepared to find the data and run the preprocessors. When ESMValTool runs a recipe, it produces preprocessed data that can be passed as input to a hydrological model. It will also store provenance and citation information to ensure transparency and reproducibility. This leads to less time spent on building custom preprocessing, more reproducible and comparable hydrological science.</p><p>In this presentation, we will give an overview of the current preprocessing pipeline of the eWaterCycle, outline ESMValTool preprocessing functions, and introduce available hydrological recipes and diagnostic scripts for the PCRGLOB, WFLOW, HYPE, MARRMOT and LISFLOOD models.</p>

scholarly journals A comparative analysis of projected impacts of climate change on river runoff from global and catchment-scale hydrological models

2010 ◽  
Vol 7 (5) ◽  
pp. 7191-7229 ◽  
Author(s):  
S. N. Gosling ◽  
R. G. Taylor ◽  
N. W. Arnell ◽  
M. C. Todd
Keyword(s):  
Climate Change ◽  
Hydrological Model ◽  
Climate Model ◽  
Annual Runoff ◽  
Structural Uncertainty ◽  
Hydrological Models ◽  
Catchment Scale ◽  
Monthly Runoff ◽  
Global Mean ◽  
Impacts Of Climate Change

Abstract. We present a comparative analysis of projected impacts of climate change on river runoff from two types of distributed hydrological model, a global hydrological model (GHM) and catchment-scale hydrological models (CHM). Analyses are conducted for six catchments that are global in coverage and feature strong contrasts in spatial scale as well as climatic and developmental conditions. These include the Liard (Canada), Mekong (SE Asia), Okavango (SW Africa), Rio Grande (Brazil), Xiangxi (China) and Harper's Brook (UK). A single GHM (Mac-PDM.09) is applied to all catchments whilst different CHMs are applied for each catchment. The CHMs include SLURP v. 12.2 (Liard), SLURP v. 12.7 (Mekong), Pitman (Okavango), MGB-IPH (Rio Grande), AV-SWAT-X 2005 (Xiangxi) and Cat-PDM (Harper's Brook). Simulations of mean annual runoff, mean monthly runoff and high (Q5) and low (Q95) monthly runoff under baseline (1961–1990) and climate change scenarios are presented. We compare the simulated runoff response of each hydrological model to (1) prescribed increases in global-mean air temperature of 1.0, 2.0, 3.0, 4.0, 5.0 and 6.0 °C relative to baseline from the UKMO HadCM3 Global Climate Model (GCM) to explore response to different amounts of climate forcing, and (2) a prescribed increase in global-mean air temperature of 2.0 °C relative to baseline for seven GCMs to explore response to climate model structural uncertainty. We find that the differences in projected changes of mean annual runoff between the two types of hydrological model can be substantial for a given GCM, and they are generally larger for indicators of high and low monthly runoff. However, they are relatively small in comparison to the range of projections across the seven GCMs. Hence, for the six catchments and seven GCMs we considered, climate model structural uncertainty is greater than the uncertainty associated with the type of hydrological model applied. Moreover, shifts in the seasonal cycle of runoff with climate change are represented similarly by both hydrological models, although for some catchments the monthly timing of high and low flows differs. This implies that for studies that seek to quantify and assess the role of climate model uncertainty on catchment-scale runoff, it may be equally as feasible to apply a GHM as it is to apply a CHM, especially when climate modelling uncertainty across the range of available GCMs is as large as it currently is. Whilst the GHM is able to represent the broad climate change signal that is represented by the CHMs, we find however, that for some catchments there are differences between GHMs and CHMs in mean annual runoff due to differences in potential evapotranspiration estimation methods, in the representation of the seasonality of runoff, and in the magnitude of changes in extreme (Q5, Q95) monthly runoff, all of which have implications for future water management issues.

Coupling Hydrological models using BMI in eWaterCycle

2020 ◽  
Author(s):  
Niels Drost ◽  
Rolf Hut ◽  
Nick Van De Giesen ◽  
Ben van Werkhoven ◽  
Jerom P.M. Aerts ◽  
...  
Keyword(s):  
Open Source ◽  
Hydrological Modeling ◽  
Hydrological Model ◽  
Open Science ◽  
Ease Of Use ◽  
Integrated Modeling ◽  
Model Output ◽  
Hydrological Models ◽  
Basic Model ◽  
Output Quality

<p>The eWaterCycle platform is a fully Open-Source platform built specifically to advance the state of FAIR and Open Science in hydrological Modeling. eWaterCycle builds on web technology, notebooks and containers to offer an integrated modeling experimentation environment for scientists. It allows scientists to run any supported hydrological model with ease, including setup and pre-processing of all data required. Common datasets such as ERA-Interim and ERA-5 forcing data and observations for verification of model output quality are available for usage by the models, and a Jupyter based interface is available for ease of use.</p><p>As the main API for models, we use the Basic Model Interface (BMI). This allows us to support models in a multitude of languages. Our gRPC based system allows coupling of models, and running of multiple instances of the same model. Our system was designed to work with higher level interfaces such as PyMT, and we are currently integrating PyMT into our platform. During my talk I will give an overview of the different elements of the eWaterCycle platform.</p><p>The BMI interface was specifically designed to make it easy to implement in any given model. During the FAIR Hydrological Modeling workshop a number of modelers worked on creating a BMI interface for their models, and making them available in the eWaterCycle system. To show the amount of effort required in common cases, I will show the BMI interface that was created for a number of these models, including SUMMA, HYPE, Marrmot, TopoFlex, LisFlood, WFLOW, and PCR-GLOBWB.</p>

scholarly journals A comparative analysis of projected impacts of climate change on river runoff from global and catchment-scale hydrological models

2011 ◽  
Vol 15 (1) ◽  
pp. 279-294 ◽  
Author(s):  
S. N. Gosling ◽  
R. G. Taylor ◽  
N. W. Arnell ◽  
M. C. Todd
Keyword(s):  
Climate Change ◽  
Hydrological Model ◽  
Climate Model ◽  
Annual Runoff ◽  
Structural Uncertainty ◽  
Hydrological Models ◽  
Catchment Scale ◽  
Monthly Runoff ◽  
Global Mean ◽  
Impacts Of Climate Change

Abstract. We present a comparative analysis of projected impacts of climate change on river runoff from two types of distributed hydrological model, a global hydrological model (GHM) and catchment-scale hydrological models (CHM). Analyses are conducted for six catchments that are global in coverage and feature strong contrasts in spatial scale as well as climatic and developmental conditions. These include the Liard (Canada), Mekong (SE Asia), Okavango (SW Africa), Rio Grande (Brazil), Xiangxi (China) and Harper's Brook (UK). A single GHM (Mac-PDM.09) is applied to all catchments whilst different CHMs are applied for each catchment. The CHMs include SLURP v. 12.2 (Liard), SLURP v. 12.7 (Mekong), Pitman (Okavango), MGB-IPH (Rio Grande), AV-SWAT-X 2005 (Xiangxi) and Cat-PDM (Harper's Brook). The CHMs typically simulate water resource impacts based on a more explicit representation of catchment water resources than that available from the GHM and the CHMs include river routing, whereas the GHM does not. Simulations of mean annual runoff, mean monthly runoff and high (Q5) and low (Q95) monthly runoff under baseline (1961–1990) and climate change scenarios are presented. We compare the simulated runoff response of each hydrological model to (1) prescribed increases in global-mean air temperature of 1.0, 2.0, 3.0, 4.0, 5.0 and 6.0 °C relative to baseline from the UKMO HadCM3 Global Climate Model (GCM) to explore response to different amounts of climate forcing, and (2) a prescribed increase in global-mean air temperature of 2.0 °C relative to baseline for seven GCMs to explore response to climate model structural uncertainty. We find that the differences in projected changes of mean annual runoff between the two types of hydrological model can be substantial for a given GCM (e.g. an absolute GHM-CHM difference in mean annual runoff percentage change for UKMO HadCM3 2 °C warming of up to 25%), and they are generally larger for indicators of high and low monthly runoff. However, they are relatively small in comparison to the range of projections across the seven GCMs. Hence, for the six catchments and seven GCMs we considered, climate model structural uncertainty is greater than the uncertainty associated with the type of hydrological model applied. Moreover, shifts in the seasonal cycle of runoff with climate change are represented similarly by both hydrological models, although for some catchments the monthly timing of high and low flows differs. This implies that for studies that seek to quantify and assess the role of climate model uncertainty on catchment-scale runoff, it may be equally as feasible to apply a GHM (Mac-PDM.09 here) as it is to apply a CHM, especially when climate modelling uncertainty across the range of available GCMs is as large as it currently is. Whilst the GHM is able to represent the broad climate change signal that is represented by the CHMs, we find however, that for some catchments there are differences between GHMs and CHMs in mean annual runoff due to differences in potential evapotranspiration estimation methods, in the representation of the seasonality of runoff, and in the magnitude of changes in extreme (Q5, Q95) monthly runoff, all of which have implications for future water management issues.

scholarly journals Quantile-Based Hydrological Modelling

Water ◽  
2021 ◽  
Vol 13 (23) ◽  
pp. 3420
Author(s):  
Hristos Tyralis ◽  
Georgia Papacharalampous
Keyword(s):  
Hydrological Model ◽  
Methodological Approach ◽  
Predictive Performance ◽  
Predictive Distribution ◽  
Hydrological Modelling ◽  
Hydrological Models ◽  
Proof Of Concept ◽  
Post Processing ◽  
Predictive Uncertainty ◽  
Quantile Loss

Predictive uncertainty in hydrological modelling is quantified by using post-processing or Bayesian-based methods. The former methods are not straightforward and the latter ones are not distribution-free (i.e., assumptions on the probability distribution of the hydrological model’s output are necessary). To alleviate possible limitations related to these specific attributes, in this work we propose the calibration of the hydrological model by using the quantile loss function. By following this methodological approach, one can directly simulate pre-specified quantiles of the predictive distribution of streamflow. As a proof of concept, we apply our method in the frameworks of three hydrological models to 511 river basins in the contiguous US. We illustrate the predictive quantiles and show how an honest assessment of the predictive performance of the hydrological models can be made by using proper scoring rules. We believe that our method can help towards advancing the field of hydrological uncertainty.

Comparing the impact for hydrology of the new ERA5 reanalyses dataset over ERA-Interim for 8 hydrological models in 6 catchments using the eWaterCycle community modelling environment.

2020 ◽  
Author(s):  
Rolf Hut ◽  
Niels Drost ◽  
Jerom Aerts ◽  
Laurene Bouaziz ◽  
Willem van Verseveld ◽  
...  
Keyword(s):  
Data Science ◽  
Conceptual Models ◽  
Hydrological Modeling ◽  
Hydrological Model ◽  
Global Climate ◽  
Hydrological Modelling ◽  
Additional Parameter ◽  
Hydrological Models ◽  
Comparison Study ◽  
The Impact

<p>The release of the European Centre for Medium-Range Weather Forecasts (ECMWF)’s Re-Analysis 5 (ERA-5) global climate forcing dataset is expected to greatly improve the quality of hydrological modeling. Following this release there is great interest in assessing the improvements of ERA-5 relative to its predecessor ERA-Interim for hydrological modeling and predictions.</p><p>In this study we compare streamflow predictions when using ERA-interim vs ERA-5 as forcing data for a suite of hydrological models from different research groups that capture the variation in modelling strategies within the hydrological modelling community. We check whether physically based models, defined as those that do not require additional parameter calibration, would lead to different conclusions in comparison to conceptual models, defined as those that require calibration. Based on the hydrological model structure we expect that conceptual models that need calibration show less difference in predicting discharge (skill) between ERA-5 and ERA-Interim, where-as the physical based (non-calibrated) models most likely will benefit from the improved accuracy of the ERA-5 input. This assessment will provide the HEPEX community with answers on how the ERA-5 dataset will improve hydrological predictions based on different hydrological modelling concepts.</p><p>An additional key objective while conducting this study is compliance to the FAIR principles of data science. To achieve this we held a workshop in Leiden, the Netherlands, where multiple hydrological models were integrated into the eWatercycle II system. eWatercycle II is a hydrological model platform containing a growing number of hydrological models. The platform facilitates research and cohesivity within the hydrological community by providing an Open-Source platform built specifically to advance the state of FAIR and Open Science in Hydrological Modeling. We also use this study to demonstrate the feasibility of eWatercycle II as a platform for FAIR hydrological models.</p><p>Preliminairy results from this comparison study were presented at the AGU Fall Meeting 2019. Here we will present the full results of the comparison study.</p>

The eWaterCycle platform for FAIR and Open Hydrological Modeling

2020 ◽  
Author(s):  
Niels Drost ◽  
Jaro Camphuijsen ◽  
Rolf Hut ◽  
Nick Van De Giesen ◽  
Ben van Werkhoven ◽  
...  
Keyword(s):  
Hydrological Modeling ◽  
Hydrological Model ◽  
Processing System ◽  
Scientific Work ◽  
Open Science ◽  
Integrated Modelling ◽  
Hydrological Models ◽  
Processing Pipeline ◽  
Output Quality ◽  
Reproducible Manner

<p>The eWaterCycle platform is a fully Open-Source platform built specifically to advance the state of FAIR and Open Science in Hydrological Modeling.</p><p>eWaterCycle builds on web technology, notebooks and containers to offer an integrated modelling experimentation environment for scientists. It allows scientists to run any supported hydrological model with ease, including setup and preprocessing of all data required. </p><p>eWaterCycle comes with an easy to use explorer, so the user can get started with the system in minutes, and uniquely lets the user generate a hydrological model notebook based on their preferences.</p><p>The eWaterCycle platform uses Jupyter as the main interface for scientific work to ensure maximum flexibility. Common datasets such as ERA-Interim and ERA-5 forcing data and observations for verification of model output quality are available for usage by the models.</p><p>To make the system capable of running any hydrological model we use docker containers coupled through gRPC. This allows us to support models in a multitude of languages, and provide fully reproducible model experiments.</p><p>Based on experiences during a FAIR Hydrological Modeling workshop in Leiden in April 2019 we have created a common pre-processing system for Hydrological modeling, based on technology from the climate sciences, in particular ESMValTool and Iris. This pre-processing pipeline can create input for a number of Hydrological models directly from the source dataset such as ERA-Interim in a fully transparent and reproducible manner.</p><p>During this pico presentation, we will explain how this platform supports creating reproducible results in an easy to use fashion.</p>

scholarly journals HESS Opinions "The art of hydrology"*

2009 ◽  
Vol 13 (2) ◽  
pp. 157-161 ◽  
Author(s):  
H. H. G. Savenije
Keyword(s):  
Physical Theory ◽  
Hydrological Model ◽  
Hydrological Modelling ◽  
Hydrological Models ◽  
Top Down ◽  
Science And Engineering ◽  
Bottom Up ◽  
Site Specific ◽  
Flexible Models ◽  
Modelling Process

Abstract. Hydrological modelling is the same as developing and encoding a hydrological theory. A hydrological model is not a tool but a hypothesis. The whole discussion about the inadequacy of hydrological models we have witnessed of late, is related to the wrong concept of what a model is. Good models don't exist. Instead of looking for the "best" model, we should aim at developing better models. The process of modelling should be top-down, learning from the data while at the same time connection should be established with underlying physical theory (bottom-up). As a result of heterogeneity occurring at all scales in hydrology, there always remains a need for calibration of models. This implies that we need tailor-made and site-specific models. Only flexible models are fit for this modelling process, as opposed to most of the established software or "one-size-fits-all" models. The process of modelling requires imagination, inspiration, creativity, ingenuity, experience and skill. These are qualities that belong to the field of art. Hydrology is an art as much as it is science and engineering.

scholarly journals <i>HESS Opinions</i> "The art of hydrology"<sup>*</sup>

2008 ◽  
Vol 5 (6) ◽  
pp. 3157-3167 ◽  
Author(s):  
H. H. G. Savenije
Keyword(s):  
Hydrological Model ◽  
Hydrological Modelling ◽  
Hydrological Models ◽  
Top Down ◽  
Science And Engineering ◽  
Site Specific ◽  
Flexible Models ◽  
Modelling Process

Abstract. Hydrological modelling is the same as developing and encoding a hydrological theory. A hydrological model is not a tool but a theory. The whole discussion about the inadequacy of hydrological models we have witnessed of late, is related to the wrong concept of what a model is. Good models don't exist. Instead, hydrological research should focus on improving models and enhancing understanding. The process of modelling should be top-down, learning from the data. There is always a need for calibration, which implies that we need tailor-made and site-specific models. Only flexible models are fit for this modelling process, as opposed to most of the "established" models, "one-size-fits-all" models or "models of everywhere". The process of modelling requires imagination, inspiration, creativity, ingenuity, experience and skill. These are qualities that belong to the field of art. Hydrology is an art as much as it is science and engineering.

Stepwise improvement of hydrological models using satellite-based evaporation and total water storage estimations

2021 ◽  
Author(s):  
Markus Hrachowitz ◽  
Petra Hulsman ◽  
Hubert Savenije
Keyword(s):  
Model Calibration ◽  
Hydrological Model ◽  
Model Development ◽  
Water Storage ◽  
Spatiotemporal Variability ◽  
Model Structure ◽  
Hydrological Models ◽  
Total Water ◽  
Spatiotemporal Model ◽  
Multiple Variables

&lt;p&gt;Hydrological models are often calibrated with respect to flow observations at the basin outlet. As a result, flow predictions may seem reliable but this is not necessarily the case for the spatiotemporal variability of system-internal processes, especially in large river basins. Satellite observations contain valuable information not only for poorly gauged basins with limited ground observations and spatiotemporal model calibration, but also for stepwise model development. This study explored the value of satellite observations to improve our understanding of hydrological processes through stepwise model structure adaption and to calibrate models both temporally and spatially. More specifically, satellite-based evaporation and total water storage anomaly observations were used to diagnose model deficiencies and to subsequently improve the hydrological model structure and the selection of feasible parameter sets. A distributed, process based hydrological model was developed for the Luangwa river basin in Zambia and calibrated with respect to discharge as benchmark. This model was modified stepwise by testing five alternative hypotheses related to the process of upwelling groundwater in wetlands, which was assumed to be negligible in the benchmark model, and the spatial discretization of the groundwater reservoir. Each model hypothesis was calibrated with respect to 1) discharge and 2) multiple variables simultaneously including discharge and the spatiotemporal variability in the evaporation and total water storage anomalies. The benchmark model calibrated with respect to discharge reproduced this variable well, as also the basin-averaged evaporation and total water storage anomalies. However, the evaporation in wetland dominated areas and the spatial variability in the evaporation and total water storage anomalies were poorly modelled. The model improved the most when introducing upwelling groundwater flow from a distributed groundwater reservoir and calibrating it with respect to multiple variables simultaneously. This study showed satellite-based evaporation and total water storage anomaly observations provide valuable information for improved understanding of hydrological processes through stepwise model development and spatiotemporal model calibration.&lt;/p&gt;

Sign in / Sign up

Export Citation Format

Share Document