Random forests-based error-correction of streamflow from a large-scale hydrological model: Using model state variables to estimate error terms

We conduct a large-scale benchmark experiment aiming to advance the use of machine-learning quantile regression algorithms for probabilistic hydrological post-processing “at scale” within operational contexts. The experiment is set up using 34-year-long daily time series of precipitation, temperature, evapotranspiration and streamflow for 511 catchments over the contiguous United States. Point hydrological predictions are obtained using the Génie Rural à 4 paramètres Journalier (GR4J) hydrological model and exploited as predictor variables within quantile regression settings. Six machine-learning quantile regression algorithms and their equal-weight combiner are applied to predict conditional quantiles of the hydrological model errors. The individual algorithms are quantile regression, generalized random forests for quantile regression, generalized random forests for quantile regression emulating quantile regression forests, gradient boosting machine, model-based boosting with linear models as base learners and quantile regression neural networks. The conditional quantiles of the hydrological model errors are transformed to conditional quantiles of daily streamflow, which are finally assessed using proper performance scores and benchmarking. The assessment concerns various levels of predictive quantiles and central prediction intervals, while it is made both independently of the flow magnitude and conditional upon this magnitude. Key aspects of the developed methodological framework are highlighted, and practical recommendations are formulated. In technical hydro-meteorological applications, the algorithms should be applied preferably in a way that maximizes the benefits and reduces the risks from their use. This can be achieved by (i) combining algorithms (e.g., by averaging their predictions) and (ii) integrating algorithms within systematic frameworks (i.e., by using the algorithms according to their identified skills), as our large-scale results point out.

Download Full-text

Assessing Annual Actual Evapotranspiration Based on Climate, Topography and Soil in Natural and Agricultural Ecosystems

Climate ◽

10.3390/cli9020020 ◽

2021 ◽

Vol 9 (2) ◽

pp. 20

Author(s):

Kleoniki Demertzi ◽

Vassilios Pisinaras ◽

Emanuel Lekakis ◽

Evangelos Tziritis ◽

Konstantinos Babakos ◽

...

Keyword(s):

Water Balance ◽

Large Scale ◽

Hydrological Model ◽

Actual Evapotranspiration ◽

Irrigated Agriculture ◽

Climate Data ◽

Natural Ecosystems ◽

Loading Effects ◽

Very High ◽

Of Greece

Simple formulas for estimating annual actual evapotranspiration (AET) based on annual climate data are widely used in large scale applications. Such formulas do not have distinct compartments related to topography, soil and irrigation, and for this reason may be limited in basins with high slopes, where runoff is the dominant water balance component, and in basins where irrigated agriculture is dominant. Thus, a simplistic method for assessing AET in both natural ecosystems and agricultural systems considering the aforementioned elements is proposed in this study. The method solves AET through water balance based on a set of formulas that estimate runoff and percolation. These formulas are calibrated by the results of the deterministic hydrological model GLEAMS (Groundwater Loading Effects of Agricultural Management Systems) for a reference surface. The proposed methodology is applied to the country of Greece and compared with the widely used climate-based methods of Oldekop, Coutagne and Turk. The results show that the proposed methodology agrees very well with the method of Turk for the lowland regions but presents significant differences in places where runoff is expected to be very high (sloppy areas and areas of high rainfall, especially during December–February), suggesting that the proposed method performs better due to its runoff compartment. The method can also be applied in a single application considering irrigation only for the irrigated lands to more accurately estimate AET in basins with a high percentage of irrigated agriculture.

Download Full-text

Parameter transferability between multiple gridded input datasets challenges hydrological model performance under changing climate

10.5194/egusphere-egu21-8782 ◽

2021 ◽

Author(s):

Moctar Dembélé ◽

Bettina Schaefli ◽

Grégoire Mariéthoz

Keyword(s):

West Africa ◽

Large Scale ◽

Hydrological Model ◽

Meteorological Data ◽

Model Performance ◽

Hydrologic Model ◽

Hydrological Modelling ◽

Climate Change Scenarios ◽

Depth Analysis ◽

Input Dataset

The diversity of remotely sensed or reanalysis-based rainfall data steadily increases, which on one hand opens new perspectives for large scale hydrological modelling in data scarce regions, but on the other hand poses challenging question regarding parameter identification and transferability under multiple input datasets. This study analyzes the variability of hydrological model performance when (1) a set of parameters is transferred from the calibration input dataset to a different meteorological datasets and reversely, when (2) an input dataset is used with a parameter set, originally calibrated for a different input dataset.The research objective is to highlight the uncertainties related to input data and the limitations of hydrological model parameter transferability across input datasets. An ensemble of 17 rainfall datasets and 6 temperature datasets from satellite and reanalysis sources (Demb&#233;l&#233; et al., 2020), corresponding to 102 combinations of meteorological data, is used to force the fully distributed mesoscale Hydrologic Model (mHM). The mHM model is calibrated for each combination of meteorological datasets, thereby resulting in 102 calibrated parameter sets, which almost all give similar model performance. Each of the 102 parameter sets is used to run the mHM model with each of the 102 input datasets, yielding 10404 scenarios to that serve for the transferability tests. The experiment is carried out for a decade from 2003 to 2012 in the large and data-scarce Volta River basin (415600 km2) in West Africa.The results show that there is a high variability in model performance for streamflow (mean CV=105%) when the parameters are transferred from the original input dataset to other input datasets (test 1 above). Moreover, the model performance is in general lower and can drop considerably when parameters obtained under all other input datasets are transferred to a selected input dataset (test 2 above). This underlines the need for model performance evaluation when different input datasets and parameter sets than those used during calibration are used to run a model. Our results represent a first step to tackle the question of parameter transferability to climate change scenarios. An in-depth analysis of the results at a later stage will shed light on which model parameterizations might be the main source of performance variability.Demb&#233;l&#233;, M., Schaefli, B., van de Giesen, N., & Mari&#233;thoz, G. (2020). Suitability of 17 rainfall and temperature gridded datasets for large-scale hydrological modelling in West Africa. Hydrology and Earth System Sciences (HESS). https://doi.org/10.5194/hess-24-5379-2020

Download Full-text

Benefits and limitations of data assimilation for discharge forecasting using an event-based rainfall–runoff model

Natural Hazards and Earth System Science ◽

10.5194/nhess-13-583-2013 ◽

2013 ◽

Vol 13 (3) ◽

pp. 583-596 ◽

Cited By ~ 15

Author(s):

M. Coustau ◽

S. Ricci ◽

V. Borrell-Estupina ◽

C. Bouvier ◽

O. Thual

Keyword(s):

Data Assimilation ◽

Hydrological Model ◽

Peak Discharge ◽

State Variables ◽

Rainfall Runoff ◽

Discharge Data ◽

Southern France ◽

Flood Peak ◽

Event Based ◽

The Impact

Abstract. Mediterranean catchments in southern France are threatened by potentially devastating fast floods which are difficult to anticipate. In order to improve the skill of rainfall-runoff models in predicting such flash floods, hydrologists use data assimilation techniques to provide real-time updates of the model using observational data. This approach seeks to reduce the uncertainties present in different components of the hydrological model (forcing, parameters or state variables) in order to minimize the error in simulated discharges. This article presents a data assimilation procedure, the best linear unbiased estimator (BLUE), used with the goal of improving the peak discharge predictions generated by an event-based hydrological model Soil Conservation Service lag and route (SCS-LR). For a given prediction date, selected model inputs are corrected by assimilating discharge data observed at the basin outlet. This study is conducted on the Lez Mediterranean basin in southern France. The key objectives of this article are (i) to select the parameter(s) which allow for the most efficient and reliable correction of the simulated discharges, (ii) to demonstrate the impact of the correction of the initial condition upon simulated discharges, and (iii) to identify and understand conditions in which this technique fails to improve the forecast skill. The correction of the initial moisture deficit of the soil reservoir proves to be the most efficient control parameter for adjusting the peak discharge. Using data assimilation, this correction leads to an average of 12% improvement in the flood peak magnitude forecast in 75% of cases. The investigation of the other 25% of cases points out a number of precautions for the appropriate use of this data assimilation procedure.

Download Full-text

Aerosol-cloud-turbulence interactions in well-coupled Arctic boundary layers over open water

10.5194/acp-2021-888 ◽

2021 ◽

Author(s):

Jan Chylik ◽

Dmitry Chechin ◽

Regis Dupuy ◽

Birte S. Kulla ◽

Christof Lüpkes ◽

...

Keyword(s):

Large Scale ◽

Convective Cloud ◽

Open Water ◽

Convective Transport ◽

Cloud Layer ◽

Mixed Phase ◽

State Variables ◽

Strongly Coupled ◽

Main Research ◽

Weather Model

Abstract. Late springtime Arctic mixed-phase convective clouds over open water in the Fram Strait as observed during the recent ACLOUD field campaign are simulated at turbulence-resolving resolutions. The main research objective is to gain more insight into the coupling of these cloud layers to the surface, and into the role played by interactions between aerosol, hydrometeors and turbulence in this process. A composite case is constructed based on data collected by two research aircraft on 18 June 2017. The boundary conditions and large-scale forcings are based on weather model analyses, yielding a simulation that freely equilibrates towards the observed thermodynamic state. The results are evaluated against a variety of independent aircraft measurements. The observed cloud macro- and microphysical structure is well reproduced, consisting of a stratiform cloud layer in mixed-phase fed by surface-driven convective transport in predominantly liquid phase. Comparison to noseboom turbulence measurements suggests that the simulated cloud-surface coupling is realistic. A joint-pdf analysis of relevant state variables is conducted, suggesting that locations where the mixed-phase cloud layer is strongly coupled to the surface by convective updrafts act as hot-spots for invigorated interactions between turbulence, clouds and aerosol. A mixing-line analysis reveals that the turbulent mixing is similar to warm convective cloud regimes, but is accompanied by hydrometeor transitions that are unique for mixed-phase cloud systems. Distinct fingerprints in the joint-pdf diagrams also explain i) the typical ring-like shape of ice mass in the outflow cloud deck, ii) its slightly elevated buoyancy, and iii) an associated local minimum in CCN.

Download Full-text

Robust estimation of hydrological model parameters

Hydrology and Earth System Sciences Discussions ◽

10.5194/hessd-5-1641-2008 ◽

2008 ◽

Vol 5 (3) ◽

pp. 1641-1675 ◽

Cited By ~ 12

Author(s):

A. Bárdossy ◽

S. K. Singh

Keyword(s):

Measurement Errors ◽

Input Data ◽

Hydrological Model ◽

Optimal Parameter ◽

Model Performance ◽

Model Parameters ◽

Geometrical Approach ◽

State Variables ◽

Parameter Vector ◽

Robust Parameter

Abstract. The estimation of hydrological model parameters is a challenging task. With increasing capacity of computational power several complex optimization algorithms have emerged, but none of the algorithms gives an unique and very best parameter vector. The parameters of hydrological models depend upon the input data. The quality of input data cannot be assured as there may be measurement errors for both input and state variables. In this study a methodology has been developed to find a set of robust parameter vectors for a hydrological model. To see the effect of observational error on parameters, stochastically generated synthetic measurement errors were applied to observed discharge and temperature data. With this modified data, the model was calibrated and the effect of measurement errors on parameters was analysed. It was found that the measurement errors have a significant effect on the best performing parameter vector. The erroneous data led to very different optimal parameter vectors. To overcome this problem and to find a set of robust parameter vectors, a geometrical approach based on the half space depth was used. The depth of the set of N randomly generated parameters was calculated with respect to the set with the best model performance (Nash-Sutclife efficiency was used for this study) for each parameter vector. Based on the depth of parameter vectors, one can find a set of robust parameter vectors. The results show that the parameters chosen according to the above criteria have low sensitivity and perform well when transfered to a different time period. The method is demonstrated on the upper Neckar catchment in Germany. The conceptual HBV model was used for this study.

Download Full-text

Distributed Weighted Least-Squares and Gaussian Belief Propagation: An Integrated Approach

10.36227/techrxiv.14781489.v1 ◽

2021 ◽

Author(s):

Dino Zivojevic ◽

Muhamed Delalic ◽

Darijo Raca ◽

Dejan Vukobratovic ◽

Mirsad Cosovic

Keyword(s):

Least Squares ◽

Large Scale ◽

Belief Propagation ◽

Weighted Least Squares ◽

Measurement Data ◽

Integrated Approach ◽

Factor Graph ◽

State Variables ◽

Large Scale Systems ◽

Distributed Approach

The purpose of a state estimation (SE) algorithm is to estimate the values of the state variables considering the available set of measurements. The centralised SE becomes impractical for large-scale systems, particularly if the measurements are spatially distributed across wide geographical areas. Dividing the large-scale systems into clusters (\ie subsystems) and distributing the computation across clusters, solves the constraints of centralised based approach. In such scenarios, using distributed SE methods brings numerous advantages over the centralised ones. In this paper, we propose a novel distributed approach to solve the linear SE model by combining local solutions obtained by applying weighted least-squares (WLS) of the given subsystems with the Gaussian belief propagation (GBP) algorithm. The proposed algorithm is based on the factor graph operating without a central coordinator, where subsystems exchange only ``beliefs", thus preserving privacy of the measurement data and state variables. Further, we propose an approach to speed-up evaluation of the local solution upon arrival of a new information to the subsystem. Finally, the proposed algorithm provides results that reach accuracy of the centralised WLS solution in a few iterations, and outperforms vanilla GBP algorithm with respect to its convergence properties.

Download Full-text

Development of a Large-Scale Hydrological Model TOPX and Its Coupling with Regional Integrated Environment Modeling System RIEMS

Chinese Journal of Geophysics ◽

10.1002/cjg2.1399 ◽

2009 ◽

Vol 52 (4) ◽

pp. 762-771 ◽

Cited By ~ 2

Author(s):

Bin YONG ◽

Li-Liang REN ◽

Xi CHEN ◽

Yun ZHANG ◽

Wan-Chang ZHANG ◽

...

Keyword(s):

Large Scale ◽

Hydrological Model ◽

Environment Modeling ◽

Integrated Environment

Download Full-text

Progress toward the Application of a Localized Particle Filter for Numerical Weather Prediction

Monthly Weather Review ◽

10.1175/mwr-d-17-0344.1 ◽

2019 ◽

Vol 147 (4) ◽

pp. 1107-1126 ◽

Cited By ~ 9

Author(s):

Jonathan Poterjoy ◽

Louis Wicker ◽

Mark Buehner

Keyword(s):

Data Assimilation ◽

Particle Filter ◽

Numerical Weather Prediction ◽

Large Scale ◽

Sequential Monte Carlo ◽

Prediction Models ◽

Weather Prediction ◽

Coupled Systems ◽

State Variables ◽

Numerical Weather

Abstract A series of papers published recently by the first author introduce a nonlinear filter that operates effectively as a data assimilation method for large-scale geophysical applications. The method uses sequential Monte Carlo techniques adopted by particle filters, which make no parametric assumptions for the underlying prior and posterior error distributions. The filter also treats the underlying dynamical system as a set of loosely coupled systems to effectively localize the effect observations have on posterior state estimates. This property greatly reduces the number of particles—or ensemble members—required for its implementation. For these reasons, the method is called the local particle filter. The current manuscript summarizes algorithmic advances made to the local particle filter following recent tests performed over a hierarchy of dynamical systems. The revised filter uses modified vector weight calculations and probability mapping techniques from earlier studies, and new strategies for improving filter stability in situations where state variables are observed infrequently with very accurate measurements. Numerical experiments performed on low-dimensional data assimilation problems provide evidence that supports the theoretical benefits of the new improvements. As a proof of concept, the revised particle filter is also tested on a high-dimensional application from a real-time weather forecasting system at the NOAA/National Severe Storms Laboratory (NSSL). The proposed changes have large implications for researchers applying the local particle filter for real applications, such as data assimilation in numerical weather prediction models.

Download Full-text