scholarly journals On the choice of calibration metrics for high flow estimation using hydrologic models

2018 ◽  
Author(s):  
Naoki Mizukami ◽  
Oldrich Rakovec ◽  
Andrew Newman ◽  
Martyn Clark ◽  
Andrew Wood ◽  
...  
Keyword(s):  
Performance Metrics ◽  
Peak Flow ◽  
Hydrologic Model ◽  
High Flow ◽  
Hydrologic Models ◽  
Peak Flows ◽  
Annual Peak ◽  
Performance Metric ◽  
Squared Error ◽  
Application Specific

Abstract. Calibration is an essential step for improving the accuracy of simulations generated using hydrologic models, and a key modeler decision is the selection of the performance metric to be optimized. It has been common to used squared error performance metrics, or normalized variants such as Nash–Sutcliffe Efficiency (NSE), based on the idea that their squared-error nature will emphasize the estimation of high flows. However, we find that NSE-based model calibrations actually result in poor reproduction of high flow events, such as the annual peak flows that are used for flood frequency estimation. Using three different types of performance metrics, we calibrate two hydrological models, the Variable Infiltration Capacity model (VIC) and the mesoscale Hydrologic Model (mHM) and evaluate their ability to simulate high flow events for 492 basins throughout the contiguous United States. The metrics investigated are (1) NSE, (2) Kling–Gupta Efficiency (KGE) and variants, and (3) Annual Peak Flow Bias (APFB), where the latter is an application-specific hydrologic signature metric that focuses on annual peak flows. As expected, the application specific APFB metric produces the best annual peak flow estimates; however, performance on other high flow related metrics is poor. In contrast, the use of NSE results in annual peak flow estimates that are more than 20 % worse, primarily due to the tendency of NSE to result in underestimation of observed flow variability. Meanwhile, the use of KGE results in annual peak flow estimates that are better than from NSE, with only a slight degradation in performance with respect to other related metrics, particularly when a non-standard weighting of the components of KGE is used. Overall this work highlights the need for a fuller understanding of performance metric behavior and design in relation to the desired goals of model calibration.

scholarly journals On the choice of calibration metrics for “high-flow” estimation using hydrologic models

2019 ◽  
Vol 23 (6) ◽  
pp. 2601-2614 ◽  
Author(s):  
Naoki Mizukami ◽  
Oldrich Rakovec ◽  
Andrew J. Newman ◽  
Martyn P. Clark ◽  
Andrew W. Wood ◽  
...  
Keyword(s):  
Performance Metrics ◽  
Peak Flow ◽  
Hydrologic Model ◽  
Flood Frequency ◽  
High Flow ◽  
Hydrologic Models ◽  
Peak Flows ◽  
Annual Peak ◽  
Performance Metric ◽  
Squared Error

Abstract. Calibration is an essential step for improving the accuracy of simulations generated using hydrologic models. A key modeling decision is selecting the performance metric to be optimized. It has been common to use squared error performance metrics, or normalized variants such as Nash–Sutcliffe efficiency (NSE), based on the idea that their squared-error nature will emphasize the estimates of high flows. However, we conclude that NSE-based model calibrations actually result in poor reproduction of high-flow events, such as the annual peak flows that are used for flood frequency estimation. Using three different types of performance metrics, we calibrate two hydrological models at a daily step, the Variable Infiltration Capacity (VIC) model and the mesoscale Hydrologic Model (mHM), and evaluate their ability to simulate high-flow events for 492 basins throughout the contiguous United States. The metrics investigated are (1) NSE, (2) Kling–Gupta efficiency (KGE) and its variants, and (3) annual peak flow bias (APFB), where the latter is an application-specific metric that focuses on annual peak flows. As expected, the APFB metric produces the best annual peak flow estimates; however, performance on other high-flow-related metrics is poor. In contrast, the use of NSE results in annual peak flow estimates that are more than 20 % worse, primarily due to the tendency of NSE to underestimate observed flow variability. On the other hand, the use of KGE results in annual peak flow estimates that are better than from NSE, owing to improved flow time series metrics (mean and variance), with only a slight degradation in performance with respect to other related metrics, particularly when a non-standard weighting of the components of KGE is used. Stochastically generated ensemble simulations based on model residuals show the ability to improve the high-flow metrics, regardless of the deterministic performances. However, we emphasize that improving the fidelity of streamflow dynamics from deterministically calibrated models is still important, as it may improve high-flow metrics (for the right reasons). Overall, this work highlights the need for a deeper understanding of performance metric behavior and design in relation to the desired goals of model calibration.

Predictably Predictable -  The Role of Catchment Characteristics and Complexity.

2021 ◽  
Author(s):  
Sophia Eugeni ◽  
Eric Vaags ◽  
Steven V. Weijs
Keyword(s):  
Water Resource Management ◽  
Performance Metrics ◽  
Complexity Analysis ◽  
Model Performance ◽  
Principal Component ◽  
The United States ◽  
Hydrologic Model ◽  
Hydrologic Models ◽  
Hydrologic Modelling

<p>Accurate hydrologic modelling is critical to effective water resource management. As catchment attributes strongly influence the hydrologic behaviors in an area, they can be used to inform hydrologic models to better predict the discharge in a basin. Some basins may be more difficult to accurately predict than others. The difficulty in predicting discharge may also be related to the complexity of the discharge signal. The study establishes the relationship between a catchment’s static attributes and hydrologic model performance in those catchments, and also investigates the link to complexity, which we quantify with measures of compressibility based in information theory. </p><p>The project analyzes a large national dataset, comprised of catchment attributes for basins across the United States, paired with established performance metrics for corresponding hydrologic models. Principal Component Analysis (PCA) was completed on the catchment attributes data to determine the strongest modes in the input. The basins were clustered according to their catchment attributes and the performance within the clusters was compared. </p><p>Significant differences in model performance emerged between the clusters of basins. For the complexity analysis, details of the implementation and technical challenges will be discussed, as well as preliminary results.</p>

A Flood Potential Index for Effective Communication of Streamflow Forecasts at Ungauged Communities

2020 ◽  
Vol 21 (4) ◽  
pp. 807-814 ◽  
Author(s):  
Felipe Quintero ◽  
Witold F. Krajewski ◽  
Marcela Rojas
Keyword(s):  
Peak Flow ◽  
Hydrologic Model ◽  
Drainage Network ◽  
Relative Importance ◽  
Proof Of Concept ◽  
Streamflow Forecasting ◽  
Annual Peak ◽  
Potential Index ◽  
Streamflow Forecasts ◽  
Forecasting System

AbstractThis study proposes a flood potential index suitable for use in streamflow forecasting at any location in a drainage network. We obtained the index by comparing the discharge magnitude derived from a hydrologic model and the expected mean annual peak flow at the spatial scale of the basin. We use the term “flood potential” to indicate that uncertainty is associated with this information. The index helps communicate flood potential alerts to communities near rivers where there are no quantitative records of historical floods to provide a reference. This method establishes a reference that we can compare to forecasted hydrographs and that facilitates communication of their relative importance. As a proof of concept, the authors present an assessment of the index as applied to the peak flows that caused severe floods in Iowa in June 2008. The Iowa Flood Center uses the proposed approach operationally as part of its real-time hydrologic forecasting system.

scholarly journals Projected Changes in the Frequency of Peak Flows along the Athabasca River: Sensitivity of Results to Statistical Methods of Analysis

Climate ◽  
2019 ◽  
Vol 7 (7) ◽  
pp. 88 ◽  
Author(s):  
Yonas Dibike ◽  
Hyung-Il Eum ◽  
Paulin Coulibaly ◽  
Joshua Hartmann
Keyword(s):  
Statistical Method ◽  
Climate Models ◽  
River Flow ◽  
Climate Model ◽  
Peak Flow ◽  
Hydrologic Model ◽  
Climate Change Scenarios ◽  
Peak Flows ◽  
Athabasca River ◽  
Projected Changes

Flows originating from alpine dominated cold region watersheds typically experience extended winter low flows followed by spring snowmelt and summer rainfall driven high flows. In a warmer climate, there will be a temperature-induced shift in precipitation from snowfall towards rain along with changes in precipitation intensity and snowmelt timing, resulting in alterations in the frequency and magnitude of peak flow events. This study examines the potential future changes in the frequency and severity of peak flow events in the Athabasca River watershed in Alberta, Canada. The analysis is based on simulated flow data by the variable infiltration capacity (VIC) hydrologic model driven by statistically downscaled climate change scenarios from the latest coupled model inter-comparison project (CMIP5). The hydrological model projections show an overall increase in mean annual streamflow in the watershed and a corresponding shift in the freshet timing to an earlier period. The river flow is projected to experience increases during the winter and spring seasons and decreases during the summer and early fall seasons, with an overall projected increase in peak flow, especially for low frequency events. Both stationary and non-stationary methods of peak flow analysis, performed at multiple points along the Athabasca River, show that projected changes in the 100-year peak flow event for the high emissions scenario by the 2080s range between 4% and 33% depending on the driving climate models and the statistical method of analysis. A closer examination of the results also reveals that the sensitivity of projected changes in peak flows to the statistical method of frequency analysis is relatively small compared to that resulting from inter-climate model variability.

scholarly journals Integrating partitioned evapotranspiration data into hydrologic models: vegetation parameterization and uncertainty quantification of simulated plant water use

2022 ◽  
Author(s):  
Adam Schreiner-McGraw ◽  
Hoori Ajami ◽  
Ray Anderson ◽  
Dong Wang
Keyword(s):  
Groundwater Recharge ◽  
Water Use ◽  
Precision Agriculture ◽  
Land Surface ◽  
Performance Metrics ◽  
Model Performance ◽  
Hydrologic Model ◽  
Plant Water ◽  
Hydrologic Models ◽  
Plant Water Use

Accurate simulation of plant water use across agricultural ecosystems is essential for various applications, including precision agriculture, quantifying groundwater recharge, and optimizing irrigation rates. Previous approaches to integrating plant water use data into hydrologic models have relied on evapotranspiration (ET) observations. Recently, the flux variance similarity approach has been developed to partition ET to transpiration (T) and evaporation, providing an opportunity to use T data to parameterize models. To explore the value of T/ET data in improving hydrologic model performance, we examined multiple approaches to incorporate these observations for vegetation parameterization. We used ET observations from 5 eddy covariance towers located in the San Joaquin Valley, California, to parameterize orchard crops in an integrated land surface – groundwater model. We find that a simple approach of selecting the best parameter sets based on ET and T performance metrics works best at these study sites. Selecting parameters based on performance relative to observed ET creates an uncertainty of 27% relative to the observed value. When parameters are selected using both T and ET data, this uncertainty drops to 24%. Similarly, the uncertainty in potential groundwater recharge drops from 63% to 58% when parameters are selected with ET or T and ET data, respectively. Additionally, using crop type parameters results in similar levels of simulated ET as using site-specific parameters. Different irrigation schemes create high amounts of uncertainty and highlight the need for accurate estimates of irrigation when performing water budget studies.

Rainfall Events as Paths of a Stochastic Process: Problems of Design Storm Analysis

1984 ◽  
Vol 16 (8-9) ◽  
pp. 131-138 ◽  
Author(s):  
Johannes Brummer
Keyword(s):  
Stochastic Process ◽  
Peak Flow ◽  
Rainfall Events ◽  
Peak Flows ◽  
Design Storm ◽  
Adequate Evaluation ◽  
Rainfall Patterns

Problems in the construction of design storms are expressed in mathematical terms. Introduced here is a concept for approximating natural peak flow values by means of the distribution of typical rainfall patterns. A comparison demonstrates the quality of this concept and the competency of some well-known design storms for the adequate evaluation of peak flows.

scholarly journals Process-Based Modeling of the High Flow of a Semi-Mountain River under Current and Future Climatic Conditions: A Case Study of the Iya River (Eastern Siberia)

Water ◽  
2021 ◽  
Vol 13 (8) ◽  
pp. 1042
Author(s):  
Andrey Kalugin
Keyword(s):  
Water Level ◽  
River Basin ◽  
21St Century ◽  
Residential Buildings ◽  
High Flow ◽  
Peak Discharge ◽  
Future Climate ◽  
Future Climate Change ◽  
Generation Model ◽  
Annual Peak

The purpose of the study was to analyze the formation conditions of catastrophic floods in the Iya River basin over the observation period, as well as a long-term forecast of the impacts of future climate change on the characteristics of the high flow in the 21st century. The semi-distributed process-based Ecological Model for Applied Geophysics (ECOMAG) was applied to the Iya River basin. Successful model testing results were obtained for daily discharge, annual peak discharge, and discharges exceeding the critical water level threshold over the multiyear period of 1970–2019. Modeling of the high flow of the Iya River was carried out according to a Kling–Gupta efficiency (KGE) of 0.91, a percent bias (PBIAS) of −1%, and a ratio of the root mean square error to the standard deviation of measured data (RSR) of 0.41. The preflood coefficient of water-saturated soil and the runoff coefficient of flood-forming precipitation in the Iya River basin were calculated in 1980, 1984, 2006, and 2019. Possible changes in the characteristics of high flow over summers in the 21st century were calculated using the atmosphere–ocean general circulation model (AOGCM) and the Hadley Centre Global Environment Model version 2-Earth System (HadGEM2-ES) as the boundary conditions in the runoff generation model. Anomalies in values were estimated for the middle and end of the current century relative to the observed runoff over the period 1990–2019. According to various Representative Concentration Pathways (RCP-scenarios) of the future climate in the Iya River basin, there will be less change in the annual peak discharge or precipitation and more change in the hazardous flow and its duration, exceeding the critical water level threshold, at which residential buildings are flooded.

scholarly journals Using satellite-based evapotranspiration estimates to improve the structure of a simple conceptual rainfall–runoff model

2017 ◽  
Vol 21 (2) ◽  
pp. 879-896 ◽  
Author(s):  
Tirthankar Roy ◽  
Hoshin V. Gupta ◽  
Aleix Serrat-Capdevila ◽  
Juan B. Valdes
Keyword(s):  
Model Performance ◽  
Hydrologic Model ◽  
Actual Evapotranspiration ◽  
Hydrologic Models ◽  
Catchment Scale ◽  
Model Driven ◽  
Alternative Approach ◽  
Precipitation Estimates ◽  
Daily Water ◽  
Rainfall Runoff Model

Abstract. Daily, quasi-global (50° N–S and 180° W–E), satellite-based estimates of actual evapotranspiration at 0.25° spatial resolution have recently become available, generated by the Global Land Evaporation Amsterdam Model (GLEAM). We investigate the use of these data to improve the performance of a simple lumped catchment-scale hydrologic model driven by satellite-based precipitation estimates to generate streamflow simulations for a poorly gauged basin in Africa. In one approach, we use GLEAM to constrain the evapotranspiration estimates generated by the model, thereby modifying daily water balance and improving model performance. In an alternative approach, we instead change the structure of the model to improve its ability to simulate actual evapotranspiration (as estimated by GLEAM). Finally, we test whether the GLEAM product is able to further improve the performance of the structurally modified model. Results indicate that while both approaches can provide improved simulations of streamflow, the second approach also improves the simulation of actual evapotranspiration significantly, which substantiates the importance of making diagnostic structural improvements to hydrologic models whenever possible.

scholarly journals Considerations for performance metrics of metagenomic next generation sequencing analyses

2020 ◽  
Author(s):  
Jason G. Kralj ◽  
Stephanie L. Servetas ◽  
Samuel P. Forry ◽  
Scott A. Jackson
Keyword(s):  
Performance Metrics ◽  
Limit Of Detection ◽  
Clinical Performance ◽  
Ground Truth ◽  
Negative Control ◽  
Fitness For Purpose ◽  
Performance Metric ◽  
The One ◽  
Sensitivity Specificity ◽  
Harmonic Means

AbstractEvaluating the performance of metagenomics analyses has proven a challenge, due in part to limited ground-truth standards, broad application space, and numerous evaluation methods and metrics. Application of traditional clinical performance metrics (i.e. sensitivity, specificity, etc.) using taxonomic classifiers do not fit the “one-bug-one-test” paradigm. Ultimately, users need methods that evaluate fitness-for-purpose and identify their analyses’ strengths and weaknesses. Within a defined cohort, reporting performance metrics by taxon, rather than by sample, will clarify this evaluation. An estimated limit of detection, positive and negative control samples, and true positive and negative true results are necessary criteria for all investigated taxa. Use of summary metrics should be restricted to comparing results of similar cohorts and data, and should employ harmonic means and continuous products for each performance metric rather than arithmetic mean. Such consideration will ensure meaningful comparisons and evaluation of fitness-for-purpose.

scholarly journals Optimal Stiffness Design for an Exhaustive Parallel Compliance Matrix in Multiactuator Robotic Limbs

2018 ◽  
Vol 10 (3) ◽  
Author(s):  
Nathan M. Cahill ◽  
Thomas Sugar ◽  
Yi Ren ◽  
Kyle Schroeder
Keyword(s):  
Energy Efficient ◽  
Peak Power ◽  
Performance Metrics ◽  
Slow Growth ◽  
Legged Robots ◽  
Power Requirement ◽  
Compliance Matrix ◽  
Performance Metric ◽  
Stiffness Design ◽  
Optimal Stiffness

Comparatively slow growth in energy density of both power storage and generation technologies has placed added emphasis on the need for energy-efficient designs in legged robots. This paper explores the potential of parallel springs in robot limb design. We start by adding what we call the exhaustive parallel compliance matrix (EPCM) to the design. The EPCM is a set of parallel springs, which includes a parallel spring for each joint and a multijoint parallel spring for all possible combinations of the robot's joints. Then, we carefully formulate and compare two performance metrics, which improve various aspects of the system performance. Each performance metric is analyzed and compared, their strengths and weaknesses being rigorously presented. The performance benefits associated with this approach are dramatic. Implementing the spring matrix reduces the sum of square power (SSP) exerted by the actuators by up to 47%, the peak power requirement by almost 40%, the sum of squared current by 55%, and the peak current by 55%. These results were generated using a planar robot limb and a gait trajectory borrowed from biology. We use a fully dynamic model of the robotic system including inertial effects. We also test the design robustness using a perturbation study, which shows that the parallel springs are effective even in the presence of trajectory perturbation.

Sign in / Sign up

Export Citation Format

Share Document