HydroGFD3: a climatological and real-time updated hydrological forcing dataset

2020 ◽  
Author(s):  
Peter Berg ◽  
Fredrik Almén ◽  
Denica Bozhinova ◽  
Riejanne Mook
Keyword(s):  
Real Time ◽  
Observational Data ◽  
Bias Correction ◽  
Hydrological Model ◽  
Initial Conditions ◽  
Correction Method ◽  
Reanalysis Data ◽  
Data Sets ◽  
Initial State ◽  
Data Set

<p>Hydrological forecasting benefits substantially from good initial conditions, which translate information into the forecast. It is therefore important to perform frequent updates of the initial state of the model before the forecast, which demands good meteorological forcing data. For a continental or global hydrological model, it is difficult to find observational data sets which fulfill the requirements of (i) long time series for calibration and spin up, (ii) consistent quality, (iii) at least daily time steps, and (iv) at least data for temperature and precipitation. HydroGFD3 is a new data set that fulfills all the criteria and provides real-time updated data.</p><p>HydroGFD3 builds upon the ERA5 reanalysis data set, and performs a bias correction for each new produced month. In contrast to earlier versions (Berg et al., 2018), HydroGFD3 is based on a multi-source climatological background, upon which individual days are produced by adding anomalies from different freely available monthly global observational data sets. These are then disaggregated based on the ERA5 reanalysis. For production redundancy and local tailoring, HydroGFD3 is produced in several tiers, each using different observational data sets originating from GPCC and CPC. Further, intermediate daily updates of the reanalysis through the source ERA5T allow the data set to be updated to within a few days of real-time.</p><p>To reach actual real-time, one tier is based on a bias correction method calibrated on the period 1980-2009, which is applied on ERA5T, and further prolonged to current day using the ECMWF deterministic forecasts. The assumption for this to work is that the forecasts have a similar bias as the reanalysis model, which is currently the case. The method also allows bias correction of the forecasts themselves; solving the issue of “drift” in the forecasts as the hydrological model adjusts to the (biased) climatological state of the forcing data.</p><p>Berg, Peter, Chantal Donnelly, and David Gustafsson. "Near-real-time adjusted reanalysis forcing data for hydrology." Hydrology and Earth System Sciences 22.2 (2018): 989-1000.</p><p> </p>

scholarly journals Near-real-time adjusted reanalysis forcing data for hydrology

2018 ◽  
Vol 22 (2) ◽  
pp. 989-1000 ◽  
Author(s):  
Peter Berg ◽  
Chantal Donnelly ◽  
David Gustafsson
Keyword(s):  
Real Time ◽  
Global Scale ◽  
Reanalysis Data ◽  
The Arctic ◽  
Data Sets ◽  
Initial State ◽  
Time Step ◽  
Data Set ◽  
Global Data Sets ◽  
Global Data

Abstract. Extending climatological forcing data to current and real-time forcing is a necessary task for hydrological forecasting. While such data are often readily available nationally, it is harder to find fit-for-purpose global data sets that span long climatological periods through to near-real time. Hydrological simulations are generally sensitive to bias in the meteorological forcing data, especially relative to the data used for the calibration of the model. The lack of high-quality daily resolution data on a global scale has previously been solved by adjusting reanalysis data with global gridded observations. However, existing data sets of this type have been produced for a fixed past time period determined by the main global observational data sets. Long delays between updates of these data sets leaves a data gap between the present day and the end of the data set. Further, hydrological forecasts require initializations of the current state of the snow, soil and lake (and sometimes river) storage. This is normally conceived by forcing the model with observed meteorological conditions for an extended spin-up period, typically at a daily time step, to calculate the initial state. Here, we present and evaluate a method named HydroGFD (Hydrological Global Forcing Data) to combine different data sets in order to produce near-real-time updated hydrological forcing data of temperature and precipitation that are compatible with the products covering the climatological period. HydroGFD resembles the already established WFDEI (WATCH Forcing Data–ERA-Interim) method (Weedon et al., 2014) closely but uses updated climatological observations, and for the near-real time it uses interim products that apply similar methods. This allows HydroGFD to produce updated forcing data including the previous calendar month around the 10th of each month. We present the HydroGFD method and therewith produced data sets, which are evaluated against global data sets, as well as with hydrological simulations with the HYPE (Hydrological Predictions for the Environment) model over Europe and the Arctic regions. We show that HydroGFD performs similarly to WFDEI and that the updated period significantly reduces the bias of the reanalysis data. For real-time updates until the current day, extending HydroGFD with operational meteorological forecasts, a large drift is present in the hydrological simulations due to the bias of the meteorological forecasting model.

scholarly journals Near real-time adjusted reanalysis forcing data for hydrology

2017 ◽  
Author(s):  
Peter Berg ◽  
Chantal Donnelly ◽  
David Gustafsson
Keyword(s):  
Real Time ◽  
Arctic Region ◽  
Analysis Data ◽  
Global Scale ◽  
Current Data ◽  
The Arctic ◽  
Data Sets ◽  
Initial State ◽  
Time Step ◽  
Data Set

Abstract. Updating climatological forcing data to near current data are compelling for impact modelling, e.g. to update model simulations or to simulate recent extreme events. Hydrological simulations are generally sensitive to bias in the meteorological forcing data, especially relative to the data used for the calibration of the model. The lack of daily resolution data at a global scale has previously been solved by adjusting re-analysis data global gridded observations. However, existing data sets of this type have been produced for a fixed past time period, determined by the main global observational data sets. Long delays between updates of these data sets leaves a data gap between present and the end of the data set. Further, hydrological forecasts require initialisations of the current state of the snow, soil, lake (and sometimes river) storage. This is normally conceived by forcing the model with observed meteorological conditions for an extended spin-up period, typically at a daily time step, to calculate the initial state. Here, we present a method named GFD (Global Forcing Data) to combine different data sets in order to produce near real-time updated hydrological forcing data that are compatible with the products covering the climatological period. GFD resembles the already established WFDEI method (Weedon et al., 2014) closely, but uses updated climatological observations, and for the near real-time it uses interim products that apply similar methods. This allows GFD to produce updated forcing data including the previous calendar month around the 10th of each month. We present the GFD method and different produced data sets, which are evaluated against the WFDEI data set, as well as with hydrological simulations with the HYPE model over Europe and the Arctic region. We show that GFD performs similarly to WFDEI and that the updated period significantly reduces the bias of the reanalysis data, although less well for the last two months of the updating cycle. For real-time updates until the current day, extending GFD with operational meteorological forecasts, a large drift is present in the hydrological simulations due to the bias of the meteorological forecasting model.

scholarly journals Comparing Meteorological Data Sets in the Evaluation of Climate Change Impact on Hydrological Indicators: A Case Study on a Mexican Basin

Water ◽  
2019 ◽  
Vol 11 (10) ◽  
pp. 2110
Author(s):  
Juan Alberto Velázquez-Zapata
Keyword(s):  
Climate Change ◽  
Bias Correction ◽  
Climate Change Impact ◽  
Hydrological Model ◽  
Meteorological Data ◽  
Data Sets ◽  
Data Set ◽  
Climate Simulations ◽  
Change Impact ◽  
Data Source

This study evaluates the choice of the meteorological data set in the simulation of the streamflow of a Mexican basin, in the bias correction of climate simulations, and in the climate change impact on hydrological indicators. The selected meteorological data sets come from stations, two interpolated data sets and one reanalysis data set. The climate simulations were taken from the five-member ensemble from the second generation Canadian Earth System Model (CanESM2) under two representative concentration pathways (RCPs), for a reference period (1981–2000) and two future periods (2041–2060 and 2081–2100). The selected lumped hydrological model is GR4J, which is a daily lumped four-parameter rainfall-runoff model. Firstly, the results show that GR4J can be calibrated and validated with the meteorological data sets to simulate daily streamflow; however, the hydrological model leads to different hydrological responses for the basin. Secondly, the bias correction procedure obtains a similar relative climate change signal for the variables, but the magnitude of the signal strongly varies with the source of meteorological data. Finally, the climate change impact on hydrological indicators also varies depending on the meteorological data source, thus, for the overall mean flow, this uncertainty is greater than the uncertainty related to the natural variability. On the other hand, mixed results were found for high flows. All in all, the selection of meteorological data source should be taken into account in the evaluation of climate change impact on water resources.

scholarly journals Hydrologic extremes – an intercomparison of multiple gridded statistical downscaling methods

2016 ◽  
Vol 20 (4) ◽  
pp. 1483-1508 ◽  
Author(s):  
Arelia T. Werner ◽  
Alex J. Cannon
Keyword(s):  
Observational Data ◽  
Statistical Downscaling ◽  
Bias Correction ◽  
Climate Model ◽  
Peak Flow ◽  
Climate Extremes ◽  
Data Sets ◽  
Hydrological Models ◽  
Data Set ◽  
Hydrologic Extremes

Abstract. Gridded statistical downscaling methods are the main means of preparing climate model data to drive distributed hydrological models. Past work on the validation of climate downscaling methods has focused on temperature and precipitation, with less attention paid to the ultimate outputs from hydrological models. Also, as attention shifts towards projections of extreme events, downscaling comparisons now commonly assess methods in terms of climate extremes, but hydrologic extremes are less well explored. Here, we test the ability of gridded downscaling models to replicate historical properties of climate and hydrologic extremes, as measured in terms of temporal sequencing (i.e. correlation tests) and distributional properties (i.e. tests for equality of probability distributions). Outputs from seven downscaling methods – bias correction constructed analogues (BCCA), double BCCA (DBCCA), BCCA with quantile mapping reordering (BCCAQ), bias correction spatial disaggregation (BCSD), BCSD using minimum/maximum temperature (BCSDX), the climate imprint delta method (CI), and bias corrected CI (BCCI) – are used to drive the Variable Infiltration Capacity (VIC) model over the snow-dominated Peace River basin, British Columbia. Outputs are tested using split-sample validation on 26 climate extremes indices (ClimDEX) and two hydrologic extremes indices (3-day peak flow and 7-day peak flow). To characterize observational uncertainty, four atmospheric reanalyses are used as climate model surrogates and two gridded observational data sets are used as downscaling target data. The skill of the downscaling methods generally depended on reanalysis and gridded observational data set. However, CI failed to reproduce the distribution and BCSD and BCSDX the timing of winter 7-day low-flow events, regardless of reanalysis or observational data set. Overall, DBCCA passed the greatest number of tests for the ClimDEX indices, while BCCAQ, which is designed to more accurately resolve event-scale spatial gradients, passed the greatest number of tests for hydrologic extremes. Non-stationarity in the observational/reanalysis data sets complicated the evaluation of downscaling performance. Comparing temporal homogeneity and trends in climate indices and hydrological model outputs calculated from downscaled reanalyses and gridded observations was useful for diagnosing the reliability of the various historical data sets. We recommend that such analyses be conducted before such data are used to construct future hydro-climatic change scenarios.

scholarly journals Handling Initial Conditions in Vector Fitting for Real Time Modeling of Power System Dynamics

Energies ◽  
2021 ◽  
Vol 14 (9) ◽  
pp. 2471
Author(s):  
Tommaso Bradde ◽  
Samuel Chevalier ◽  
Marco De Stefano ◽  
Stefano Grivet-Talocia ◽  
Luca Daniel
Keyword(s):  
Dynamical Systems ◽  
Power System ◽  
System Dynamics ◽  
Real Time ◽  
Initial Conditions ◽  
Test System ◽  
Initial State ◽  
Power System Dynamics ◽  
Vector Fitting ◽  
Time Modeling

This paper develops a predictive modeling algorithm, denoted as Real-Time Vector Fitting (RTVF), which is capable of approximating the real-time linearized dynamics of multi-input multi-output (MIMO) dynamical systems via rational transfer function matrices. Based on a generalization of the well-known Time-Domain Vector Fitting (TDVF) algorithm, RTVF is suitable for online modeling of dynamical systems which experience both initial-state decay contributions in the measured output signals and concurrently active input signals. These adaptations were specifically contrived to meet the needs currently present in the electrical power systems community, where real-time modeling of low frequency power system dynamics is becoming an increasingly coveted tool by power system operators. After introducing and validating the RTVF scheme on synthetic test cases, this paper presents a series of numerical tests on high-order closed-loop generator systems in the IEEE 39-bus test system.

Fast and accurate detection of surface defect based on improved YOLOv4

2021 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Jiawei Lian ◽  
Junhong He ◽  
Yun Niu ◽  
Tianze Wang
Keyword(s):  
Feature Extraction ◽  
Real Time ◽  
Surface Defect ◽  
Steel Ingot ◽  
Industrial Applications ◽  
Data Sets ◽  
Data Set ◽  
Processing Technologies ◽  
Content Type ◽  
Public Data

Purpose The current popular image processing technologies based on convolutional neural network have the characteristics of large computation, high storage cost and low accuracy for tiny defect detection, which is contrary to the high real-time and accuracy, limited computing resources and storage required by industrial applications. Therefore, an improved YOLOv4 named as YOLOv4-Defect is proposed aim to solve the above problems. Design/methodology/approach On the one hand, this study performs multi-dimensional compression processing on the feature extraction network of YOLOv4 to simplify the model and improve the feature extraction ability of the model through knowledge distillation. On the other hand, a prediction scale with more detailed receptive field is added to optimize the model structure, which can improve the detection performance for tiny defects. Findings The effectiveness of the method is verified by public data sets NEU-CLS and DAGM 2007, and the steel ingot data set collected in the actual industrial field. The experimental results demonstrated that the proposed YOLOv4-Defect method can greatly improve the recognition efficiency and accuracy and reduce the size and computation consumption of the model. Originality/value This paper proposed an improved YOLOv4 named as YOLOv4-Defect for the detection of surface defect, which is conducive to application in various industrial scenarios with limited storage and computing resources, and meets the requirements of high real-time and precision.

scholarly journals A Distributed Content-Based Video Retrieval System for Large Data-sets

2021 ◽  
Author(s):  
ElMehdi SAOUDI ◽  
Said Jai Andaloussi
Keyword(s):  
Real Time ◽  
Retrieval System ◽  
Video Retrieval ◽  
Multimedia Databases ◽  
Video Data ◽  
Multimedia Data ◽  
Machine Learning Techniques ◽  
Data Sets ◽  
Data Set ◽  
Content Based Video Retrieval

Abstract With the rapid growth of the volume of video data and the development of multimedia technologies, it has become necessary to have the ability to accurately and quickly browse and search through information stored in large multimedia databases. For this purpose, content-based video retrieval ( CBVR ) has become an active area of research over the last decade. In this paper, We propose a content-based video retrieval system providing similar videos from a large multimedia data-set based on a query video. The approach uses vector motion-based signatures to describe the visual content and uses machine learning techniques to extract key-frames for rapid browsing and efficient video indexing. We have implemented the proposed approach on both, single machine and real-time distributed cluster to evaluate the real-time performance aspect, especially when the number and size of videos are large. Experiments are performed using various benchmark action and activity recognition data-sets and the results reveal the effectiveness of the proposed method in both accuracy and processing time compared to state-of-the-art methods.

scholarly journals High resolution atmospheric reconstruction for Europe 1948–2012: coastDat2

2013 ◽  
Vol 6 (2) ◽  
pp. 779-809 ◽  
Author(s):  
B. Geyer
Keyword(s):  
Climate Model ◽  
Numerical Models ◽  
Regional Climate ◽  
Reanalysis Data ◽  
Data Sets ◽  
Data Set ◽  
Horizontal Grid ◽  
Global Reanalysis ◽  
Marine Climate

Abstract. The coastDat data sets were produced to give a consistent and homogeneous database mainly for assessing weather statistics and long-term changes for Europe, especially in data sparse regions. A sequence of numerical models was employed to reconstruct all aspects of marine climate (such as storms, waves, surges etc.) over many decades. Here, we describe the atmospheric part of coastDat2 (Geyer and Rockel, 2013, doi:10.1594/WDCC/coastDat-2_COSMO-CLM). It consists of a regional climate reconstruction for entire Europe, including Baltic and North Sea and parts of the Atlantic. The simulation was done for 1948 to 2012 with a regional climate model and a horizontal grid size of 0.22° in rotated coordinates. Global reanalysis data were used as forcing and spectral nudging was applied. To meet the demands on the coastDat data set about 70 variables are stored hourly.

Sign in / Sign up

Export Citation Format

Share Document