Hydrostats: A Python Package for Characterizing Errors between Observed and Predicted Time Series

Hydrologists use a number of tools to compare model results to observed flows. These include tools to pre-process the data, data frames to store and access data, visualization and plotting routines, error metrics for single realizations, and ensemble metrics for stochastic realizations to calibrate and evaluate hydrologic models. We present an open-source Python package to help characterize predicted and observed hydrologic time series data called hydrostats which has three main capabilities: Data storage and retrieval based on the Python Data Analysis Library (pandas), visualization and plotting routines using Matplotlib, and a metrics library that currently contains routines to compute over 70 different error metrics and routines for ensemble forecast skill scores. Hydrostats data storage and retrieval functions allow hydrologists to easily compare all, or portions of, a time series. For example, it makes it easy to compare observed and modeled data only during April over a 30-year period. The package includes literature references, explanations, examples, and source code. In this note, we introduce the hydrostats package, provide short examples of the various capabilities, and provide some background on programming issues and practices. The hydrostats package provides a range of tools to make characterizing and analyzing model data easy and efficient. The electronic supplement provides working hydrostats examples.

Download Full-text

Extending HydroShare to enable hydrologic time series data as social media

Journal of Hydroinformatics ◽

10.2166/hydro.2015.331 ◽

2015 ◽

Vol 18 (2) ◽

pp. 198-209 ◽

Cited By ~ 5

Author(s):

Jeffrey M. Sadler ◽

Daniel P. Ames ◽

Shaun J. Livingston

Keyword(s):

Social Media ◽

Time Series ◽

Web Services ◽

Data Management ◽

Data Storage ◽

Time Series Data ◽

Series Data ◽

Design Development ◽

Scientific Discussion ◽

Hydrologic Time Series

The Consortium of Universities for the Advancement of Hydrologic Science Inc. (CUAHSI) hydrologic information system (HIS) is a widely used service oriented system for time series data management. While this system is intended to empower the hydrologic sciences community with better data storage and distribution, it lacks support for the kind of ‘Web 2.0’ collaboration and social-networking capabilities being used in other fields. This paper presents the design, development, and testing of a software extension of CUAHSI's newest product, HydroShare. The extension integrates the existing CUAHSI HIS into HydroShare's social hydrology architecture. With this extension, HydroShare provides integrated HIS time series with efficient archiving, discovery, and retrieval of the data, extensive creator and science metadata, scientific discussion and collaboration around the data and other basic social media features. HydroShare provides functionality for online social interaction and collaboration while the existing HIS provides the distributed data management and web services framework. The extension is expected to enable scientists to access and share both national- and laboratory-scale hydrologic time series datasets in a standards-based web services architecture combined with social media functionality developed specifically for the hydrologic sciences.

Download Full-text

Use of hydrologic time series data for identification of recharge mechanism in a fractured bedrock aquifer system

Journal of Hydrology ◽

10.1016/s0022-1694(00)00158-x ◽

2000 ◽

Vol 229 (3-4) ◽

pp. 190-201 ◽

Cited By ~ 144

Author(s):

Jin-Yong Lee ◽

Kang-Kun Lee

Keyword(s):

Time Series ◽

Time Series Data ◽

Series Data ◽

Aquifer System ◽

Fractured Bedrock ◽

Hydrologic Time Series ◽

Fractured Bedrock Aquifer ◽

Bedrock Aquifer

Download Full-text

HYDROLOGIC TIME SERIES DATA MODELING USING MULTIPLICATIVE ARIMA

Advances in Geosciences ◽

10.1142/9789812836144_0008 ◽

2010 ◽

pp. 81-94

Author(s):

CHAKKRAPONG TAEWICHIT ◽

SUWATANA CHITTALADAKORN

Keyword(s):

Time Series ◽

Time Series Data ◽

Data Modeling ◽

Series Data ◽

Hydrologic Time Series

Download Full-text

Use of Hydrologic Time Series Data for Analysis the Karstic Drought Characteristic

Applied Mechanics and Materials ◽

10.4028/www.scientific.net/amm.641-642.127 ◽

2014 ◽

Vol 641-642 ◽

pp. 127-131

Author(s):

Li Hong Liu ◽

Da Sheng Wang ◽

He Huang ◽

Guang Quan Xu

Keyword(s):

Time Series ◽

Time Series Data ◽

Series Data ◽

Water Volume ◽

Significant Heterogeneity ◽

Drought Duration ◽

Hydrologic Time Series ◽

Drought Hazard ◽

Groundwater Drought ◽

Drought Frequency Analysis

Karstification creates significant heterogeneity of hydraulic conductivity within the aquifer, where flows are organized to a hierarchical structure, from the surface to the spring. A karstic aquifer subjected to groundwater flood and drought, as a site for the occurrence of karst groundwater, is the main or unique focus for groundwater development and utilization in southwestern China. The present paper introduces a methodology devoted to groundwater drought hazard assessment. It focuses on groundwater drought by applying of the spring time series for an estimate and categorization of operating resources of groundwater. The results show that a permit for use of water for ER1+ER2 up to 0.48 m3/s, with the exceeding probability 80% selected for representing dry. The longest drought duration time was happened in the year 1993 with the 2.9×106m3 shortage of water volume. Groundwater drought frequency analysis provides a useful tool for water management.

Download Full-text

Time Series Outlier Detection Based on Sliding Window Prediction

Mathematical Problems in Engineering ◽

10.1155/2014/879736 ◽

2014 ◽

Vol 2014 ◽

pp. 1-14 ◽

Cited By ~ 23

Author(s):

Yufeng Yu ◽

Yuelong Zhu ◽

Shijin Li ◽

Dingsheng Wan

Keyword(s):

Time Series ◽

Outlier Detection ◽

Time Series Data ◽

Series Data ◽

Forecasting Model ◽

Data Series ◽

Hydrologic Time Series ◽

Prediction Confidence ◽

Evaluation Of Data ◽

Operation And Management

In order to detect outliers in hydrological time series data for improving data quality and decision-making quality related to design, operation, and management of water resources, this research develops a time series outlier detection method for hydrologic data that can be used to identify data that deviate from historical patterns. The method first built a forecasting model on the history data and then used it to predict future values. Anomalies are assumed to take place if the observed values fall outside a given prediction confidence interval (PCI), which can be calculated by the predicted value and confidence coefficient. The use ofPCIas threshold is mainly on the fact that it considers the uncertainty in the data series parameters in the forecasting model to address the suitable threshold selection problem. The method performs fast, incremental evaluation of data as it becomes available, scales to large quantities of data, and requires no preclassification of anomalies. Experiments with different hydrologic real-world time series showed that the proposed methods are fast and correctly identify abnormal data and can be used for hydrologic time series analysis.

Download Full-text

MVTS-Data Toolkit: A Python package for preprocessing multivariate time series data

SoftwareX ◽

10.1016/j.softx.2020.100518 ◽

2020 ◽

Vol 12 ◽

pp. 100518 ◽

Cited By ~ 1

Author(s):

Azim Ahmadzadeh ◽

Kankana Sinha ◽

Berkay Aydin ◽

Rafal A. Angryk

Keyword(s):

Time Series ◽

Time Series Data ◽

Multivariate Time Series ◽

Series Data ◽

Python Package

Download Full-text

Linear Time Complexity Time Series Clustering with Symbolic Pattern Forest

Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2019/406 ◽

2019 ◽

Author(s):

Xiaosheng Li ◽

Jessica Lin ◽

Liang Zhao

Keyword(s):

Time Series ◽

Data Storage ◽

Time Complexity ◽

Clustering Algorithm ◽

Time Series Data ◽

Linear Time ◽

Series Data ◽

Data Generation ◽

Time Series Clustering ◽

Group Structures

With increasing powering of data storage and advances in data generation and collection technologies, large volumes of time series data become available and the content is changing rapidly. This requires the data mining methods to have low time complexity to handle the huge and fast-changing data. This paper presents a novel time series clustering algorithm that has linear time complexity. The proposed algorithm partitions the data by checking some randomly selected symbolic patterns in the time series. Theoretical analysis is provided to show that group structures in the data can be revealed from this process. We evaluate the proposed algorithm extensively on all 85 datasets from the well-known UCR time series archive, and compare with the state-of-the-art approaches with statistical analysis. The results show that the proposed method is faster, and achieves better accuracy compared with other rival methods.

Download Full-text

Multiscale Analysis of Hydrologic Time Series Data using the Hilbert-Huang Transform

Vadose Zone Journal ◽

10.2136/vzj2009.0163 ◽

2010 ◽

Vol 9 (4) ◽

pp. 925-942 ◽

Cited By ~ 12

Author(s):

J. Rudi ◽

R. Pabel ◽

G. Jager ◽

R. Koch ◽

A. Kunoth ◽

...

Keyword(s):

Time Series ◽

Multiscale Analysis ◽

Time Series Data ◽

Series Data ◽

Hilbert Huang Transform ◽

Hydrologic Time Series

Download Full-text

State space modelling and forecasting of sugarcane yield in Haryana, India

Journal of Applied and Natural Science ◽

10.31018/jans.v9i4.1485 ◽

2017 ◽

Vol 9 (4) ◽

pp. 2036-2042 ◽

Cited By ~ 1

Author(s):

Suman Suman ◽

Urmil Verma

Keyword(s):

Time Series ◽

State Space ◽

Time Series Data ◽

Moving Average ◽

State Space Models ◽

Series Data ◽

Arima Models ◽

Department Of Agriculture ◽

Error Metrics ◽

Sugarcane Yield

Box and Jenkins’ Autoregressive Integrated Moving Average (ARIMA) models are widely used for analyzing and forecasting the time-series data. In this approach, the underlying parameters are assumed to be constant however the data in agriculture are generally collected over time and thus have the time-dependency in parameters. Such data can be analyzed using state space (SS) procedures by the application of Kalman filtering technique. The purpose of this article is to illustrate the usefulness of state space models in sugarcane yield forecasting and to pro-vide some empirical evidence for its superiority over the classical time-series analysis. ARIMA and state space models individually could provide the suitable relationship(s) to reliably forecast the sugarcane yield in Karnal, Ambala, Kurukshetra, Yamunanagar and Panipat districts of Haryana (India). However, the state space models with lower error metrics showed the superiority over ARIMA models for this empirical study. The sugarcane yield forecasts based on SS models in the districts under consideration showed good agreement with State Department of Agriculture (DOA) yields by showing 3-6 percent average absolute deviations.

Download Full-text