Automated Data-Driven Approaches to Evaluating and Interpreting Water Quality Time Series Data from Water Distribution Systems

The sampling frequency and quantity of time series data collected from water distribution systems has been increasing in recent years, giving rise to the potential for improving system knowledge if suitable automated techniques can be applied, in particular, machine learning. Novelty (or anomaly) detection refers to the automatic identification of novel or abnormal patterns embedded in large amounts of “normal” data. When dealing with time series data (transformed into vectors), this means abnormal events embedded amongst many normal time series points. The support vector machine is a data-driven statistical technique that has been developed as a tool for classification and regression. The key features include statistical robustness with respect to non-Gaussian errors and outliers, the selection of the decision boundary in a principled way, and the introduction of nonlinearity in the feature space without explicitly requiring a nonlinear algorithm by means of kernel functions. In this research, support vector regression is used as a learning method for anomaly detection from water flow and pressure time series data. No use is made of past event histories collected through other information sources. The support vector regression methodology, whose robustness derives from the training error function, is applied to a case study.

Download Full-text

WDSchain: A Toolbox for Enhancing the Security Using Blockchain Technology in Water Distribution System

Water ◽

10.3390/w13141944 ◽

2021 ◽

Vol 13 (14) ◽

pp. 1944

Author(s):

Haitham H. Mahmoud ◽

Wenyan Wu ◽

Yonghao Wang

Keyword(s):

Time Series ◽

Distribution Systems ◽

Water Distribution ◽

Time Series Data ◽

Series Data ◽

Process Time ◽

Time Interval ◽

Security Level ◽

Data Validation ◽

Simulated Time

This work develops a toolbox called WDSchain on MATLAB that can simulate blockchain on water distribution systems (WDS). WDSchain can import data from Excel and EPANET water modelling software. It extends the EPANET to enable simulation blockchain of the hydraulic data at any intended nodes. Using WDSchain will strengthen network automation and the security in WDS. WDSchain can process time-series data with two simulation modes: (1) static blockchain, which takes a snapshot of one-time interval data of all nodes in WDS as input and output into chained blocks at a time, and (2) dynamic blockchain, which takes all simulated time-series data of all the nodes as input and establishes chained blocks at the simulated time. Five consensus mechanisms are developed in WDSchain to provide data at different security levels using PoW, PoT, PoV, PoA, and PoAuth. Five different sizes of WDS are simulated in WDSchain for performance evaluation. The results show that a trade-off is needed between the system complexity and security level for data validation. The WDSchain provides a methodology to further explore the data validation using Blockchain to WDS. The limitations of WDSchain do not consider selection of blockchain nodes and broadcasting delay compared to commercial blockchain platforms.

Download Full-text

Event Detection in Water Distribution Systems from Multivariate Water Quality Time Series

Environmental Science & Technology ◽

10.1021/es3014024 ◽

2012 ◽

Vol 46 (15) ◽

pp. 8212-8219 ◽

Cited By ~ 80

Author(s):

Lina Perelman ◽

Jonathan Arad ◽

Mashor Housh ◽

Avi Ostfeld

Keyword(s):

Water Quality ◽

Time Series ◽

Event Detection ◽

Distribution Systems ◽

Water Distribution ◽

Water Distribution Systems

Download Full-text

Pattern matching and associative artificial neural networks for water distribution system time series data analysis

Journal of Hydroinformatics ◽

10.2166/hydro.2013.057 ◽

2013 ◽

Vol 16 (3) ◽

pp. 617-632 ◽

Cited By ~ 18

Author(s):

S. R. Mounce ◽

R. B. Mounce ◽

T. Jackson ◽

J. Austin ◽

J. B. Boxall

Keyword(s):

Neural Network ◽

Neural Networks ◽

Water Quality ◽

Time Series ◽

Pattern Matching ◽

Distribution Systems ◽

Water Distribution ◽

Quality Data ◽

Series Data ◽

Artificial Neural

Water distribution systems, and other infrastructures, are increasingly being pervaded by sensing technologies, collecting a growing volume of data aimed at supporting operational and investment decisions. These sensors monitor system characteristics, i.e. flows, pressures and water quality, such as in pipes. This paper presents the application of pattern matching techniques and binary associative neural networks for novelty detection in such data. A protocol for applying pattern matching to automatically recognise specific waveforms in time series based on their shapes is described together with a system called Advanced Uncertain Reasoning Architecture (AURA) Alert for autonomous determination of novelty. AURA is a class of binary neural network that has a number of advantages over standard artificial neural network techniques for condition monitoring including a sound theoretical basis to determine the bounds of the system operation. Results from application to several case studies are provided including both hydraulic and water quality data. In the case of pattern matching, the results demonstrated some transferability of burst patterns across District Metered Areas; however limitations in performance and difficulties with assembling pattern libraries were found. Results for the AURA system demonstrate the potential for robust event detection across multiple parameters providing valuable information for diagnosis; one example also demonstrates the potential for detection of precursor information, vital for proactive management.

Download Full-text

Multiobjective Water Distribution Systems Control of Pumping Cost, Water Quality, and Storage-Reliability Constraints

Journal of Water Resources Planning and Management ◽

10.1061/(asce)wr.1943-5452.0000309 ◽

2014 ◽

Vol 140 (2) ◽

pp. 184-193 ◽

Cited By ~ 31

Author(s):

Wojciech Kurek ◽

Avi Ostfeld

Keyword(s):

Water Quality ◽

Distribution Systems ◽

Water Distribution ◽

Water Distribution Systems ◽

Pumping Cost ◽

Systems Control ◽

And Storage

Download Full-text

A review of data-driven approaches for burst detection in water distribution systems

Urban Water Journal ◽

10.1080/1573062x.2017.1279191 ◽

2017 ◽

Vol 14 (9) ◽

pp. 972-983 ◽

Cited By ~ 38

Author(s):

Yipeng Wu ◽

Shuming Liu

Keyword(s):

Distribution Systems ◽

Water Distribution ◽

Water Distribution Systems ◽

Data Driven ◽

Burst Detection

Download Full-text

Estimating infection-related human mobility networks based on time series data of COVID-19 infection in Japan

10.1101/2021.08.02.21261486 ◽

2021 ◽

Author(s):

Tetsuya Yamada ◽

Shoi Shi

Keyword(s):

Time Series ◽

Infectious Diseases ◽

Time Series Data ◽

Human Mobility ◽

Emerging Infectious Diseases ◽

Human Movement ◽

Data Driven ◽

Disease Spread ◽

Series Data ◽

Data Infrastructure

Comprehensive and evidence-based countermeasures against emerging infectious diseases have become increasingly important in recent years. COVID-19 and many other infectious diseases are spread by human movement and contact, but complex transportation networks in 21 century make it difficult to predict disease spread in rapidly changing situations. It is especially challenging to estimate the network of infection transmission in the countries that the traffic and human movement data infrastructure is not yet developed. In this study, we devised a method to estimate the network of transmission of COVID-19 from the time series data of its infection and applied it to determine its spread across areas in Japan. We incorporated the effects of soft lockdowns, such as the declaration of a state of emergency, and changes in the infection network due to government-sponsored travel promotion, and predicted the spread of infection using the Tokyo Olympics as a model. The models used in this study are available online, and our data-driven infection network models are scalable, whether it be at the level of a city, town, country, or continent, and applicable anywhere in the world, as long as the time-series data of infections per region is available. These estimations of effective distance and the depiction of infectious disease networks based on actual infection data are expected to be useful in devising data-driven countermeasures against emerging infectious diseases worldwide.

Download Full-text

Partial Correlation-Based Attention for Multivariate Time Series Forecasting

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v34i10.7132 ◽

2020 ◽

Vol 34 (10) ◽

pp. 13720-13721

Author(s):

Won Kyung Lee

Keyword(s):

Time Series ◽

Partial Correlation ◽

Time Series Data ◽

Multivariate Time Series ◽

Time Series Forecasting ◽

Data Driven ◽

Series Data ◽

Time Lags ◽

Dependency Structure ◽

Agnostic Learning

A multivariate time-series forecasting has great potentials in various domains. However, it is challenging to find dependency structure among the time-series variables and appropriate time-lags for each variable, which change dynamically over time. In this study, I suggest partial correlation-based attention mechanism which overcomes the shortcomings of existing pair-wise comparisons-based attention mechanisms. Moreover, I propose data-driven series-wise multi-resolution convolutional layers to represent the input time-series data for domain agnostic learning.

Download Full-text