Rainfall prediction optimization model in ten-day time step based on sliding window mechanism and zero sum game

Abstract Rainfall is a precious water resource, especially for Shenzhen with scarce local water resources. Therefore, an effective rainfall prediction model is essential for improvement of water supply efficiency and water resources planning in Shenzhen. In this study, a deep learning model based on zero sum game (ZSG) was proposed to predict ten-day rainfall, the regular models were constructed for comparison, and the cross-validation was performed to further compare the generalization ability of the models. Meanwhile, the sliding window mechanism, differential evolution genetic algorithm, and discrete wavelet transform were developed to solve the problem of data non-stationarity, local optimal solutions, and noise filtration, respectively. The k-means clustering algorithm was used to discover the potential laws of the dataset to provide reference for sliding window. Mean square error (MSE), Nash–Sutcliffe efficiency coefficient (NSE) and mean absolute error (MAE) were applied for model evaluation. The results indicated that ZSG could better optimize the parameter adjustment process of models, and improved generalization ability of models. The generalization ability of the bidirectional model was superior to that of the unidirectional model. The ZSG-based models showed stronger superiority compared with regular models, and provided the lowest MSE (1.29%), NSE (21.75%), and MAE (7.5%) in the ten-day rainfall prediction.

Download Full-text

Non Stationary Multi-Armed Bandit: Empirical Evaluation of a New Concept Drift-Aware Algorithm

Entropy ◽

10.3390/e23030380 ◽

2021 ◽

Vol 23 (3) ◽

pp. 380

Author(s):

Emanuele Cavenaghi ◽

Gabriele Sottocornola ◽

Fabio Stella ◽

Markus Zanker

Keyword(s):

Real World ◽

Concept Drift ◽

Empirical Evaluation ◽

Sliding Window ◽

Discount Factor ◽

Data Streaming ◽

Sources Of Information ◽

Sequential Decision ◽

Time Step ◽

Thompson Sampling

The Multi-Armed Bandit (MAB) problem has been extensively studied in order to address real-world challenges related to sequential decision making. In this setting, an agent selects the best action to be performed at time-step t, based on the past rewards received by the environment. This formulation implicitly assumes that the expected payoff for each action is kept stationary by the environment through time. Nevertheless, in many real-world applications this assumption does not hold and the agent has to face a non-stationary environment, that is, with a changing reward distribution. Thus, we present a new MAB algorithm, named f-Discounted-Sliding-Window Thompson Sampling (f-dsw TS), for non-stationary environments, that is, when the data streaming is affected by concept drift. The f-dsw TS algorithm is based on Thompson Sampling (TS) and exploits a discount factor on the reward history and an arm-related sliding window to contrast concept drift in non-stationary environments. We investigate how to combine these two sources of information, namely the discount factor and the sliding window, by means of an aggregation function f(.). In particular, we proposed a pessimistic (f=min), an optimistic (f=max), as well as an averaged (f=mean) version of the f-dsw TS algorithm. A rich set of numerical experiments is performed to evaluate the f-dsw TS algorithm compared to both stationary and non-stationary state-of-the-art TS baselines. We exploited synthetic environments (both randomly-generated and controlled) to test the MAB algorithms under different types of drift, that is, sudden/abrupt, incremental, gradual and increasing/decreasing drift. Furthermore, we adapt four real-world active learning tasks to our framework—a prediction task on crimes in the city of Baltimore, a classification task on insects species, a recommendation task on local web-news, and a time-series analysis on microbial organisms in the tropical air ecosystem. The f-dsw TS approach emerges as the best performing MAB algorithm. At least one of the versions of f-dsw TS performs better than the baselines in synthetic environments, proving the robustness of f-dsw TS under different concept drift types. Moreover, the pessimistic version (f=min) results as the most effective in all real-world tasks.

Download Full-text

Analyzing catchment behavior through catchment modeling in the Gilgel Abay, Upper Blue Nile River Basin, Ethiopia

Hydrology and Earth System Sciences ◽

10.5194/hess-14-2153-2010 ◽

2010 ◽

Vol 14 (10) ◽

pp. 2153-2165 ◽

Cited By ~ 46

Author(s):

S. Uhlenbrook ◽

Y. Mohamed ◽

A. S. Gragne

Keyword(s):

Water Resources ◽

Model Performance ◽

Computation Time ◽

Hydrological Processes ◽

Runoff Generation ◽

Time Step ◽

Blue Nile ◽

Model Transferability ◽

Catchment Modeling ◽

Data Limitations

Abstract. Understanding catchment hydrological processes is essential for water resources management, in particular in data scarce regions. The Gilgel Abay catchment (a major tributary into Lake Tana, source of the Blue Nile) is undergoing intensive plans for water management, which is part of larger development plans in the Blue Nile basin in Ethiopia. To obtain a better understanding of the water balance dynamics and runoff generation mechanisms and to evaluate model transferability, catchment modeling has been conducted using the conceptual hydrological model HBV. Accordingly, the catchment of the Gilgel Abay has been divided into two gauged sub-catchments (Upper Gilgel Abay and Koga) and the un-gauged part of the catchment. All available data sets were tested for stationarity, consistency and homogeneity and the data limitations (quality and quantity) are discussed. Manual calibration of the daily models for three different catchment representations, i.e. (i) lumped, (ii) lumped with multiple vegetation zones, and (iii) semi-distributed with multiple vegetation and elevation zones, showed good to satisfactory model performances with Nash-Sutcliffe efficiencies Reff > 0.75 and > 0.6 for the Upper Gilgel Abay and Koga sub-catchments, respectively. Better model results could not be obtained with manual calibration, very likely due to the limited data quality and model insufficiencies. Increasing the computation time step to 15 and 30 days improved the model performance in both sub-catchments to Reff > 0.8. Model parameter transferability tests have been conducted by interchanging parameters sets between the two gauged sub-catchments. Results showed poor performances for the daily models (0.30 < Reff < 0.67), but better performances for the 15 and 30 days models, Reff > 0.80. The transferability tests together with a sensitivity analysis using Monte Carlo simulations (more than 1 million model runs per catchment representation) explained the different hydrologic responses of the two sub-catchments, which seems to be mainly caused by the presence of dambos in Koga sub-catchment. It is concluded that daily model transferability is not feasible, while it can produce acceptable results for the 15 and 30 days models. This is very useful for water resources planning and management, but not sufficient to capture detailed hydrological processes in an ungauged area.

Download Full-text

Data–Driven Indirect Adaptive Model Predictive Control

Jurnal Teknologi ◽

10.11113/jt.v54.807 ◽

2012 ◽

Cited By ~ 2

Author(s):

Norhaliza Wahab ◽

Mohamed Reza Katebi ◽

Mohd Fua’ad Rahmat ◽

Salinda Bunyamin

Keyword(s):

Model Predictive Control ◽

Activated Sludge ◽

State Space ◽

Predictive Control ◽

Sliding Window ◽

Subspace Identification ◽

Activated Sludge Process ◽

Adaptive Model ◽

Time Step ◽

Adaptive Model Predictive Control

Kertas kerja ini membincangkan tentang reka bentuk Pengawal Ramalan Model Suai menggunakan kaedah Pengenalpastian Model Keadaan Ruang Sub–ruang bagi proses enapcemar teraktif. Penggunaan teknik Pengenalpastian Model Keadaan Ruang Sub–ruang di dalam kaedah kawalan tingkat gelangsar suai dibincangkan di mana pengenalpastian sub–ruang dalam talian menggunakan algoritma N4SID di perkenalkan bersama dengan rekabentuk Pengawal ramalan model. Pembangunan N4SID dalam talian di dalam kertas kerja ini menggunakan pengemaskini QR di mana gabungan di antara teknik kemaskini dan kemasbawah membolehkan pengadaptasi tingkap gelangsar. Di sini, untuk setiap langkah masa, bagi setiap data baru akan dimasukkan ke faktor R manakala data yang lama dibuang. Begitu juga, strategi bagi uraian nilai tunggal diperkenalkan ke dalam Pengawal Ramalan Model Suai tak langsung untuk masukan tambahan kawalan bagi sistem terkekang tak lelurus. Beberapa kajian simulasi bagi parameter kawalan berlainan di dalam pengawal/pengenalpastian algoritma dilaksanakan. Bagi reka bentuk Pengawal Ramalan Model Suai tak langsung, pengiraan masa yang terlibat dengan menggunakan pendekatan uraian nilai tunggal kurang berbanding dengan kaedah perancangan kuadratik dan keputusan yang memberangsangkan ini adalah sumbangan utama di dalam kertas kerja ini. Kata kunci: Pengawal suai; proses enapcemar teraktif; pengawal ramalan model; pengenalpastian sub–ruang This paper explores the design of Adaptive Model Predictive Control (AMPC) using Subspace State–space Model Identification (SMI) techniques for an activated sludge process. The implementation of SMI techniques in the adaptive sliding window control methods are discussed where the online subspace identification using Numerical State–space Subspace System Identification (N4SID) algorithm is proposed along with Model Predictive Control (MPC) design method. The online N4SID algorithm developed in this study makes use of the QR–updating where the combination of update and down date techniques enables sliding window adaptation. Here, at each time step, for the new experimental data added into R factor, the oldest data are removed. Also, the Singular Value Decomposition (SVD–based) strategy is proposed into Indirect AMPC (IAMPC) for the control increment input constrained nonlinear system. Several simulation studies for different control parameters in control/identification algorithm are performed. For the IAMPC control design, the computational times involved using an SVD approach shows less burdensome compared to Quadratic Programming (QP) method and such an interesting result is considered as one of the main contribution in this paper. Key words: Adaptive control; activated sludge process; model predictive control; subspace identification

Download Full-text

Application of Machine Learning for Fault Classification and Location in a Radial Distribution Grid

Applied Sciences ◽

10.3390/app10144965 ◽

2020 ◽

Vol 10 (14) ◽

pp. 4965

Author(s):

Yordanos Dametw Mamuya ◽

Yih-Der Lee ◽

Jing-Wen Shen ◽

Md Shafiullah ◽

Cheng-Chien Kuo

Keyword(s):

Machine Learning ◽

Fault Detection ◽

Radial Distribution ◽

Fault Classification ◽

Discrete Wavelet ◽

Efficiency Coefficient ◽

Distribution Grid ◽

Phase Current ◽

Three Phase ◽

Types Of Faults

Fault location with the highest possible accuracy has a significant role in expediting the restoration process, after being exposed to any kind of fault in power distribution grids. This paper provides fault detection, classification, and location methods using machine learning tools and advanced signal processing for a radial distribution grid. The three-phase current signals, one cycle before and one cycle after the inception of the fault are measured at the sending end of the grid. A discrete wavelet transform (DWT) is employed to extract useful features from the three-phase current signal. Standard statistical techniques are then applied onto DWT coefficients to extract the useful features. Among many features, mean, standard deviation (SD), energy, skewness, kurtosis, and entropy are evaluated and fed into the artificial neural network (ANN), Multilayer perceptron (MLP), and extreme learning machine (ELM), to identify the fault type and its location. During the training process, all types of faults with variations in the loading and fault resistance are considered. The performance of the proposed fault locating methods is evaluated in terms of root mean absolute percentage error (MAPE), root mean squared error (RMSE), Willmott’s index of agreement (WIA), coefficient of determination ( R 2 ), and Nash-Sutcliffe model efficiency coefficient (NSEC). The time it takes for training and testing are also considered. The proposed method that discrete wavelet transforms with machine learning is a very accurate and reliable method for fault classifying and locating in both a balanced and unbalanced radial system. 100% fault detection accuracy is achieved for all types of faults. Except for the slight confusion of three line to ground (3LG) and three line (3L) faults, 100% classification accuracy is also achieved. The performance measures show that both MLP and ELM are very accurate and comparative in locating faults. The method can be further applied for meshed networks with multiple distributed generators. Renewable generations in the form of distributed generation units can also be studied.

Download Full-text

Paleo-hydrologic reconstruction of 400 years of past flows at a weekly time step for major rivers of Western Canada

10.5194/essd-2019-57 ◽

2019 ◽

Author(s):

Andrew R. Slaughter ◽

Saman Razavi

Keyword(s):

Time Series ◽

Water Resources ◽

Tree Ring ◽

Flow Distribution ◽

Future Climate Change ◽

Western Canada ◽

Tree Ring Chronology ◽

Time Step ◽

Water Resources Systems ◽

Tree Ring Data

Abstract. The assumption of stationarity in water resources no longer holds, particularly within the context of future climate change. Plausible scenarios of flows that fluctuate outside the envelope of variability of the gauging data are required to assess the robustness of water resources systems to future conditions. This study presents a novel method of generating weekly-time-step flows based on tree-ring chronology data. Specifically, this method addresses two long-standing challenges with paleo-reconstruction: (1) the typically limited predictive power of tree-ring data at the annual and sub-annual scale, and (2) the inflated short-term persistence in tree-ring time series and improper use of prewhitening. Unlike the conventional approach, this method establishes relationships between tree-ring chronologies and naturalised flow at a biennial scale to preserve persistence properties and variability of hydrological time series. Biennial flow reconstructions are further disaggregated to weekly, according to the weekly flow distribution of reference two-year instrumental periods, identified as periods with broadly similar tree-ring properties to that of every two-year paleo-period. The Saskatchewan River Basin (SaskRB), a major river in Western Canada, is selected as a study area, and weekly flows in its four major tributaries are extended back to the year 1600. The study shows that the reconstructed flows properly preserve the statistical properties of the reference flows, particularly, short- to long-term persistence and the structure of variability across time scales. An ensemble approach is presented to represent the uncertainty inherent in the statistical relationships and disaggregation method. The ensemble of reconstructed weekly flows are publically available for download from https://doi.org/10.20383/101.0139 (Slaughter and Razavi, 2019).

Download Full-text

Gated Recurrent Unit Networks and Discrete Wavelet Transforms Applied to Forecasting and Trading in the Stock Market

10.5753/eniac.2020.12167 ◽

2020 ◽

Author(s):

Victor Biazon ◽

Reinaldo Bianchi

Keyword(s):

Stock Market ◽

Wavelet Transforms ◽

Stock Price ◽

Discrete Wavelet ◽

Discrete Wavelet Transforms ◽

Time Step ◽

Proposed Model ◽

Gated Recurrent Unit ◽

Novel Model ◽

Unit Network

Trading in the stock market always comes with the challenge of deciding the best action to take on each time step. The problem is intensified by the theory that it is not possible to predict stock market time series as all information related to the stock price is already contained in it. In this work we propose a novel model called Discrete Wavelet Transform Gated Recurrent Unit Network (DWT-GRU). The model learns from the data to choose between buying, holding and selling, and when to execute them. The proposed model was compared to other recurrent neural networks, with and without wavelets preprocessing, and the buy and hold strategy. The results shown that the DWT-GRU outperformed all the set baselines in the analysed stocks of the Brazilian stock market.

Download Full-text

Information content in the rainfall observed by INSAT-3D and IMERG: An intercomparison study over the north eastern region of India

10.5194/egusphere-egu21-14100 ◽

2021 ◽

Author(s):

Aniket Chakravorty ◽

Shyam Sundar Kundu ◽

Penumetcha Lakshmi Narasa Raju

Keyword(s):

Information Content ◽

Permutation Entropy ◽

Eastern Region ◽

Redundant Information ◽

Time Step ◽

Wavelet Entropy ◽

Rainfall Prediction ◽

The North ◽

New Information ◽

North Eastern

<p>There has been a noticeable increase in the application of artificial intelligence (AI) algorithms in various areas, in the recent past. One such area is the prediction of rainfall over a region. This application has seen crucial advancement with the use of deep sequential learning algorithms. This new approach to rainfall prediction has also helped increase the utilization of satellite data for prediction. As, AI based prediction algorithms are based on data, the characteristics of it dominates the accuracy of the prediction. And one such characteristic is the information content in the data being used. This information content is classified into redundant information (information of past states in the current state) and new information. The performance of the AI based rainfall prediction depends on the amount of redundant information present in the data being used for training the AI model, more the redundant information (less the new information content) more accurate will be the prediction. Various entropy based measure have been used to quantify the new information content in the data, like permutation entropy, sample entropy, wavelet entropy, etc. This study uses a new measure called the Wavelet Entropy Energy Measure (WEEM). One of the advantages of WEEM is that it considers the dynamics of the process spread across different time scales, which other information measures have not considered explicitly. Since, the dynamics of rainfall is multi-scalar in nature, WEEM is a suitable measure for it. The main goal of this study is to find out the amount of information being generated by INSAT-3D and IMERG rainfall at each time step over the North Eastern Region of India, which will dictate the suitability of the two rainfall product to be used for AI based rainfall prediction.</p>

Download Full-text

Hydrological Modelling and Water Resources Assessment of Chongwe River Catchment using WEAP Model

Water ◽

10.3390/w11040839 ◽

2019 ◽

Vol 11 (4) ◽

pp. 839 ◽

Cited By ~ 4

Author(s):

Tewodros M. Tena ◽

Phenny Mwaanga ◽

Alick Nguvulu

Keyword(s):

Water Resources ◽

Water Resource Management ◽

Measurement Data ◽

Annual Precipitation ◽

Mean Annual Precipitation ◽

Efficiency Coefficient ◽

Management Options ◽

Weap Model ◽

River Catchment ◽

Monthly Streamflow

The Chongwe River Catchment (CRC) is located in Zambia. It receives a mean annual precipitation of 889 mm. The catchment is facing growing anthropogenic and socio-economic activities leading to severe water shortages in recent years, particularly from July to October. The objective of this study was to assess the available water resources by investigating the important hydrological components and estimating the catchment water balance using the Water Evaluation and Planning (WEAP) model. The average precipitation over a 52 year period and a 34 year period of streamflow measurement data for four stations were used in the hydrological balance model. The results revealed that the catchment received an estimated mean annual precipitation of 4603.12 Mm3. It also released an estimated mean annual runoff and evapotranspiration of 321.94 Mm3 and 4063.69 Mm3, respectively. The estimated mean annual total abstractions in the catchment was 119.87 Mm3. The average annual change in the catchment storage was 120.18 Mm3. The study also determined an external inflow of 22.55 Mm3 from the Kafue River catchment. The simulated mean monthly streamflow at the outlet of the CRC was 10.32 m3/s. The estimated minimum and maximum streamflow volume of the Chongwe River was about 1.01 Mm3 in September and 79.7 Mm3 in February, respectively. The performance of the WEAP model simulation was assessed statistically using the coefficient of determination (R2 = 0.97) and the Nash–Sutcliffe model efficiency coefficient (NSE = 0.64). The R2 and NSE values indicated a satisfactory model fit and result. Meeting the water demand of the growing population and associated socio-economic development activities in the CRC is possible but requires appropriate water resource management options.

Download Full-text

Improving Image Search through MKFCM Clustering Strategy-Based Re-ranking Measure

Journal of Intelligent Systems ◽

10.1515/jisys-2017-0227 ◽

2018 ◽

Vol 29 (1) ◽

pp. 497-514

Author(s):

A.K. Naveena ◽

N.K. Narayanan

Keyword(s):

Clustering Algorithm ◽

Data Retrieval ◽

Initial Step ◽

Discrete Wavelet ◽

Image Search ◽

Clustering Methods ◽

Fuzzy C Means ◽

Image Retrieval System ◽

Fuzzy C Means Clustering ◽

Search Approach

Abstract The main intention of this research is to develop a novel ranking measure for content-based image retrieval system. Owing to the achievement of data retrieval, most commercial search engines still utilize a text-based search approach for image search by utilizing encompassing textual information. As the text information is, in some cases, noisy and even inaccessible, the drawback of such a recovery strategy is to the extent that it cannot depict the contents of images precisely, subsequently hampering the execution of image search. In order to improve the performance of image search, we propose in this work a novel algorithm for improving image search through a multi-kernel fuzzy c-means (MKFCM) algorithm. In the initial step of our method, images are retrieved using four-level discrete wavelet transform-based features and the MKFCM clustering algorithm. Next, the retrieved images are analyzed using fuzzy c-means clustering methods, and the rank of the results is adjusted according to the distance of a cluster from a query. To improve the ranking performance, we combine the retrieved result and ranking result. At last, we obtain the ranked retrieved images. In addition, we analyze the effects of different clustering methods. The effectiveness of the proposed methodology is analyzed with the help of precision, recall, and F-measures.

Download Full-text

Quantifying the Effect of Land Use Change and Climate Variability on Green Water Resources in the Xihe River Basin, Northeast China

Sustainability ◽

10.3390/su11020338 ◽

2019 ◽

Vol 11 (2) ◽

pp. 338 ◽

Cited By ~ 2

Author(s):

Leting Lyu ◽

Xiaorui Wang ◽

Caizhi Sun ◽

Tiantian Ren ◽

Defeng Zheng

Keyword(s):

Land Use ◽

Water Resources ◽

Land Use Change ◽

Climate Variability ◽

River Basin ◽

Water Flow ◽

Water Storage ◽

Green Water ◽

Efficiency Coefficient ◽

Relative Change

Based on a land use interpretation and distributed hydrological model, soil and water assessment tool (SWAT), this study simulated the hydrological cycle in Xihe River Basin in northern China. In addition, the influence of climate variability and land use change on green water resources in the basin from 1995 to 2015 was analyzed. The results show that (1) The ENS (Nash-Sutcliffe model efficiency coefficient) and R2 (coefficient of determination) were 0.94 and 0.89, respectively, in the calibration period, and 0.89 and 0.88, respectively, in the validation period. These indicate high simulation accuracy; (2) Changes in green water flow and green water storage due to climate variability accounted for increases of 2.07 mm/a and 1.28 mm/a, respectively. The relative change rates were 0.49% and 0.9%, respectively, and the green water coefficient decreased by 1%; (3) Changes in green water flow and green water storage due to land use change accounted for increases of 69.15 mm and 48.82 mm, respectively. The relative change rates were 16.4% and 37.2%, respectively, and the green water coefficient increased by 10%; (4) Affected by both climate variability and land use change, green water resources increased by 121.3 mm and the green water coefficient increased by 9% in the Xihe River Basin. It is noteworthy that the influence of land use change was greater than that of climate variability.

Download Full-text