Runoff Prediction Based on Deep Belief Networks

2021 ◽  
Vol 2021 (2) ◽  
pp. 35-46
Author(s):  
Thanh Hiên Nguyễn ◽  
Thi Tran ◽  
Hieu Duong ◽  
Hoai Tran
Keyword(s):  
Gradient Descent ◽  
Stochastic Gradient Descent ◽  
Belief Networks ◽  
Deep Belief Networks ◽  
Nonlinear Functions ◽  
Mountain Regions ◽  
Swarm Optimization ◽  
Central Highland ◽  
Runoff Prediction ◽  
The Impact

Runoff prediction has recently become an essential task with respect to assessing the impact of climate change to people’s livelihoods and production. However, the runoff time series always exhibits nonlinear and non-stationary features, which makes it very difficult to be accurately predicted. Machine learning have been recently proved to be a powerful tool in helping society adapt to a changing climate and its subfield, deep learning, showed the power in approximate nonlinear functions. In this study, we propose a method based on deep belief networks (DBN) for runoff prediction. In order to evaluate the proposed method, we collected runoff datasets from Srepok and Dak Nong rivers located in mountain regions of the Central Highland of Vietnam in the periods of 2001-2007 at Dak Nong hydrology station and 1990-2011 at Buon Don hydrology station, respectively. Experimental results show that DBN outperforms, respectively, LSTM, BiLSTM, multi-layer perceptron (MLP) trained by particle swarm optimization (PSO) and MLP trained by stochastic gradient descent (SGD) in which gradients are computed using the backpropagation (BP) procedure. The results also confirm that DBN is suitable to employ for the task of runoff prediction.

scholarly journals Study of Precipitation Forecast Based on Deep Belief Networks

Algorithms ◽  
2018 ◽  
Vol 11 (9) ◽  
pp. 132 ◽  
Author(s):  
Jinglin Du ◽  
Yayun Liu ◽  
Zhijun Liu
Keyword(s):  
Weather Forecasting ◽  
Human Life ◽  
Current Trend ◽  
Predictive Performance ◽  
Feature Representation ◽  
Precipitation Forecast ◽  
Semantic Features ◽  
Belief Networks ◽  
Deep Belief Networks ◽  
The Impact

Due to the impact of weather forecasting on global human life, and to better reflect the current trend of weather changes, it is necessary to conduct research about the prediction of precipitation and provide timely and complete precipitation information for climate prediction and early warning decisions to avoid serious meteorological disasters. For the precipitation prediction problem in the era of climate big data, we propose a new method based on deep learning. In this paper, we will apply deep belief networks in weather precipitation forecasting. Deep belief networks transform the feature representation of data in the original space into a new feature space, with semantic features to improve the predictive performance. The experimental results show, compared with other forecasting methods, the feasibility of deep belief networks in the field of weather forecasting.

scholarly journals Optimal survey schemes for stochastic gradient descent with applications to M-estimation

2019 ◽  
Vol 23 ◽  
pp. 310-337 ◽  
Author(s):  
Stephan Clémençon ◽  
Patrice Bertail ◽  
Emilie Chautru ◽  
Guillaume Papa
Keyword(s):  
Gradient Descent ◽  
A Priori ◽  
Stochastic Gradient ◽  
Stochastic Gradient Descent ◽  
Estimation Methods ◽  
Full Data ◽  
Predictive Learning ◽  
Inclusion Probabilities ◽  
Estimation Problems ◽  
The Impact

Iterative stochastic approximation methods are widely used to solve M-estimation problems, in the context of predictive learning in particular. In certain situations that shall be undoubtedly more and more common in the Big Data era, the datasets available are so massive that computing statistics over the full sample is hardly feasible, if not unfeasible. A natural and popular approach to gradient descent in this context consists in substituting the “full data” statistics with their counterparts based on subsamples picked at random of manageable size. It is the main purpose of this paper to investigate the impact of survey sampling with unequal inclusion probabilities on stochastic gradient descent-based M-estimation methods. Precisely, we prove that, in presence of some a priori information, one may significantly increase statistical accuracy in terms of limit variance, when choosing appropriate first order inclusion probabilities. These results are described by asymptotic theorems and are also supported by illustrative numerical experiments.

scholarly journals A Convergence Analysis of Distributed SGD with Communication-Efficient Gradient Sparsification

2019 ◽  
Author(s):  
Shaohuai Shi ◽  
Kaiyong Zhao ◽  
Qiang Wang ◽  
Zhenheng Tang ◽  
Xiaowen Chu
Keyword(s):  
Gradient Descent ◽  
Communication Complexity ◽  
Stochastic Gradient Descent ◽  
Communication Overhead ◽  
Data Sets ◽  
Objective Functions ◽  
Promising Technique ◽  
Convergence Performance ◽  
The Impact ◽  
Theoretical Results

Gradient sparsification is a promising technique to significantly reduce the communication overhead in decentralized synchronous stochastic gradient descent (S-SGD) algorithms. Yet, many existing gradient sparsification schemes (e.g., Top-k sparsification) have a communication complexity of O(kP), where k is the number of selected gradients by each worker and P is the number of workers. Recently, the gTop-k sparsification scheme has been proposed to reduce the communication complexity from O(kP) to O(k logP), which significantly boosts the system scalability. However, it remains unclear whether the gTop-k sparsification scheme can converge in theory. In this paper, we first provide theoretical proofs on the convergence of the gTop-k scheme for non-convex objective functions under certain analytic assumptions. We then derive the convergence rate of gTop-k S-SGD, which is at the same order as the vanilla mini-batch SGD. Finally, we conduct extensive experiments on different machine learning models and data sets to verify the soundness of the assumptions and theoretical results, and discuss the impact of the compression ratio on the convergence performance.

Comparison of SVM, RF and SGD Methods for Determination of Programmer's Performance Classification Model in Social Media Activities

2020 ◽  
Vol 4 (2) ◽  
pp. 329-335
Author(s):  
Rusydi Umar ◽  
Imam Riadi ◽  
Purwono
Keyword(s):  
Social Media ◽  
Gradient Descent ◽  
Classification Model ◽  
Stochastic Gradient Descent ◽  
Support Vector ◽  
Svm Algorithm ◽  
Vector Machines ◽  
Performance Patterns ◽  
A Company

The failure of most startups in Indonesia is caused by team performance that is not solid and competent. Programmers are an integral profession in a startup team. The development of social media can be used as a strategic tool for recruiting the best programmer candidates in a company. This strategic tool is in the form of an automatic classification system of social media posting from prospective programmers. The classification results are expected to be able to predict the performance patterns of each candidate with a predicate of good or bad performance. The classification method with the best accuracy needs to be chosen in order to get an effective strategic tool so that a comparison of several methods is needed. This study compares classification methods including the Support Vector Machines (SVM) algorithm, Random Forest (RF) and Stochastic Gradient Descent (SGD). The classification results show the percentage of accuracy with k = 10 cross validation for the SVM algorithm reaches 81.3%, RF at 74.4%, and SGD at 80.1% so that the SVM method is chosen as a model of programmer performance classification on social media activities.

Sign in / Sign up

Export Citation Format

Share Document