Daily runoff forecasting based on data-augmented neural network model

Abstract Accurate daily runoff prediction plays an important role in the management and utilization of water resources. In order to improve the accuracy of prediction, this paper proposes a deep neural network (CAGANet) composed of a convolutional layer, an attention mechanism, a gated recurrent unit (GRU) neural network, and an autoregressive (AR) model. Given that the daily runoff sequence is abrupt and unstable, it is difficult for a single model and combined model to obtain high-precision daily runoff predictions directly. Therefore, this paper uses a linear interpolation method to enhance the stability of hydrological data and apply the augmented data to the CAGANet model, the support vector machine (SVM) model, the long short-term memory (LSTM) neural network and the attention-mechanism-based LSTM model (AM-LSTM). The comparison results show that among the four models based on data augmentation, the CAGANet model proposed in this paper has the best prediction accuracy. Its Nash–Sutcliffe efficiency can reach 0.993. Therefore, the CAGANet model based on data augmentation is a feasible daily runoff forecasting scheme.

Download Full-text

An End-to-End Model Based on Multiple Neural Networks with Data Augmentation for Keyword Spotting

International Journal of Asian Language Processing ◽

10.1142/s271755452050006x ◽

2020 ◽

pp. 2050006

Author(s):

Shuzhou Chai ◽

Wei-Qiang Zhang ◽

Changsheng Lv ◽

Zhenye Yang

Keyword(s):

Neural Network ◽

Data Augmentation ◽

Short Term Memory ◽

Attention Mechanism ◽

Keyword Spotting ◽

Data Set ◽

Positive Rate ◽

Memory Network ◽

Hidden Layer ◽

High Level

In this paper, we propose a network for small footprint keyword spotting. It includes four parts, data augmentation, Time-Delay Neural Network (TDNN) and Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), and attention mechanism. Data augmentation is Google SpecAugment with time warping and frequency mask and time mask to the spectrum on clean data set and noisy data set. TDNN and CNN model the spectrogram features from the time and space dimensions. RNN-type networks include RNN, Long Short-Term Memory network (LSTM), Gated Recurrent Unit (GRU), Bidirectional Long Short-Term Memory network (BiLSTM), and Bidirectional Gated Recurrent Unit (BiGRU). The RNN extracts hidden layer features and transforms them into high-level representations. The attention mechanism is selected to generate different weights and multiplied by the high-level representation generated by the RNN to obtain a fixed-length vector. Finally, we use a linear transformation and softmax function to generate scores. We also explored the size of attention mechanism, two attention mechanisms, rectified linear unit and hidden layer of RNN. Our model has achieved a true positive rate of 99.81% at a 5% false positive rate.

Download Full-text

Gated Hierarchical LSTMs for Target-Based Sentiment Analysis

International Journal of Software Engineering and Knowledge Engineering ◽

10.1142/s0218194018400259 ◽

2018 ◽

Vol 28 (11n12) ◽

pp. 1719-1737

Author(s):

Hao Wang ◽

Xiaofang Zhang ◽

Bin Liang ◽

Qian Zhou ◽

Baowen Xu

Keyword(s):

Neural Network ◽

Sentiment Analysis ◽

Short Term Memory ◽

Network Models ◽

Neural Model ◽

Attention Mechanism ◽

Support Vector ◽

Neural Network Models ◽

Long Distance ◽

Sentence Level

In the field of target-based sentiment analysis, the deep neural model combining attention mechanism is a remarkable success. In current research, it is commonly seen that attention mechanism is combined with Long Short-Term Memory (LSTM) networks. However, such neural network-based architectures generally rely on complex computation and only focus on single target. In this paper, we propose a gated hierarchical LSTM (GH-LSTMs) model which combines regional LSTM and sentence-level LSTM via a gated operation for the task of target-based sentiment analysis. This approach can distinguish different polarities of sentiment of different targets in the same sentence through a regional LSTM. Furthermore, it is able to concentrate on the long-distance dependency of target in the whole sentence via a sentence-level LSTM. The final results of our experiments on multi-domain datasets of two languages from SemEval 2016 indicate that our approach yields better performance than Support Vector Machine (SVM) and several typical neural network models. A case study of some typical examples also makes a supplement to this conclusion.

Download Full-text

Morphological residual convolutional neural network (M-RCNN) for intelligent recognition of wear particles from artificial joints

Friction ◽

10.1007/s40544-021-0516-2 ◽

2021 ◽

Author(s):

Xiaobin Hu ◽

Jian Song ◽

Zhenhua Liao ◽

Yuhong Liu ◽

Jian Gao ◽

...

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Data Augmentation ◽

Support Vector ◽

Wear Particles ◽

Artificial Joints ◽

Performance Deterioration ◽

Overall Performance ◽

Correct Category ◽

Intelligent Recognition

AbstractFinding the correct category of wear particles is important to understand the tribological behavior. However, manual identification is tedious and time-consuming. We here propose an automatic morphological residual convolutional neural network (M-RCNN), exploiting the residual knowledge and morphological priors between various particle types. We also employ data augmentation to prevent performance deterioration caused by the extremely imbalanced problem of class distribution. Experimental results indicate that our morphological priors are distinguishable and beneficial to largely boosting overall performance. M-RCNN demonstrates a much higher accuracy (0.940) than the deep residual network (0.845) and support vector machine (0.821). This work provides an effective solution for automatically identifying wear particles and can be a powerful tool to further analyze the failure mechanisms of artificial joints.

Download Full-text

Study on Lidar Data Interpolation Method Based on GA-BP

Advanced Materials Research ◽

10.4028/www.scientific.net/amr.588-589.1312 ◽

2012 ◽

Vol 588-589 ◽

pp. 1312-1315

Author(s):

Yi Kun Zhang ◽

Ming Hui Zhang ◽

Xin Hong Hei ◽

Deng Xin Hua ◽

Hao Chen

Keyword(s):

Neural Network ◽

Bp Neural Network ◽

Interpolation Method ◽

Linear Interpolation ◽

Lidar Data ◽

Data Interpolation ◽

Interpolation Model ◽

Genetic Method ◽

Interpolation Accuracy ◽

Linear Interpolation Method

Aiming at building a Lidar data interpolation model, this paper designs and implements a GA-BP interpolation method. The proposed method uses genetic method to optimize BP neural network, which greatly improves the calculation accuracy and convergence rate of BP neural network. Experimental results show that the proposed method has a higher interpolation accuracy compared with BP neural network as well as linear interpolation method.

Download Full-text

HARTH: A Human Activity Recognition Dataset for Machine Learning

Sensors ◽

10.3390/s21237853 ◽

2021 ◽

Vol 21 (23) ◽

pp. 7853

Author(s):

Aleksej Logacjov ◽

Kerstin Bach ◽

Atle Kongsvold ◽

Hilde Bremseth Bårdstu ◽

Paul Jarle Mork

Keyword(s):

Neural Network ◽

Machine Learning ◽

Support Vector Machine ◽

Convolutional Neural Network ◽

Activity Recognition ◽

Human Activity ◽

Short Term Memory ◽

Human Activity Recognition ◽

Support Vector ◽

Free Living

Existing accelerometer-based human activity recognition (HAR) benchmark datasets that were recorded during free living suffer from non-fixed sensor placement, the usage of only one sensor, and unreliable annotations. We make two contributions in this work. First, we present the publicly available Human Activity Recognition Trondheim dataset (HARTH). Twenty-two participants were recorded for 90 to 120 min during their regular working hours using two three-axial accelerometers, attached to the thigh and lower back, and a chest-mounted camera. Experts annotated the data independently using the camera’s video signal and achieved high inter-rater agreement (Fleiss’ Kappa =0.96). They labeled twelve activities. The second contribution of this paper is the training of seven different baseline machine learning models for HAR on our dataset. We used a support vector machine, k-nearest neighbor, random forest, extreme gradient boost, convolutional neural network, bidirectional long short-term memory, and convolutional neural network with multi-resolution blocks. The support vector machine achieved the best results with an F1-score of 0.81 (standard deviation: ±0.18), recall of 0.85±0.13, and precision of 0.79±0.22 in a leave-one-subject-out cross-validation. Our highly professional recordings and annotations provide a promising benchmark dataset for researchers to develop innovative machine learning approaches for precise HAR in free living.

Download Full-text

Categorizing Natural Language-Based Customer Satisfaction: An Implementation Method Using Support Vector Machine and Long Short-Term Memory Neural Network

International Journal of Integrated Engineering ◽

10.30880/ijie.2021.13.04.007 ◽

2021 ◽

Vol 13 (4) ◽

Author(s):

Ralph Sherwin A. Corpuz ◽

Keyword(s):

Neural Network ◽

Support Vector Machine ◽

Natural Language ◽

Text Categorization ◽

Short Term Memory ◽

Support Vector ◽

Feature Engineering ◽

Short Term ◽

Term Memory ◽

Long Short Term Memory

Analyzing natural language-based Customer Satisfaction (CS) is a tedious process. This issue is practically true if one is to manually categorize large datasets. Fortunately, the advent of supervised machine learning techniques has paved the way toward the design of efficient categorization systems used for CS. This paper presents the feasibility of designing a text categorization model using two popular and robust algorithms – the Support Vector Machine (SVM) and Long Short-Term Memory (LSTM) Neural Network, in order to automatically categorize complaints, suggestions, feedbacks, and commendations. The study found that, in terms of training accuracy, SVM has best rating of 98.63% while LSTM has best rating of 99.32%. Such results mean that both SVM and LSTM algorithms are at par with each other in terms of training accuracy, but SVM is significantly faster than LSTM by approximately 35.47s. The training performance results of both algorithms are attributed on the limitations of the dataset size, high-dimensionality of both English and Tagalog languages, and applicability of the feature engineering techniques used. Interestingly, based on the results of actual implementation, both algorithms are found to be 100% effective in accurately predicting the correct CS categories. Hence, the extent of preference between the two algorithms boils down on the available dataset and the skill in optimizing these algorithms through feature engineering techniques and in implementing them toward actual text categorization applications.

Download Full-text

Multi-Regional Online Car-Hailing Order Quantity Forecasting Based on the Convolutional Neural Network

Information ◽

10.3390/info10060193 ◽

2019 ◽

Vol 10 (6) ◽

pp. 193 ◽

Cited By ~ 1

Author(s):

Zihao Huang ◽

Gang Huang ◽

Zhijun Chen ◽

Chaozhong Wu ◽

Xiaofeng Ma ◽

...

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Travel Demand ◽

Short Term Memory ◽

Demand Forecasting ◽

Image Feature ◽

Support Vector ◽

Data Set ◽

Demand Distribution ◽

Demand Forecasting Model

With the development of online cars, the demand for travel prediction is increasing in order to reduce the information asymmetry between passengers and drivers of online car-hailing. This paper proposes a travel demand forecasting model named OC-CNN based on the convolutional neural network to forecast the travel demand. In order to make full use of the spatial characteristics of the travel demand distribution, this paper meshes the prediction area and creates a travel demand data set of the graphical structure to preserve its spatial properties. Taking advantage of the convolutional neural network in image feature extraction, the historical demand data of the first twenty-five minutes of the entire region are used as a model input to predict the travel demand for the next five minutes. In order to verify the performance of the proposed method, one-month data from online car-hailing of the Chengdu Fourth Ring Road are used. The results show that the model successfully extracts the spatiotemporal features of the data, and the prediction accuracies of the proposed method are superior to those of the representative methods, including the Bayesian Ridge Model, Linear Regression, Support Vector Regression, and Long Short-Term Memory networks.

Download Full-text

Patient visit forecasting in an emergency department using a deep neural network approach

Kybernetes ◽

10.1108/k-10-2018-0520 ◽

2019 ◽

Vol 49 (9) ◽

pp. 2335-2348 ◽

Cited By ~ 4

Author(s):

Milad Yousefi ◽

Moslem Yousefi ◽

Masood Fathi ◽

Flavio S. Fogliatto

Keyword(s):

Neural Network ◽

Emergency Department ◽

Linear Regression ◽

Deep Neural Network ◽

Short Term Memory ◽

Demand Forecasting ◽

Machine Learning Algorithms ◽

Support Vector ◽

Neural Network Approach ◽

Content Type

Purpose This study aims to investigate the factors affecting daily demand in an emergency department (ED) and to provide a forecasting tool in a public hospital for horizons of up to seven days. Design/methodology/approach In this study, first, the important factors to influence the demand in EDs were extracted from literature then the relevant factors to the study are selected. Then, a deep neural network is applied to constructing a reliable predictor. Findings Although many statistical approaches have been proposed for tackling this issue, better forecasts are viable by using the abilities of machine learning algorithms. Results indicate that the proposed approach outperforms statistical alternatives available in the literature such as multiple linear regression, autoregressive integrated moving average, support vector regression, generalized linear models, generalized estimating equations, seasonal ARIMA and combined ARIMA and linear regression. Research limitations/implications The authors applied this study in a single ED to forecast patient visits. Applying the same method in different EDs may give a better understanding of the performance of the model to the authors. The same approach can be applied in any other demand forecasting after some minor modifications. Originality/value To the best of the knowledge, this is the first study to propose the use of long short-term memory for constructing a predictor of the number of patient visits in EDs.

Download Full-text

A Machine Learning View on Momentum and Reversal Trading

Algorithms ◽

10.3390/a11110170 ◽

2018 ◽

Vol 11 (11) ◽

pp. 170 ◽

Cited By ~ 2

Author(s):

Zhixi Li ◽

Vincent Tam

Keyword(s):

Neural Network ◽

Machine Learning ◽

Stock Market ◽

Short Term Memory ◽

Predictive Ability ◽

Trading Strategies ◽

Machine Learning Techniques ◽

Support Vector ◽

Learning Approaches ◽

Learning Techniques

Momentum and reversal effects are important phenomena in stock markets. In academia, relevant studies have been conducted for years. Researchers have attempted to analyze these phenomena using statistical methods and to give some plausible explanations. However, those explanations are sometimes unconvincing. Furthermore, it is very difficult to transfer the findings of these studies to real-world investment trading strategies due to the lack of predictive ability. This paper represents the first attempt to adopt machine learning techniques for investigating the momentum and reversal effects occurring in any stock market. In the study, various machine learning techniques, including the Decision Tree (DT), Support Vector Machine (SVM), Multilayer Perceptron Neural Network (MLP), and Long Short-Term Memory Neural Network (LSTM) were explored and compared carefully. Several models built on these machine learning approaches were used to predict the momentum or reversal effect on the stock market of mainland China, thus allowing investors to build corresponding trading strategies. The experimental results demonstrated that these machine learning approaches, especially the SVM, are beneficial for capturing the relevant momentum and reversal effects, and possibly building profitable trading strategies. Moreover, we propose the corresponding trading strategies in terms of market states to acquire the best investment returns.

Download Full-text

sEMG-Based Neural Network Prediction Model Selection of Gesture Fatigue and Dataset Optimization

Computational Intelligence and Neuroscience ◽

10.1155/2020/8853314 ◽

2020 ◽

Vol 2020 ◽

pp. 1-17

Author(s):

Fujun Ma ◽

Fanghao Song ◽

Yan Liu ◽

Jiahui Niu

Keyword(s):

Neural Network ◽

Energy Consumption ◽

Prediction Model ◽

Mean Square Error ◽

Short Term Memory ◽

Experimental Results ◽

Support Vector ◽

Mean Square ◽

Semg Signals ◽

Research Studies

The fatigue energy consumption of independent gestures can be obtained by calculating the power spectrum of surface electromyography (sEMG) signals. The existing research studies focus on the fatigue of independent gestures, while the research studies on integrated gestures are few. However, the actual gesture operation mode is usually integrated by multiple independent gestures, so the fatigue degree of integrated gestures can be predicted by training neural network of independent gestures. Three natural gestures including browsing information, playing games, and typing are divided into nine independent gestures in this paper, and the predicted model is established and trained by calculating the energy consumption of independent gestures. The artificial neural networks (ANNs) including backpropagation (BP) neural network, recurrent neural network (RNN), and long short-term memory (LSTM) are used to predict the fatigue of gesture. The support vector machine (SVM) is used to assist verification. Mean square error (MSE), root mean square error (RMSE), and mean absolute error (MAE) are utilized to evaluate the optimal prediction model. Furthermore, the different datasets of the processed sEMG signal and its decomposed wavelet coefficients are trained, respectively, and the changes of error functions of them are compared. The experimental results show that LSTM model is more suitable for gesture fatigue prediction. The processed sEMG signals are appropriate for using as the training set the fatigue degree of one-handed gesture. It is better to use wavelet decomposition coefficients as datasets to predict the high-dimensional sEMG signals of two-handed gestures. The experimental results can be applied to predict the fatigue degree of complex human-machine interactive gestures, help to avoid unreasonable gestures, and improve the user’s interactive experience.

Download Full-text