scholarly journals Deep Learning Based Recurrent Neural Networks to Enhance the Performance of Wind Energy Forecasting: A Review

2021 ◽  
Vol 35 (1) ◽  
pp. 1-10
Author(s):  
Senthil Kumar Paramasivan
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Wind Energy ◽  
Recurrent Neural Networks ◽  
Time Series Data ◽  
Short Term Memory ◽  
Function Optimization ◽  
Series Data ◽  
Energy Forecasting

In the modern era, deep learning is a powerful technique in the field of wind energy forecasting. The deep neural network effectively handles the seasonal variation and uncertainty characteristics of wind speed by proper structural design, objective function optimization, and feature learning. The present paper focuses on the critical analysis of wind energy forecasting using deep learning based Recurrent neural networks (RNN) models. It explores RNN and its variants, such as simple RNN, Long Short Term Memory (LSTM), Gated Recurrent Unit (GRU), and Bidirectional RNN models. The recurrent neural network processes the input time series data sequentially and captures well the temporal dependencies exist in the successive input data. This review investigates the RNN models of wind energy forecasting, the data sources utilized, and the performance achieved in terms of the error measures. The overall review shows that the deep learning based RNN improves the performance of wind energy forecasting compared to the conventional techniques.

Application of Deep Learning for Characterization of Drivers’ Engagement in Secondary Tasks in In-Vehicle Systems

2020 ◽  
Vol 2674 (8) ◽  
pp. 429-440
Author(s):  
Osama A. Osman ◽  
Hesham Rakha
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Time Series Data ◽  
Short Term Memory ◽  
Network Models ◽  
Sensor Technology ◽  
Series Data ◽  
Neural Network Models ◽  
Secondary Tasks ◽  
Driving Assistance

Distracted driving (i.e., engaging in secondary tasks) is an epidemic that threatens the lives of thousands every year. Data collected from vehicular sensor technologies and through connectivity provide comprehensive information that, if used to detect driver engagement in secondary tasks, could save thousands of lives and millions of dollars. This study investigates the possibility of achieving this goal using promising deep learning tools. Specifically, two deep neural network models (a multilayer perceptron neural network model and a long short-term memory networks [LSTMN] model) were developed to identify three secondary tasks: cellphone calling, cellphone texting, and conversation with adjacent passengers. The Second Strategic Highway Research Program Naturalistic Driving Study (SHRP 2 NDS) time series data, collected using vehicle sensor technology, were used to train and test the model. The results show excellent performance for the developed models, with a slight improvement for the LSTMN model, with overall classification accuracies ranging between 95 and 96%. Specifically, the models are able to identify the different types of secondary tasks with high accuracies of 100% for calling, 96%–97% for texting, 90%–91% for conversation, and 95%–96% for the normal driving. Based on this performance, the developed models improve on the results of a previous model developed by the author to classify the same three secondary tasks, which had an accuracy of 82%. The model is promising for use in in-vehicle driving assistance technology to report engagement in unlawful tasks or alert drivers to take over control in level 1 and 2 automated vehicles.

scholarly journals API Call-Based Malware Classification Using Recurrent Neural Networks

2021 ◽  
Author(s):  
Chen Li ◽  
Junjun Zheng
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Time Series Data ◽  
Short Term Memory ◽  
Application Programming Interface ◽  
Classification Model ◽  
Series Data ◽  
Process Time ◽  
Economic Damage ◽  
Malware Classification

Malicious software, called malware, can perform harmful actions on computer systems, which may cause economic damage and information leakage. Therefore, malware classification is meaningful and required to prevent malware attacks. Application programming interface (API) call sequences are easily observed and are good choices as features for malware classification. However, one of the main issues is how to generate a suitable feature for the algorithms of classification to achieve a high classification accuracy. Different malware sample brings API call sequence with different lengths, and these lengths may reach millions, which may cause computation cost and time complexities. Recurrent neural networks (RNNs) is one of the most versatile approaches to process time series data, which can be used to API call-based Malware calssification. In this paper, we propose a malware classification model with RNN, especially the long short-term memory (LSTM) and the gated recurrent unit (GRU), to classify variants of malware by using long-sequences of API calls. In numerical experiments, a benchmark dataset is used to illustrate the proposed approach and validate its accuracy. The numerical results show that the proposed RNN model works well on the malware classification.

scholarly journals Toward Smart Lockdown: A Novel Approach for COVID-19 Hotspots Prediction Using a Deep Hybrid Neural Network

Computers ◽  
2020 ◽  
Vol 9 (4) ◽  
pp. 99
Author(s):  
Sultan Daud Khan ◽  
Louai Alarabi ◽  
Saleh Basalamah
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Time Scale ◽  
Prediction Models ◽  
Time Series Data ◽  
Short Term Memory ◽  
Economic Recession ◽  
Series Data ◽  
Economic Activities ◽  
Novel Approach

COVID-19 caused the largest economic recession in the history by placing more than one third of world’s population in lockdown. The prolonged restrictions on economic and business activities caused huge economic turmoil that significantly affected the financial markets. To ease the growing pressure on the economy, scientists proposed intermittent lockdowns commonly known as “smart lockdowns”. Under smart lockdown, areas that contain infected clusters of population, namely hotspots, are placed on lockdown, while economic activities are allowed to operate in un-infected areas. In this study, we proposed a novel deep learning prediction framework for the accurate prediction of hotpots. We exploit the benefits of two deep learning models, i.e., Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) and propose a hybrid framework that has the ability to extract multi time-scale features from convolutional layers of CNN. The multi time-scale features are then concatenated and provide as input to 2-layers LSTM model. The LSTM model identifies short, medium and long-term dependencies by learning the representation of time-series data. We perform a series of experiments and compare the proposed framework with other state-of-the-art statistical and machine learning based prediction models. From the experimental results, we demonstrate that the proposed framework beats other existing methods with a clear margin.

Deep Learning in the Healthcare Industry

2020 ◽  
pp. 220-245 ◽  
Author(s):  
Zahra A. Shirazi ◽  
Camila P. E. de Souza ◽  
Rasha Kashef ◽  
Felipe F. Rodrigues
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Input Data ◽  
Time Series Data ◽  
Short Term Memory ◽  
Healthcare Industry ◽  
Neural Network Models ◽  
Input Layer ◽  
Hidden Layer

Artificial Neural networks (ANN) are composed of nodes that are joint to each other through weighted connections. Deep learning, as an extension of ANN, is a neural network model, but composed of different categories of layers: input layer, hidden layers, and output layers. Input data is fed into the first (input) layer. But the main process of the neural network models is done within the hidden layers, ranging from a single hidden layer to multiple ones. Depending on the type of model, the structure of the hidden layers is different. Depending on the type of input data, different models are applied. For example, for image data, convolutional neural networks are the most appropriate. On the other hand, for text or sequential and time series data, recurrent neural networks or long short-term memory models are the better choices. This chapter summarizes the state-of-the-art deep learning methods applied to the healthcare industry.

scholarly journals A Fault Prediction and Cause Identification Approach in Complex Industrial Processes Based on Deep Learning

2021 ◽  
Vol 2021 ◽  
pp. 1-13
Author(s):  
Yao Li
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Production Line ◽  
Prediction Accuracy ◽  
Time Series Data ◽  
Short Term Memory ◽  
Fault Prediction ◽  
Industrial Processes ◽  
Series Data ◽  
Process Data

Faults occurring in the production line can cause many losses. Predicting the fault events before they occur or identifying the causes can effectively reduce such losses. A modern production line can provide enough data to solve the problem. However, in the face of complex industrial processes, this problem will become very difficult depending on traditional methods. In this paper, we propose a new approach based on a deep learning (DL) algorithm to solve the problem. First, we regard these process data as a spatial sequence according to the production process, which is different from traditional time series data. Second, we improve the long short-term memory (LSTM) neural network in an encoder-decoder model to adapt to the branch structure, corresponding to the spatial sequence. Meanwhile, an attention mechanism (AM) algorithm is used in fault detection and cause identification. Third, instead of traditional biclassification, the output is defined as a sequence of fault types. The approach proposed in this article has two advantages. On the one hand, treating data as a spatial sequence rather than a time sequence can overcome multidimensional problems and improve prediction accuracy. On the other hand, in the trained neural network, the weight vectors generated by the AM algorithm can represent the correlation between faults and the input data. This correlation can help engineers identify the cause of faults. The proposed approach is compared with some well-developed fault diagnosing methods in the Tennessee Eastman process. Experimental results show that the approach has higher prediction accuracy, and the weight vector can accurately label the factors that cause faults.

Development and Comparative Analysis of Advanced Deep Learning Techniques for Crash Prediction in Advanced Driver Support Systems

2021 ◽  
pp. 036119812110312
Author(s):  
Osama A Osman ◽  
Mustafa Hajij
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Time Series Data ◽  
Short Term Memory ◽  
Motor Vehicle ◽  
Series Data ◽  
False Alarms ◽  
Driver Assistance Systems ◽  
Prediction Horizon ◽  
Promising Tool

Motor vehicle crashes claimed 38,800 lives and caused 4.4 million injuries in 2019 alone. Studies have shown that 94% of these crashes are because of driver errors. Such a huge contribution of driver errors to crashes indicates that efforts at improving safety should be directed toward both vehicles and drivers through advanced driver assistance systems (ADAS) and vehicular technologies. This study investigates the potential that real-time driver behavior data collected through vehicular technologies offer to predict crashes, as the first line of defense to avoid them. Three deep learning models were developed including multilayer perceptron neural networks (MLP-NN), long-short-term memory networks (LSTMN), and convolutional neural networks (CNN) using vehicle kinematics time series data extracted from the Second Strategic Highway Research Program Naturalistic Driving Study (SHRP2 NDS) dataset. The study builds on the hypothesis that crashes are preceded by turbulences that take place over time (turbulence horizon). If these turbulences are detected promptly they can help predict and avoid crashes. Several values were tested for the turbulence horizon and the prediction horizon (how long before the crash impact it can be predicted) to identify the optimal values. The results showed that the CNN model can predict all crashes with a 100% accuracy and zero false alarms 3 s before the crash impact time when a 6-s turbulence horizon is used. This outstanding performance demonstrates the developed model is a promising tool for implementation in ADAS.

scholarly journals Perbandingan Model Deep Learning untuk Klasifikasi Sentiment Analysis dengan Teknik Natural Languange Processing

2021 ◽  
Vol 7 (2) ◽  
pp. 113-121
Author(s):  
Firman Pradana Rachman
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Neural Networks ◽  
Deep Learning ◽  
Sentiment Analysis ◽  
Recurrent Neural Networks ◽  
Short Term Memory ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory

Setiap orang mempunyai pendapat atau opini terhadap suatu produk, tokoh masyarakat, atau pun sebuah kebijakan pemerintah yang tersebar di media sosial. Pengolahan data opini itu di sebut dengan sentiment analysis. Dalam pengolahan data opini yang besar tersebut tidak hanya cukup menggunakan machine learning, namun bisa juga menggunakan deep learning yang di kombinasikan dengan teknik NLP (Natural Languange Processing). Penelitian ini membandingkan beberapa model deep learning seperti CNN (Convolutional Neural Network), RNN (Recurrent Neural Networks), LSTM (Long Short-Term Memory) dan beberapa variannya untuk mengolah data sentiment analysis dari review produk amazon dan yelp.

Using deep convolutional neural networks to forecast spatial patterns of Amazonian deforestation

2021 ◽  
Author(s):  
James George Clifford Ball ◽  
Katerina Petrova ◽  
David Coomes ◽  
Seth Flaxman
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Tropical Forest ◽  
Convolutional Neural Networks ◽  
Time Series Data ◽  
Short Term Memory ◽  
Series Data ◽  
Surface Model ◽  
Observation Data ◽  
Deep Convolutional Neural Networks

1. Tropical forests are subject to diverse deforestation pressures but their conservation is essential to achieve global climate goals. Predicting the location of deforestation is challenging due to the complexity of the natural and human systems involved but accurate and timely forecasts could enable effective planning and on-the-ground enforcement practices to curb deforestation rates. New computer vision technologies based on deep learning can be applied to the increasing volume of Earth observation data to generate novel insights and make predictions with unprecedented accuracy. 2. Here, we demonstrate the ability of deep convolutional neural networks to learn spatiotemporal patterns of deforestation from a limited set of freely available global data layers, including multispectral satellite imagery, the Hansen maps of historic deforestation (2001-2020) and the ALOS JAXA digital surface model, to forecast future deforestation (2021). We designed four original deep learning model architectures, based on 2D Convolutional Neural Networks (2DCNN), 3D Convolutional Neural Networks (3DCNN), and Long Short-Term Memory (LSTM) Recurrent Neural Networks (RNN) to produce spatial maps that indicate the risk to each forested pixel (~30 m) in the landscape of becoming deforested within the next year.. They were trained and tested on data from two ~80,000 km2 tropical forest regions in the Southern Peruvian Amazon. 3. We found that the networks could predict the likely location of future deforestation to a high degree of accuracy. Our best performing model - a 3DCNN - had the highest pixel-wise accuracy (80-90%) when validated on 2020 deforestation based 2014-2019 training. Visual examination of the forecasts indicated that the 3DCNN network could automatically discern the drivers of forest loss from the input data. For example, pixels around new access routes (e.g. roads) were assigned high risk whereas this was not the case for recent, concentrated natural loss events (e.g. remote landslides). 4. CNNs can harness limited time-series data to predict near-future deforestation patterns, an important step in using the growing volume of satellite remote sensing data to curb global deforestation. The modelling framework can be readily applied to any tropical forest location and used by governments and conservation organisations to prevent deforestation and plan protected areas.

scholarly journals Hybrid Neural Networks for Learning the Trend in Time Series

2017 ◽  
Author(s):  
Tao Lin ◽  
Tian Guo ◽  
Karl Aberer
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Time Series ◽  
Time Series Data ◽  
Short Term Memory ◽  
Feature Fusion ◽  
Series Data ◽  
Hybrid Neural Network ◽  
Load Schedule ◽  
Hybrid Neural Networks

The trend of time series characterizes the intermediate upward and downward behaviour of time series. Learning and forecasting the trend in time series data play an important role in many real applications, ranging from resource allocation in data centers, load schedule in smart grid, and so on. Inspired by the recent successes of neural networks, in this paper we propose TreNet, a novel end-to-end hybrid neural network to learn local and global contextual features for predicting the trend of time series. TreNet leverages convolutional neural networks (CNNs) to extract salient features from local raw data of time series. Meanwhile, considering the long-range dependency existing in the sequence of historical trends of time series, TreNet uses a long-short term memory recurrent neural network (LSTM) to capture such dependency. Then, a feature fusion layer is to learn joint representation for predicting the trend. TreNet demonstrates its effectiveness by outperforming CNN, LSTM, the cascade of CNN and LSTM, Hidden Markov Model based method and various kernel based baselines on real datasets.

Neural Network Techniques for Time Series Prediction: A Review

2019 ◽  
Vol 3 (3) ◽  
Author(s):  
Muhammad Faheem Mushtaq ◽  
Urooj Akram ◽  
Muhammad Aamir ◽  
Haseeb Ali ◽  
Muhammad Zulqarnain
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Time Series ◽  
Time Series Data ◽  
Weather Prediction ◽  
Time Series Prediction ◽  
Series Data ◽  
Prediction Problem ◽  
Neural Network Models ◽  
Physical Time

It is important to predict a time series because many problems that are related to prediction such as health prediction problem, climate change prediction problem and weather prediction problem include a time component. To solve the time series prediction problem various techniques have been developed over many years to enhance the accuracy of forecasting. This paper presents a review of the prediction of physical time series applications using the neural network models. Neural Networks (NN) have appeared as an effective tool for forecasting of time series.  Moreover, to resolve the problems related to time series data, there is a need of network with single layer trainable weights that is Higher Order Neural Network (HONN) which can perform nonlinearity mapping of input-output. So, the developers are focusing on HONN that has been recently considered to develop the input representation spaces broadly. The HONN model has the ability of functional mapping which determined through some time series problems and it shows the more benefits as compared to conventional Artificial Neural Networks (ANN). The goal of this research is to present the reader awareness about HONN for physical time series prediction, to highlight some benefits and challenges using HONN.

Sign in / Sign up

Export Citation Format

Share Document