A Hybrid GLM Model for Predicting Citywide Spatio-Temporal Metro Passenger Flow

Accurate prediction of citywide short-term metro passenger flow is essential to urban management and transport scheduling. Recently, an increasing number of researchers have applied deep learning models to passenger flow prediction. Nevertheless, the task is still challenging due to the complex spatial dependency on the metro network and the time-varying traffic patterns. Therefore, we propose a novel deep learning architecture combining graph attention networks (GAT) with long short-term memory (LSTM) networks, which is called the hybrid GLM (hybrid GAT and LSTM Model). The proposed model captures the spatial dependency via the graph attention layers and learns the temporal dependency via the LSTM layers. Moreover, some external factors are embedded. We tested the hybrid GLM by predicting the metro passenger flow in Shanghai, China. The results are compared with the forecasts from some typical data-driven models. The hybrid GLM gets the smallest root-mean-square error (RMSE) and mean absolute percentage error (MAPE) in different time intervals (TIs), which exhibits the superiority of the proposed model. In particular, in the TI 10 min, the hybrid GLM brings about 6–30% extra improvements in terms of RMSE. We additionally explore the sensitivity of the model to its parameters, which will aid the application of this model.

Download Full-text

Forecasting Day-Ahead Hourly Photovoltaic Power Generation Using Convolutional Self-Attention Based Long Short-Term Memory

Energies ◽

10.3390/en13154017 ◽

2020 ◽

Vol 13 (15) ◽

pp. 4017 ◽

Cited By ~ 2

Author(s):

Dukhwan Yu ◽

Wonik Choi ◽

Myoungsoo Kim ◽

Ling Liu

Keyword(s):

Power Generation ◽

Deep Learning ◽

Prediction Accuracy ◽

Short Term Memory ◽

Local Context ◽

Short Term ◽

Term Memory ◽

Proposed Model ◽

Long Short Term Memory ◽

Pv Power Generation

The problem of Photovoltaic (PV) power generation forecasting is becoming crucial as the penetration level of Distributed Energy Resources (DERs) increases in microgrids and Virtual Power Plants (VPPs). In order to improve the stability of power systems, a fair amount of research has been proposed for increasing prediction performance in practical environments through statistical, machine learning, deep learning, and hybrid approaches. Despite these efforts, the problem of forecasting PV power generation remains to be challenging in power system operations since existing methods show limited accuracy and thus are not sufficiently practical enough to be widely deployed. Many existing methods using long historical data suffer from the long-term dependency problem and are not able to produce high prediction accuracy due to their failure to fully utilize all features of long sequence inputs. To address this problem, we propose a deep learning-based PV power generation forecasting model called Convolutional Self-Attention based Long Short-Term Memory (LSTM). By using the convolutional self-attention mechanism, we can significantly improve prediction accuracy by capturing the local context of the data and generating keys and queries that fit the local context. To validate the applicability of the proposed model, we conduct extensive experiments on both PV power generation forecasting using a real world dataset and power consumption forecasting. The experimental results of power generation forecasting using the real world datasets show that the MAPEs of the proposed model are much lower, in fact by 7.7%, 6%, 3.9% compared to the Deep Neural Network (DNN), LSTM and LSTM with the canonical self-attention, respectively. As for power consumption forecasting, the proposed model exhibits 32%, 17% and 44% lower Mean Absolute Percentage Error (MAPE) than the DNN, LSTM and LSTM with the canonical self-attention, respectively.

Download Full-text

Short-Term Prediction of Bus Passenger Flow Based on a Hybrid Optimized LSTM Network

ISPRS International Journal of Geo-Information ◽

10.3390/ijgi8090366 ◽

2019 ◽

Vol 8 (9) ◽

pp. 366 ◽

Cited By ~ 5

Author(s):

Yong Han ◽

Cheng Wang ◽

Yibin Ren ◽

Shukang Wang ◽

Huangcheng Zheng ◽

...

Keyword(s):

High Efficiency ◽

Short Term Memory ◽

Spatial Scales ◽

Optimization Procedure ◽

Stochastic Gradient Descent ◽

Short Term ◽

Passenger Flow ◽

Proposed Model ◽

Gradient Descent Algorithm ◽

Lstm Network

The accurate prediction of bus passenger flow is the key to public transport management and the smart city. A long short-term memory network, a deep learning method for modeling sequences, is an efficient way to capture the time dependency of passenger flow. In recent years, an increasing number of researchers have sought to apply the LSTM model to passenger flow prediction. However, few of them pay attention to the optimization procedure during model training. In this article, we propose a hybrid, optimized LSTM network based on Nesterov accelerated adaptive moment estimation (Nadam) and the stochastic gradient descent algorithm (SGD). This method trains the model with high efficiency and accuracy, solving the problems of inefficient training and misconvergence that exist in complex models. We employ a hybrid optimized LSTM network to predict the actual passenger flow in Qingdao, China and compare the prediction results with those obtained by non-hybrid LSTM models and conventional methods. In particular, the proposed model brings about a 4%–20% extra performance improvements compared with those of non-hybrid LSTM models. We have also tried combinations of other optimization algorithms and applications in different models, finding that optimizing LSTM by switching Nadam to SGD is the best choice. The sensitivity of the model to its parameters is also explored, which provides guidance for applying this model to bus passenger flow data modelling. The good performance of the proposed model in different temporal and spatial scales shows that it is more robust and effective, which can provide insightful support and guidance for dynamic bus scheduling and regional coordination scheduling.

Download Full-text

Malicious Text Identification: Deep Learning from Public Comments and Emails

Information ◽

10.3390/info11060312 ◽

2020 ◽

Vol 11 (6) ◽

pp. 312 ◽

Cited By ~ 1

Author(s):

Asma Baccouche ◽

Sadaf Ahmed ◽

Daniel Sierra-Sosa ◽

Adel Elmaghraby

Keyword(s):

Social Media ◽

Deep Learning ◽

Language Processing ◽

Short Term Memory ◽

Good Alternative ◽

Classification Problems ◽

Short Term ◽

Independent Dataset ◽

Proposed Model ◽

Long Short Term Memory

Identifying internet spam has been a challenging problem for decades. Several solutions have succeeded to detect spam comments in social media or fraudulent emails. However, an adequate strategy for filtering messages is difficult to achieve, as these messages resemble real communications. From the Natural Language Processing (NLP) perspective, Deep Learning models are a good alternative for classifying text after being preprocessed. In particular, Long Short-Term Memory (LSTM) networks are one of the models that perform well for the binary and multi-label text classification problems. In this paper, an approach merging two different data sources, one intended for Spam in social media posts and the other for Fraud classification in emails, is presented. We designed a multi-label LSTM model and trained it on the joint datasets including text with common bigrams, extracted from each independent dataset. The experiment results show that our proposed model is capable of identifying malicious text regardless of the source. The LSTM model trained with the merged dataset outperforms the models trained independently on each dataset.

Download Full-text

CONVOLUTIONAL LONG-SHORT TERM MEMORY NETWORKS MODEL FOR LONG DURATION EEG SIGNAL CLASSIFICATION

Journal of Mechanics in Medicine and Biology ◽

10.1142/s0219519419400050 ◽

2019 ◽

Vol 19 (01) ◽

pp. 1940005 ◽

Cited By ~ 3

Author(s):

ULAS BARAN BALOGLU ◽

ÖZAL YILDIRIM

Keyword(s):

Deep Learning ◽

Short Term Memory ◽

Learning Algorithms ◽

Eeg Signals ◽

Short Term ◽

Term Memory ◽

Long Duration ◽

Proposed Model ◽

Clinical Scenarios ◽

Long Short Term Memory

Background and objective: Deep learning structures have recently achieved remarkable success in the field of machine learning. Convolutional neural networks (CNN) in image processing and long-short term memory (LSTM) in the time-series analysis are commonly used deep learning algorithms. Healthcare applications of deep learning algorithms provide important contributions for computer-aided diagnosis research. In this study, convolutional long-short term memory (CLSTM) network was used for automatic classification of EEG signals and automatic seizure detection. Methods: A new nine-layer deep network model consisting of convolutional and LSTM layers was designed. The signals processed in the convolutional layers were given as an input to the LSTM network whose outputs were processed in densely connected neural network layers. The EEG data is appropriate for a model having 1-D convolution layers. A bidirectional model was employed in the LSTM layer. Results: Bonn University EEG database with five different datasets was used for experimental studies. In this database, each dataset contains 23.6[Formula: see text]s duration 100 single channel EEG segments which consist of 4097 dimensional samples (173.61[Formula: see text]Hz). Eight two-class and three three-class clinical scenarios were examined. When the experimental results were evaluated, it was seen that the proposed model had high accuracy on both binary and ternary classification tasks. Conclusions: The proposed end-to-end learning structure showed a good performance without using any hand-crafted feature extraction or shallow classifiers to detect the seizures. The model does not require filtering, and also automatically learns to filter the input as well. As a result, the proposed model can process long duration EEG signals without applying segmentation, and can detect epileptic seizures automatically by using the correlation of ictal and interictal signals of raw data.

Download Full-text

COVID-19 in Iran: Forecasting Pandemic Using Deep Learning

Computational and Mathematical Methods in Medicine ◽

10.1155/2021/6927985 ◽

2021 ◽

Vol 2021 ◽

pp. 1-16

Author(s):

Rahele Kafieh ◽

Roya Arian ◽

Narges Saeedizadeh ◽

Zahra Amini ◽

Nasim Dadashi Serej ◽

...

Keyword(s):

Deep Learning ◽

Short Term Memory ◽

Percentage Error ◽

Mean Square ◽

Short Term ◽

Average Percentage ◽

The Usa ◽

Long Short Term Memory ◽

Average Percentage Error ◽

Almost All

COVID-19 has led to a pandemic, affecting almost all countries in a few months. In this work, we applied selected deep learning models including multilayer perceptron, random forest, and different versions of long short-term memory (LSTM), using three data sources to train the models, including COVID-19 occurrences, basic information like coded country names, and detailed information like population, and area of different countries. The main goal is to forecast the outbreak in nine countries (Iran, Germany, Italy, Japan, Korea, Switzerland, Spain, China, and the USA). The performances of the models are measured using four metrics, including mean average percentage error (MAPE), root mean square error (RMSE), normalized RMSE (NRMSE), and R 2 . The best performance was found for a modified version of LSTM, called M-LSTM (winner model), to forecast the future trajectory of the pandemic in the mentioned countries. For this purpose, we collected the data from January 22 till July 30, 2020, for training, and from 1 August 2020 to 31 August 2020, for the testing phase. Through experimental results, the winner model achieved reasonably accurate predictions (MAPE, RMSE, NRMSE, and R 2 are 0.509, 458.12, 0.001624, and 0.99997, respectively). Furthermore, we stopped the training of the model on some dates related to main country actions to investigate the effect of country actions on predictions by the model.

Download Full-text

A Deep Learning Model with Conv-LSTM Networks for Subway Passenger Congestion Delay Prediction

Journal of Advanced Transportation ◽

10.1155/2021/6645214 ◽

2021 ◽

Vol 2021 ◽

pp. 1-10

Author(s):

Wei Chen ◽

Zongping Li ◽

Can Liu ◽

Yi Ai

Keyword(s):

Deep Learning ◽

Short Term Memory ◽

Urban Rail Transit ◽

Rail Transit ◽

Learning Technology ◽

Short Term ◽

Average Delay ◽

Passenger Flow ◽

Subway Stations ◽

Delay Prediction

When urban rail transit is faced with a large number of commuter passengers during peak periods, passengers are often waiting for the next train because the subway is running at full load, which causes delays to the overall travel time of passengers. The calculation and prediction of the congestion delay in subway stations can guide the operation department and passengers to make better planning and selection. In this paper, we use a new method based on deep learning technology to evaluate the congestion delay of subway stations. Firstly, we use automatic fare collection (AFC) system data to evaluate the congestion delays of stations. Then, we use a convolutional long short-term memory (Conv-LSTM) network to extract spatial and temporal characteristics to solve the short-term prediction problem of the subway congestion delay in the network structure. The spatiotemporal variables include inbound passenger flow, outbound passenger flow, number of passengers delayed, and average delay time. As a spatiotemporal sequence, the input and prediction targets are both spatiotemporal three-dimensional tensors in the end-to-end training model. The effectiveness of the method is verified by a case study of the Chongqing Rail Transit. Experimental results show that Conv-LSTM is better than the benchmark models in capturing spatial and temporal correlation.

Download Full-text

Lane-Changing Behavior Prediction Based on Game Theory and Deep Learning

Journal of Advanced Transportation ◽

10.1155/2021/6634960 ◽

2021 ◽

Vol 2021 ◽

pp. 1-12

Author(s):

Shuo Jia ◽

Fei Hui ◽

Cheng Wei ◽

Xiangmo Zhao ◽

Jianbei Liu

Keyword(s):

Game Theory ◽

Deep Learning ◽

Short Term Memory ◽

Accurate Prediction ◽

Behavior Prediction ◽

Short Term ◽

Lane Changing ◽

Proposed Model ◽

The Game Theory ◽

Long Short Term Memory

Lane changing is an important scenario in traffic environments, and accurate prediction of lane-changing behavior is essential to ensure traffic and driver safety. To achieve this goal, a vehicle lane-changing prediction model based on game theory and deep learning is developed. In the game theory component, the interaction between vehicles during lane changing is analyzed according to the running state of the vehicle, with the probability of lane changing as its output. For the deep-learning component, long short-term memory and a convolutional neural network are used to extract and learn data features during the lane-changing process as well as combine the output of the game theory component to obtain the prediction result of whether the vehicle will change lanes. By using an open-source traffic dataset to train and verify the proposed model, the verification results show that the prediction accuracy can reach 94.56% within 0.4 s of lane-changing operation and that the model can achieve timely and accurate prediction of the lane-changing behavior of vehicles.

Download Full-text

Deep Learning Approach for Estimating Workability of Self-Compacting Concrete from Mixing Image Sequences

Advances in Materials Science and Engineering ◽

10.1155/2018/6387930 ◽

2018 ◽

Vol 2018 ◽

pp. 1-16 ◽

Cited By ~ 1

Author(s):

Zhongcong Ding ◽

Xuehui An

Keyword(s):

Deep Learning ◽

Short Term Memory ◽

Image Sequences ◽

Learning Approach ◽

Self Compacting Concrete ◽

Short Term ◽

Slump Flow ◽

Proposed Model ◽

Long Short Term Memory ◽

Funnel Flow

We propose a deep learning approach to better utilize the spatial and temporal information obtained from image sequences of the self-compacting concrete- (SCC-) mixing process to recover SCC characteristics in terms of the predicted slump flow value (SF) and V-funnel flow time (VF). The proposed model integrates features of the convolutional neural network and long short-term memory and is trained to extract features and compute an estimate. The performance of the method is evaluated using the testing set. The results indicate that the proposed method could potentially be used to automatically estimate SCC workability.

Download Full-text

Non-Intrusive Load Identification Method Based on Improved Long Short Term Memory Network

Energies ◽

10.3390/en14030684 ◽

2021 ◽

Vol 14 (3) ◽

pp. 684

Author(s):

Jiateng Song ◽

Hongbin Wang ◽

Mingxing Du ◽

Lei Peng ◽

Shuai Zhang ◽

...

Keyword(s):

Deep Learning ◽

Short Term Memory ◽

Rapid Development ◽

Series Data ◽

Load Identification ◽

Short Term ◽

Term Memory ◽

Identification Method ◽

Proposed Model ◽

Long Short Term Memory

Non-intrusive load monitoring (NILM) is an important research direction and development goal on the distribution side of smart grid, which can significantly improve the timeliness of demand side response and users’ awareness of load. Due to rapid development, deep learning becomes an effective way to optimize NILM. In this paper, we propose a novel load identification method based on long short term memory (LSTM) on deep learning. Sequence-to-point (seq2point) learning is introduced into LSTM. The innovative combination of the LSTM and the seq2point brings their respective advantages together, so that the proposed model can accurately identify the load in process of time series data. In this paper, we proved the feature of reducing identification error in the experimental data, from three datasets, UK-DALE dataset, REDD dataset, and REFIT dataset. In terms of mean absolute error (MAE), the three datasets have increased by 15%, 14%, and 18% respectively; in terms of normalized signal aggregate error (SAE), the three datasets have increased by 21%, 24%, and 30% respectively. Compared with the existing models, the proposed model has better accuracy and generalization in identifying three open source datasets.

Download Full-text

A COMBINED DEEP LEARNING MODEL FOR PERSIAN SENTIMENT ANALYSIS

IIUM Engineering Journal ◽

10.31436/iiumej.v20i1.1036 ◽

2019 ◽

Vol 20 (1) ◽

pp. 129-139 ◽

Cited By ~ 2

Author(s):

Zahra Bokaee Nezhad ◽

Mohammad Ali Deihimi

Keyword(s):

Deep Learning ◽

Natural Language Processing ◽

Sentiment Analysis ◽

Language Processing ◽

Short Term Memory ◽

Short Term ◽

Term Memory ◽

Proposed Model ◽

Long Short Term Memory ◽

Deep Learning Model

With increasing members in social media sites today, people tend to share their views about everything online. It is a convenient way to convey their messages to end users on a specific subject. Sentiment Analysis is a subfield of Natural Language Processing (NLP) that refers to the identification of users’ opinions toward specific topics. It is used in several fields such as marketing, customer services, etc. However, limited works have been done on Persian Sentiment Analysis. On the other hand, deep learning has recently become popular because of its successful role in several Natural Language Processing tasks. The objective of this paper is to propose a novel hybrid deep learning architecture for Persian Sentiment Analysis. According to the proposed model, local features are extracted by Convolutional Neural Networks (CNN) and long-term dependencies are learned by Long Short Term Memory (LSTM). Therefore, the model can harness both CNN's and LSTM's abilities. Furthermore, Word2vec is used for word representation as an unsupervised learning step. To the best of our knowledge, this is the first attempt where a hybrid deep learning model is used for Persian Sentiment Analysis. We evaluate the model on a Persian dataset that is introduced in this study. The experimental results show the effectiveness of the proposed model with an accuracy of 85%. ABSTRAK: Hari ini dengan ahli yang semakin meningkat di laman media sosial, orang cenderung untuk berkongsi pandangan mereka tentang segala-galanya dalam talian. Ini adalah cara mudah untuk menyampaikan mesej mereka kepada pengguna akhir mengenai subjek tertentu. Analisis Sentimen adalah subfield Pemprosesan Bahasa Semula Jadi yang merujuk kepada pengenalan pendapat pengguna ke arah topik tertentu. Ia digunakan dalam beberapa bidang seperti pemasaran, perkhidmatan pelanggan, dan sebagainya. Walau bagaimanapun, kerja-kerja terhad telah dilakukan ke atas Analisis Sentimen Parsi. Sebaliknya, pembelajaran mendalam baru menjadi popular kerana peranannya yang berjaya dalam beberapa tugas Pemprosesan Bahasa Asli (NLP). Objektif makalah ini adalah mencadangkan senibina pembelajaran hibrid yang baru dalam Analisis Sentimen Parsi. Menurut model yang dicadangkan, ciri-ciri tempatan ditangkap oleh Rangkaian Neural Convolutional (CNN) dan ketergantungan jangka panjang dipelajari oleh Long Short Term Memory (LSTM). Oleh itu, model boleh memanfaatkan kebolehan CNN dan LSTM. Selain itu, Word2vec digunakan untuk perwakilan perkataan sebagai langkah pembelajaran tanpa pengawasan. Untuk pengetahuan yang terbaik, ini adalah percubaan pertama di mana model pembelajaran mendalam hibrid digunakan untuk Analisis Sentimen Persia. Kami menilai model pada dataset Persia yang memperkenalkan dalam kajian ini. Keputusan eksperimen menunjukkan keberkesanan model yang dicadangkan dengan ketepatan 85%.

Download Full-text