scholarly journals LSTM Networks for Online Cross-Network Recommendations

2018 ◽  
Author(s):  
Dilruk Perera ◽  
Roger Zimmermann
Keyword(s):  
Short Term Memory ◽  
Auxiliary Information ◽  
User Preference ◽  
Time Intervals ◽  
User Interactions ◽  
Linear Relationships ◽  
Proposed Model ◽  
Target Network ◽  
Lstm Network

Cross-network recommender systems use auxiliary information from multiple source networks to create holistic user profiles and improve recommendations in a target network. However, we find two major limitations in existing cross-network solutions that reduce overall recommender performance. Existing models (1) fail to capture complex non-linear relationships in user interactions, and (2) are designed for offline settings hence, not updated online with incoming interactions to capture the dynamics in the recommender environment. We propose a novel multi-layered Long Short-Term Memory (LSTM) network based online solution to mitigate these issues. The proposed model contains three main extensions to the standard LSTM: First, an attention gated mechanism to capture long-term user preference changes. Second, a higher order interaction layer to alleviate data sparsity. Third, time aware LSTM cell gates to capture irregular time intervals between user interactions. We illustrate our solution using auxiliary information from Twitter and Google Plus to improve recommendations on YouTube. Extensive experiments show that the proposed model consistently outperforms state-of-the-art in terms of accuracy, diversity and novelty.

scholarly journals Automatic Generation of Meta-Path Graph for Concept Recommendation in MOOCs

Electronics ◽  
2021 ◽  
Vol 10 (14) ◽  
pp. 1671
Author(s):  
Jibing Gong ◽  
Cheng Wang ◽  
Zhiyong Zhao ◽  
Xinghao Zhang
Keyword(s):  
Auxiliary Information ◽  
Automatic Generation ◽  
Information Network ◽  
Heterogeneous Information Network ◽  
Heterogeneous Information ◽  
Online Study ◽  
Path Graph ◽  
Proposed Model ◽  
Meta Path

In MOOCs, generally speaking, curriculum designing, course selection, and knowledge concept recommendation are the three major steps that systematically instruct users to learn. This paper focuses on the knowledge concept recommendation in MOOCs, which recommends related topics to users to facilitate their online study. The existing approaches only consider the historical behaviors of users, but ignore various kinds of auxiliary information, which are also critical for user embedding. In addition, traditional recommendation models only consider the immediate user response to the recommended items, and do not explicitly consider the long-term interests of users. To deal with the above issues, this paper proposes AGMKRec, a novel reinforced concept recommendation model with a heterogeneous information network. We first clarify the concept recommendation in MOOCs as a reinforcement learning problem to offer a personalized and dynamic knowledge concept label list to users. To consider more auxiliary information of users, we construct a heterogeneous information network among users, courses, and concepts, and use a meta-path-based method which can automatically identify useful meta-paths and multi-hop connections to learn a new graph structure for learning effective node representations on a graph. Comprehensive experiments and analyses on a real-world dataset collected from XuetangX show that our proposed model outperforms some state-of-the-art methods.

scholarly journals A Context-Aware Location Recommendation System for Tourists Using Hierarchical LSTM Model

2020 ◽  
Vol 12 (10) ◽  
pp. 4107
Author(s):  
Wafa Shafqat ◽  
Yung-Cheol Byun
Keyword(s):  
Recommendation System ◽  
Short Term Memory ◽  
Contextual Information ◽  
Optimal Size ◽  
Effective Parameters ◽  
Travel History ◽  
Proposed Model ◽  
Processing Information ◽  
Lstm Network ◽  
Gated Recurrent Unit

The significance of contextual data has been recognized by analysts and specialists in numerous disciplines such as customization, data recovery, ubiquitous and versatile processing, information mining, and management. While a generous research has just been performed in the zone of recommender frameworks, by far most of the existing approaches center on prescribing the most relevant items to customers. It usually neglects extra-contextual information, for example time, area, climate or the popularity of different locations. Therefore, we proposed a deep long-short term memory (LSTM) based context-enriched hierarchical model. This proposed model had two levels of hierarchy and each level comprised of a deep LSTM network. In each level, the task of the LSTM was different. At the first level, LSTM learned from user travel history and predicted the next location probabilities. A contextual learning unit was active between these two levels. This unit extracted maximum possible contexts related to a location, the user and its environment such as weather, climate and risks. This unit also estimated other effective parameters such as the popularity of a location. To avoid feature congestion, XGBoost was used to rank feature importance. The features with no importance were discarded. At the second level, another LSTM framework was used to learn these contextual features embedded with location probabilities and resulted into top ranked places. The performance of the proposed approach was elevated with an accuracy of 97.2%, followed by gated recurrent unit (GRU) (96.4%) and then Bidirectional LSTM (94.2%). We also performed experiments to find the optimal size of travel history for effective recommendations.

scholarly journals Short-Term Prediction of Bus Passenger Flow Based on a Hybrid Optimized LSTM Network

2019 ◽  
Vol 8 (9) ◽  
pp. 366 ◽  
Author(s):  
Yong Han ◽  
Cheng Wang ◽  
Yibin Ren ◽  
Shukang Wang ◽  
Huangcheng Zheng ◽  
...  
Keyword(s):  
High Efficiency ◽  
Short Term Memory ◽  
Spatial Scales ◽  
Optimization Procedure ◽  
Stochastic Gradient Descent ◽  
Short Term ◽  
Passenger Flow ◽  
Proposed Model ◽  
Gradient Descent Algorithm ◽  
Lstm Network

The accurate prediction of bus passenger flow is the key to public transport management and the smart city. A long short-term memory network, a deep learning method for modeling sequences, is an efficient way to capture the time dependency of passenger flow. In recent years, an increasing number of researchers have sought to apply the LSTM model to passenger flow prediction. However, few of them pay attention to the optimization procedure during model training. In this article, we propose a hybrid, optimized LSTM network based on Nesterov accelerated adaptive moment estimation (Nadam) and the stochastic gradient descent algorithm (SGD). This method trains the model with high efficiency and accuracy, solving the problems of inefficient training and misconvergence that exist in complex models. We employ a hybrid optimized LSTM network to predict the actual passenger flow in Qingdao, China and compare the prediction results with those obtained by non-hybrid LSTM models and conventional methods. In particular, the proposed model brings about a 4%–20% extra performance improvements compared with those of non-hybrid LSTM models. We have also tried combinations of other optimization algorithms and applications in different models, finding that optimizing LSTM by switching Nadam to SGD is the best choice. The sensitivity of the model to its parameters is also explored, which provides guidance for applying this model to bus passenger flow data modelling. The good performance of the proposed model in different temporal and spatial scales shows that it is more robust and effective, which can provide insightful support and guidance for dynamic bus scheduling and regional coordination scheduling.

scholarly journals An LSTM-Based Method with Attention Mechanism for Travel Time Prediction

Sensors ◽  
2019 ◽  
Vol 19 (4) ◽  
pp. 861 ◽  
Author(s):  
Xiangdong Ran ◽  
Zhiguang Shan ◽  
Yufei Fang ◽  
Chuang Lin
Keyword(s):  
Short Term Memory ◽  
Attention Mechanism ◽  
Traffic Prediction ◽  
Travel Time Prediction ◽  
Short Term ◽  
Term Memory ◽  
Proposed Model ◽  
Departure Time ◽  
Long Short Term Memory

Traffic prediction is based on modeling the complex non-linear spatiotemporal traffic dynamics in road network. In recent years, Long Short-Term Memory has been applied to traffic prediction, achieving better performance. The existing Long Short-Term Memory methods for traffic prediction have two drawbacks: they do not use the departure time through the links for traffic prediction, and the way of modeling long-term dependence in time series is not direct in terms of traffic prediction. Attention mechanism is implemented by constructing a neural network according to its task and has recently demonstrated success in a wide range of tasks. In this paper, we propose an Long Short-Term Memory-based method with attention mechanism for travel time prediction. We present the proposed model in a tree structure. The proposed model substitutes a tree structure with attention mechanism for the unfold way of standard Long Short-Term Memory to construct the depth of Long Short-Term Memory and modeling long-term dependence. The attention mechanism is over the output layer of each Long Short-Term Memory unit. The departure time is used as the aspect of the attention mechanism and the attention mechanism integrates departure time into the proposed model. We use AdaGrad method for training the proposed model. Based on the datasets provided by Highways England, the experimental results show that the proposed model can achieve better accuracy than the Long Short-Term Memory and other baseline methods. The case study suggests that the departure time is effectively employed by using attention mechanism.

scholarly journals Long Short-Term Memory with Dynamic Skip Connections

2019 ◽  
Vol 33 ◽  
pp. 6481-6488 ◽  
Author(s):  
Tao Gui ◽  
Qi Zhang ◽  
Lujun Zhao ◽  
Yaosong Lin ◽  
Minlong Peng ◽  
...  
Keyword(s):  
Language Processing ◽  
Short Term Memory ◽  
Training Data ◽  
Sequential Data ◽  
Short Term ◽  
Term Memory ◽  
Transition Functions ◽  
Proposed Model ◽  
Long Short Term Memory

In recent years, long short-term memory (LSTM) has been successfully used to model sequential data of variable length. However, LSTM can still experience difficulty in capturing long-term dependencies. In this work, we tried to alleviate this problem by introducing a dynamic skip connection, which can learn to directly connect two dependent words. Since there is no dependency information in the training data, we propose a novel reinforcement learning-based method to model the dependency relationship and connect dependent words. The proposed model computes the recurrent transition functions based on the skip connections, which provides a dynamic skipping advantage over RNNs that always tackle entire sentences sequentially. Our experimental results on three natural language processing tasks demonstrate that the proposed method can achieve better performance than existing methods. In the number prediction experiment, the proposed model outperformed LSTM with respect to accuracy by nearly 20%.

scholarly journals Descriptive Answer Script Grading System using CNN-BiLSTM Network

2021 ◽  
Vol 9 (5) ◽  
pp. 139-144
Author(s):  
Shirien K A ◽  
Neethu George ◽  
Surekha Mariam Varghese
Keyword(s):  
Short Term Memory ◽  
Student Evaluation ◽  
Semantic Representation ◽  
Similarity Score ◽  
Test Dataset ◽  
Learning Framework ◽  
Proposed Model ◽  
Public Dataset ◽  
Long Short Term Memory

Descriptive answer script assessment and rating program is an automated framework to evaluate the answer scripts correctly. There are several classification schemes in which a piece of text is evaluated on the basis of spelling, semantics and meaning. But, lots of these aren’t successful. Some of the models available to rate the response scripts include Simple Long Short Term Memory (LSTM), Deep LSTM. In addition to that Convolution Neural Network and Bi-directional LSTM is considered here to refine the result. The model uses convolutional neural networks and bidirectional LSTM networks to learn local information of words and capture long-term dependency information of contexts on the Tensorflow and Keras deep learning framework. The embedding semantic representation of texts can be used for computing semantic similarities between pieces of texts and to grade them based on the similarity score. The experiment used methods for data optimization, such as data normalization and dropout, and tested the model on an Automated Student Evaluation Short Response Scoring, a commonly used public dataset. By comparing with the existing systems, the proposed model has achieved the state-of-the-art performance and achieves better results in the accuracy of the test dataset.

scholarly journals Prediction of Hot Topics of Agricultural Public Opinion Based on Attention Mechanism LSTM Model

2021 ◽  
Vol 12 (4) ◽  
pp. 1-16
Author(s):  
Lifang Fu ◽  
Feifei Zhao
Keyword(s):  
Public Opinion ◽  
Short Term Memory ◽  
Classical Swine Fever ◽  
Attention Mechanism ◽  
Evaluation Indexes ◽  
The Public ◽  
Changing Trend ◽  
Lstm Network ◽  
Long Term Trend

In order to timely and accurately analyze the focus and appeal of public opinion on the Internet, A LSTM-ATTN model was proposed to extract the hot topics and predict their changing trend based on tens of thousands of news and commentary messages. First, an improved LDA model was used to extract hot words and classify the hot topics. Aimed to more accurately describe the detailed characteristics and long-term trend of topic popularity, a prediction model is proposed based on attention mechanism Long Short-Term Memory (LSTM) network, which named LSTM-ATTN model. A large number of numerical experiments were carried out using the public opinion information of "African classical swine fever" event in China. According to results of evaluation indexes, the relative superiority of LSTM-ATTN model was demonstrated. It can capture and reflect the inherent characteristics and periodic fluctuations of the agricultural public opinion information. Also, it has higher convergence efficiency and prediction accuracy.

scholarly journals Predicting Sentiment Polarity of Microblogs using an LSTM – CNN Deep Learning Model

2019 ◽  
Vol 8 (6) ◽  
pp. 4368-4373
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Prediction Accuracy ◽  
Short Term Memory ◽  
Learning Model ◽  
Supervised Machine Learning ◽  
Machine Learning Model ◽  
Proposed Model ◽  
Network Approaches

In this paper we propose a novel supervised machine learning model to predict the polarity of sentiments expressed in microblogs. The proposed model has a stacked neural network structure consisting of Long Short Term Memory (LSTM) and Convolutional Neural Network (CNN) layers. In order to capture the long-term dependencies of sentiments in the text ordering of a microblog, the proposed model employs an LSTM layer. The encodings produced by the LSTM layer are then fed to a CNN layer, which generates localized patterns of higher accuracy. These patterns are capable of capturing both local and global long-term dependences in the text of the microblogs. It was observed that the proposed model performs better and gives improved prediction accuracy when compared to semantic, machine learning and deep neural network approaches such as SVM, CNN, LSTM, CNN-LSTM, etc. This paper utilizes the benchmark Stanford Large Movie Review dataset to show the significance of the new approach. The prediction accuracy of the proposed approach is comparable to other state-of-art approaches.

Attention-Based Deep Recurrent Model for Survival Prediction

2021 ◽  
Vol 2 (4) ◽  
pp. 1-18
Author(s):  
Zhaohong Sun ◽  
Wei Dong ◽  
Jinlong Shi ◽  
Kunlun He ◽  
Zhengxing Huang
Keyword(s):  
Risk Factors ◽  
Survival Analysis ◽  
Deep Learning ◽  
Service Management ◽  
Short Term Memory ◽  
Survival Prediction ◽  
Learning Techniques ◽  
Proposed Model ◽  
Traditional Approaches

Survival analysis exhibits profound effects on health service management. Traditional approaches for survival analysis have a pre-assumption on the time-to-event probability distribution and seldom consider sequential visits of patients on medical facilities. Although recent studies leverage the merits of deep learning techniques to capture non-linear features and long-term dependencies within multiple visits for survival analysis, the lack of interpretability prevents deep learning models from being applied to clinical practice. To address this challenge, this article proposes a novel attention-based deep recurrent model, named AttenSurv , for clinical survival analysis. Specifically, a global attention mechanism is proposed to extract essential/critical risk factors for interpretability improvement. Thereafter, Bi-directional Long Short-Term Memory is employed to capture the long-term dependency on data from a series of visits of patients. To further improve both the prediction performance and the interpretability of the proposed model, we propose another model, named GNNAttenSurv , by incorporating a graph neural network into AttenSurv, to extract the latent correlations between risk factors. We validated our solution on three public follow-up datasets and two electronic health record datasets. The results demonstrated that our proposed models yielded consistent improvement compared to the state-of-the-art baselines on survival analysis.

On the Use of Generative Deep Learning Approaches for Generating Hidden Test Scripts

2021 ◽  
Vol 31 (10) ◽  
pp. 1447-1468
Author(s):  
Mert Oz ◽  
Caner Kaya ◽  
Erdi Olmezogullari ◽  
Mehmet S. Aktas
Keyword(s):  
Web Application ◽  
Web Applications ◽  
Short Term Memory ◽  
Learning Approaches ◽  
Generation Task ◽  
Clickstream Data ◽  
Test Script ◽  
Proposed Model ◽  
Action Sequences ◽  
Lstm Network

With the advent of web 2.0, web application architectures have been evolved, and their complexity has grown enormously. Due to the complexity, testing of web applications is getting time-consuming and intensive process. In today’s web applications, users can achieve the same goal by performing different actions. To ensure that the entire system is safe and robust, developers try to test all possible user action sequences in the testing phase. Since the space of all the possibilities is enormous, covering all user action sequences can be impossible. To automate the test script generation task and reduce the space of the possible user action sequences, we propose a novel method based on long short-term memory (LSTM) network for generating test scripts from user clickstream data. The experiment results clearly show that generated hidden test sequences are user-like sequences, and the process of generating test scripts with the proposed model is less time-consuming than writing them manually.

Sign in / Sign up

Export Citation Format

Share Document