Memory Augmented Graph Neural Networks for Sequential Recommendation

The chronological order of user-item interactions can reveal time-evolving and sequential user behaviors in many recommender systems. The items that users will interact with may depend on the items accessed in the past. However, the substantial increase of users and items makes sequential recommender systems still face non-trivial challenges: (1) the hardness of modeling the short-term user interests; (2) the difficulty of capturing the long-term user interests; (3) the effective modeling of item co-occurrence patterns. To tackle these challenges, we propose a memory augmented graph neural network (MA-GNN) to capture both the long- and short-term user interests. Specifically, we apply a graph neural network to model the item contextual information within a short-term period and utilize a shared memory network to capture the long-range dependencies between items. In addition to the modeling of user interests, we employ a bilinear function to capture the co-occurrence patterns of related items. We extensively evaluate our model on five real-world datasets, comparing with several state-of-the-art methods and using a variety of performance metrics. The experimental results demonstrate the effectiveness of our model for the task of Top-K sequential recommendation.

Download Full-text

MR-GNN: Multi-Resolution and Dual Graph Neural Network for Predicting Structured Entity Interactions

Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2019/551 ◽

2019 ◽

Author(s):

Nuo Xu ◽

Pinghui Wang ◽

Long Chen ◽

Jing Tao ◽

Junzhou Zhao

Keyword(s):

Neural Network ◽

Short Term Memory ◽

Dual Graph ◽

Material Design ◽

Drug Regimen ◽

Short Term ◽

New Material ◽

Long Short Term Memory ◽

Real World Datasets ◽

Graph Neural Networks

Predicting interactions between structured entities lies at the core of numerous tasks such as drug regimen and new material design. In recent years, graph neural networks have become attractive. They represent structured entities as graphs, and then extract features from each individual graph using graph convolution operations. However, these methods have some limitations: i) their networks only extract features from a fix-sized subgraph structure (i.e., a fix-sized receptive field) of each node, and ignore features in substructures of different sizes, and ii) features are extracted by considering each entity independently, which may not effectively reflect the interaction between two entities. To resolve these problems, we present {\em MR-GNN}, an end-to-end graph neural network with the following features: i) it uses a multi-resolution based architecture to extract node features from different neighborhoods of each node, and, ii) it uses dual graph-state long short-term memory networks (LSTMs) to summarize local features of each graph and extracts the interaction features between pairwise graphs. Experiments conducted on real-world datasets show that MR-GNN improves the prediction of state-of-the-art methods.

Download Full-text

Attention-Enhanced Graph Neural Networks for Session-Based Recommendation

Mathematics ◽

10.3390/math8091607 ◽

2020 ◽

Vol 8 (9) ◽

pp. 1607 ◽

Cited By ~ 1

Author(s):

Baocheng Wang ◽

Wentao Cai

Keyword(s):

Critical Role ◽

User Needs ◽

Short Term ◽

User Interests ◽

Recommendation Algorithms ◽

Multiple Levels ◽

Real World Datasets ◽

Graph Neural Networks ◽

Novel Model

Session-based recommendation, which aims to match user needs with rich resources based on anonymous sessions, nowadays plays a critical role in various online platforms (e.g., media streaming sites, search and e-commerce). Existing recommendation algorithms usually model a session as a sequence or a session graph to model transitions between items. Despite their effectiveness, we would argue that the performance of these methods is still flawed: (1) Using only fixed session item embedding without considering the diversity of users’ interests and target items. (2) For user’s long-term interest, the difficulty of capturing the different priorities for different items accurately. To tackle these defects, we propose a novel model which leverages both the target attentive network and self-attention network to improve the graph-neural-network (GNN)-based recommender. In our model, we first model user’s interaction sequences as session graphs which serves as the input of the GNN, and each node vector involved in session graph can be obtained via the GNN. Next, target attentive network can activates different user interests corresponding to varied target items (i.e., the session embedding learned varies with different target items), which can reveal the relevance between users’ interests and target items. At last, after applying the self-attention mechanism, the different priorities for different items can be captured to improve the precision of the long-term session representation. By using a hybrid of long-term and short-term session representation, we can capture users’ comprehensive interests at multiple levels. Extensive experiments demonstrate the effectiveness of our algorithm on two real-world datasets for session-based recommendation.

Download Full-text

What to Do Next: Modeling User Behaviors by Time-LSTM

Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2017/504 ◽

2017 ◽

Cited By ~ 59

Author(s):

Yu Zhu ◽

Hao Li ◽

Yikang Liao ◽

Beidou Wang ◽

Ziyu Guan ◽

...

Keyword(s):

Neural Network ◽

Recurrent Neural Network ◽

Language Modeling ◽

Sequential Data ◽

Sequential Order ◽

Short Term ◽

Time Intervals ◽

User Behaviors ◽

Real World Datasets

Recently, Recurrent Neural Network (RNN) solutions for recommender systems (RS) are becoming increasingly popular. The insight is that, there exist some intrinsic patterns in the sequence of users' actions, and RNN has been proved to perform excellently when modeling sequential data. In traditional tasks such as language modeling, RNN solutions usually only consider the sequential order of objects without the notion of interval. However, in RS, time intervals between users' actions are of significant importance in capturing the relations of users' actions and the traditional RNN architectures are not good at modeling them. In this paper, we propose a new LSTM variant, i.e. Time-LSTM, to model users' sequential actions. Time-LSTM equips LSTM with time gates to model time intervals. These time gates are specifically designed, so that compared to the traditional RNN solutions, Time-LSTM better captures both of users' short-term and long-term interests, so as to improve the recommendation performance. Experimental results on two real-world datasets show the superiority of the recommendation method using Time-LSTM over the traditional methods.

Download Full-text

Attention Enhanced Serial Unet++ Network for Removing Unevenly Distributed Haze

Electronics ◽

10.3390/electronics10222868 ◽

2021 ◽

Vol 10 (22) ◽

pp. 2868

Author(s):

Wenxuan Zhao ◽

Yaqin Zhao ◽

Liqi Feng ◽

Jiaxi Tang

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Real World ◽

Large Scale ◽

Learning Strategy ◽

Contextual Information ◽

Small Scale ◽

Image Dehazing ◽

Atmospheric Scattering ◽

Real World Datasets

The purpose of image dehazing is the reduction of the image degradation caused by suspended particles for supporting high-level visual tasks. Besides the atmospheric scattering model, convolutional neural network (CNN) has been used for image dehazing. However, the existing image dehazing algorithms are limited in face of unevenly distributed haze and dense haze in real-world scenes. In this paper, we propose a novel end-to-end convolutional neural network called attention enhanced serial Unet++ dehazing network (AESUnet) for single image dehazing. We attempt to build a serial Unet++ structure that adopts a serial strategy of two pruned Unet++ blocks based on residual connection. Compared with the simple Encoder–Decoder structure, the serial Unet++ module can better use the features extracted by encoders and promote contextual information fusion in different resolutions. In addition, we take some improvement measures to the Unet++ module, such as pruning, introducing the convolutional module with ResNet structure, and a residual learning strategy. Thus, the serial Unet++ module can generate more realistic images with less color distortion. Furthermore, following the serial Unet++ blocks, an attention mechanism is introduced to pay different attention to haze regions with different concentrations by learning weights in the spatial domain and channel domain. Experiments are conducted on two representative datasets: the large-scale synthetic dataset RESIDE and the small-scale real-world datasets I-HAZY and O-HAZY. The experimental results show that the proposed dehazing network is not only comparable to state-of-the-art methods for the RESIDE synthetic datasets, but also surpasses them by a very large margin for the I-HAZY and O-HAZY real-world dataset.

Download Full-text

A Deep Learning-Based Course Recommender System for Sustainable Development in Education

Applied Sciences ◽

10.3390/app11198993 ◽

2021 ◽

Vol 11 (19) ◽

pp. 8993

Author(s):

Qinglong Li ◽

Jaekyeong Kim

Keyword(s):

Deep Learning ◽

Online Education ◽

Recommender Systems ◽

Recommender System ◽

Information Overload ◽

Personalized Recommendation ◽

Course Content ◽

User Behaviors ◽

Real World Datasets ◽

High Level

Recently, the worldwide COVID-19 pandemic has led to an increasing demand for online education platforms. However, it is challenging to correctly choose course content from among many online education resources due to the differences in users’ knowledge structures. Therefore, a course recommender system has the essential role of improving the learning efficiency of users. At present, many online education platforms have built diverse recommender systems that utilize traditional data mining methods, such as Collaborative Filtering (CF). Despite the development and contributions of many recommender systems based on CF, diverse deep learning models for personalized recommendation are being studied because of problems such as sparsity and scalability. Therefore, to solve traditional recommendation problems, this study proposes a novel deep learning-based course recommender system (DECOR), which elaborately captures high-level user behaviors and course attribute features. The DECOR model can reduce information overload, solve high-dimensional data sparsity problems, and achieve high feature information extraction performance. We perform several experiments utilizing real-world datasets to evaluate the DECOR model’s performance compared with that of traditional recommendation approaches. The experimental results indicate that the DECOR model offers better and more robust recommendation performance than the traditional methods.

Download Full-text

Streamflow prediction using an integrated methodology based on convolutional neural network and long short-term memory networks

Scientific Reports ◽

10.1038/s41598-021-96751-4 ◽

2021 ◽

Vol 11 (1) ◽

Author(s):

Sujan Ghimire ◽

Zaher Mundher Yaseen ◽

Aitazaz A. Farooque ◽

Ravinesh C. Deo ◽

Ji Zhang ◽

...

Keyword(s):

Neural Network ◽

Time Series ◽

Flood Control ◽

Performance Metrics ◽

Short Term Memory ◽

Short Term ◽

Neural Network Models ◽

Time Intervals ◽

Term Memory ◽

Long Short Term Memory

AbstractStreamflow (Qflow) prediction is one of the essential steps for the reliable and robust water resources planning and management. It is highly vital for hydropower operation, agricultural planning, and flood control. In this study, the convolution neural network (CNN) and Long-Short-term Memory network (LSTM) are combined to make a new integrated model called CNN-LSTM to predict the hourly Qflow (short-term) at Brisbane River and Teewah Creek, Australia. The CNN layers were used to extract the features of Qflow time-series, while the LSTM networks use these features from CNN for Qflow time series prediction. The proposed CNN-LSTM model is benchmarked against the standalone model CNN, LSTM, and Deep Neural Network models and several conventional artificial intelligence (AI) models. Qflow prediction is conducted for different time intervals with the length of 1-Week, 2-Weeks, 4-Weeks, and 9-Months, respectively. With the help of different performance metrics and graphical analysis visualization, the experimental results reveal that with small residual error between the actual and predicted Qflow, the CNN-LSTM model outperforms all the benchmarked conventional AI models as well as ensemble models for all the time intervals. With 84% of Qflow prediction error below the range of 0.05 m3 s−1, CNN-LSTM demonstrates a better performance compared to 80% and 66% for LSTM and DNN, respectively. In summary, the results reveal that the proposed CNN-LSTM model based on the novel framework yields more accurate predictions. Thus, CNN-LSTM has significant practical value in Qflow prediction.

Download Full-text

A Rating Prediction Recommendation Model Combined with the Optimizing Allocation for Information Granularity of Attributes

Information ◽

10.3390/info13010021 ◽

2022 ◽

Vol 13 (1) ◽

pp. 21

Author(s):

Jianfei Li ◽

Yongbin Wang ◽

Zhulin Tao

Keyword(s):

Neural Network ◽

Latent Factors ◽

Latent Factor ◽

Information Granularity ◽

Node Attribute ◽

Item Attribute ◽

Rating Prediction ◽

Real World Datasets ◽

Graph Neural Networks ◽

Granularity Distribution

In recent years, graph neural networks (GNNS) have been demonstrated to be a powerful way to learn graph data. The existing recommender systems based on the implicit factor models mainly use the interactive information between users and items for training and learning. A user–item graph, a user–attribute graph, and an item–attribute graph are constructed according to the interactions between users and items. The latent factors of users and items can be learned in these graph structure data. There are many methods for learning the latent factors of users and items. Still, they do not fully consider the influence of node attribute information on the representation of the latent factors of users and items. We propose a rating prediction recommendation model, short for LNNSR, utilizing the level of information granularity allocated on each attribute by developing a granular neural network. The different granularity distribution proportion weights of each attribute can be learned in the granular neural network. The learned granularity allocation proportion weights are integrated into the latent factor representation of users and items. Thus, we can capture user-embedding representations and item-embedding representations more accurately, and it can also provide a reasonable explanation for the recommendation results. Finally, we concatenate the user latent factor-embedding and the item latent factor-embedding and then feed it into a multi-layer perceptron for rating prediction. Extensive experiments on two real-world datasets demonstrate the effectiveness of the proposed framework.

Download Full-text

Context-Aware Short-Term Interest First Model for Session-Based Recommendation

10.5121/csit.2021.110404 ◽

2021 ◽

Author(s):

Haomei Duan ◽

Jinghua Zhu

Keyword(s):

Neural Network ◽

General Interest ◽

Long Term Memory ◽

Graph Structure ◽

Context Aware ◽

User Profiles ◽

Short Term ◽

Term Memory ◽

Real World Datasets

In the case that user profiles are not available, the recommendation based on anonymous session is particularly important, which aims to predict the items that the user may click at the next moment based on the user's access sequence over a while. In recent years, with the development of recurrent neural network, attention mechanism, and graph neural network, the performance of session-based recommendation has been greatly improved. However, the previous methods did not comprehensively consider the context dependencies and short-term interest first of the session. Therefore, we propose a context-aware short-term interest first model (CASIF).The aim of this paper is improve the accuracy of recommendations by combining context and short-term interest. In CASIF, we dynamically construct a graph structure for session sequences and capture rich context dependencies via graph neural network (GNN), latent feature vectors are captured as inputs of the next step. Then we build the shortterm interest first module, which can to capture the user's general interest from the session in the context of long-term memory, at the same time get the user's current interest from the item of the last click. In the end, the short-term and long-term interest are combined as the final interest and multiplied by the candidate vector to obtain the recommendation probability. Finally, a large number of experiments on two real-world datasets demonstrate the effectiveness of our proposed method.

Download Full-text

Exploiting Heterogeneous Graph Neural Networks with Latent Worker/Task Correlation Information for Label Aggregation in Crowdsourcing

ACM Transactions on Knowledge Discovery from Data ◽

10.1145/3460865 ◽

2022 ◽

Vol 16 (2) ◽

pp. 1-18

Author(s):

Hanlu Wu ◽

Tengfei Ma ◽

Lingfei Wu ◽

Fangli Xu ◽

Shouling Ji

Keyword(s):

Neural Network ◽

Neural Networks ◽

State Of The Art ◽

Superior Performance ◽

True Label ◽

Label Aggregation ◽

Correlation Information ◽

Real World Datasets ◽

Graph Neural Networks ◽

High Level

Crowdsourcing has attracted much attention for its convenience to collect labels from non-expert workers instead of experts. However, due to the high level of noise from the non-experts, a label aggregation model that infers the true label from noisy crowdsourced labels is required. In this article, we propose a novel framework based on graph neural networks for aggregating crowd labels. We construct a heterogeneous graph between workers and tasks and derive a new graph neural network to learn the representations of nodes and the true labels. Besides, we exploit the unknown latent interaction between the same type of nodes (workers or tasks) by adding a homogeneous attention layer in the graph neural networks. Experimental results on 13 real-world datasets show superior performance over state-of-the-art models.

Download Full-text

An Attention-Based Recommender System to Predict Contextual Intent Based on Choice Histories across and within Sessions

Applied Sciences ◽

10.3390/app8122426 ◽

2018 ◽

Vol 8 (12) ◽

pp. 2426 ◽

Cited By ~ 2

Author(s):

Ruo Huang ◽

Shelby McIntyre ◽

Meina Song ◽

Haihong E ◽

Zhonghong Ou

Keyword(s):

Recommender Systems ◽

State Of The Art ◽

User Profile ◽

Vital Role ◽

Short Term ◽

Novel Approach ◽

Learning Techniques ◽

Real World Datasets ◽

Near Future

Recent years have witnessed the growth of recommender systems, with the help of deep learning techniques. Recurrent Neural Networks (RNNs) play an increasingly vital role in various session-based recommender systems, since they use the user’s sequential history to build a comprehensive user profile, which helps improve the recommendation. However, a problem arises regarding how to be aware of the variation in the user’s contextual preference, especially the short-term intent in the near future, and make the best use of it to produce a precise recommendation at the start of a session. We propose a novel approach named Attention-based Short-term and Long-term Model (ASLM), to improve the next-item recommendation, by using an attention-based RNNs integrating both the user’s short-term intent and the long-term preference at the same time with a two-layer network. The experimental study on three real-world datasets and two sub-datasets demonstrates that, compared with other state-of-the-art methods, the proposed approach can significantly improve the next-item recommendation, especially at the start of sessions. As a result, our proposed approach is capable of coping with the cold-start problem at the beginning of each session.

Download Full-text