Collaborative Co-Attention Network for Session-Based Recommendation

Session-based recommendation aims to model a user’s intent and predict an item that the user may interact with in the next step based on an ongoing session. Existing session-based recommender systems mainly aim to model the sequential signals based on Recurrent Neural Network (RNN) structures or the item transition relations between items with Graph Neural Network (GNN) based frameworks to identify a user’s intent for recommendation. However, in real scenarios, there may be strong sequential signals existing in users’ adjacent behaviors or multi-step transition relations among different items. Thus, either RNN- or GNN-based methods can only capture limited information for modeling complex user behavior patterns. RNNs pay attention to the sequential relations among consecutive items, while GNNs focus on structural information, i.e., how to enrich the item embedding with its adjacent items. In this paper, we propose a Collaborative Co-attention Network for Session-based Recommendation (CCN-SR) to incorporate both sequential and structural information, as well as capture the co-relations between them for obtaining an accurate session representation. To be specific, we first model the ongoing session with an RNN structure to capture the sequential information among items. Meanwhile, we also construct a session graph to learn the item representations with a GNN structure. Then, we design a co-attention network upon these two structures to capture the mutual information between them. The designed co-attention network can enrich the representation of each node in the session with both sequential and structural information, and thus generate a more comprehensive representation for each session. Extensive experiments are conducted on two public e-commerce datasets, and the results demonstrate that our proposed model outperforms state-of-the-art baseline model for session based recommendation in terms of both Recall and MRR. We also investigate different combination strategies and the experimental results verify the effectiveness of our proposed co-attention mechanism. Besides, our CCN-SR model achieves better performance than baseline models with different session lengths.

Download Full-text

An Attention-Based Graph Neural Network for Heterogeneous Structural Learning

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v34i04.5833 ◽

2020 ◽

Vol 34 (04) ◽

pp. 4132-4139

Author(s):

Huiting Hong ◽

Hantao Guo ◽

Yucheng Lin ◽

Xiaoqing Yang ◽

Zang Li ◽

...

Keyword(s):

Neural Network ◽

Structural Information ◽

Representation Learning ◽

Graph Representation ◽

Heterogeneous Information ◽

Domain Experts ◽

Proposed Model ◽

Meta Path ◽

Low Dimensional ◽

Public Datasets

In this paper, we focus on graph representation learning of heterogeneous information network (HIN), in which various types of vertices are connected by various types of relations. Most of the existing methods conducted on HIN revise homogeneous graph embedding models via meta-paths to learn low-dimensional vector space of HIN. In this paper, we propose a novel Heterogeneous Graph Structural Attention Neural Network (HetSANN) to directly encode structural information of HIN without meta-path and achieve more informative representations. With this method, domain experts will not be needed to design meta-path schemes and the heterogeneous information can be processed automatically by our proposed model. Specifically, we implicitly represent heterogeneous information using the following two methods: 1) we model the transformation between heterogeneous vertices through a projection in low-dimensional entity spaces; 2) afterwards, we apply the graph neural network to aggregate multi-relational information of projected neighborhood by means of attention mechanism. We also present three extensions of HetSANN, i.e., voices-sharing product attention for the pairwise relationships in HIN, cycle-consistency loss to retain the transformation between heterogeneous entity spaces, and multi-task learning with full use of information. The experiments conducted on three public datasets demonstrate that our proposed models achieve significant and consistent improvements compared to state-of-the-art solutions.

Download Full-text

An English Teaching Resource Recommendation System Based on Network Behavior Analysis

Scientific Programming ◽

10.1155/2021/6191543 ◽

2021 ◽

Vol 2021 ◽

pp. 1-9

Author(s):

Jing Zhang

Keyword(s):

Neural Network ◽

Deep Learning ◽

Recommendation System ◽

User Behavior ◽

Teaching Resources ◽

English Teaching ◽

Collaborative Recommendation ◽

Proposed Model ◽

Deep Learning Model ◽

Local User

The sharing of English teaching resources has always been a concern. In order to further improve the value of different English teaching resources, this paper proposes a resource management system based on an improved collaborative recommendation algorithm. The proposed model can predict user behavior based on deep learning models of graph neural network (GNN) and recurrent neural network (RNN). The graph neural network can capture the hidden state of local user behavior and be used as a preprocessing step. Recurrent neural networks can capture time series information. Therefore, the model is constructed by combining GNN and RNN to obtain the advantages of both. In order to prove the effectiveness of the model, we used CNGrid’s real user behavior dataset in the experiment and finally compared the results with other methods. The different deep learning-based models achieved a precision of up to 88% and outperformed other traditional models. The experimental results show that this new deep learning model has good sharing value.

Download Full-text

Recipe Recommendation With Hierarchical Graph Attention Network

Frontiers in Big Data ◽

10.3389/fdata.2021.778417 ◽

2022 ◽

Vol 4 ◽

Author(s):

Yijun Tian ◽

Chuxu Zhang ◽

Ronald Metoyer ◽

Nitesh V. Chawla

Keyword(s):

Neural Network ◽

Eating Habits ◽

Learning Approach ◽

Attention Network ◽

Relational Information ◽

Food Items ◽

Hierarchical Graph ◽

Proposed Model ◽

Network Modules ◽

User History

Recipe recommendation systems play an important role in helping people find recipes that are of their interest and fit their eating habits. Unlike what has been developed for recommending recipes using content-based or collaborative filtering approaches, the relational information among users, recipes, and food items is less explored. In this paper, we leverage the relational information into recipe recommendation and propose a graph learning approach to solve it. In particular, we propose HGAT, a novel hierarchical graph attention network for recipe recommendation. The proposed model can capture user history behavior, recipe content, and relational information through several neural network modules, including type-specific transformation, node-level attention, and relation-level attention. We further introduce a ranking-based objective function to optimize the model. Thorough experiments demonstrate that HGAT outperforms numerous baseline methods.

Download Full-text

Nodule Detection with Convolutional Neural Network Using Apache Spark and GPU Frameworks

Applied Sciences ◽

10.3390/app11062838 ◽

2021 ◽

Vol 11 (6) ◽

pp. 2838

Author(s):

Nikitha Johnsirani Venkatesan ◽

Dong Ryeol Shin ◽

Choon Sung Nam

Keyword(s):

Neural Network ◽

Radiation Dose ◽

Convolutional Neural Network ◽

Model Performance ◽

Performance Comparison ◽

Apache Spark ◽

Training Time ◽

Learning Framework ◽

Proposed Model

In the pharmaceutical field, early detection of lung nodules is indispensable for increasing patient survival. We can enhance the quality of the medical images by intensifying the radiation dose. High radiation dose provokes cancer, which forces experts to use limited radiation. Using abrupt radiation generates noise in CT scans. We propose an optimal Convolutional Neural Network model in which Gaussian noise is removed for better classification and increased training accuracy. Experimental demonstration on the LUNA16 dataset of size 160 GB shows that our proposed method exhibit superior results. Classification accuracy, specificity, sensitivity, Precision, Recall, F1 measurement, and area under the ROC curve (AUC) of the model performance are taken as evaluation metrics. We conducted a performance comparison of our proposed model on numerous platforms, like Apache Spark, GPU, and CPU, to depreciate the training time without compromising the accuracy percentage. Our results show that Apache Spark, integrated with a deep learning framework, is suitable for parallel training computation with high accuracy.

Download Full-text

Natural Disasters Intensity Analysis and Classification Based on Multispectral Images Using Multi-Layered Deep Convolutional Neural Network

Sensors ◽

10.3390/s21082648 ◽

2021 ◽

Vol 21 (8) ◽

pp. 2648

Author(s):

Muhammad Aamir ◽

Tariq Ali ◽

Muhammad Irfan ◽

Ahmad Shaf ◽

Muhammad Zeeshan Azam ◽

...

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Natural Disasters ◽

Deep Convolutional Neural Network ◽

Multispectral Images ◽

Learning Techniques ◽

Proposed Model ◽

Disaster Intensity ◽

And Performance

Natural disasters not only disturb the human ecological system but also destroy the properties and critical infrastructures of human societies and even lead to permanent change in the ecosystem. Disaster can be caused by naturally occurring events such as earthquakes, cyclones, floods, and wildfires. Many deep learning techniques have been applied by various researchers to detect and classify natural disasters to overcome losses in ecosystems, but detection of natural disasters still faces issues due to the complex and imbalanced structures of images. To tackle this problem, we propose a multilayered deep convolutional neural network. The proposed model works in two blocks: Block-I convolutional neural network (B-I CNN), for detection and occurrence of disasters, and Block-II convolutional neural network (B-II CNN), for classification of natural disaster intensity types with different filters and parameters. The model is tested on 4428 natural images and performance is calculated and expressed as different statistical values: sensitivity (SE), 97.54%; specificity (SP), 98.22%; accuracy rate (AR), 99.92%; precision (PRE), 97.79%; and F1-score (F1), 97.97%. The overall accuracy for the whole model is 99.92%, which is competitive and comparable with state-of-the-art algorithms.

Download Full-text

Predicting the Energy Consumption of a Robot in an Exploration Task Using Optimized Neural Networks

Electronics ◽

10.3390/electronics10080920 ◽

2021 ◽

Vol 10 (8) ◽

pp. 920

Author(s):

Liesle Caballero ◽

Álvaro Perafan ◽

Martha Rinaldy ◽

Winston Percybrooks

Keyword(s):

Neural Network ◽

Energy Consumption ◽

Mobile Robot ◽

Energy Budget ◽

Dynamic Models ◽

Pearson Correlation ◽

Experimental Conditions ◽

Grid Map ◽

Proposed Model ◽

Exploration Task

This paper deals with the problem of determining a useful energy budget for a mobile robot in a given environment without having to carry out experimental measures for every possible exploration task. The proposed solution uses machine learning models trained on a subset of possible exploration tasks but able to make predictions on untested scenarios. Additionally, the proposed model does not use any kinematic or dynamic models of the robot, which are not always available. The method is based on a neural network with hyperparameter optimization to improve performance. Tabu List optimization strategy is used to determine the hyperparameter values (number of layers and number of neurons per layer) that minimize the percentage relative absolute error (%RAE) while maximize the Pearson correlation coefficient (R) between predicted data and actual data measured under a number of experimental conditions. Once the optimized artificial neural network is trained, it can be used to predict the performance of an exploration algorithm on arbitrary variations of a grid map scenario. Based on such prediction, it is possible to know the energy needed for the robot to complete the exploration task. A total of 128 tests were carried out using a robot executing two exploration algorithms in a grid map with the objective of locating a target whose location is not known a priori by the robot. The experimental energy consumption was measured and compared with the prediction of our model. A success rate of 96.093% was obtained, measured as the percentage of tests where the energy budget suggested by the model was enough to actually carry out the task when compared to the actual energy consumed in the test, suggesting that the proposed model could be useful for energy budgeting in actual mobile robot applications.

Download Full-text

Tomato Leaf Disease Diagnosis Based on Improved Convolution Neural Network by Attention Module

Agriculture ◽

10.3390/agriculture11070651 ◽

2021 ◽

Vol 11 (7) ◽

pp. 651

Author(s):

Shengyi Zhao ◽

Yun Peng ◽

Jizhan Liu ◽

Shuo Wu

Keyword(s):

Neural Network ◽

High Performance ◽

Model Comparison ◽

Research Direction ◽

Disease Diagnosis ◽

Tomato Leaf ◽

Identification Accuracy ◽

Main Research ◽

Proposed Model ◽

Complex Features

Crop disease diagnosis is of great significance to crop yield and agricultural production. Deep learning methods have become the main research direction to solve the diagnosis of crop diseases. This paper proposed a deep convolutional neural network that integrates an attention mechanism, which can better adapt to the diagnosis of a variety of tomato leaf diseases. The network structure mainly includes residual blocks and attention extraction modules. The model can accurately extract complex features of various diseases. Extensive comparative experiment results show that the proposed model achieves the average identification accuracy of 96.81% on the tomato leaf diseases dataset. It proves that the model has significant advantages in terms of network complexity and real-time performance compared with other models. Moreover, through the model comparison experiment on the grape leaf diseases public dataset, the proposed model also achieves better results, and the average identification accuracy of 99.24%. It is certified that add the attention module can more accurately extract the complex features of a variety of diseases and has fewer parameters. The proposed model provides a high-performance solution for crop diagnosis under the real agricultural environment.

Download Full-text

Recurrent neural networks with long term temporal dependencies in machine tool wear diagnosis and prognosis

SN Applied Sciences ◽

10.1007/s42452-021-04427-5 ◽

2021 ◽

Vol 3 (4) ◽

Author(s):

Jianlei Zhang ◽

Yukun Zeng ◽

Binil Starly

Keyword(s):

Neural Network ◽

Tool Wear ◽

Machine Tool ◽

Recurrent Neural Network ◽

Machine Tools ◽

Prediction Performance ◽

Sequential Data ◽

Diagnosis And Prognosis ◽

Proposed Model

AbstractData-driven approaches for machine tool wear diagnosis and prognosis are gaining attention in the past few years. The goal of our study is to advance the adaptability, flexibility, prediction performance, and prediction horizon for online monitoring and prediction. This paper proposes the use of a recent deep learning method, based on Gated Recurrent Neural Network architecture, including Long Short Term Memory (LSTM), which try to captures long-term dependencies than regular Recurrent Neural Network method for modeling sequential data, and also the mechanism to realize the online diagnosis and prognosis and remaining useful life (RUL) prediction with indirect measurement collected during the manufacturing process. Existing models are usually tool-specific and can hardly be generalized to other scenarios such as for different tools or operating environments. Different from current methods, the proposed model requires no prior knowledge about the system and thus can be generalized to different scenarios and machine tools. With inherent memory units, the proposed model can also capture long-term dependencies while learning from sequential data such as those collected by condition monitoring sensors, which means it can be accommodated to machine tools with varying life and increase the prediction performance. To prove the validity of the proposed approach, we conducted multiple experiments on a milling machine cutting tool and applied the model for online diagnosis and RUL prediction. Without loss of generality, we incorporate a system transition function and system observation function into the neural net and trained it with signal data from a minimally intrusive vibration sensor. The experiment results showed that our LSTM-based model achieved the best overall accuracy among other methods, with a minimal Mean Square Error (MSE) for tool wear prediction and RUL prediction respectively.

Download Full-text

Sentence similarity evaluation using Sent2Vec and siamese neural network with parallel structure

Journal of Intelligent & Fuzzy Systems ◽

10.3233/jifs-189593 ◽

2021 ◽

pp. 1-10

Author(s):

Hye-Jeong Song ◽

Tak-Sung Heo ◽

Jong-Dae Kim ◽

Chan-Young Park ◽

Yu-Seop Kim

Keyword(s):

Neural Network ◽

Language Processing ◽

Short Term Memory ◽

Parallel Structure ◽

Short Term ◽

Similarity Estimation ◽

Accurate Judgment ◽

Proposed Model ◽

Sentence Similarity ◽

Long Short Term Memory

Sentence similarity evaluation is a significant task used in machine translation, classification, and information extraction in the field of natural language processing. When two sentences are given, an accurate judgment should be made whether the meaning of the sentences is equivalent even if the words and contexts of the sentences are different. To this end, existing studies have measured the similarity of sentences by focusing on the analysis of words, morphemes, and letters. To measure sentence similarity, this study uses Sent2Vec, a sentence embedding, as well as morpheme word embedding. Vectors representing words are input to the 1-dimension convolutional neural network (1D-CNN) with various sizes of kernels and bidirectional long short-term memory (Bi-LSTM). Self-attention is applied to the features transformed through Bi-LSTM. Subsequently, vectors undergoing 1D-CNN and self-attention are converted through global max pooling and global average pooling to extract specific values, respectively. The vectors generated through the above process are concatenated to the vector generated through Sent2Vec and are represented as a single vector. The vector is input to softmax layer, and finally, the similarity between the two sentences is determined. The proposed model can improve the accuracy by up to 5.42% point compared with the conventional sentence similarity estimation models.

Download Full-text

Graph contextualized attention network for predicting synthetic lethality in human cancers

Bioinformatics ◽

10.1093/bioinformatics/btab110 ◽

2021 ◽

Author(s):

Yahui Long ◽

Min Wu ◽

Yong Liu ◽

Jie Zheng ◽

Chee Keong Kwoh ◽

...

Keyword(s):

Synthetic Lethality ◽

Critical Role ◽

Design Feature ◽

Cost Effective ◽

Attention Network ◽

New Genes ◽

Lab Experiments ◽

Proposed Model ◽

Multiple Feature ◽

Wet Lab

Abstract Motivation Synthetic Lethality (SL) plays an increasingly critical role in the targeted anticancer therapeutics. In addition, identifying SL interactions can create opportunities to selectively kill cancer cells without harming normal cells. Given the high cost of wet-lab experiments, in silico prediction of SL interactions as an alternative can be a rapid and cost-effective way to guide the experimental screening of candidate SL pairs. Several matrix factorization-based methods have recently been proposed for human SL prediction. However, they are limited in capturing the dependencies of neighbors. In addition, it is also highly challenging to make accurate predictions for new genes without any known SL partners. Results In this work, we propose a novel graph contextualized attention network named GCATSL to learn gene representations for SL prediction. First, we leverage different data sources to construct multiple feature graphs for genes, which serve as the feature inputs for our GCATSL method. Second, for each feature graph, we design node-level attention mechanism to effectively capture the importance of local and global neighbors and learn local and global representations for the nodes, respectively. We further exploit multi-layer perceptron (MLP) to aggregate the original features with the local and global representations and then derive the feature-specific representations. Third, to derive the final representations, we design feature-level attention to integrate feature-specific representations by taking the importance of different feature graphs into account. Extensive experimental results on three datasets under different settings demonstrated that our GCATSL model outperforms 14 state-of-the-art methods consistently. In addition, case studies further validated the effectiveness of our proposed model in identifying novel SL pairs. Availability Python codes and dataset are freely available on GitHub (https://github.com/longyahui/GCATSL) and Zenodo (https://zenodo.org/record/4522679) under the MIT license.

Download Full-text