scholarly journals SessNet: A Hybrid Session-Based Recommender System

2021 ◽  
Author(s):  
Omar Nada
Keyword(s):  
Neural Network ◽  
Recommender System ◽  
State Of The Art ◽  
User Preferences ◽  
Feed Forward Neural Network ◽  
Two Stage ◽  
Past Behavior ◽  
Benchmark Datasets ◽  
User History ◽  
User Actions

<div>Session-based recommendation is the task of predicting user actions during short online sessions. Previous work considers the user to be anonymous in this setting, with no past behavior history available. In reality, this is often not the case, and none of the existing approaches are flexible enough to seamlessly integrate user history when available. In this thesis, we propose a novel hybrid session-based recommender system to perform next-click prediction, which is able to take advantage of historical user preferences when accessible. Specifically, we propose SessNet, a deep profiling session-based recommender system, with a two-stage dichotomy. First, we use bidirectional transformers to model local and global session intent. Second, we concatenate any user information with the current session representation to feed to a feed-forward neural network to identify the next click. Historical user preferences are computed using the sequence-aware embeddings obtained from the first step, allowing us to better understand the users. We evaluate the efficacy of the proposed method using two benchmark datasets, YooChoose1/64 and Dignetica. Our experimental results show that SessNet outperforms state-of-the-art session-based recommenders on P@20 for both datasets.</div>

scholarly journals SessNet: A Hybrid Session-Based Recommender System

2021 ◽  
Author(s):  
Omar Nada
Keyword(s):  
Neural Network ◽  
Recommender System ◽  
State Of The Art ◽  
User Preferences ◽  
Feed Forward Neural Network ◽  
Two Stage ◽  
Past Behavior ◽  
Benchmark Datasets ◽  
User History ◽  
User Actions

<div>Session-based recommendation is the task of predicting user actions during short online sessions. Previous work considers the user to be anonymous in this setting, with no past behavior history available. In reality, this is often not the case, and none of the existing approaches are flexible enough to seamlessly integrate user history when available. In this thesis, we propose a novel hybrid session-based recommender system to perform next-click prediction, which is able to take advantage of historical user preferences when accessible. Specifically, we propose SessNet, a deep profiling session-based recommender system, with a two-stage dichotomy. First, we use bidirectional transformers to model local and global session intent. Second, we concatenate any user information with the current session representation to feed to a feed-forward neural network to identify the next click. Historical user preferences are computed using the sequence-aware embeddings obtained from the first step, allowing us to better understand the users. We evaluate the efficacy of the proposed method using two benchmark datasets, YooChoose1/64 and Dignetica. Our experimental results show that SessNet outperforms state-of-the-art session-based recommenders on P@20 for both datasets.</div>

scholarly journals Collaborative Knowledge-Enhanced Recommendation with Self-Supervisions

Mathematics ◽  
2021 ◽  
Vol 9 (17) ◽  
pp. 2129
Author(s):  
Zhiqiang Pan ◽  
Honghui Chen
Keyword(s):  
Neural Network ◽  
State Of The Art ◽  
User Preferences ◽  
The State ◽  
The Self ◽  
Knowledge Graph ◽  
Model Optimization ◽  
Interaction Graph ◽  
Benchmark Datasets ◽  
Collaborative Knowledge

Knowledge-enhanced recommendation (KER) aims to integrate the knowledge graph (KG) into collaborative filtering (CF) for alleviating the sparsity and cold start problems. The state-of-the-art graph neural network (GNN)–based methods mainly focus on exploiting the connectivity between entities in the knowledge graph, while neglecting the interaction relation between items reflected in the user-item interactions. Moreover, the widely adopted BPR loss for model optimization fails to provide sufficient supervisions for learning discriminative representation of users and items. To address these issues, we propose the collaborative knowledge-enhanced recommendation (CKER) method. Specifically, CKER proposes a collaborative graph convolution network (CGCN) to learn the user and item representations from the connection between items in the constructed interaction graph and the connectivity between entities in the knowledge graph. Moreover, we introduce the self-supervised learning to maximize the mutual information between the interaction- and knowledge-aware user preferences by deriving additional supervision signals. We conduct comprehensive experiments on two benchmark datasets, namely Amazon-Book and Last-FM, and the experimental results show that CKER can outperform the state-of-the-art baselines in terms of recall and NDCG on knowledge-enhanced recommendation.

scholarly journals Multi-Term Attention Networks for Skeleton-Based Action Recognition

2020 ◽  
Vol 10 (15) ◽  
pp. 5326
Author(s):  
Xiaolei Diao ◽  
Xiaoqiang Li ◽  
Chen Huang
Keyword(s):  
Neural Network ◽  
Time Scales ◽  
Action Recognition ◽  
State Of The Art ◽  
Attention Networks ◽  
Weighted Fusion ◽  
Temporal Features ◽  
Benchmark Datasets ◽  
Spatio Temporal ◽  
Different Time Scales

The same action takes different time in different cases. This difference will affect the accuracy of action recognition to a certain extent. We propose an end-to-end deep neural network called “Multi-Term Attention Networks” (MTANs), which solves the above problem by extracting temporal features with different time scales. The network consists of a Multi-Term Attention Recurrent Neural Network (MTA-RNN) and a Spatio-Temporal Convolutional Neural Network (ST-CNN). In MTA-RNN, a method for fusing multi-term temporal features are proposed to extract the temporal dependence of different time scales, and the weighted fusion temporal feature is recalibrated by the attention mechanism. Ablation research proves that this network has powerful spatio-temporal dynamic modeling capabilities for actions with different time scales. We perform extensive experiments on four challenging benchmark datasets, including the NTU RGB+D dataset, UT-Kinect dataset, Northwestern-UCLA dataset, and UWA3DII dataset. Our method achieves better results than the state-of-the-art benchmarks, which demonstrates the effectiveness of MTANs.

scholarly journals InteractionNN: A Neural Network for Learning Hidden Features in Sparse Prediction

2019 ◽  
Author(s):  
Xiaowang Zhang ◽  
Qiang Gao ◽  
Zhiyong Feng
Keyword(s):  
Neural Network ◽  
Nonlinear Interaction ◽  
Feature Interaction ◽  
Feed Forward Neural Network ◽  
Low Level ◽  
Feature Interactions ◽  
Proposed Model ◽  
Benchmark Datasets ◽  
High Level ◽  
Dense Features

In this paper, we present a neural network (InteractionNN) for sparse predictive analysis where hidden features of sparse data can be learned by multilevel feature interaction. To characterize multilevel interaction of features, InteractionNN consists of three modules, namely, nonlinear interaction pooling, layer-lossing, and embedding. Nonlinear interaction pooling (NI pooling) is a hierarchical structure and, by shortcut connection, constructs low-level feature interactions from basic dense features to elementary features. Layer-lossing is a feed-forward neural network where high-level feature interactions can be learned from low-level feature interactions via correlation of all layers with target. Moreover, embedding is to extract basic dense features from sparse features of data which can help in reducing our proposed model computational complex. Finally, our experiment evaluates on the two benchmark datasets and the experimental results show that InteractionNN performs better than most of state-of-the-art models in sparse regression.

scholarly journals A transformer-based approach to irony and sarcasm detection

2020 ◽  
Vol 32 (23) ◽  
pp. 17309-17320
Author(s):  
Rolandos Alexandros Potamias ◽  
Georgios Siolas ◽  
Andreas - Georgios Stafylopatis
Keyword(s):  
Neural Network ◽  
Language Processing ◽  
Network Architecture ◽  
Figurative Language ◽  
State Of The Art ◽  
Unresolved Issue ◽  
Discussion Forums ◽  
Large Margin ◽  
Neural Architecture ◽  
Benchmark Datasets

AbstractFigurative language (FL) seems ubiquitous in all social media discussion forums and chats, posing extra challenges to sentiment analysis endeavors. Identification of FL schemas in short texts remains largely an unresolved issue in the broader field of natural language processing, mainly due to their contradictory and metaphorical meaning content. The main FL expression forms are sarcasm, irony and metaphor. In the present paper, we employ advanced deep learning methodologies to tackle the problem of identifying the aforementioned FL forms. Significantly extending our previous work (Potamias et al., in: International conference on engineering applications of neural networks, Springer, Berlin, pp 164–175, 2019), we propose a neural network methodology that builds on a recently proposed pre-trained transformer-based network architecture which is further enhanced with the employment and devise of a recurrent convolutional neural network. With this setup, data preprocessing is kept in minimum. The performance of the devised hybrid neural architecture is tested on four benchmark datasets, and contrasted with other relevant state-of-the-art methodologies and systems. Results demonstrate that the proposed methodology achieves state-of-the-art performance under all benchmark datasets, outperforming, even by a large margin, all other methodologies and published studies.

scholarly journals FLEX: Faithful Linguistic Explanations for Neural Net Based Model Decisions

2019 ◽  
Vol 33 ◽  
pp. 2539-2546
Author(s):  
Sandareka Wickramanayake ◽  
Wynne Hsu ◽  
Mong Li Lee
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
State Of The Art ◽  
Neural Net ◽  
End User ◽  
Learning Network ◽  
Benchmark Datasets ◽  
Deep Learning Network ◽  
Post Hoc ◽  
User Trust

Explaining the decisions of a Deep Learning Network is imperative to safeguard end-user trust. Such explanations must be intuitive, descriptive, and faithfully explain why a model makes its decisions. In this work, we propose a framework called FLEX (Faithful Linguistic EXplanations) that generates post-hoc linguistic justifications to rationalize the decision of a Convolutional Neural Network. FLEX explains a model’s decision in terms of features that are responsible for the decision. We derive a novel way to associate such features to words, and introduce a new decision-relevance metric that measures the faithfulness of an explanation to a model’s reasoning. Experiment results on two benchmark datasets demonstrate that the proposed framework can generate discriminative and faithful explanations compared to state-of-the-art explanation generators. We also show how FLEX can generate explanations for images of unseen classes as well as automatically annotate objects in images.

Neural Arabic Text Diacritization: State-of-the-Art Results and a Novel Approach for Arabic NLP Downstream Tasks

2022 ◽  
Vol 21 (1) ◽  
pp. 1-25
Author(s):  
Ali Fadel ◽  
Ibraheem Tuffaha ◽  
Mahmoud Al-Ayyoub
Keyword(s):  
Neural Network ◽  
State Of The Art ◽  
Conditional Random Field ◽  
Arabic Text ◽  
Learning Models ◽  
Post Processing ◽  
Feed Forward Neural Network ◽  
Novel Approach ◽  
Normalized Gradient ◽  
Processing Steps

In this work, we present several deep learning models for the automatic diacritization of Arabic text. Our models are built using two main approaches, viz. Feed-Forward Neural Network (FFNN) and Recurrent Neural Network (RNN), with several enhancements such as 100-hot encoding, embeddings, Conditional Random Field (CRF), and Block-Normalized Gradient (BNG). The models are tested on the only freely available benchmark dataset and the results show that our models are either better or on par with other models even those requiring human-crafted language-dependent post-processing steps, unlike ours. Moreover, we show how diacritics in Arabic can be used to enhance the models of downstream NLP tasks such as Machine Translation (MT) and Sentiment Analysis (SA) by proposing novel Translation over Diacritization (ToD) and Sentiment over Diacritization (SoD) approaches.

scholarly journals Exploiting Positional Information for Session-Based Recommendation

2022 ◽  
Vol 40 (2) ◽  
pp. 1-24
Author(s):  
Ruihong Qiu ◽  
Zi Huang ◽  
Tong Chen ◽  
Hongzhi Yin
Keyword(s):  
Neural Network ◽  
State Of The Art ◽  
Positional Information ◽  
Network Module ◽  
Historical Records ◽  
Encoding Scheme ◽  
Depth Analysis ◽  
Benchmark Datasets ◽  
Local Shift ◽  
Current Preference

For present e-commerce platforms, it is important to accurately predict users’ preference for a timely next-item recommendation. To achieve this goal, session-based recommender systems are developed, which are based on a sequence of the most recent user-item interactions to avoid the influence raised from outdated historical records. Although a session can usually reflect a user’s current preference, a local shift of the user’s intention within the session may still exist. Specifically, the interactions that take place in the early positions within a session generally indicate the user’s initial intention, while later interactions are more likely to represent the latest intention. Such positional information has been rarely considered in existing methods, which restricts their ability to capture the significance of interactions at different positions. To thoroughly exploit the positional information within a session, a theoretical framework is developed in this paper to provide an in-depth analysis of the positional information. We formally define the properties of forward-awareness and backward-awareness to evaluate the ability of positional encoding schemes in capturing the initial and the latest intention. According to our analysis, existing positional encoding schemes are generally forward-aware only, which can hardly represent the dynamics of the intention in a session. To enhance the positional encoding scheme for the session-based recommendation, a dual positional encoding (DPE) is proposed to account for both forward-awareness and backward-awareness . Based on DPE, we propose a novel Positional Recommender (PosRec) model with a well-designed Position-aware Gated Graph Neural Network module to fully exploit the positional information for session-based recommendation tasks. Extensive experiments are conducted on two e-commerce benchmark datasets, Yoochoose and Diginetica and the experimental results show the superiority of the PosRec by comparing it with the state-of-the-art session-based recommender models.

scholarly journals Transductive Relation-Propagation Network for Few-shot Learning

2020 ◽  
Author(s):  
Yuqing Ma ◽  
Shihao Bai ◽  
Shan An ◽  
Wei Liu ◽  
Aishan Liu ◽  
...  
Keyword(s):  
Neural Network ◽  
Learning Strategy ◽  
State Of The Art ◽  
Challenging Problem ◽  
Learning Problem ◽  
Graph Node ◽  
Learning Methods ◽  
Transductive Learning ◽  
Benchmark Datasets ◽  
Propagation Network

Few-shot learning, aiming to learn novel concepts from few labeled examples, is an interesting and very challenging problem with many practical advantages. To accomplish this task, one should concentrate on revealing the accurate relations of the support-query pairs. We propose a transductive relation-propagation graph neural network (TRPN) to explicitly model and propagate such relations across support-query pairs. Our TRPN treats the relation of each support-query pair as a graph node, named relational node, and resorts to the known relations between support samples, including both intra-class commonality and inter-class uniqueness, to guide the relation propagation in the graph, generating the discriminative relation embeddings for support-query pairs. A pseudo relational node is further introduced to propagate the query characteristics, and a fast, yet effective transductive learning strategy is devised to fully exploit the relation information among different queries. To the best of our knowledge, this is the first work that explicitly takes the relations of support-query pairs into consideration in few-shot learning, which might offer a new way to solve the few-shot learning problem. Extensive experiments conducted on several benchmark datasets demonstrate that our method can significantly outperform a variety of state-of-the-art few-shot learning methods.

scholarly journals k-Nearest Neighbors by Means of Sequence to Sequence Deep Neural Networks and Memory Networks

2021 ◽  
Author(s):  
Yiming Xu ◽  
Diego Klabjan
Keyword(s):  
Neural Network ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
Sampling Technique ◽  
Nearest Neighbors ◽  
Classification Models ◽  
Feed Forward Neural Network ◽  
K Nearest Neighbors ◽  
Out Of Sample ◽  
Memory Network

k-Nearest Neighbors is one of the most fundamental but effective classification models. In this paper, we propose two families of models built on a sequence to sequence model and a memory network model to mimic the k-Nearest Neighbors model, which generate a sequence of labels, a sequence of out-of-sample feature vectors and a final label for classification, and thus they could also function as oversamplers. We also propose 'out-of-core' versions of our models which assume that only a small portion of data can be loaded into memory. Computational experiments show that our models on structured datasets outperform k-Nearest Neighbors, a feed-forward neural network, XGBoost, lightGBM, random forest and a memory network, due to the fact that our models must produce additional output and not just the label. On image and text datasets, the performance of our model is close to many state-of-the-art deep models. As an oversampler on imbalanced datasets, the sequence to sequence kNN model often outperforms Synthetic Minority Over-sampling Technique and Adaptive Synthetic Sampling.

Sign in / Sign up

Export Citation Format

Share Document