HSRec: Hierarchical self-attention incorporating knowledge graph for sequential recommendation

2021 ◽  
pp. 1-12
Author(s):  
Zuoxi Yang ◽  
Shoubin Dong
Keyword(s):  
Semantic Information ◽  
High Order ◽  
Attention Mechanism ◽  
Sequential Patterns ◽  
Knowledge Graph ◽  
Intrinsic Interest ◽  
Fine Grained ◽  
Potential Interest ◽  
Preference Evolution ◽  
Real World Datasets

Modeling user’s fine-grained preferences and dynamic preference evolution from their chronological behaviors are challenging and crucial for sequential recommendation. In this paper, we develop a Hierarchical Self-Attention Incorporating Knowledge Graph for Sequential Recommendation (HSRec). HSRec models not only the user’s intrinsic preferences but also the user’s external potential interests to capture the user’s fine-grained preferences. Specifically, the intrinsic interest module and potential interest module are designed to capture these two preferences respectively. In the intrinsic interest module, user’s sequential patterns are characterized from their behaviors via the self-attention mechanism. As for the potential interest module, high-order paths can be generated with the help of the knowledge graph. Therefore, a hierarchical self-attention mechanism is designed to aggregate the semantic information of user interaction from these paths. Specifically, an entity-level self-attention mechanism is applied to capture the sequential patterns contained in the high-order paths while an interaction-level self-attention mechanism is designed to further capture the semantic information from user interactions. Moreover, according to the high-order semantic relevance, HSRec can explore the user’s dynamic preferences at each time, thus describing the user’s dynamic preference evolution. Finally, experiments conducted on three real world datasets demonstrate the state-of-the-art performance of the HSRec.

scholarly journals An Attention-Based Latent Information Extraction Network (ALIEN) for High-Order Feature Interactions

2020 ◽  
Vol 10 (16) ◽  
pp. 5468
Author(s):  
Ruo Huang ◽  
Shelby McIntyre ◽  
Meina Song ◽  
Haihong E ◽  
Zhonghong Ou
Keyword(s):  
Information Extraction ◽  
Recommender Systems ◽  
Vital Role ◽  
High Order ◽  
Sequential Patterns ◽  
Sequence Information ◽  
Recommendation Algorithm ◽  
Feature Interactions ◽  
Real World Datasets ◽  
Click Through Rate

One of the primary tasks for commercial recommender systems is to predict the probabilities of users clicking items, e.g., advertisements, music and products. This is because such predictions have a decisive impact on profitability. The classic recommendation algorithm, collaborative filtering (CF), still plays a vital role in many industrial recommender systems. However, although straight CF is good at capturing similar users’ preferences for items based on their past interactions, it lacks regarding (1) modeling the influences of users’ sequential patterns from their individual history interaction sequences and (2) the relevance of users’ and items’ attributes. In this work, we developed an attention-based latent information extraction network (ALIEN) for click-through rate prediction, to integrate (1) implicit user similarity in terms of click patterns (analogous to CF), and (2) modeling the low and high-order feature interactions and (3) historical sequence information. The new model is based on the deep learning, which goes beyond the capabilities of econometric approaches, such as matrix factorization (MF) and k-means. In addition, the approach provides explainability to the recommendation by interpreting the contributions of different features and historical interactions. We have conducted experiments on real-world datasets that demonstrate considerable improvements over strong baselines.

scholarly journals Multi-Layer Attention Approach for Aspect based Sentiment Analysis

2020 ◽  
Author(s):  
Xinzhi Ai ◽  
Xiaoge Li ◽  
Feixiong Hu ◽  
Shuting Zhi ◽  
Likun Hu
Keyword(s):  
Sentiment Analysis ◽  
Semantic Information ◽  
Short Term Memory ◽  
Attention Mechanism ◽  
Training Dataset ◽  
Emotion Classification ◽  
Data Set ◽  
New Model ◽  
Fine Grained ◽  
Proposed Model

Based on the aspect-level sentiment analysis is typical of fine-grained emotional classification that assigns sentiment polarity for each of the aspects in a review. For better handle the emotion classification task, this paper put forward a new model which apply Long Short-Term Memory network combine multiple attention with aspect context. Where multiple attention mechanism (i.e., location attention, content attention and class attention) refers to takes the factors of context location, content semantics and class balancing into consideration. Therefore, the proposed model can adaptively integrate location and semantic information between the aspect targets and their contexts into sentimental features, and overcome the model data variance introduced by the imbalanced training dataset. In addition, the aspect context is encoded on both sides of the aspect target, so as to enhance the ability of the model to capture semantic information. The Multi-Attention mechanism (MATT) and Aspect Context (AC) allow our model to perform better when facing reviews with more complicated structures. The result of this experiment indicate that the accuracy of the new model is up to 80.6% and 75.1% for two datasets in SemEval-2014 Task 4 respectively, While the accuracy of the data set on twitter 71.1%, and 81.6% for the Chinese automotive-domain dataset. Compared with some previous models for sentiment analysis, our model shows a higher accuracy.

scholarly journals TransET: Knowledge Graph Embedding with Entity Types

Electronics ◽  
2021 ◽  
Vol 10 (12) ◽  
pp. 1407
Author(s):  
Peng Wang ◽  
Jing Zhou ◽  
Yuzhang Liu ◽  
Xingchen Zhou
Keyword(s):  
Link Prediction ◽  
State Of The Art ◽  
Score Function ◽  
Graph Embedding ◽  
Vector Spaces ◽  
Knowledge Graph ◽  
Semantic Features ◽  
Knowledge Graphs ◽  
Real World Datasets ◽  
Low Dimensional

Knowledge graph embedding aims to embed entities and relations into low-dimensional vector spaces. Most existing methods only focus on triple facts in knowledge graphs. In addition, models based on translation or distance measurement cannot fully represent complex relations. As well-constructed prior knowledge, entity types can be employed to learn the representations of entities and relations. In this paper, we propose a novel knowledge graph embedding model named TransET, which takes advantage of entity types to learn more semantic features. More specifically, circle convolution based on the embeddings of entity and entity types is utilized to map head entity and tail entity to type-specific representations, then translation-based score function is used to learn the presentation triples. We evaluated our model on real-world datasets with two benchmark tasks of link prediction and triple classification. Experimental results demonstrate that it outperforms state-of-the-art models in most cases.

A deep embedding model for knowledge graph completion based on attention mechanism

2021 ◽  
Author(s):  
Jin Huang ◽  
TingHua Zhang ◽  
Jia Zhu ◽  
Weihao Yu ◽  
Yong Tang ◽  
...  
Keyword(s):  
Attention Mechanism ◽  
Knowledge Graph ◽  
Deep Embedding

Semi-Supervised Aspect-Based Sentiment Analysis for Case-Related Microblog Reviews Using Case Knowledge Graph Embedding

2021 ◽  
pp. 2050012
Author(s):  
Peilian Zhao ◽  
Cunli Mao ◽  
Zhengtao Yu
Keyword(s):  
Sentiment Analysis ◽  
Domain Knowledge ◽  
Opinion Mining ◽  
Data Augmentation ◽  
Training Data ◽  
Knowledge Graph ◽  
Fine Grained ◽  
Learning Framework ◽  
Proposed Model ◽  
Real World Applications

Aspect-Based Sentiment Analysis (ABSA), a fine-grained task of opinion mining, which aims to extract sentiment of specific target from text, is an important task in many real-world applications, especially in the legal field. Therefore, in this paper, we study the problem of limitation of labeled training data required and ignorance of in-domain knowledge representation for End-to-End Aspect-Based Sentiment Analysis (E2E-ABSA) in legal field. We proposed a new method under deep learning framework, named Semi-ETEKGs, which applied E2E framework using knowledge graph (KG) embedding in legal field after data augmentation (DA). Specifically, we pre-trained the BERT embedding and in-domain KG embedding for unlabeled data and labeled data with case elements after DA, and then we put two embeddings into the E2E framework to classify the polarity of target-entity. Finally, we built a case-related dataset based on a popular benchmark for ABSA to prove the efficiency of Semi-ETEKGs, and experiments on case-related dataset from microblog comments show that our proposed model outperforms the other compared methods significantly.

Fine-Grained Evaluation of Rule- and Embedding-Based Systems for Knowledge Graph Completion

2018 ◽  
pp. 3-20 ◽  
Author(s):  
Christian Meilicke ◽  
Manuel Fink ◽  
Yanjie Wang ◽  
Daniel Ruffinelli ◽  
Rainer Gemulla ◽  
...  
Keyword(s):  
Knowledge Graph ◽  
Fine Grained

scholarly journals Aspect-level sentiment analysis merged with knowledge graph and graph convolutional neural network

2021 ◽  
Vol 2083 (4) ◽  
pp. 042044
Author(s):  
Zuhua Dai ◽  
Yuanyuan Liu ◽  
Shilong Di ◽  
Qi Fan
Keyword(s):  
Neural Network ◽  
Sentiment Analysis ◽  
Structural Information ◽  
Knowledge Graph ◽  
Convolutional Network ◽  
Text Data ◽  
Short Text ◽  
Fine Grained ◽  
Syntactic Information ◽  
Text Information

Abstract Aspect level sentiment analysis belongs to fine-grained sentiment analysis, w hich has caused extensive research in academic circles in recent years. For this task, th e recurrent neural network (RNN) model is usually used for feature extraction, but the model cannot effectively obtain the structural information of the text. Recent studies h ave begun to use the graph convolutional network (GCN) to model the syntactic depen dency tree of the text to solve this problem. For short text data, the text information is not enough to accurately determine the emotional polarity of the aspect words, and the knowledge graph is not effectively used as external knowledge that can enrich the sem antic information. In order to solve the above problems, this paper proposes a graph co nvolutional neural network (GCN) model that can process syntactic information, know ledge graphs and text semantic information. The model works on the “syntax-knowled ge” graph to extract syntactic information and common sense information at the same t ime. Compared with the latest model, the model in this paper can effectively improve t he accuracy of aspect-level sentiment classification on two datasets.

scholarly journals Correction: Constructing High-Fidelity Phenotype Knowledge Graphs for Infectious Diseases With a Fine-Grained Semantic Information Model: Development and Usability Study (Preprint)

2021 ◽  
Author(s):  
Lizong Deng ◽  
Luming Chen ◽  
Tao Yang ◽  
Mi Liu ◽  
Shicheng Li ◽  
...  
Keyword(s):  
Infectious Diseases ◽  
Semantic Information ◽  
Model Development ◽  
Information Model ◽  
High Fidelity ◽  
Usability Study ◽  
Systems Medicine ◽  
Medical Sciences ◽  
Fine Grained ◽  
Knowledge Graphs

UNSTRUCTURED In “Constructing High-Fidelity Phenotype Knowledge Graphs for Infectious Diseases With a Fine-Grained Semantic Information Model: Development and Usability Study” (J Med Internet Res 2021;23(6):e26892) the authors noted one error. The institution name of affiliation “Suzhou Institute of Systems Medicine” was not correct. It should be corrected from “Suzhou Institute of Systems Medicine” to “Center of Systems Medicine, Institute of Basic Medical Sciences, Chinese Academy of Medical Sciences & Peking Union Medical College; Suzhou Institute of Systems Medicine”

scholarly journals Network Embedding via a Bi-Mode and Deep Neural Network Model

2017 ◽  
Author(s):  
Yang Fang ◽  
Xiang Zhao ◽  
Zhen Tan
Keyword(s):  
Neural Network ◽  
Deep Neural Network ◽  
Semantic Information ◽  
Dimensional Space ◽  
Relation Extraction ◽  
Network Embedding ◽  
Structure Information ◽  
Second Mode ◽  
Real World Datasets ◽  
Low Dimensional

Network Embedding (NE) is an important method to learn the representations of network via a low-dimensional space. Conventional NE models focus on capturing the structure information and semantic information of vertices while neglecting such information for edges. In this work, we propose a novel NE model named BimoNet to capture both the structure and semantic information of edges. BimoNet is composed of two parts, i.e., the bi-mode embedding part and the deep neural network part. For bi-mode embedding part, the first mode named add-mode is used to express the entity-shared features of edges and the second mode named subtract-mode is employed to represent the entity-specific features of edges. These features actually reflect the semantic information. For deep neural network part, we firstly regard the edges in a network as nodes, and the vertices as links, which will not change the overall structure of the whole network. Then we take the nodes' adjacent matrix as the input of the deep neural network as it can obtain similar representations for nodes with similar structure. Afterwards, by jointly optimizing the objective function of these two parts, BimoNet could preserve both the semantic and structure information of edges. In experiments, we evaluate BimoNet on three real-world datasets and task of relation extraction, and BimoNet is demonstrated to outperform state-of-the-art baseline models consistently and significantly.

Sign in / Sign up

Export Citation Format

Share Document