scholarly journals A Novel Negative Sampling Based on Frequency of Relational Association Entities for Knowledge Graph Embedding

2021 ◽  
Author(s):  
Wanhua Cao ◽  
Yi Zhang ◽  
Juntao Liu ◽  
Ziyun Rao
Keyword(s):  
Link Prediction ◽  
State Of The Art ◽  
Evaluation Criteria ◽  
Relation Extraction ◽  
Graph Embedding ◽  
Semantic Space ◽  
Knowledge Graph ◽  
Knowledge Reasoning ◽  
Relation Prediction ◽  
Low Dimensional

Knowledge graph embedding improves the performance of relation extraction and knowledge reasoning by encoding entities and relationships in low-dimensional semantic space. During training, negative samples are usually constructed by replacing the head/tail entity. And the different replacing relationships lead to different accuracy of the prediction results. This paper develops a negative triplets construction framework according to the frequency of relational association entities. The proposed construction framework can fully consider the quantitative of relations and entities in the dataset to assign the proportion of relation and entity replacement and the frequency of the entities associated with each relationship to set reasonable proportions for different relations. To verify the validity of the proposed construction framework, it is integrated into the state-of-the-art knowledge graph embedding models, such as TransE, TransH, DistMult, ComplEx, and Analogy. And both the evaluation criteria of relation prediction and entity prediction are used to evaluate the performance of link prediction more comprehensively. The experimental results on two commonly used datasets, WN18 and FB15K, show that the proposed method improves entity link and triplet classification accuracy, especially the accuracy of relational link prediction.

scholarly journals TransET: Knowledge Graph Embedding with Entity Types

Electronics ◽  
2021 ◽  
Vol 10 (12) ◽  
pp. 1407
Author(s):  
Peng Wang ◽  
Jing Zhou ◽  
Yuzhang Liu ◽  
Xingchen Zhou
Keyword(s):  
Link Prediction ◽  
State Of The Art ◽  
Score Function ◽  
Graph Embedding ◽  
Vector Spaces ◽  
Knowledge Graph ◽  
Semantic Features ◽  
Knowledge Graphs ◽  
Real World Datasets ◽  
Low Dimensional

Knowledge graph embedding aims to embed entities and relations into low-dimensional vector spaces. Most existing methods only focus on triple facts in knowledge graphs. In addition, models based on translation or distance measurement cannot fully represent complex relations. As well-constructed prior knowledge, entity types can be employed to learn the representations of entities and relations. In this paper, we propose a novel knowledge graph embedding model named TransET, which takes advantage of entity types to learn more semantic features. More specifically, circle convolution based on the embeddings of entity and entity types is utilized to map head entity and tail entity to type-specific representations, then translation-based score function is used to learn the presentation triples. We evaluated our model on real-world datasets with two benchmark tasks of link prediction and triple classification. Experimental results demonstrate that it outperforms state-of-the-art models in most cases.

scholarly journals A Co-Embedding Model with Variational Auto-Encoder for Knowledge Graphs

2022 ◽  
Vol 12 (2) ◽  
pp. 715
Author(s):  
Luodi Xie ◽  
Huimin Huang ◽  
Qing Du
Keyword(s):  
State Of The Art ◽  
Relation Extraction ◽  
Semantic Space ◽  
Knowledge Graph ◽  
High Quality ◽  
Gaussian Distributions ◽  
Benchmark Datasets ◽  
Semantic Spaces ◽  
Knowledge Graphs ◽  
Low Dimensional

Knowledge graph (KG) embedding has been widely studied to obtain low-dimensional representations for entities and relations. It serves as the basis for downstream tasks, such as KG completion and relation extraction. Traditional KG embedding techniques usually represent entities/relations as vectors or tensors, mapping them in different semantic spaces and ignoring the uncertainties. The affinities between entities and relations are ambiguous when they are not embedded in the same latent spaces. In this paper, we incorporate a co-embedding model for KG embedding, which learns low-dimensional representations of both entities and relations in the same semantic space. To address the issue of neglecting uncertainty for KG components, we propose a variational auto-encoder that represents KG components as Gaussian distributions. In addition, compared with previous methods, our method has the advantages of high quality and interpretability. Our experimental results on several benchmark datasets demonstrate our model’s superiority over the state-of-the-art baselines.

scholarly journals Understanding Negative Sampling in Knowledge Graph Embedding

2021 ◽  
Vol 12 (1) ◽  
pp. 71-81
Author(s):  
Jing Qian ◽  
Gangmin Li ◽  
Katie Atkinson ◽  
Yong Yue
Keyword(s):  
Link Prediction ◽  
Graph Embedding ◽  
Knowledge Graph ◽  
Direct Impact ◽  
Dimensional Vector Space ◽  
Dynamic Distribution ◽  
Space Efficiency ◽  
Node Classification ◽  
Low Dimensional

Knowledge graph embedding (KGE) is to project entities and relations of a knowledge graph (KG) into a low-dimensional vector space, which has made steady progress in recent years. Conventional KGE methods, especially translational distance-based models, are trained through discriminating positive samples from negative ones. Most KGs store only positive samples for space efficiency. Negative sampling thus plays a crucial role in encoding triples of a KG. The quality of generated negative samples has a direct impact on the performance of learnt knowledge representation in a myriad of downstream tasks, such as recommendation, link prediction and node classification. We summarize current negative sampling approaches in KGE into three categories, static distribution-based, dynamic distribution-based and custom cluster-based respectively. Based on this categorization we discuss the most prevalent existing approaches and their characteristics. It is a hope that this review can provide some guidelines for new thoughts about negative sampling in KGE.

scholarly journals Bootstrapping Entity Alignment with Knowledge Graph Embedding

2018 ◽  
Author(s):  
Zequn Sun ◽  
Wei Hu ◽  
Qingheng Zhang ◽  
Yuzhong Qu
Keyword(s):  
Performance Improvement ◽  
Real World ◽  
State Of The Art ◽  
Graph Embedding ◽  
Training Data ◽  
Knowledge Graph ◽  
Error Accumulation ◽  
Knowledge Graphs ◽  
Real World Datasets ◽  
Low Dimensional

Embedding-based entity alignment represents different knowledge graphs (KGs) as low-dimensional embeddings and finds entity alignment by measuring the similarities between entity embeddings. Existing approaches have achieved promising results, however, they are still challenged by the lack of enough prior alignment as labeled training data. In this paper, we propose a bootstrapping approach to embedding-based entity alignment. It iteratively labels likely entity alignment as training data for learning alignment-oriented KG embeddings. Furthermore, it employs an alignment editing method to reduce error accumulation during iterations. Our experiments on real-world datasets showed that the proposed approach significantly outperformed the state-of-the-art embedding-based ones for entity alignment. The proposed alignment-oriented KG embedding, bootstrapping process and alignment editing method all contributed to the performance improvement.

A quaternion-group knowledge graph embedding model

2021 ◽  
pp. 1-10
Author(s):  
Heng Chen ◽  
Guanyu Li ◽  
Yunhao Sun ◽  
Wei Jiang
Keyword(s):  
Link Prediction ◽  
State Of The Art ◽  
Graph Embedding ◽  
Knowledge Graph ◽  
Complex Vector ◽  
Quaternion Group ◽  
Rotation Matrices ◽  
Group Knowledge ◽  
Benchmark Datasets ◽  
Complete Relation

Capturing the composite embedding representation of a multi-hop relation path is an extremely vital task in knowledge graph completion. Recently, rotation-based relation embedding models have been widely studied to embed composite relations into complex vector space. However, these models make some over-simplified assumptions on the composite relations, resulting the relations to be commutative. To tackle this problem, this paper proposes a novel knowledge graph embedding model, named QuatGE, which can provide sufficient modeling capabilities for complex composite relations. In particular, our method models each relation as a rotation operator in quaternion group-based space. The advantages of our model are twofold: (1) Since the quaternion group is a non-commutative group (i.e., non-Abelian group), the corresponding rotation matrices of composite relations can be non-commutative; (2) The model has a more expressive setting with stronger modeling capabilities, which is flexible to model and infer the complete relation patterns, including: symmetry/anti-symmetry, inversion and commutative/non-commutative composition. Experimental results on four benchmark datasets show that the proposed method outperforms the existing state-of-the-art models for link prediction, especially on composite relations.

scholarly journals ParamE: Regarding Neural Network Parameters as Relation Embeddings for Knowledge Graph Completion

2020 ◽  
Vol 34 (03) ◽  
pp. 2774-2781
Author(s):  
Feihu Che ◽  
Dawei Zhang ◽  
Jianhua Tao ◽  
Mingyue Niu ◽  
Bocheng Zhao
Keyword(s):  
Neural Network ◽  
Link Prediction ◽  
State Of The Art ◽  
Graph Embedding ◽  
Feature Space ◽  
Knowledge Graph ◽  
Network Parameters ◽  
New Knowledge ◽  
Fitting Ability ◽  
Translational Models

We study the task of learning entity and relation embeddings in knowledge graphs for predicting missing links. Previous translational models on link prediction make use of translational properties but lack enough expressiveness, while the convolution neural network based model (ConvE) takes advantage of the great nonlinearity fitting ability of neural networks but overlooks translational properties. In this paper, we propose a new knowledge graph embedding model called ParamE which can utilize the two advantages together. In ParamE, head entity embeddings, relation embeddings and tail entity embeddings are regarded as the input, parameters and output of a neural network respectively. Since parameters in networks are effective in converting input to output, taking neural network parameters as relation embeddings makes ParamE much more expressive and translational. In addition, the entity and relation embeddings in ParamE are from feature space and parameter space respectively, which is in line with the essence that entities and relations are supposed to be mapped into two different spaces. We evaluate the performances of ParamE on standard FB15k-237 and WN18RR datasets, and experiments show ParamE can significantly outperform existing state-of-the-art models, such as ConvE, SACN, RotatE and D4-STE/Gumbel.

scholarly journals Improving Distant Supervised Relation Extraction with Noise Detection Strategy

2021 ◽  
Vol 11 (5) ◽  
pp. 2046
Author(s):  
Xiaoyan Meng ◽  
Tonghai Jiang ◽  
Xi Zhou ◽  
Bo Ma ◽  
Yi Wang ◽  
...  
Keyword(s):  
State Of The Art ◽  
Relation Extraction ◽  
Knowledge Graph ◽  
Noise Detection ◽  
Plain Text ◽  
Distant Supervision ◽  
Detection Strategy ◽  
Relation Prediction ◽  
Noisy Labels ◽  
The Web

Distant supervised relation extraction (DSRE) is widely used to extract novel relational facts from plain text, so as to improve the knowledge graph. However, distant supervision inevitably suffers from the noisy labeling problem that will severely damage the performance of relation extraction. Currently, most DSRE methods are mainly focused on reducing the weights of noisy sentences, ignoring the bag-level noise where all sentences in a bag are wrongly labeled. In this paper, we present a novel noise detection-based relation extraction approach (NDRE) to automatically detect noisy labels with entity information and dynamically correct them, which can alleviate both instance-level and bag-level noisy problems. By this means, we can extend the dataset from the Web tables without introducing more noise. In this approach, to embed the semantics of sentences from corpus and web tables, we firstly propose a powerful sentence coder that employs an internal multi-head self-attention mechanism between the piecewise max-pooling convolutional neural network. Second, we adopt a noise detection strategy, which is expected to dynamically detect and correct the original noisy label according to the similarity between sentence representation and entity-aware embeddings. Then, we aggregate the information from corpus and web tables to make the final relation prediction. Experimental results on a public benchmark dataset demonstrate that our proposed approach achieves significant improvements over the state-of-the-art baselines and can effectively reduce the noisy labeling problem.

scholarly journals Persona2vec: a flexible multi-role representations learning framework for graphs

2021 ◽  
Vol 7 ◽  
pp. e439
Author(s):  
Jisung Yoon ◽  
Kai-Cheng Yang ◽  
Woo-Sung Jung ◽  
Yong-Yeol Ahn
Keyword(s):  
Community Structure ◽  
Link Prediction ◽  
Graph Mining ◽  
State Of The Art ◽  
Multiple Representations ◽  
Graph Embedding ◽  
Learning Framework ◽  
Overlapping Community ◽  
Art Performance ◽  
Low Dimensional

Graph embedding techniques, which learn low-dimensional representations of a graph, are achieving state-of-the-art performance in many graph mining tasks. Most existing embedding algorithms assign a single vector to each node, implicitly assuming that a single representation is enough to capture all characteristics of the node. However, across many domains, it is common to observe pervasively overlapping community structure, where most nodes belong to multiple communities, playing different roles depending on the contexts. Here, we propose persona2vec, a graph embedding framework that efficiently learns multiple representations of nodes based on their structural contexts. Using link prediction-based evaluation, we show that our framework is significantly faster than the existing state-of-the-art model while achieving better performance.

scholarly journals How much topological structure is preserved by graph embeddings?

2019 ◽  
Vol 16 (2) ◽  
pp. 597-614 ◽  
Author(s):  
Xin Liu ◽  
Chenyi Zhuang ◽  
Tsuyoshi Murata ◽  
Kyoung-Sook Kim ◽  
Natthawut Kertkeidkachorn
Keyword(s):  
Topological Structure ◽  
Link Prediction ◽  
State Of The Art ◽  
Graph Embedding ◽  
Graph Embeddings ◽  
Dimensional Vector ◽  
Original Graph ◽  
Dimensional Vector Space ◽  
Low Dimensional ◽  
Link Distribution

Graph embedding aims at learning representations of nodes in a low dimensional vector space. Good embeddings should preserve the graph topological structure. To study how much such structure can be preserved, we propose evaluation methods from four aspects: 1) How well the graph can be reconstructed based on the embeddings, 2) The divergence of the original link distribution and the embedding-derived distribution, 3) The consistency of communities discovered from the graph and embeddings, and 4) To what extent we can employ embeddings to facilitate link prediction. We find that it is insufficient to rely on the embeddings to reconstruct the original graph, to discover communities, and to predict links at a high precision. Thus, the embeddings by the state-of-the-art approaches can only preserve part of the topological structure.

scholarly journals Dual Quaternion Embeddings for Link Prediction

2021 ◽  
Vol 11 (12) ◽  
pp. 5572
Author(s):  
Liming Gao ◽  
Huiling Zhu ◽  
Hankz Hankui Zhuo ◽  
Jin Xu 
Keyword(s):  
Artificial Intelligence ◽  
Link Prediction ◽  
State Of The Art ◽  
Knowledge Graph ◽  
Dual Quaternion ◽  
Novel Approach ◽  
Quaternion Space ◽  
Knowledge Graphs ◽  
Low Dimensional

The applications of knowledge graph have received much attention in the field of artificial intelligence. The quality of knowledge graphs is, however, often influenced by missing facts. To predict the missing facts, various solid transformation based models have been proposed by mapping knowledge graphs into low dimensional spaces. However, most of the existing transformation based approaches ignore that there are multiple relations between two entities, which is common in the real world. In order to address this challenge, we propose a novel approach called DualQuatE that maps entities and relations into a dual quaternion space. Specifically, entities are represented by pure quaternions and relations are modeled based on the combination of rotation and translation from head to tail entities. After that we utilize interactions of different translations and rotations to distinguish various relations between head and tail entities. Experimental results exhibit that the performance of DualQuatE is competitive compared to the existing state-of-the-art models.

Sign in / Sign up

Export Citation Format

Share Document