scholarly journals Contextual Parameter Generation for Knowledge Graph Link Prediction

2020 ◽  
Vol 34 (03) ◽  
pp. 3000-3008
Author(s):  
George Stoica ◽  
Otilia Stretcu ◽  
Emmanouil Antonios Platanios ◽  
Tom Mitchell ◽  
Barnabás Póczos
Keyword(s):  
Link Prediction ◽  
State Of The Art ◽  
Linear Functions ◽  
Convergence Time ◽  
Knowledge Graph ◽  
Complex Interactions ◽  
Significant Performance ◽  
Representational Power ◽  
Graph Link ◽  
Contextual Parameter

We consider the task of knowledge graph link prediction. Given a question consisting of a source entity and a relation (e.g., Shakespeare and BornIn), the objective is to predict the most likely answer entity (e.g., England). Recent approaches tackle this problem by learning entity and relation embeddings. However, they often constrain the relationship between these embeddings to be additive (i.e., the embeddings are concatenated and then processed by a sequence of linear functions and element-wise non-linearities). We show that this type of interaction significantly limits representational power. For example, such models cannot handle cases where a different projection of the source entity is used for each relation. We propose to use contextual parameter generation to address this limitation. More specifically, we treat relations as the context in which source entities are processed to produce predictions, by using relation embeddings to generate the parameters of a model operating over source entity embeddings. This allows models to represent more complex interactions between entities and relations. We apply our method on two existing link prediction methods, including the current state-of-the-art, resulting in significant performance gains and establishing a new state-of-the-art for this task. These gains are achieved while also reducing convergence time by up to 28 times.

scholarly journals Neighborhood Intervention Consistency: Measuring Confidence for Knowledge Graph Link Prediction

2021 ◽  
Author(s):  
Kai Wang ◽  
Yu Liu ◽  
Quan Z. Sheng
Keyword(s):  
Link Prediction ◽  
State Of The Art ◽  
Confidence Score ◽  
Measurement Methods ◽  
Knowledge Graph ◽  
Critical Gap ◽  
Optimal Score ◽  
Causal Intervention ◽  
Graph Link ◽  
Confidence Measurement

Link prediction based on knowledge graph embeddings (KGE) has recently drawn a considerable momentum. However, existing KGE models suffer from insufficient accuracy and hardly evaluate the confidence probability of each predicted triple. To fill this critical gap, we propose a novel confidence measurement method based on causal intervention, called Neighborhood Intervention Consistency (NIC). Unlike previous confidence measurement methods that focus on the optimal score in a prediction, NIC actively intervenes in the input entity vector to measure the robustness of the prediction result. The experimental results on ten popular KGE models show that our NIC method can effectively estimate the confidence score of each predicted triple. The top 10% triples with high NIC confidence can achieve 30% higher accuracy in the state-of-the-art KGE models.

scholarly journals TransET: Knowledge Graph Embedding with Entity Types

Electronics ◽  
2021 ◽  
Vol 10 (12) ◽  
pp. 1407
Author(s):  
Peng Wang ◽  
Jing Zhou ◽  
Yuzhang Liu ◽  
Xingchen Zhou
Keyword(s):  
Link Prediction ◽  
State Of The Art ◽  
Score Function ◽  
Graph Embedding ◽  
Vector Spaces ◽  
Knowledge Graph ◽  
Semantic Features ◽  
Knowledge Graphs ◽  
Real World Datasets ◽  
Low Dimensional

Knowledge graph embedding aims to embed entities and relations into low-dimensional vector spaces. Most existing methods only focus on triple facts in knowledge graphs. In addition, models based on translation or distance measurement cannot fully represent complex relations. As well-constructed prior knowledge, entity types can be employed to learn the representations of entities and relations. In this paper, we propose a novel knowledge graph embedding model named TransET, which takes advantage of entity types to learn more semantic features. More specifically, circle convolution based on the embeddings of entity and entity types is utilized to map head entity and tail entity to type-specific representations, then translation-based score function is used to learn the presentation triples. We evaluated our model on real-world datasets with two benchmark tasks of link prediction and triple classification. Experimental results demonstrate that it outperforms state-of-the-art models in most cases.

scholarly journals Enhanced Knowledge Graph Embedding by Jointly Learning Soft Rules and Facts

Algorithms ◽  
2019 ◽  
Vol 12 (12) ◽  
pp. 265 ◽  
Author(s):  
Jindou Zhang ◽  
Jing Li
Keyword(s):  
Link Prediction ◽  
State Of The Art ◽  
Background Knowledge ◽  
Background Information ◽  
Knowledge Graph ◽  
Logical Inference ◽  
Training Algorithm ◽  
First Order ◽  
Forward Chaining ◽  
Logical Rules

Combining first order logic rules with a Knowledge Graph (KG) embedding model has recently gained increasing attention, as rules introduce rich background information. Among such studies, models equipped with soft rules, which are extracted with certain confidences, achieve state-of-the-art performance. However, the existing methods either cannot support the transitivity and composition rules or take soft rules as regularization terms to constrain derived facts, which is incapable of encoding the logical background knowledge about facts contained in soft rules. In addition, previous works performed one time logical inference over rules to generate valid groundings for modeling rules, ignoring forward chaining inference, which can further generate more valid groundings to better model rules. To these ends, this paper proposes Soft Logical rules enhanced Embedding (SoLE), a novel KG embedding model equipped with a joint training algorithm over soft rules and KG facts to inject the logical background knowledge of rules into embeddings, as well as forward chaining inference over rules. Evaluations on Freebase and DBpedia show that SoLE not only achieves improvements of 11.6%/5.9% in Mean Reciprocal Rank (MRR) and 18.4%/15.9% in HITS@1 compared to the model on which SoLE is based, but also significantly and consistently outperforms the state-of-the-art baselines in the link prediction task.

Complex-InversE: Improving bilinear knowledge graph embeddings by mapping into complex space

2021 ◽  
pp. 1-11
Author(s):  
Yukun Cao ◽  
Zeyu Miao
Keyword(s):  
Link Prediction ◽  
Complex Space ◽  
Prediction Method ◽  
Knowledge Graph ◽  
The Public ◽  
Low Dimensional ◽  
Link Information ◽  
Prediction Problems ◽  
Benchmark Evaluation ◽  
Graph Link

Knowledge graph link prediction uses known fact links to infer the missing link information in the knowledge graph, which is of great significance to the completion of the knowledge graph. Generating low-dimensional embeddings of entities and relations which are used to make inferences is a popular way for such link prediction problems. This paper proposes a knowledge graph link prediction method called Complex-InversE in the complex space, which maps entities and relations into the complex space. The composition of complex embeddings can handle a large variety of binary relations, among them symmetric and antisymmetric relations. The Complex-InversE effectively captures the antisymmetric relations and introduces Dropout and Early-Stopping technologies into deal with the problem of small numbers of relationships and entities, thus effectively alleviates the model’s overfitting. The results of comparison experiment on the public knowledge graph datasets show that the Complex-InversE achieves good results on multiple benchmark evaluation indicators and outperforms previous methods. Complex-InversE’s code is available on GitHub at https://github.com/ZeyuMiao97/Complex-InversE.

The research of clinical temporal knowledge graph based on depth learning

2021 ◽  
pp. 1-10
Author(s):  
Lijuan Diao ◽  
Shoujun Song ◽  
Gaofang Cao ◽  
Yang Kong
Keyword(s):  
Deep Learning ◽  
Knowledge Base ◽  
Link Prediction ◽  
Prediction Models ◽  
Temporal Characteristic ◽  
Diabetic Complication ◽  
Transformation Model ◽  
Sequential Learning ◽  
Knowledge Graph ◽  
Graph Link

Temporal knowledge base exists on various fields. Take medical medicine field as example, diabetes is a typical chronic disease which evolves slowly. This paper starts from actual EMR data of hospitals by combination of experience and knowledge of clinical doctors. Link prediction on clinical knowledge base such as diabetic complication requires the analysis on temporal characteristic of temporal knowledge base, which is a great challenge for traditional link prediction models. This paper proposes temporal knowledge graph link prediction model based on deep learning. This model selects the TransR transformation model suitable for big data and makes entity projection in relation space containing different semantic meanings, so as to vector the entities and complex semantic relations in graph. Then it adopts LSTM recursive neural network and adds the top-bottom relational information of the graph for sequential learning. Finally it constantly carries out deep learning through incremental calculation and LSTM recursive network to improve the accuracy of prediction. The incremental LSTM model highlights the hidden semantic and clinical temporal information and effectively utilizes sequential learning to mining forward-backward dependent information. It compensates the deficiency of lower prediction accuracy on timely knowledge graph caused by the traditional link prediction models. Finally, it is proved that the new model has better performance over temporal knowledge graph link prediction.

A quaternion-group knowledge graph embedding model

2021 ◽  
pp. 1-10
Author(s):  
Heng Chen ◽  
Guanyu Li ◽  
Yunhao Sun ◽  
Wei Jiang
Keyword(s):  
Link Prediction ◽  
State Of The Art ◽  
Graph Embedding ◽  
Knowledge Graph ◽  
Complex Vector ◽  
Quaternion Group ◽  
Rotation Matrices ◽  
Group Knowledge ◽  
Benchmark Datasets ◽  
Complete Relation

Capturing the composite embedding representation of a multi-hop relation path is an extremely vital task in knowledge graph completion. Recently, rotation-based relation embedding models have been widely studied to embed composite relations into complex vector space. However, these models make some over-simplified assumptions on the composite relations, resulting the relations to be commutative. To tackle this problem, this paper proposes a novel knowledge graph embedding model, named QuatGE, which can provide sufficient modeling capabilities for complex composite relations. In particular, our method models each relation as a rotation operator in quaternion group-based space. The advantages of our model are twofold: (1) Since the quaternion group is a non-commutative group (i.e., non-Abelian group), the corresponding rotation matrices of composite relations can be non-commutative; (2) The model has a more expressive setting with stronger modeling capabilities, which is flexible to model and infer the complete relation patterns, including: symmetry/anti-symmetry, inversion and commutative/non-commutative composition. Experimental results on four benchmark datasets show that the proposed method outperforms the existing state-of-the-art models for link prediction, especially on composite relations.

scholarly journals ParamE: Regarding Neural Network Parameters as Relation Embeddings for Knowledge Graph Completion

2020 ◽  
Vol 34 (03) ◽  
pp. 2774-2781
Author(s):  
Feihu Che ◽  
Dawei Zhang ◽  
Jianhua Tao ◽  
Mingyue Niu ◽  
Bocheng Zhao
Keyword(s):  
Neural Network ◽  
Link Prediction ◽  
State Of The Art ◽  
Graph Embedding ◽  
Feature Space ◽  
Knowledge Graph ◽  
Network Parameters ◽  
New Knowledge ◽  
Fitting Ability ◽  
Translational Models

We study the task of learning entity and relation embeddings in knowledge graphs for predicting missing links. Previous translational models on link prediction make use of translational properties but lack enough expressiveness, while the convolution neural network based model (ConvE) takes advantage of the great nonlinearity fitting ability of neural networks but overlooks translational properties. In this paper, we propose a new knowledge graph embedding model called ParamE which can utilize the two advantages together. In ParamE, head entity embeddings, relation embeddings and tail entity embeddings are regarded as the input, parameters and output of a neural network respectively. Since parameters in networks are effective in converting input to output, taking neural network parameters as relation embeddings makes ParamE much more expressive and translational. In addition, the entity and relation embeddings in ParamE are from feature space and parameter space respectively, which is in line with the essence that entities and relations are supposed to be mapped into two different spaces. We evaluate the performances of ParamE on standard FB15k-237 and WN18RR datasets, and experiments show ParamE can significantly outperform existing state-of-the-art models, such as ConvE, SACN, RotatE and D4-STE/Gumbel.

scholarly journals Robust Embedding with Multi-Level Structures for Link Prediction

2019 ◽  
Author(s):  
Zihan Wang ◽  
Zhaochun Ren ◽  
Chunyu He ◽  
Peng Zhang ◽  
Yue Hu
Keyword(s):  
Neural Network ◽  
Recent Work ◽  
Link Prediction ◽  
Robustness Analysis ◽  
Hierarchical Structures ◽  
Knowledge Graph ◽  
Structure Information ◽  
Robustness To Noise ◽  
Multi Level ◽  
Representational Power

Knowledge Graph (KG) embedding has become crucial for the task of link prediction. Recent work applies encoder-decoder models to tackle this problem, where an encoder is formulated as a graph neural network (GNN) and a decoder is represented by an embedding method. These approaches enforce embedding techniques with structure information. Unfortunately, existing GNN-based frameworks still confront 3 severe problems: low representational power, stacking in a flat way, and poor robustness to noise. In this work, we propose a novel multi-level graph neural network (M-GNN) to address the above challenges. We first identify an injective aggregate scheme and design a powerful GNN layer using multi-layer perceptrons (MLPs). Then, we define graph coarsening schemes for various kinds of relations, and stack GNN layers on a series of coarsened graphs, so as to model hierarchical structures. Furthermore, attention mechanisms are adopted so that our approach can make predictions accurately even on the noisy knowledge graph. Results on WN18 and FB15k datasets show that our approach is effective in the standard link prediction task, significantly and consistently outperforming competitive baselines. Furthermore, robustness analysis on FB15k-237 dataset demonstrates that our proposed M-GNN is highly robust to sparsity and noise. 

Sign in / Sign up

Export Citation Format

Share Document