Label-Free Distant Supervision for Relation Extraction via Knowledge Graph Embedding

Knowledge Graph is a semantic network that reveals the relationship between entities, which construction is to describe various entities, concepts and their relationships in the real world. Since knowledge graph can effectively reveal the relationship between the different knowledge items, it has been widely utilized in the intelligent education. In particular, relation extraction is the critical part of knowledge graph and plays a very important role in the construction of knowledge graph. According to the different magnitude of data labeling, entity relationship extraction tasks of deep learning can be divided into two categories: supervised and distant supervised. Supervised learning approaches can extract effective entity relationships. However, these approaches rely on labeled data heavily resulting in the time-consuming and laborconsuming. The distant supervision approach is widely concerned by researchers because it can generate the entity relation extraction automatically. However, the development and application of the distant supervised approach has been seriously hindered due to the noises, lack of information and disequilibrium in the relation extraction tasks. Inspired by the above analysis, the paper proposes a novel curriculum points relationship extraction model based on the distant supervision. In particular, firstly the research of the distant supervised relationship extraction model based on the sentence bag attention mechanism to extract the relationship of curriculum points. Secondly, the research of knowledge graph construction based on the knowledge ontology. Thirdly, the development of curriculum semantic retrieval platform based on Web. Compared with the existing advanced models, the AUC of this system is increased by 14.2%; At the same time, taking "big data processing" course in computer field as an example, the relationship extraction result with F1 value of 88.1% is realized. The experimental results show that the proposed model provides an effective solution for the development and application of knowledge graph in the field of intelligent education.

Download Full-text

Data-Driven Cybersecurity Knowledge Graph Construction for Industrial Control System Security

Wireless Communications and Mobile Computing ◽

10.1155/2020/8883696 ◽

2020 ◽

Vol 2020 ◽

pp. 1-13

Author(s):

Guowei Shen ◽

Wanling Wang ◽

Qilin Mu ◽

Yanhong Pu ◽

Ya Qin ◽

...

Keyword(s):

Control System ◽

Security Analysis ◽

Relation Extraction ◽

Data Driven ◽

Knowledge Graph ◽

Semantic Features ◽

Industrial Control System ◽

Process Data ◽

Industrial Control ◽

Distant Supervision

Industrial control systems (ICS) involve many key industries, which once attacked will cause heavy losses. However, traditional passive defense methods of cybersecurity have difficulty effectively dealing with increasingly complex threats; a knowledge graph is a new idea to analyze and process data in cybersecurity analysis. We propose a novel overall framework of data-driven industrial control network security defense, which integrated fragmented multisource threat data with an industrial network layout by a cybersecurity knowledge graph. In order to better correlate data to construct a knowledge graph, we propose a distant supervised relation extraction model ResPCNN-ATT; it is based on a deep residual convolutional neural network and attention mechanism, reduces the influence of noisy data in distant supervision, and better extracts deep semantic features in sentences by using deep residuals. We empirically demonstrate the performance of the proposed method in the field of general cybersecurity by using dataset CSER; the model proposed in this paper achieves higher accuracy than other models. And then, the dataset ICSER was used to construct a cybersecurity knowledge graph (CSKG) on the basis of analyzing specific industrial control scenarios, visualizing the knowledge graph for further security analysis to the industrial control system.

Download Full-text

Improving Distant Supervised Relation Extraction with Noise Detection Strategy

Applied Sciences ◽

10.3390/app11052046 ◽

2021 ◽

Vol 11 (5) ◽

pp. 2046

Author(s):

Xiaoyan Meng ◽

Tonghai Jiang ◽

Xi Zhou ◽

Bo Ma ◽

Yi Wang ◽

...

Keyword(s):

State Of The Art ◽

Relation Extraction ◽

Knowledge Graph ◽

Noise Detection ◽

Plain Text ◽

Distant Supervision ◽

Detection Strategy ◽

Relation Prediction ◽

Noisy Labels ◽

The Web

Distant supervised relation extraction (DSRE) is widely used to extract novel relational facts from plain text, so as to improve the knowledge graph. However, distant supervision inevitably suffers from the noisy labeling problem that will severely damage the performance of relation extraction. Currently, most DSRE methods are mainly focused on reducing the weights of noisy sentences, ignoring the bag-level noise where all sentences in a bag are wrongly labeled. In this paper, we present a novel noise detection-based relation extraction approach (NDRE) to automatically detect noisy labels with entity information and dynamically correct them, which can alleviate both instance-level and bag-level noisy problems. By this means, we can extend the dataset from the Web tables without introducing more noise. In this approach, to embed the semantics of sentences from corpus and web tables, we firstly propose a powerful sentence coder that employs an internal multi-head self-attention mechanism between the piecewise max-pooling convolutional neural network. Second, we adopt a noise detection strategy, which is expected to dynamically detect and correct the original noisy label according to the similarity between sentence representation and entity-aware embeddings. Then, we aggregate the information from corpus and web tables to make the final relation prediction. Experimental results on a public benchmark dataset demonstrate that our proposed approach achieves significant improvements over the state-of-the-art baselines and can effectively reduce the noisy labeling problem.

Download Full-text

Distant Supervision Relation Extraction via adaptive dependency-path and additional knowledge graph supervision

Neural Networks ◽

10.1016/j.neunet.2020.10.012 ◽

2021 ◽

Vol 134 ◽

pp. 42-53

Author(s):

Yong Shi ◽

Yang Xiao ◽

Pei Quan ◽

MingLong Lei ◽

Lingfeng Niu

Keyword(s):

Relation Extraction ◽

Knowledge Graph ◽

Distant Supervision

Download Full-text

GREG: A Global Level Relation Extraction with Knowledge Graph Embedding

Applied Sciences ◽

10.3390/app10031181 ◽

2020 ◽

Vol 10 (3) ◽

pp. 1181 ◽

Cited By ~ 1

Author(s):

Kuekyeng Kim ◽

Yuna Hur ◽

Gyeongmin Kim ◽

Heuiseok Lim

Keyword(s):

Local Level ◽

Relation Extraction ◽

Graph Embedding ◽

Structured Data ◽

Unstructured Data ◽

Knowledge Graph ◽

Global Level ◽

Proposed Model ◽

Vector Representations ◽

Document Level

In an age overflowing with information, the task of converting unstructured data into structured data are a vital task of great need. Currently, most relation extraction modules are more focused on the extraction of local mention-level relations—usually from short volumes of text. However, in most cases, the most vital and important relations are those that are described in length and detail. In this research, we propose GREG: A Global level Relation Extractor model using knowledge graph embeddings for document-level inputs. The model uses vector representations of mention-level ‘local’ relation’s to construct knowledge graphs that can represent the input document. The knowledge graph is then used to predict global level relations from documents or large bodies of text. The proposed model is largely divided into two modules which are synchronized during their training. Thus, each of the model’s modules is designed to deal with local relations and global relations separately. This allows the model to avoid the problem of struggling against loss of information due to too much information crunched into smaller sized representations when attempting global level relation extraction. Through evaluation, we have shown that the proposed model yields high performances in both predicting global level relations and local level relations consistently.

Download Full-text

A Novel Negative Sampling Based on Frequency of Relational Association Entities for Knowledge Graph Embedding

Journal of Web Engineering ◽

10.13052/jwe1540-9589.2068 ◽

2021 ◽

Author(s):

Wanhua Cao ◽

Yi Zhang ◽

Juntao Liu ◽

Ziyun Rao

Keyword(s):

Link Prediction ◽

State Of The Art ◽

Evaluation Criteria ◽

Relation Extraction ◽

Graph Embedding ◽

Semantic Space ◽

Knowledge Graph ◽

Knowledge Reasoning ◽

Relation Prediction ◽

Low Dimensional

Knowledge graph embedding improves the performance of relation extraction and knowledge reasoning by encoding entities and relationships in low-dimensional semantic space. During training, negative samples are usually constructed by replacing the head/tail entity. And the different replacing relationships lead to different accuracy of the prediction results. This paper develops a negative triplets construction framework according to the frequency of relational association entities. The proposed construction framework can fully consider the quantitative of relations and entities in the dataset to assign the proportion of relation and entity replacement and the frequency of the entities associated with each relationship to set reasonable proportions for different relations. To verify the validity of the proposed construction framework, it is integrated into the state-of-the-art knowledge graph embedding models, such as TransE, TransH, DistMult, ComplEx, and Analogy. And both the evaluation criteria of relation prediction and entity prediction are used to evaluate the performance of link prediction more comprehensively. The experimental results on two commonly used datasets, WN18 and FB15K, show that the proposed method improves entity link and triplet classification accuracy, especially the accuracy of relational link prediction.

Download Full-text