Syntactic and Semantic Features Based Relation Extraction in Agriculture Domain

2018 ◽  
pp. 252-258
Author(s):  
Zhanghui Liu ◽  
Yiyan Chen ◽  
Yuanfei Dai ◽  
Chenhao Guo ◽  
Zuwen Zhang ◽  
...  
Keyword(s):  
Relation Extraction ◽  
Semantic Features

A Hybrid Method Based on Semi-Supervised Learning for Relation Extraction in Chinese EMRs (Preprint)

2021 ◽  
Author(s):  
ChunMing Yang
Keyword(s):  
Supervised Learning ◽  
Learning Algorithm ◽  
Medical Knowledge ◽  
Relation Extraction ◽  
Small Scale ◽  
Semantic Features ◽  
Training Process ◽  
Network Layers ◽  
Relation Prediction ◽  
The Cost

BACKGROUND Extracting relations between the entities from Chinese electronic medical records(EMRs) is the key to automatically constructing medical knowledge graphs. Due to the less available labeled corpus, most of the current researches are based on shallow networks, which cannot fully capture the complex semantic features in the text of Chinese EMRs. OBJECTIVE In this study, a hybrid deep learning method based on semi-supervised learning is proposed to extract the entity relations from small-scale complex Chinese EMRs. METHODS The semantic features of sentences are extracted by residual network (ResNet) and the long dependent information is captured by bidirectional GRU (Gated Recurrent Unit). Then the attention mechanism is used to assign weights to the extracted features respectively, and the output of the two attention mechanisms is integrated for relation prediction. We adjusted the training process with manually annotated small-scale relational corpus and bootstrapping semi-supervised learning algorithm, and continuously expanded the datasets during the training process. RESULTS The experimental results show that the best F1-score of the proposed method on the overall relation categories reaches 89.78%, which is 13.07% higher than the baseline CNN model. The F1-score on DAP, SAP, SNAP, TeRD, TeAP, TeCP, TeRS, TeAS, TrAD, TrRD and TrAP 11 relation categories reaches 80.95%, 93.91%, 92.96%, 88.43%, 86.54%, 85.58%, 87.96%, 94.74%, 93.01%, 87.58% and 95.48%, respectively. CONCLUSIONS The hybrid neural network method strengthens the feature transfer and reuse between different network layers and reduces the cost of manual tagging relations. The results demonstrate that our proposed method is effective for the relation extraction in Chinese EMRs.

scholarly journals MRE: A Military Relation Extraction Model Based on BiGRU and Multi-Head Attention

Symmetry ◽  
2021 ◽  
Vol 13 (9) ◽  
pp. 1742
Author(s):  
Yiwei Lu ◽  
Ruopeng Yang ◽  
Xuping Jiang ◽  
Dan Zhou ◽  
Changshen Yin ◽  
...  
Keyword(s):  
Comprehensive Evaluation ◽  
Language Model ◽  
Relation Extraction ◽  
Attention Mechanism ◽  
Semantic Features ◽  
Chinese Word Segmentation ◽  
Conceptual Foundation ◽  
Attention Model ◽  
Operational Information ◽  
Military Relations

A great deal of operational information exists in the form of text. Therefore, extracting operational information from unstructured military text is of great significance for assisting command decision making and operations. Military relation extraction is one of the main tasks of military information extraction, which aims at identifying the relation between two named entities from unstructured military texts. However, the traditional methods of extracting military relations cannot easily resolve problems such as inadequate manual features and inaccurate Chinese word segmentation in military fields, failing to make full use of symmetrical entity relations in military texts. With our approach, based on the pre-trained language model, we present a Chinese military relation extraction method, which combines the bi-directional gate recurrent unit (BiGRU) and multi-head attention mechanism (MHATT). More specifically, the conceptual foundation of our method lies in constructing an embedding layer and combining word embedding with position embedding, based on the pre-trained language model; the output vectors of BiGRU neural networks are symmetrically spliced to learn the semantic features of context, and they fuse the multi-head attention mechanism to improve the ability of expressing semantic information. On the military text corpus that we have built, we conduct extensive experiments. We demonstrate the superiority of our method over the traditional non-attention model, attention model, and improved attention model, and the comprehensive evaluation value F1-score of the model is improved by about 4%.

scholarly journals Relation Extraction Between Bacteria and Biotopes from Biomedical Texts with Attention Mechanisms and Domain-Specific Contextual Representations

2019 ◽  
Author(s):  
Amarin Jettakul ◽  
Duangdao Wichadakul ◽  
Peerapon Vateekul
Keyword(s):  
Deep Learning ◽  
Relation Extraction ◽  
Semantic Features ◽  
Significant Information ◽  
Domain Specific ◽  
Applied Microbiology ◽  
Feature Based ◽  
Concise Information ◽  
Biomedical Texts ◽  
Deep Learning Model

AbstractThe Bacteria Biotope (BB) task is biomedical relation extraction (RE) that aims to study the interaction between bacteria and their locations. This task is considered to pertain to fundamental knowledge in applied microbiology. Some previous investigations have used feature-based models; others have presented deep-learning-based models such as convolutional and recurrent neural networks used with the shortest dependency paths (SDPs). Although SDPs contain valuable and concise information, sections of significant information necessary to define bacterial location relationships are often neglected. In addition, the traditional word embedding used in previous studies may suffer from word ambiguation across linguistic contexts.Here, we present a deep learning model for biomedical RE. The model incorporates feature combinations of SDPs and full sentences with various attention mechanisms. We also used pre-trained contextual representations based on domain-specific vocabularies. In order to assess the model’s robustness, we introduced a mean F1 score on many models using different random seeds. The experiments were conducted on the standard BB corpus in BioNLP-ST’16. Our experimental results revealed that the model performed better (in terms of both maximum and average F1 scores; 60.77% and 57.63%, respectively) compared with other existing models.We demonstrated that our proposed contributions to this task can be used to extract rich lexical, syntactic, and semantic features that effectively boost the model’s performance. Moreover, we analyzed the trade-off between precision and recall in order to choose the proper cut-off to use in real-world applications.

scholarly journals Distant Supervision for Relation Extraction with Sentence Selection and Interaction Representation

2021 ◽  
Vol 2021 ◽  
pp. 1-16
Author(s):  
Tiantian Chen ◽  
Nianbin Wang ◽  
Hongbin Wang ◽  
Haomin Zhan
Keyword(s):  
Large Scale ◽  
Semantic Information ◽  
State Of The Art ◽  
Relation Extraction ◽  
Semantic Features ◽  
Distant Supervision ◽  
Word Level ◽  
Proposed Model ◽  
Relation Prediction ◽  
Better Than

Distant supervision (DS) has been widely used for relation extraction (RE), which automatically generates large-scale labeled data. However, there is a wrong labeling problem, which affects the performance of RE. Besides, the existing method suffers from the lack of useful semantic features for some positive training instances. To address the above problems, we propose a novel RE model with sentence selection and interaction representation for distantly supervised RE. First, we propose a pattern method based on the relation trigger words as a sentence selector to filter out noisy sentences to alleviate the wrong labeling problem. After clean instances are obtained, we propose the interaction representation using the word-level attention mechanism-based entity pairs to dynamically increase the weights of the words related to entity pairs, which can provide more useful semantic information for relation prediction. The proposed model outperforms the strongest baseline by 2.61 in F1-score on a widely used dataset, which proves that our model performs significantly better than the state-of-the-art RE systems.

scholarly journals Relation extraction between bacteria and biotopes from biomedical texts with attention mechanisms and domain-specific contextual representations

2019 ◽  
Vol 20 (1) ◽  
Author(s):  
Amarin Jettakul ◽  
Duangdao Wichadakul ◽  
Peerapon Vateekul
Keyword(s):  
Deep Learning ◽  
Relation Extraction ◽  
Semantic Features ◽  
Domain Specific ◽  
Applied Microbiology ◽  
Feature Based ◽  
Crucial Information ◽  
Concise Information ◽  
Biomedical Texts ◽  
Deep Learning Model

Abstract Background The Bacteria Biotope (BB) task is a biomedical relation extraction (RE) that aims to study the interaction between bacteria and their locations. This task is considered to pertain to fundamental knowledge in applied microbiology. Some previous investigations conducted the study by applying feature-based models; others have presented deep-learning-based models such as convolutional and recurrent neural networks used with the shortest dependency paths (SDPs). Although SDPs contain valuable and concise information, some parts of crucial information that is required to define bacterial location relationships are often neglected. Moreover, the traditional word-embedding used in previous studies may suffer from word ambiguation across linguistic contexts. Results Here, we present a deep learning model for biomedical RE. The model incorporates feature combinations of SDPs and full sentences with various attention mechanisms. We also used pre-trained contextual representations based on domain-specific vocabularies. To assess the model’s robustness, we introduced a mean F1 score on many models using different random seeds. The experiments were conducted on the standard BB corpus in BioNLP-ST’16. Our experimental results revealed that the model performed better (in terms of both maximum and average F1 scores; 60.77% and 57.63%, respectively) compared with other existing models. Conclusions We demonstrated that our proposed contributions to this task can be used to extract rich lexical, syntactic, and semantic features that effectively boost the model’s performance. Moreover, we analyzed the trade-off between precision and recall to choose the proper cut-off to use in real-world applications.

scholarly journals Drug Disease Relation Extraction from Biomedical Literature Using NLP and Machine Learning

2021 ◽  
Vol 2021 ◽  
pp. 1-10
Author(s):  
Wahiba Ben Abdessalem Karaa ◽  
Eman H. Alkhammash ◽  
Aida Bchir
Keyword(s):  
Language Processing ◽  
Relation Extraction ◽  
Relevant Information ◽  
Biomedical Literature ◽  
Semantic Relations ◽  
Polynomial Kernel ◽  
Support Vector ◽  
Semantic Features ◽  
Semantic Relationships ◽  
Medical Concepts

Extracting the relations between medical concepts is very valuable in the medical domain. Scientists need to extract relevant information and semantic relations between medical concepts, including protein and protein, gene and protein, drug and drug, and drug and disease. These relations can be extracted from biomedical literature available on various databases. This study examines the extraction of semantic relations that can occur between diseases and drugs. Findings will help specialists make good decisions when administering a medication to a patient and will allow them to continuously be up to date in their field. The objective of this work is to identify different features related to drugs and diseases from medical texts by applying Natural Language Processing (NLP) techniques and UMLS ontology. The Support Vector Machine classifier uses these features to extract valuable semantic relationships among text entities. The contributing factor of this research is the combination of the strength of a suggested NLP technique, which takes advantage of UMLS ontology and enables the extraction of correct and adequate features (frequency features, lexical features, morphological features, syntactic features, and semantic features), and Support Vector Machines with polynomial kernel function. These features are manipulated to pinpoint the relations between drug and disease. The proposed approach was evaluated using a standard corpus extracted from MEDLINE. The finding considerably improves the performance and outperforms similar works, especially the f-score for the most important relation “cure,” which is equal to 98.19%. The accuracy percentage is better than those in all the existing works for all the relations.

scholarly journals Data-Driven Cybersecurity Knowledge Graph Construction for Industrial Control System Security

2020 ◽  
Vol 2020 ◽  
pp. 1-13
Author(s):  
Guowei Shen ◽  
Wanling Wang ◽  
Qilin Mu ◽  
Yanhong Pu ◽  
Ya Qin ◽  
...  
Keyword(s):  
Control System ◽  
Security Analysis ◽  
Relation Extraction ◽  
Data Driven ◽  
Knowledge Graph ◽  
Semantic Features ◽  
Industrial Control System ◽  
Process Data ◽  
Industrial Control ◽  
Distant Supervision

Industrial control systems (ICS) involve many key industries, which once attacked will cause heavy losses. However, traditional passive defense methods of cybersecurity have difficulty effectively dealing with increasingly complex threats; a knowledge graph is a new idea to analyze and process data in cybersecurity analysis. We propose a novel overall framework of data-driven industrial control network security defense, which integrated fragmented multisource threat data with an industrial network layout by a cybersecurity knowledge graph. In order to better correlate data to construct a knowledge graph, we propose a distant supervised relation extraction model ResPCNN-ATT; it is based on a deep residual convolutional neural network and attention mechanism, reduces the influence of noisy data in distant supervision, and better extracts deep semantic features in sentences by using deep residuals. We empirically demonstrate the performance of the proposed method in the field of general cybersecurity by using dataset CSER; the model proposed in this paper achieves higher accuracy than other models. And then, the dataset ICSER was used to construct a cybersecurity knowledge graph (CSKG) on the basis of analyzing specific industrial control scenarios, visualizing the knowledge graph for further security analysis to the industrial control system.

scholarly journals Combining entity co-occurrence information and sentence semantic features for relation extraction

2018 ◽  
Vol 48 (11) ◽  
pp. 1533-1545
Author(s):  
Yudan MA ◽  
Yi ZHAO ◽  
Jing JIN ◽  
Huaiyu WAN
Keyword(s):  
Relation Extraction ◽  
Semantic Features

scholarly journals Semantic Enhanced Distantly Supervised Relation Extraction via Graph Attention Network

Information ◽  
2020 ◽  
Vol 11 (11) ◽  
pp. 528
Author(s):  
Xiaoye Ouyang ◽  
Shudong Chen ◽  
Rong Wang
Keyword(s):  
New York ◽  
New York Times ◽  
Relation Extraction ◽  
Extraction Methods ◽  
Semantic Features ◽  
Attention Networks ◽  
Word Position ◽  
Supervised Methods ◽  
Dependency Trees ◽  
Type Information

Distantly Supervised relation extraction methods can automatically extract the relation between entity pairs, which are essential for the construction of a knowledge graph. However, the automatically constructed datasets comprise amounts of low-quality sentences and noisy words, and the current Distantly Supervised methods ignore these noisy data, resulting in unacceptable accuracy. To mitigate this problem, we present a novel Distantly Supervised approach SEGRE (Semantic Enhanced Graph attention networks Relation Extraction) for improved relation extraction. Our model first uses word position and entity type information to provide abundant local features and background knowledge. Then it builds the dependency trees to remove noisy words that are irrelevant to relations and employs Graph Attention Networks (GATs) to encode syntactic information, which also captures the important semantic features of relational words in each instance. Furthermore, to make our model more robust against noisy words, the intra-bag attention module is used to weight the bag representation and mitigate noise in the bag. Through extensive experiments on Riedel New York Times (NYT) and Google IISc Distantly Supervised (GIDS) datasets, we demonstrate SEGRE’s effectiveness.

Sign in / Sign up

Export Citation Format

Share Document