scholarly journals Attention as Relation: Learning Supervised Multi-head Self-Attention for Relation Extraction

2020 ◽  
Author(s):  
Jie Liu ◽  
Shaowei Chen ◽  
Bingquan Wang ◽  
Jiaxin Zhang ◽  
Na Li ◽  
...  
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
State Of The Art ◽  
Relation Extraction ◽  
Attention Mechanism ◽  
Entity Extraction ◽  
Relation Type ◽  
Benchmark Datasets ◽  
Relation Learning

Joint entity and relation extraction is critical for many natural language processing (NLP) tasks, which has attracted increasing research interest. However, it is still faced with the challenges of identifying the overlapping relation triplets along with the entire entity boundary and detecting the multi-type relations. In this paper, we propose an attention-based joint model, which mainly contains an entity extraction module and a relation detection module, to address the challenges. The key of our model is devising a supervised multi-head self-attention mechanism as the relation detection module to learn the token-level correlation for each relation type separately. With the attention mechanism, our model can effectively identify overlapping relations and flexibly predict the relation type with its corresponding intensity. To verify the effectiveness of our model, we conduct comprehensive experiments on two benchmark datasets. The experimental results demonstrate that our model achieves state-of-the-art performances.

scholarly journals A Relation-Specific Attention Network for Joint Entity and Relation Extraction

2020 ◽  
Author(s):  
Yue Yuan ◽  
Xiaofei Zhou ◽  
Shirui Pan ◽  
Qiannan Zhu ◽  
Zeliang Song ◽  
...  
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
State Of The Art ◽  
Relation Extraction ◽  
Attention Mechanism ◽  
Important Task ◽  
Entity Recognition ◽  
Attention Network ◽  
Public Datasets

Joint extraction of entities and relations is an important task in natural language processing (NLP), which aims to capture all relational triplets from plain texts. This is a big challenge due to some of the triplets extracted from one sentence may have overlapping entities. Most existing methods perform entity recognition followed by relation detection between every possible entity pairs, which usually suffers from numerous redundant operations. In this paper, we propose a relation-specific attention network (RSAN) to handle the issue. Our RSAN utilizes relation-aware attention mechanism to construct specific sentence representations for each relation, and then performs sequence labeling to extract its corresponding head and tail entities. Experiments on two public datasets show that our model can effectively extract overlapping triplets and achieve state-of-the-art performance.

scholarly journals Key2Vec: Automatic Ranked Keyphrase Extraction from Scientific Articles using Phrase Embeddings

2018 ◽  
Author(s):  
Debanjan Mahata ◽  
John Kuriakose ◽  
Rajiv Ratn Shah ◽  
Roger Zimmermann
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
State Of The Art ◽  
Keyphrase Extraction ◽  
Text Documents ◽  
Benchmark Datasets

Keyphrase extraction is a fundamental task in natural language processing that facilitates mapping of documents to a set of representative phrases. In this paper, we present an unsupervised technique (Key2Vec) that leverages phrase embeddings for ranking keyphrases extracted from scientific articles. Specifically, we propose an effective way of processing text documents for training multi-word phrase embeddings that are used for thematic representation of scientific articles and ranking of keyphrases extracted from them using theme-weighted PageRank. Evaluations are performed on benchmark datasets producing state-of-the-art results.

scholarly journals Few-Shot Knowledge Graph Completion

2020 ◽  
Vol 34 (03) ◽  
pp. 3041-3048 ◽  
Author(s):  
Chuxu Zhang ◽  
Huaxiu Yao ◽  
Chao Huang ◽  
Meng Jiang ◽  
Zhenhui Li ◽  
...  
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
State Of The Art ◽  
Knowledge Graph ◽  
Graph Structure ◽  
Reference Set ◽  
Knowledge Graphs ◽  
Public Datasets ◽  
Relation Learning

Knowledge graphs (KGs) serve as useful resources for various natural language processing applications. Previous KG completion approaches require a large number of training instances (i.e., head-tail entity pairs) for every relation. The real case is that for most of the relations, very few entity pairs are available. Existing work of one-shot learning limits method generalizability for few-shot scenarios and does not fully use the supervisory information; however, few-shot KG completion has not been well studied yet. In this work, we propose a novel few-shot relation learning model (FSRL) that aims at discovering facts of new relations with few-shot references. FSRL can effectively capture knowledge from heterogeneous graph structure, aggregate representations of few-shot references, and match similar entity pairs of reference set for every relation. Extensive experiments on two public datasets demonstrate that FSRL outperforms the state-of-the-art.

scholarly journals Drop Redundant, Shrink Irrelevant: Selective Knowledge Injection for Language Pretraining

2021 ◽  
Author(s):  
Ningyu Zhang ◽  
Shumin Deng ◽  
Xu Cheng ◽  
Xi Chen ◽  
Yichi Zhang ◽  
...  
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Prior Knowledge ◽  
Language Processing ◽  
Empirical Investigation ◽  
State Of The Art ◽  
Knowledge Bases ◽  
Experimental Results ◽  
Benchmark Datasets ◽  
Injection Methods

Previous research has demonstrated the power of leveraging prior knowledge to improve the performance of deep models in natural language processing. However, traditional methods neglect the fact that redundant and irrelevant knowledge exists in external knowledge bases. In this study, we launched an in-depth empirical investigation into downstream tasks and found that knowledge-enhanced approaches do not always exhibit satisfactory improvements. To this end, we investigate the fundamental reasons for ineffective knowledge infusion and present selective injection for language pretraining, which constitutes a model-agnostic method and is readily pluggable into previous approaches. Experimental results on benchmark datasets demonstrate that our approach can enhance state-of-the-art knowledge injection methods.

Report on the 4th Joint Workshop on Bibliometric-Enhanced Information Retrieval and Natural Language Processing for Digital Libraries at SIGIR 2019

2019 ◽  
Vol 53 (2) ◽  
pp. 3-10
Author(s):  
Muthu Kumar Chandrasekaran ◽  
Philipp Mayr
Keyword(s):  
Information Retrieval ◽  
Natural Language Processing ◽  
Natural Language ◽  
Research And Development ◽  
Language Processing ◽  
Digital Libraries ◽  
State Of The Art ◽  
Shared Task ◽  
Processing Information ◽  
Joint Workshop

The 4 th joint BIRNDL workshop was held at the 42nd ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2019) in Paris, France. BIRNDL 2019 intended to stimulate IR researchers and digital library professionals to elaborate on new approaches in natural language processing, information retrieval, scientometrics, and recommendation techniques that can advance the state-of-the-art in scholarly document understanding, analysis, and retrieval at scale. The workshop incorporated different paper sessions and the 5 th edition of the CL-SciSumm Shared Task.

scholarly journals A Natural Language Processing System for Extracting Evidence of Drug Repurposing from Scientific Publications

2020 ◽  
Vol 34 (08) ◽  
pp. 13369-13381
Author(s):  
Shivashankar Subramanian ◽  
Ioana Baldini ◽  
Sushma Ravichandran ◽  
Dmitriy A. Katz-Rogozhnikov ◽  
Karthikeyan Natesan Ramamurthy ◽  
...  
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Generic Drugs ◽  
Low Cost ◽  
Processing System ◽  
Drug Repurposing ◽  
Cancer Type ◽  
Entity Extraction ◽  
Scientific Publications

More than 200 generic drugs approved by the U.S. Food and Drug Administration for non-cancer indications have shown promise for treating cancer. Due to their long history of safe patient use, low cost, and widespread availability, repurposing of these drugs represents a major opportunity to rapidly improve outcomes for cancer patients and reduce healthcare costs. In many cases, there is already evidence of efficacy for cancer, but trying to manually extract such evidence from the scientific literature is intractable. In this emerging applications paper, we introduce a system to automate non-cancer generic drug evidence extraction from PubMed abstracts. Our primary contribution is to define the natural language processing pipeline required to obtain such evidence, comprising the following modules: querying, filtering, cancer type entity extraction, therapeutic association classification, and study type classification. Using the subject matter expertise on our team, we create our own datasets for these specialized domain-specific tasks. We obtain promising performance in each of the modules by utilizing modern language processing techniques and plan to treat them as baseline approaches for future improvement of individual components.

LIS4: Lesk Inspired Sense Specific Semantic Similarity using WordNet

2021 ◽  
pp. 2150006
Author(s):  
Saravanakumar Kandasamy ◽  
Aswani Kumar Cherukuri
Keyword(s):  
Information Retrieval ◽  
Natural Language Processing ◽  
Natural Language ◽  
Semantic Similarity ◽  
Language Processing ◽  
Gold Standard ◽  
Question Answering ◽  
Knowledge Based ◽  
Benchmark Datasets ◽  
Processing Information

Semantic similarity quantification between concepts is one of the inevitable parts in domains like Natural Language Processing, Information Retrieval, Question Answering, etc. to understand the text and their relationships better. Last few decades, many measures have been proposed by incorporating various corpus-based and knowledge-based resources. WordNet and Wikipedia are two of the Knowledge-based resources. The contribution of WordNet in the above said domain is enormous due to its richness in defining a word and all of its relationship with others. In this paper, we proposed an approach to quantify the similarity between concepts that exploits the synsets and the gloss definitions of different concepts using WordNet. Our method considers the gloss definitions, contextual words that are helping in defining a word, synsets of contextual word and the confidence of occurrence of a word in other word’s definition for calculating the similarity. The evaluation based on different gold standard benchmark datasets shows the efficiency of our system in comparison with other existing taxonomical and definitional measures.

SANTM: Efficient Self-attention-driven Network for Text Matching

2022 ◽  
Vol 22 (3) ◽  
pp. 1-21
Author(s):  
Prayag Tiwari ◽  
Amit Kumar Jaiswal ◽  
Sahil Garg ◽  
Ilsun You
Keyword(s):  
Natural Language ◽  
State Of The Art ◽  
The State ◽  
Attention Mechanism ◽  
Matching Problems ◽  
Attention Model ◽  
Extra Information ◽  
Textual Entailment ◽  
Benchmark Datasets ◽  
Text Matching

Self-attention mechanisms have recently been embraced for a broad range of text-matching applications. Self-attention model takes only one sentence as an input with no extra information, i.e., one can utilize the final hidden state or pooling. However, text-matching problems can be interpreted either in symmetrical or asymmetrical scopes. For instance, paraphrase detection is an asymmetrical task, while textual entailment classification and question-answer matching are considered asymmetrical tasks. In this article, we leverage attractive properties of self-attention mechanism and proposes an attention-based network that incorporates three key components for inter-sequence attention: global pointwise features, preceding attentive features, and contextual features while updating the rest of the components. Our model follows evaluation on two benchmark datasets cover tasks of textual entailment and question-answer matching. The proposed efficient Self-attention-driven Network for Text Matching outperforms the state of the art on the Stanford Natural Language Inference and WikiQA datasets with much fewer parameters.

Sign in / Sign up

Export Citation Format

Share Document