scholarly journals GRN: Gated Relation Network to Enhance Convolutional Neural Network for Named Entity Recognition

2019 ◽  
Vol 33 ◽  
pp. 6236-6243 ◽  
Author(s):  
Hui Chen ◽  
Zijia Lin ◽  
Guiguang Ding ◽  
Jianguang Lou ◽  
Yusen Zhang ◽  
...  
Keyword(s):  
Neural Networks ◽  
Short Term Memory ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Local Context ◽  
Named Entity ◽  
Entire Sentence ◽  
Context Features ◽  
Gpu Parallelism

The dominant approaches for named entity recognitionm (NER) mostly adopt complex recurrent neural networks (RNN), e.g., long-short-term-memory (LSTM). However, RNNs are limited by their recurrent nature in terms of computational efficiency. In contrast, convolutional neural networks (CNN) can fully exploit the GPU parallelism with their feedforward architectures. However, little attention has been paid to performing NER with CNNs, mainly owing to their difficulties in capturing the long-term context information in a sequence. In this paper, we propose a simple but effective CNN-based network for NER, i.e., gated relation network (GRN), which is more capable than common CNNs in capturing long-term context. Specifically, in GRN we firstly employ CNNs to explore the local context features of each word. Then we model the relations between words and use them as gates to fuse local context features into global ones for predicting labels. Without using recurrent layers that process a sentence in a sequential manner, our GRN allows computations to be performed in parallel across the entire sentence. Experiments on two benchmark NER datasets (i.e., CoNLL2003 and Ontonotes 5.0) show that, our proposed GRN can achieve state-of-the-art performance with or without external knowledge. It also enjoys lower time costs to train and test.

scholarly journals An Improved Method for Named Entity Recognition and Its Application to CEMR

2019 ◽  
Vol 11 (9) ◽  
pp. 185
Author(s):  
Ming Gao ◽  
Qifeng Xiao ◽  
Shaochun Wu ◽  
Kun Deng
Keyword(s):  
Word Order ◽  
Medical Records ◽  
Semantic Information ◽  
Short Term Memory ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Local Context ◽  
Network Computing ◽  
Named Entity ◽  
Gpu Parallelism

Named Entity Recognition (NER) on Clinical Electronic Medical Records (CEMR) is a fundamental step in extracting disease knowledge by identifying specific entity terms such as diseases, symptoms, etc. However, the state-of-the-art NER methods based on Long Short-Term Memory (LSTM) fail to exploit GPU parallelism fully under the massive medical records. Although a novel NER method based on Iterated Dilated CNNs (ID-CNNs) can accelerate network computing, it tends to ignore the word-order feature and semantic information of the current word. In order to enhance the performance of ID-CNNs-based models on NER tasks, an attention-based ID-CNNs-CRF model, which combines the word-order feature and local context, is proposed. Firstly, position embedding is utilized to fuse word-order information. Secondly, the ID-CNNs architecture is used to extract global semantic information rapidly. Simultaneously, the attention mechanism is employed to pay attention to the local context. Finally, we apply the CRF to obtain the optimal tag sequence. Experiments conducted on two CEMR datasets show that our model outperforms traditional ones. The F1-scores of 94.55% and 91.17% are obtained respectively on these two datasets, and both are better than LSTM-based models.

scholarly journals CNN-Based Chinese NER with Lexicon Rethinking

2019 ◽  
Author(s):  
Tao Gui ◽  
Ruotian Ma ◽  
Qi Zhang ◽  
Lujun Zhao ◽  
Yu-Gang Jiang ◽  
...  
Keyword(s):  
Short Term Memory ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Great Success ◽  
Short Term ◽  
Named Entity ◽  
Word Level ◽  
Long Short Term Memory ◽  
High Level ◽  
Gpu Parallelism

Character-level Chinese named entity recognition (NER) that applies long short-term memory (LSTM) to incorporate lexicons has achieved great success. However, this method fails to fully exploit GPU parallelism and candidate lexicons can conflict. In this work, we propose a faster alternative to Chinese NER: a convolutional neural network (CNN)-based method that incorporates lexicons using a rethinking mechanism. The proposed method can model all the characters and potential words that match the sentence in parallel. In addition, the rethinking mechanism can address the word conflict by feeding back the high-level features to refine the networks. Experimental results on four datasets show that the proposed method can achieve better performance than both word-level and character-level baseline methods. In addition, the proposed method performs up to 3.21 times faster than state-of-the-art methods, while realizing better performance.

scholarly journals Biomedical Named Entity Recognition Using the SVM Methodologies and bio Tagging Schemes

2020 ◽  
Vol 72 (4) ◽  
pp. 52-64
Author(s):  
Thiyagu Meenachisundaram ◽  
Manjula Dhanabalachandran
Keyword(s):  
Machine Learning ◽  
Named Entity Recognition ◽  
Machine Learning Algorithms ◽  
Entity Recognition ◽  
Local Context ◽  
Named Entity ◽  
Conventional Machine ◽  
Context Features ◽  
Linear Text ◽  
Biomedical Named Entity Recognition

Biomedical Named Entity Recognition (BNER) is identification of entities such as drugs, genes, and chemicals from biomedical text, which help in information extraction from the domain literature. It would allow extracting information such as drug profiles, similar or related drugs and associations between drugs and their targets. This venue presents opportunities for improvement even though many machine learning methods have been applied. The efficiency can be improved in case of biological related chemical entities as there are varied structure and properties. This new approach combines two state-of-the-art algorithms and aims to improve the performance by applying it to varied sets of features including linguistic, orthographic, Morphological, domain features and local context features. It uses the sequence tagging capability of CRF to identify the boundary of the entity and classification efficiency of SVM to detect subtypes in BNER. The method is tested on two different datasets 1) GENIA and 2) CHEMDNER corpus with different types of entities. The result shows that proposed hybrid method enhances the BNER compared to the conventional machine learning algorithms. Moreover the detailed study of SVM and the methodologies has been discussed clearly. The linear and non linear text classification can be mapped clearly in the section 3. The final section describes the results and the evaluation of the proposed method.

scholarly journals Building and Evaluating an Annotated Corpus for Automated Recognition of Chat-Based Social Engineering Attacks

2021 ◽  
Vol 11 (22) ◽  
pp. 10871
Author(s):  
Nikolaos Tsinganos ◽  
Ioannis Mavridis
Keyword(s):  
Short Term Memory ◽  
Early Stage ◽  
Named Entity Recognition ◽  
Social Engineering ◽  
Cyber Attacks ◽  
Entity Recognition ◽  
Automated Recognition ◽  
Named Entity ◽  
Key Factor ◽  
Context Features

Chat-based Social Engineering (CSE) is widely recognized as a key factor to successful cyber-attacks, especially in small and medium-sized enterprise (SME) environments. Despite the interest in preventing CSE attacks, few studies have considered the specific features of the language used by the attackers. This work contributes to the area of early-stage automated CSE attack recognition by proposing an approach for building and annotating a specific-purpose corpus and presenting its application in the CSE domain. The resulting CSE corpus is then evaluated by training a bi-directional long short-term memory (bi-LSTM) neural network for the purpose of named entity recognition (NER). The results of this study emphasize the importance of adding a plethora of metadata to a dataset to provide critical in-context features and produce a corpus that broadens our understanding of the tactics used by social engineers. The outcomes can be applied to dedicated cyber-defence mechanisms utilized to protect SME employees using Electronic Medium Communication (EMC) software.

Advances in Computational Linguistics and Text Processing Frameworks

2020 ◽  
pp. 217-244
Author(s):  
Ayush Srivastav ◽  
Hera Khan ◽  
Amit Kumar Mishra
Keyword(s):  
Neural Networks ◽  
Natural Language Processing ◽  
Natural Language ◽  
Computational Linguistics ◽  
Language Processing ◽  
Text Processing ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Named Entity ◽  
Part Of Speech

The chapter provides an eloquent account of the major methodologies and advances in the field of Natural Language Processing. The most popular models that have been used over time for the task of Natural Language Processing have been discussed along with their applications in their specific tasks. The chapter begins with the fundamental concepts of regex and tokenization. It provides an insight to text preprocessing and its methodologies such as Stemming and Lemmatization, Stop Word Removal, followed by Part-of-Speech tagging and Named Entity Recognition. Further, this chapter elaborates the concept of Word Embedding, its various types, and some common frameworks such as word2vec, GloVe, and fastText. A brief description of classification algorithms used in Natural Language Processing is provided next, followed by Neural Networks and its advanced forms such as Recursive Neural Networks and Seq2seq models that are used in Computational Linguistics. A brief description of chatbots and Memory Networks concludes the chapter.

Sign in / Sign up

Export Citation Format

Share Document