Techniques for Named Entity Recognition

2012 ◽  
pp. 191-217 ◽  
Author(s):  
Girish Keshav Palshikar
Keyword(s):  
Semantic Processing ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Named Entities ◽  
Named Entity ◽  
Domain Specific ◽  
Number Of Factors ◽  
Web Contents ◽  
Biological Domain

While building and using a fully semantic understanding of Web contents is a distant goal, named entities (NEs) provide a small, tractable set of elements carrying a well-defined semantics. Generic named entities are names of persons, locations, organizations, phone numbers, and dates, while domain-specific named entities includes names of for example, proteins, enzymes, organisms, genes, cells, et cetera, in the biological domain. An ability to automatically perform named entity recognition (NER) – i.e., identify occurrences of NE in Web contents – can have multiple benefits, such as improving the expressiveness of queries and also improving the quality of the search results. A number of factors make building highly accurate NER a challenging task. Given the importance of NER in semantic processing of text, this chapter presents a detailed survey of NER techniques for English text.

Techniques for Named Entity Recognition

2013 ◽  
pp. 400-426 ◽  
Author(s):  
Girish Keshav Palshikar
Keyword(s):  
Semantic Processing ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Named Entities ◽  
Named Entity ◽  
Domain Specific ◽  
Number Of Factors ◽  
Web Contents ◽  
Biological Domain

While building and using a fully semantic understanding of Web contents is a distant goal, named entities (NEs) provide a small, tractable set of elements carrying a well-defined semantics. Generic named entities are names of persons, locations, organizations, phone numbers, and dates, while domain-specific named entities includes names of for example, proteins, enzymes, organisms, genes, cells, et cetera, in the biological domain. An ability to automatically perform named entity recognition (NER) – i.e., identify occurrences of NE in Web contents – can have multiple benefits, such as improving the expressiveness of queries and also improving the quality of the search results. A number of factors make building highly accurate NER a challenging task. Given the importance of NER in semantic processing of text, this chapter presents a detailed survey of NER techniques for English text.

scholarly journals A Probability based Classification of Named Entities for Malayalam Language combining Word, Part of Speech and Lexicalized features

2019 ◽  
Vol 8 (2) ◽  
pp. 839-842
Keyword(s):  
Named Entity Recognition ◽  
Entity Recognition ◽  
Supervised Machine Learning ◽  
Named Entities ◽  
Named Entity ◽  
Domain Specific ◽  
Part Of Speech ◽  
Classification Probability ◽  
Malayalam Language

Named Entity Recognition is the process wherein named entities which are designators of a sentence are identified. Designators of a sentence are domain specific. The proposed system identifies named entities in Malayalam language belonging to tourism domain which generally includes names of persons, places, organizations, dates etc. The system uses word, part of speech and lexicalized features to find the probability of a word belonging to a named entity category and to do the appropriate classification. Probability is calculated based on supervised machine learning using word and part of speech features present in a tagged training corpus and using certain rules applied based on lexicalized features.

scholarly journals Domain-Transferable Method for Named Entity Recognition Task

2020 ◽  
Author(s):  
Vladislav Mikhailov ◽  
Tatiana Shavrina
Keyword(s):  
Language Processing ◽  
Question Answering ◽  
Named Entity Recognition ◽  
Recognition Task ◽  
Entity Recognition ◽  
Neural Models ◽  
Named Entities ◽  
Named Entity ◽  
Domain Specific ◽  
Human Effort

Named Entity Recognition (NER) is a fundamental task in the fields of natural language processing and information extraction. NER has been widely used as a standalone tool or an essential component in a variety of applications such as question answering, dialogue assistants and knowledge graphs development. However, training reliable NER models requires a large amount of labelled data which is expensive to obtain, particularly in specialized domains. This paper describes a method to learn a domain-specific NER model for an arbitrary set of named entities when domain-specific supervision is not available. We assume that the supervision can be obtained with no human effort, and neural models can learn from each other. The code, data and models are publicly available.

scholarly journals PROJECTING NAMED ENTITY TAGS FROM A RESOURCE RICH LANGUAGE TO A RESOURCE POOR LANGUAGE

2013 ◽  
Author(s):  
Norshuhani Zamin ◽  
Alan Oxley ◽  
Zainab Abu Bakar
Keyword(s):  
Named Entity Recognition ◽  
Entity Recognition ◽  
Projection Algorithm ◽  
Coefficient Function ◽  
Scoring Method ◽  
Named Entities ◽  
String Similarity ◽  
Parallel Corpora ◽  
Named Entity ◽  
Domain Specific

Named Entities (NE) are the prominent entities appearing in textual documents. Automatic classification of NE in a textual corpus is a vital process in Information Extraction and Information Retrieval research. Named Entity Recognition (NER) is the identification of words in text that correspond to a pre-defined taxonomy such as person, organization, location, date, time, etc. This article focuses on the person (PER), organization (ORG) and location (LOC) entities for a Malay journalistic corpus of terrorism. A projection algorithm, using the Dice Coefficient function and bigram scoring method with domain-specific rules, is suggested to map the NE information from the English corpus to the Malay corpus of terrorism. The English corpus is the translated version of the Malay corpus. Hence, these two corpora are treated as parallel corpora. The method computes the string similarity between the English words and the list of available lexemes in a pre-built lexicon that approximates the best NE mapping. The algorithm has been effectively evaluated using our own terrorism tagged corpus; it achieved satisfactory results in terms of precision, recall, and F-measure. An evaluation of the selected open source NER tool for English is also presented.  

scholarly journals Medical Named Entity Recognition from Un-labelled Medical Records based on Pre-trained Language Models and Domain Dictionary

2021 ◽  
pp. 1-13
Author(s):  
Chaojie Wen ◽  
Tao Chen ◽  
Xudong Jia ◽  
Jiang Zhu
Keyword(s):  
Medical Records ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Language Models ◽  
Medical Texts ◽  
Named Entities ◽  
Named Entity ◽  
Domain Specific ◽  
Medical Entity ◽  
Medical Documents

Abstract Medical named entity recognition (NER) is an area in which medical named entities are recognized from medical texts, such as diseases, drugs, surgery reports, anatomical parts, examination documents, and so on. Conventional medical NER methods do not make full use of un-labelled medical texts embedded in medical documents. To address this issue, we propose a medical NER approach based on pre-trained language models and a domain dictionary. First, we construct a medical entity dictionary by extracting medical entities from labelled medical texts and collecting medical entities from other resources, such as the Yidu-N4K dataset. Second, we employ this dictionary to train domain-specific pre-trained language models using un-labelled medical texts. Third, we employ a pseudo labelling mechanism in un-labelled medical texts to automatically annotate texts and create pseudo labels. Fourth, the BiLSTM-CRF sequence tagging model is used to fine-tune the pre-trained language models. Our experiments on the un-labelled medical texts, which are extracted from Chinese electronic medical records, show that the proposed NER approach enables the strict and relaxed F1 scores to be 88.7% and 95.3%, respectively.

scholarly journals DeIDNER Corpus: Annotation of Clinical Discharge Summary Notes for Named Entity Recognition Using BRAT Tool

2021 ◽  
Author(s):  
Mahanazuddin Syed ◽  
Shaymaa Al-Shukri ◽  
Shorabuddin Syed ◽  
Kevin Sexton ◽  
Melody L. Greer ◽  
...  
Keyword(s):  
Language Processing ◽  
Named Entity Recognition ◽  
Discharge Summary ◽  
Training Data ◽  
Entity Recognition ◽  
Named Entities ◽  
Named Entity ◽  
Domain Specific ◽  
Model Training ◽  
Clinical Domain

Named Entity Recognition (NER) aims to identify and classify entities into predefined categories is a critical pre-processing task in Natural Language Processing (NLP) pipeline. Readily available off-the-shelf NER algorithms or programs are trained on a general corpus and often need to be retrained when applied on a different domain. The end model’s performance depends on the quality of named entities generated by these NER models used in the NLP task. To improve NER model accuracy, researchers build domain-specific corpora for both model training and evaluation. However, in the clinical domain, there is a dearth of training data because of privacy reasons, forcing many studies to use NER models that are trained in the non-clinical domain to generate NER feature-set. Thus, influencing the performance of the downstream NLP tasks like information extraction and de-identification. In this paper, our objective is to create a high quality annotated clinical corpus for training NER models that can be easily generalizable and can be used in a downstream de-identification task to generate named entities feature-set.

scholarly journals ACE-ADP: Adversarial Contextual Embeddings Based Named Entity Recognition for Agricultural Diseases and Pests

Agriculture ◽  
2021 ◽  
Vol 11 (10) ◽  
pp. 912
Author(s):  
Xuchao Guo ◽  
Xia Hao ◽  
Zhan Tang ◽  
Lei Diao ◽  
Zhao Bai ◽  
...  
Keyword(s):  
Web Application ◽  
Question Answering ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Text Representation ◽  
Named Entity ◽  
Domain Specific ◽  
Adversarial Training ◽  
Ablation Study

Entity recognition tasks, which aim to utilize the deep learning-based models to identify the agricultural diseases and pests-related nouns such as the names of diseases, pests, and drugs from the texts collected on the internet or input by users, are a fundamental component for agricultural knowledge graph construction and question-answering, which will be implemented as a web application and provide the general public with solutions for agricultural diseases and pest control. Nonetheless, there are still challenges: (1) the polysemous problem needs to be further solved, (2) the quality of the text representation needs to be further enhanced, (3) the performance for rare entities needs to be further improved. We proposed an adversarial contextual embeddings-based model named ACE-ADP for named entity recognition in Chinese agricultural diseases and pests domain (CNER-ADP). First, we enhanced the text representation and overcame the polysemy problem by using the fine-tuned BERT model to generate the contextual character-level embedded representation with the specific knowledge. Second, adversarial training was also introduced to enhance the generalization and robustness in terms of identifying the rare entities. The experimental results showed that our model achieved an F1 of 98.31% with 4.23% relative improvement compared to the baseline model (i.e., word2vec-based BiLSTM-CRF) on the self-annotated corpus named Chinese named entity recognition dataset for agricultural diseases and pests (AgCNER). Besides, the ablation study and discussion demonstrated that ACE-ADP could not only effectively extract rare entities but also maintain a powerful ability to predict new entities in new datasets with high accuracy. It could be used as a basis for further research on other domain-specific named entity recognition.

scholarly journals Unsupervised Named Entity Recognition for Hi-Tech Domain

2021 ◽  
Author(s):  
Abinaya Govindan ◽  
Gyan Ranjan ◽  
Amit Verma
Keyword(s):  
Named Entity Recognition ◽  
Entity Recognition ◽  
Manual Annotation ◽  
Data Generation ◽  
Named Entities ◽  
Named Entity ◽  
Free Data ◽  
Minimal Effort ◽  
Unique Context

This paper presents named entity recognition as a multi-answer QA task combined with contextual natural-language-inference based noise reduction. This method allows us to use pre-trained models that have been trained for certain downstream tasks to generate unsupervised data, reducing the need for manual annotation to create named entity tags with tokens. For each entity, we provide a unique context, such as entity types, definitions, questions and a few empirical rules along with the target text to train a named entity model for the domain of our interest. This formulation (a) allows the system to jointly learn NER-specific features from the datasets provided, and (b) can extract multiple NER-specific features, thereby boosting the performance of existing NER models (c) provides business-contextualized definitions to reduce ambiguity among similar entities. We conducted numerous tests to determine the quality of the created data, and we find that this method of data generation allows us to obtain clean, noise-free data with minimal effort and time. This approach has been demonstrated to be successful in extracting named entities, which are then used in subsequent components.

scholarly journals An Annotated Corpus of Crime-Related Portuguese Documents for NLP and Machine Learning Processing

Data ◽  
2021 ◽  
Vol 6 (7) ◽  
pp. 71
Author(s):  
Gonçalo Carnaz ◽  
Mário Antunes ◽  
Vitor Beires Nogueira
Keyword(s):  
Machine Learning ◽  
Language Processing ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Automatic Identification ◽  
Named Entities ◽  
Related Data ◽  
Named Entity ◽  
Chain Of Custody ◽  
Evidence Collection

Criminal investigations collect and analyze the facts related to a crime, from which the investigators can deduce evidence to be used in court. It is a multidisciplinary and applied science, which includes interviews, interrogations, evidence collection, preservation of the chain of custody, and other methods and techniques of investigation. These techniques produce both digital and paper documents that have to be carefully analyzed to identify correlations and interactions among suspects, places, license plates, and other entities that are mentioned in the investigation. The computerized processing of these documents is a helping hand to the criminal investigation, as it allows the automatic identification of entities and their relations, being some of which difficult to identify manually. There exists a wide set of dedicated tools, but they have a major limitation: they are unable to process criminal reports in the Portuguese language, as an annotated corpus for that purpose does not exist. This paper presents an annotated corpus, composed of a collection of anonymized crime-related documents, which were extracted from official and open sources. The dataset was produced as the result of an exploratory initiative to collect crime-related data from websites and conditioned-access police reports. The dataset was evaluated and a mean precision of 0.808, recall of 0.722, and F1-score of 0.733 were obtained with the classification of the annotated named-entities present in the crime-related documents. This corpus can be employed to benchmark Machine Learning (ML) and Natural Language Processing (NLP) methods and tools to detect and correlate entities in the documents. Some examples are sentence detection, named-entity recognition, and identification of terms related to the criminal domain.

scholarly journals TEI-friendly annotation scheme for medieval named entities: a case on a Spanish medieval corpus

2021 ◽  
Author(s):  
Elena Álvarez-Mellado ◽  
María Luisa Díez-Platas ◽  
Pablo Ruiz-Fabo ◽  
Helena Bermúdez ◽  
Salvador Ros ◽  
...  
Keyword(s):  
Historical Data ◽  
Named Entity Recognition ◽  
Rich Source ◽  
Entity Recognition ◽  
Historical Evidence ◽  
Annotation Scheme ◽  
Named Entities ◽  
General Domain ◽  
Named Entity ◽  
Entity Annotation

AbstractMedieval documents are a rich source of historical data. Performing named-entity recognition (NER) on this genre of texts can provide us with valuable historical evidence. However, traditional NER categories and schemes are usually designed with modern documents in mind (i.e. journalistic text) and the general-domain NER annotation schemes fail to capture the nature of medieval entities. In this paper we explore the challenges of performing named-entity annotation on a corpus of Spanish medieval documents: we discuss the mismatches that arise when applying traditional NER categories to a corpus of Spanish medieval documents and we propose a novel humanist-friendly TEI-compliant annotation scheme and guidelines intended to capture the particular nature of medieval entities.

Sign in / Sign up

Export Citation Format

Share Document