Low Resource Named Entity Recognition Using Contextual Word Representation and Neural Cross-Lingual Knowledge Transfer

2019 ◽  
pp. 299-311
Author(s):  
Soyeon Caren Han ◽  
Yingru Lin ◽  
Siqu Long ◽  
Josiah Poon
Keyword(s):  
Knowledge Transfer ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Low Resource ◽  
Named Entity ◽  
Word Representation ◽  
Cross Lingual

scholarly journals Named Entity Recognition in Low-resource Languages using Cross-lingual distributional word representation

2020 ◽  
Vol Volume 33 - 2020 - Special... ◽  
Author(s):  
Paulin Melatagia Yonta ◽  
Michael Franklin Mbouopda
Keyword(s):  
Named Entity Recognition ◽  
Low Frequency ◽  
Entity Recognition ◽  
Nous Obtenons ◽  
Named Entities ◽  
Low Resource ◽  
Named Entity ◽  
Word Representation ◽  
International Audience ◽  
Cross Lingual

International audience Named Entity Recognition (NER) is a fundamental task in many NLP applications that seek to identify and classify expressions such as people, location, and organization names. Many NER systems have been developed, but the annotated data needed for good performances are not available for low-resource languages, such as Cameroonian languages. In this paper we exploit the low frequency of named entities in text to define a new suitable cross-lingual distributional representation for named entity recognition. We build the first Ewondo (a Bantu low-resource language of Cameroon) named entities recognizer by projecting named entity tags from English using our word representation. In terms of Recall, Precision and F-score, the obtained results show the effectiveness of the proposed distributional representation of words La reconnaissance des entités nommées (REN) est une tâche fondamentale du TALN dont le but est d'identifier les expressions telles que les noms de personnes, de lieux et d'organisations dans un texte. Il existe de nos jours plusieurs systèmes de REN, cependant les données nécessaires pour les utiliser dans le traitement des langues peu dotées telles que les langues camerounaises ne sont pas disponibles. Nous exploitons le fait que les entités nommées apparaissent rarement dans les textes pour définir une nouvelle représentation distributionnelle interlingue des mots, qui soit adaptée à la REN. En utilisant notre représentation, nous projectons les entités nommées de l'anglais vers l'ewondo (une langue bantou du Cameroun); nous obtenons donc le tout premier modèle de reconnaissance des entités nommées en langue ewondo. Les résultats en terme de précision, rappel et f-mesure montrent l'efficacité de notre représentation

Improving Low Resource Named Entity Recognition using Cross-lingual Knowledge Transfer

2018 ◽  
Author(s):  
Xiaocheng Feng ◽  
Xiachong Feng ◽  
Bing Qin ◽  
Zhangyin Feng ◽  
Ting Liu
Keyword(s):  
Neural Networks ◽  
Knowledge Transfer ◽  
State Of The Art ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Semantic Representations ◽  
Low Resource ◽  
Named Entity ◽  
High Resource ◽  
Distribution Features

Neural networks have been widely used for high resource language (e.g. English) named entity recognition (NER) and have shown state-of-the-art results.However, for low resource languages, such as Dutch, Spanish, due to the limitation of resources and lack of annotated data, taggers tend to have lower performances.To narrow this gap, we propose three novel strategies to enrich the semantic representations of low resource languages: we first develop neural networks to improve low resource word representations by knowledge transfer from high resource language using bilingual lexicons. Further, a lexicon extension strategy is designed to address out-of lexicon problem by automatically learning semantic projections.Thirdly, we regard word-level entity type distribution features as an external language-independent knowledge and incorporate them into our neural architecture. Experiments on two low resource languages (including Dutch and Spanish) demonstrate the effectiveness of these additional semantic representations (average 4.8\% improvement). Moreover, on Chinese OntoNotes 4.0 dataset, our approach achieved an F-score of 83.07\% with 2.91\% absolute gain compared to the state-of-the-art results.

scholarly journals Learning Task-Specific Representation for Novel Words in Sequence Labeling

2019 ◽  
Author(s):  
Minlong Peng ◽  
Qi Zhang ◽  
Xiaoyu Xing ◽  
Tao Gui ◽  
Jinlan Fu ◽  
...  
Keyword(s):  
Empirical Studies ◽  
Named Entity Recognition ◽  
Learning Task ◽  
Training Data ◽  
Entity Recognition ◽  
Named Entity ◽  
Part Of Speech Tagging ◽  
Sequence Labeling ◽  
Part Of Speech ◽  
Word Representation

Word representation is a key component in neural-network-based sequence labeling systems. However, representations of unseen or rare words trained on the end task are usually poor for appreciable performance. This is commonly referred to as the out-of-vocabulary (OOV) problem. In this work, we address the OOV problem in sequence labeling using only training data of the task. To this end, we propose a novel method to predict representations for OOV words from their surface-forms (e.g., character sequence) and contexts. The method is specifically designed to avoid the error propagation problem suffered by existing approaches in the same paradigm. To evaluate its effectiveness, we performed extensive empirical studies on four part-of-speech tagging (POS) tasks and four named entity recognition (NER) tasks. Experimental results show that the proposed method can achieve better or competitive performance on the OOV problem compared with existing state-of-the-art methods.

scholarly journals Enhanced Meta-Learning for Cross-Lingual Named Entity Recognition with Minimal Resources

2020 ◽  
Vol 34 (05) ◽  
pp. 9274-9281
Author(s):  
Qianhui Wu ◽  
Zijia Lin ◽  
Guoxin Wang ◽  
Hui Chen ◽  
Börje F. Karlsson ◽  
...  
Keyword(s):  
Learning Algorithm ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Target Language ◽  
Test Case ◽  
Named Entity ◽  
Meta Learning ◽  
Target Languages ◽  
Cross Lingual ◽  
The Given

For languages with no annotated resources, transferring knowledge from rich-resource languages is an effective solution for named entity recognition (NER). While all existing methods directly transfer from source-learned model to a target language, in this paper, we propose to fine-tune the learned model with a few similar examples given a test case, which could benefit the prediction by leveraging the structural and semantic information conveyed in such similar examples. To this end, we present a meta-learning algorithm to find a good model parameter initialization that could fast adapt to the given test case and propose to construct multiple pseudo-NER tasks for meta-training by computing sentence similarities. To further improve the model's generalization ability across different languages, we introduce a masking scheme and augment the loss function with an additional maximum term during meta-training. We conduct extensive experiments on cross-lingual named entity recognition with minimal resources over five target languages. The results show that our approach significantly outperforms existing state-of-the-art methods across the board.

Finding next of kin: Cross-lingual embedding spaces for related languages

2019 ◽  
Vol 26 (2) ◽  
pp. 163-182 ◽  
Author(s):  
Serge Sharoff
Keyword(s):  
Similarity Measure ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Levenshtein Distance ◽  
Next Of Kin ◽  
Named Entity ◽  
Genre Classification ◽  
Lexical Similarity ◽  
Cross Lingual ◽  
Embedding Methods

AbstractSome languages have very few NLP resources, while many of them are closely related to better-resourced languages. This paper explores how the similarity between the languages can be utilised by porting resources from better- to lesser-resourced languages. The paper introduces a way of building a representation shared across related languages by combining cross-lingual embedding methods with a lexical similarity measure which is based on the weighted Levenshtein distance. One of the outcomes of the experiments is a Panslavonic embedding space for nine Balto-Slavonic languages. The paper demonstrates that the resulting embedding space helps in such applications as morphological prediction, named-entity recognition and genre classification.

scholarly journals Evaluating Word Representation Features in Biomedical Named Entity Recognition Tasks

2014 ◽  
Vol 2014 ◽  
pp. 1-6 ◽  
Author(s):  
Buzhou Tang ◽  
Hongxin Cao ◽  
Xiaolong Wang ◽  
Qingcai Chen ◽  
Hua Xu
Keyword(s):  
Machine Learning ◽  
Language Processing ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Biomedical Domain ◽  
Crucial Step ◽  
Named Entity ◽  
Different Types ◽  
Word Representation ◽  
Biomedical Named Entity Recognition

Biomedical Named Entity Recognition (BNER), which extracts important entities such as genes and proteins, is a crucial step of natural language processing in the biomedical domain. Various machine learning-based approaches have been applied to BNER tasks and showed good performance. In this paper, we systematically investigated three different types of word representation (WR) features for BNER, including clustering-based representation, distributional representation, and word embeddings. We selected one algorithm from each of the three types of WR features and applied them to the JNLPBA and BioCreAtIvE II BNER tasks. Our results showed that all the three WR algorithms were beneficial to machine learning-based BNER systems. Moreover, combining these different types of WR features further improved BNER performance, indicating that they are complementary to each other. By combining all the three types of WR features, the improvements inF-measure on the BioCreAtIvE II GM and JNLPBA corpora were 3.75% and 1.39%, respectively, when compared with the systems using baseline features. To the best of our knowledge, this is the first study to systematically evaluate the effect of three different types of WR features for BNER tasks.

scholarly journals Named Entity Recognition for a Low Resource Language

2019 ◽  
Vol 8 (3) ◽  
pp. 587-590
Keyword(s):  
Machine Learning ◽  
Named Entity Recognition ◽  
Training Data ◽  
Entity Recognition ◽  
Linguistic Knowledge ◽  
Rule Based ◽  
Low Resource ◽  
Named Entity ◽  
The North ◽  
Rule Based Approach

Kokborok named entity recognition using the rules based approach is being studied in this paper. Named entity recognition is one of the applications of natural language processing. It is considered a subtask for information extraction. Named entity recognition is the means of identifying the named entity for some specific task. We have studied the named entity recognition system for the Kokborok language. Kokborok is the official language of the state of Tripura situated in the north eastern part of India. It is also widely spoken in other part of the north eastern state of India and adjoining areas of Bangladesh. The named entities are like the name of person, organization, location etc. Named entity recognitions are studied using the machine learning approach, rule based approach or the hybrid approach combining the machine learning and rule based approaches. Rule based named entity recognitions are influence by the linguistic knowledge of the language. Machine learning approach requires a large number of training data. Kokborok being a low resource language has very limited number of training data. The rule based approach requires linguistic rules and the results are not depended on the size of data available. We have framed a heuristic rules for identifying the named entity based on linguistic knowledge of the language. An encouraging result is obtained after we test our data with the rule based approach. We also tried to study and frame the rules for the counting system in Kokborok in this paper. The rule based approach to named entity recognition is found suitable for low resource language with limited digital work and absence of named entity tagged data. We have framed a suitable algorithm using the rules for solving the named entity recognition task for obtaining a desirable result.

Sign in / Sign up

Export Citation Format

Share Document