scholarly journals Parameter Space Factorization for Zero-Shot Learning across Tasks and Languages

2021 ◽  
Vol 9 ◽  
pp. 410-428
Author(s):  
Edoardo M. Ponti ◽  
Ivan Vulić ◽  
Ryan Cotterell ◽  
Marinela Parovic ◽  
Roi Reichart ◽  
...  
Keyword(s):  
Latent Variables ◽  
Named Entity Recognition ◽  
Training Data ◽  
Entity Recognition ◽  
Language Varieties ◽  
Named Entity ◽  
Shot Classification ◽  
Pos Tagging ◽  
Part Of Speech ◽  
Cross Lingual

Abstract Most combinations of NLP tasks and language varieties lack in-domain examples for supervised training because of the paucity of annotated data. How can neural models make sample-efficient generalizations from task–language combinations with available data to low-resource ones? In this work, we propose a Bayesian generative model for the space of neural parameters. We assume that this space can be factorized into latent variables for each language and each task. We infer the posteriors over such latent variables based on data from seen task–language combinations through variational inference. This enables zero-shot classification on unseen combinations at prediction time. For instance, given training data for named entity recognition (NER) in Vietnamese and for part-of-speech (POS) tagging in Wolof, our model can perform accurate predictions for NER in Wolof. In particular, we experiment with a typologically diverse sample of 33 languages from 4 continents and 11 families, and show that our model yields comparable or better results than state-of-the-art, zero-shot cross-lingual transfer methods. Our code is available at github.com/cambridgeltl/parameter-factorization.

scholarly journals Sparse Coding of Neural Word Embeddings for Multilingual Sequence Labeling

2017 ◽  
Vol 5 ◽  
pp. 247-261 ◽  
Author(s):  
Gáabor Berend
Keyword(s):  
Sparse Coding ◽  
Named Entity Recognition ◽  
Training Data ◽  
Entity Recognition ◽  
Named Entity ◽  
Part Of Speech Tagging ◽  
Pos Tagging ◽  
Sequence Labeling ◽  
Part Of Speech ◽  
Proposed Model

In this paper we propose and carefully evaluate a sequence labeling framework which solely utilizes sparse indicator features derived from dense distributed word representations. The proposed model obtains (near) state-of-the art performance for both part-of-speech tagging and named entity recognition for a variety of languages. Our model relies only on a few thousand sparse coding-derived features, without applying any modification of the word representations employed for the different tasks. The proposed model has favorable generalization properties as it retains over 89.8% of its average POS tagging accuracy when trained at 1.2% of the total available training data, i.e. 150 sentences per language.

scholarly journals Learning Task-Specific Representation for Novel Words in Sequence Labeling

2019 ◽  
Author(s):  
Minlong Peng ◽  
Qi Zhang ◽  
Xiaoyu Xing ◽  
Tao Gui ◽  
Jinlan Fu ◽  
...  
Keyword(s):  
Empirical Studies ◽  
Named Entity Recognition ◽  
Learning Task ◽  
Training Data ◽  
Entity Recognition ◽  
Named Entity ◽  
Part Of Speech Tagging ◽  
Sequence Labeling ◽  
Part Of Speech ◽  
Word Representation

Word representation is a key component in neural-network-based sequence labeling systems. However, representations of unseen or rare words trained on the end task are usually poor for appreciable performance. This is commonly referred to as the out-of-vocabulary (OOV) problem. In this work, we address the OOV problem in sequence labeling using only training data of the task. To this end, we propose a novel method to predict representations for OOV words from their surface-forms (e.g., character sequence) and contexts. The method is specifically designed to avoid the error propagation problem suffered by existing approaches in the same paradigm. To evaluate its effectiveness, we performed extensive empirical studies on four part-of-speech tagging (POS) tasks and four named entity recognition (NER) tasks. Experimental results show that the proposed method can achieve better or competitive performance on the OOV problem compared with existing state-of-the-art methods.

scholarly journals Ry/Rk-Lex: A Computational Lexicon for Runyankore and Rukiga Languages

2021 ◽  
Author(s):  
David Sabiiti Bamutura
Keyword(s):  
Computational Linguistics ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Semantic Features ◽  
Language Resources ◽  
Bantu Languages ◽  
Pos Tagging ◽  
Part Of Speech ◽  
Cross Lingual ◽  
Computational Resources

Current research in computational linguistics and NLP requires the existence of language resources. Whereas these resources are available for only a few well-resourced languages, there are many languages that have been neglected. Among the neglected and / or under-resourced languages are Runyankore and Rukiga (henceforth referred to as Ry/Rk). In this paper, we report on Ry/Rk-Lex, a moderately large computational lexicon for Ry/Rk that we constructed from various existing data sources. Ry/Rk are two under-resourced Bantu languages with virtually no computational resources. About 9,400 lemmata have been entered so far. Ry/Rk-Lex has been enriched with syntactic and lexical semantic features, with the intent of providing a reference computational lexicon for Ry/Rk in other NLP (1) tasks such as: morphological analysis and generation; part of speech (POS) tagging; named entity recognition (NER); and (2) applications such as: spell and grammar checking; and cross-lingual information retrieval (CLIR). We have used Ry/Rk-Lex to dramatically increase the lexical coverage of previously developed computational resource grammars for Ry/Rk.

scholarly journals Named entity recognition in texts with the help of part of speech tagging

2018 ◽  
pp. 74-83
Author(s):  
M. Bevza
Keyword(s):  
State Of The Art ◽  
Named Entity Recognition ◽  
Recognition Task ◽  
Entity Recognition ◽  
Named Entity ◽  
Part Of Speech Tagging ◽  
Pos Tagging ◽  
Part Of Speech ◽  
Recent Developments ◽  
Future Work

We analyze neural network architectures that yield state of the art results on named entity recognition task and propose a number of new architectures for improving results even further. We have analyzed a number of ideas and approaches that researchers have used to achieve state of the art results in a variety of NLP tasks. In this work, we present a few architectures which we consider to be most likely to improve the existing state of the art solutions for named entity recognition task and part of speech tasks. The architectures are inspired by recent developments in multi-task learning. This work tests the hypothesis that NER and POS are related tasks and adding information about POS tags as input to the network can help achieve better NER results. And vice versa, information about NER tags can help solve the task of POS tagging. This work also contains the implementation of the network and results of the experiments together with the conclusions and future work.

scholarly journals A Comparison of Architectures and Pretraining Methods for Contextualized Multilingual Word Embeddings

2020 ◽  
Vol 34 (05) ◽  
pp. 9090-9097
Author(s):  
Niels Van der Heijden ◽  
Samira Abnar ◽  
Ekaterina Shutova
Keyword(s):  
Language Processing ◽  
State Of The Art ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Word Embeddings ◽  
Named Entity ◽  
Pos Tagging ◽  
Part Of Speech ◽  
Joint Training ◽  
Comprehensive Comparison

The lack of annotated data in many languages is a well-known challenge within the field of multilingual natural language processing (NLP). Therefore, many recent studies focus on zero-shot transfer learning and joint training across languages to overcome data scarcity for low-resource languages. In this work we (i) perform a comprehensive comparison of state-of-the-art multilingual word and sentence encoders on the tasks of named entity recognition (NER) and part of speech (POS) tagging; and (ii) propose a new method for creating multilingual contextualized word embeddings, compare it to multiple baselines and show that it performs at or above state-of-the-art level in zero-shot transfer settings. Finally, we show that our method allows for better knowledge sharing across languages in a joint training setting.

scholarly journals Zero-Resource Cross-Lingual Named Entity Recognition

2020 ◽  
Vol 34 (05) ◽  
pp. 7415-7423
Author(s):  
M Saiful Bari ◽  
Shafiq Joty ◽  
Prathyusha Jwalapuram
Keyword(s):  
Named Entity Recognition ◽  
Training Data ◽  
Fine Tuning ◽  
Entity Recognition ◽  
Adversarial Learning ◽  
Named Entity ◽  
Word Level ◽  
Parallel Data ◽  
Feature Augmentation ◽  
Cross Lingual

Recently, neural methods have achieved state-of-the-art (SOTA) results in Named Entity Recognition (NER) tasks for many languages without the need for manually crafted features. However, these models still require manually annotated training data, which is not available for many languages. In this paper, we propose an unsupervised cross-lingual NER model that can transfer NER knowledge from one language to another in a completely unsupervised way without relying on any bilingual dictionary or parallel data. Our model achieves this through word-level adversarial learning and augmented fine-tuning with parameter sharing and feature augmentation. Experiments on five different languages demonstrate the effectiveness of our approach, outperforming existing models by a good margin and setting a new SOTA for each language pair.

scholarly journals Towards the Construction of a Gold Standard Biomedical Corpus for the Romanian Language

Data ◽  
2018 ◽  
Vol 3 (4) ◽  
pp. 53 ◽  
Author(s):  
Maria Mitrofan ◽  
Verginica Barbu Mititelu ◽  
Grigorina Mitrofan
Keyword(s):  
Language Processing ◽  
Gold Standard ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Language Resources ◽  
Named Entities ◽  
Named Entity ◽  
Pos Tagging ◽  
Part Of Speech ◽  
Biomedical Named Entity Recognition

Gold standard corpora (GSCs) are essential for the supervised training and evaluation of systems that perform natural language processing (NLP) tasks. Currently, most of the resources used in biomedical NLP tasks are mainly in English. Little effort has been reported for other languages including Romanian and, thus, access to such language resources is poor. In this paper, we present the construction of the first morphologically and terminologically annotated biomedical corpus of the Romanian language (MoNERo), meant to serve as a gold standard for biomedical part-of-speech (POS) tagging and biomedical named entity recognition (bioNER). It contains 14,012 tokens distributed in three medical subdomains: cardiology, diabetes and endocrinology, extracted from books, journals and blogposts. In order to automatically annotate the corpus with POS tags, we used a Romanian tag set which has 715 labels, while diseases, anatomy, procedures and chemicals and drugs labels were manually annotated for bioNER with a Cohen Kappa coefficient of 92.8% and revealed the occurrence of 1877 medical named entities. The automatic annotation of the corpus has been manually checked. The corpus is publicly available and can be used to facilitate the development of NLP algorithms for the Romanian language.

South China Sea Conflicts Classification Using Named Entity Recognition (NER) and Part-of-Speech (POS) Tagging

2020 ◽  
Vol 10 (1) ◽  
Author(s):  
Nur Rafeeqkha Sulaiman ◽  
Maheyzah Md Siraj
Keyword(s):  
South China Sea ◽  
Language Processing ◽  
South China ◽  
Named Entity Recognition ◽  
Online News ◽  
Entity Recognition ◽  
China Sea ◽  
Named Entity ◽  
Pos Tagging ◽  
Part Of Speech

Internet connects everyone to everything globally. The existence of Internet eases people in completing daily tasks. Thanks to Internet, information is being digitalized and spread openly to the public. Online news articles not only provide us with useful and reliable information and reports, it also eases information extraction and gathering for research purposes especially in Natural Language Processing (NLP) and Machine Learning (ML). The topics regarding the South China Sea have been popular lately due to the rise of conflicts between several countries claim on the islands in the sea. Gathering data through Internet and online sources proves to be easy, but to process a huge amount data and to identify only useful information manually takes a longer time to complete. Extracting important features from a text document can be done by using one or a combination of feature extraction methods. Relevant information and the classification of news articles in relation to the conflicts in South China Sea need to be done. In this paper, a model is proposed to use Named Entity Recognition (NER) that search for and classifies important information regarding to the conflicts. In order to do that, a combination of Part-of-Speech (POS) and NER are needed to extract type of conflicts from the news.  This study also claims to classify news by using Conditional Random Field (CRF) algorithm and Multinomial Naïve Bayes (MNB) as classification methods by training and testing the data. 

scholarly journals A Probability based Classification of Named Entities for Malayalam Language combining Word, Part of Speech and Lexicalized features

2019 ◽  
Vol 8 (2) ◽  
pp. 839-842
Keyword(s):  
Named Entity Recognition ◽  
Entity Recognition ◽  
Supervised Machine Learning ◽  
Named Entities ◽  
Named Entity ◽  
Domain Specific ◽  
Part Of Speech ◽  
Classification Probability ◽  
Malayalam Language

Named Entity Recognition is the process wherein named entities which are designators of a sentence are identified. Designators of a sentence are domain specific. The proposed system identifies named entities in Malayalam language belonging to tourism domain which generally includes names of persons, places, organizations, dates etc. The system uses word, part of speech and lexicalized features to find the probability of a word belonging to a named entity category and to do the appropriate classification. Probability is calculated based on supervised machine learning using word and part of speech features present in a tagged training corpus and using certain rules applied based on lexicalized features.

Sign in / Sign up

Export Citation Format

Share Document