scholarly journals Named Entity Correction in Neural Machine Translation Using the Attention Alignment Map

2021 ◽  
Vol 11 (15) ◽  
pp. 7026
Author(s):  
Jangwon Lee ◽  
Jungi Lee  ◽  
Minho Lee  ◽  
Gil-Jin Jang
Keyword(s):  
Machine Translation ◽  
Named Entity Recognition ◽  
Network Models ◽  
Entity Recognition ◽  
Neural Network Models ◽  
Named Entities ◽  
Neural Machine Translation ◽  
Named Entity ◽  
Input And Output ◽  
Artificial Neural Network Models

Neural machine translation (NMT) methods based on various artificial neural network models have shown remarkable performance in diverse tasks and have become mainstream for machine translation currently. Despite the recent successes of NMT applications, a predefined vocabulary is still required, meaning that it cannot cope with out-of-vocabulary (OOV) or rarely occurring words. In this paper, we propose a postprocessing method for correcting machine translation outputs using a named entity recognition (NER) model to overcome the problem of OOV words in NMT tasks. We use attention alignment mapping (AAM) between the named entities of input and output sentences, and mistranslated named entities are corrected using word look-up tables. The proposed method corrects named entities only, so it does not require retraining of existing NMT models. We carried out translation experiments on a Chinese-to-Korean translation task for Korean historical documents, and the evaluation results demonstrated that the proposed method improved the bilingual evaluation understudy (BLEU) score by 3.70 from the baseline.

An Experimental Study of Hybrid Machine Learning Models for Extracting Named Entities

10.29007/dp5m ◽  
2019 ◽  
Author(s):  
Lei Jiang ◽  
Elena Bolshakova
Keyword(s):  
Neural Network ◽  
Conditional Random Fields ◽  
Named Entity Recognition ◽  
Network Models ◽  
Entity Recognition ◽  
Neural Network Models ◽  
Named Entities ◽  
Hybrid Neural Network ◽  
Named Entity ◽  
Two Hybrid

The paper describes two hybrid neural network models for named entity recognition (NER) in texts, as well as results of experiments with them. The first model, namely Bi-LSTM-CRF, is known and used for NER, while the other model named Gated-CNN- CRF is proposed in this work. It combines convolutional neural network (CNN), gated linear units, and conditional random fields (CRF). Both models were tested for NER on three different language datasets, for English, Russian, and Chinese. All resulted scores of precision, recall and F1-measure for both models are close to the state-of-the-art for NER, and for the English dataset CoNLL-2003, Gated-CNN-CRF model achieves 92.66 of F1-measure, outperforming the known result.

scholarly journals Quantifying Uncertainties in Natural Language Processing Tasks

2019 ◽  
Vol 33 ◽  
pp. 7322-7329 ◽  
Author(s):  
Yijun Xiao ◽  
William Yang Wang
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Intelligent Systems ◽  
Named Entity Recognition ◽  
Network Models ◽  
Entity Recognition ◽  
Neural Network Models ◽  
Named Entity ◽  
Modeling Uncertainties

Reliable uncertainty quantification is a first step towards building explainable, transparent, and accountable artificial intelligent systems. Recent progress in Bayesian deep learning has made such quantification realizable. In this paper, we propose novel methods to study the benefits of characterizing model and data uncertainties for natural language processing (NLP) tasks. With empirical experiments on sentiment analysis, named entity recognition, and language modeling using convolutional and recurrent neural network models, we show that explicitly modeling uncertainties is not only necessary to measure output confidence levels, but also useful at enhancing model performances in various NLP tasks.

scholarly journals Korean-Vietnamese Neural Machine Translation with Named Entity Recognition and Part-of-Speech Tags

2020 ◽  
Vol E103.D (4) ◽  
pp. 866-873
Author(s):  
Van-Hai VU ◽  
Quang-Phuoc NGUYEN ◽  
Kiem-Hieu NGUYEN ◽  
Joon-Choul SHIN ◽  
Cheol-Young OCK
Keyword(s):  
Machine Translation ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Neural Machine Translation ◽  
Named Entity ◽  
Part Of Speech

scholarly journals Cross-type Biomedical Named Entity Recognition with Deep Multi-Task Learning

2018 ◽  
Author(s):  
Xuan Wang ◽  
Yu Zhang ◽  
Xiang Ren ◽  
Yuhao Zhang ◽  
Marinka Zitnik ◽  
...  
Keyword(s):  
State Of The Art ◽  
Named Entity Recognition ◽  
Network Models ◽  
Training Data ◽  
Entity Recognition ◽  
Neural Network Models ◽  
Named Entity ◽  
Task Learning ◽  
Level Information ◽  
Biomedical Named Entity Recognition

AbstractMotivationState-of-the-art biomedical named entity recognition (BioNER) systems often require handcrafted features specific to each entity type, such as genes, chemicals and diseases. Although recent studies explored using neural network models for BioNER to free experts from manual feature engineering, the performance remains limited by the available training data for each entity type.ResultsWe propose a multi-task learning framework for BioNER to collectively use the training data of different types of entities and improve the performance on each of them. In experiments on 15 benchmark BioNER datasets, our multi-task model achieves substantially better performance compared with state-of-the-art BioNER systems and baseline neural sequence labeling models. Further analysis shows that the large performance gains come from sharing character- and word-level information among relevant biomedical entities across differently labeled corpora.AvailabilityOur source code is available at https://github.com/yuzhimanhua/[email protected], [email protected].

scholarly journals Coreference Aware Representation Learning for Neural Named Entity Recognition

2019 ◽  
Author(s):  
Zeyu Dai ◽  
Hongliang Fei ◽  
Ping Li
Keyword(s):  
Neural Network ◽  
State Of The Art ◽  
Structural Information ◽  
Named Entity Recognition ◽  
Network Models ◽  
Third Party ◽  
Entity Recognition ◽  
Neural Network Models ◽  
Named Entity ◽  
Art Performance

Recent neural network models have achieved state-of-the-art performance on the task of named entity recognition (NER). However, previous neural network models typically treat the input sentences as a linear sequence of words but ignore rich structural information, such as the coreference relations among non-adjacent words, phrases or entities. In this paper, we propose a novel approach to learn coreference-aware word representations for the NER task at the document level. In particular, we enrich the well-known neural architecture ``CNN-BiLSTM-CRF'' with a coreference layer on top of the BiLSTM layer to incorporate coreferential relations. Furthermore, we introduce the coreference regularization to ensure the coreferential entities to share similar representations and consistent predictions within the same coreference cluster. Our proposed model achieves new state-of-the-art performance on two NER benchmarks: CoNLL-2003 and OntoNotes v5.0. More importantly, we demonstrate that our framework does not rely on gold coreference knowledge, and can still work well even when the coreferential relations are generated by a third-party toolkit.

Language Model Pre-training Method in Machine Translation Based on Named Entity Recognition

2020 ◽  
Vol 29 (07n08) ◽  
pp. 2040021
Author(s):  
Zhen Li ◽  
Dan Qu ◽  
Chaojie Xie ◽  
Wenlin Zhang ◽  
Yanxia Li
Keyword(s):  
Machine Translation ◽  
Semantic Information ◽  
Language Model ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Model Parameters ◽  
Training Method ◽  
Neural Machine Translation ◽  
Parallel Corpora ◽  
Named Entity

Neural Machine Translation (NMT) model has become the mainstream technology in machine translation. The supervised neural machine translation model trains with abundant of sentence-level parallel corpora. But for low-resources language or dialect with no such corpus available, it is difficult to achieve good performance. Researchers began to focus on unsupervised neural machine translation (UNMT) that monolingual corpus as training data. UNMT need to construct the language model (LM) which learns semantic information from the monolingual corpus. This paper focuses on the pre-training of LM in unsupervised machine translation and proposes a pre-training method, NER-MLM (named entity recognition masked language model). Through performing NER, the proposed method can obtain better semantic information and language model parameters with better training results. In the unsupervised machine translation task, the BLEU scores on the WMT’16 English–French, English–German, data sets are 35.30, 27.30 respectively. To the best of our knowledge, this is the highest results in the field of UNMT reported so far.

Sign in / Sign up

Export Citation Format

Share Document