scholarly journals Towards Making the Most of Context in Neural Machine Translation

2020 ◽  
Author(s):  
Zaixiang Zheng ◽  
Xiang Yue ◽  
Shujian Huang ◽  
Jiajun Chen ◽  
Alexandra Birch
Keyword(s):  
Machine Translation ◽  
State Of The Art ◽  
Local Context ◽  
Unified Approach ◽  
Neural Machine Translation ◽  
Global Context ◽  
Level Data ◽  
Sentence Level ◽  
Target Languages ◽  
Document Level

Document-level machine translation manages to outperform sentence level models by a small margin, but have failed to be widely adopted. We argue that previous research did not make a clear use of the global context, and propose a new document-level NMT framework that deliberately models the local context of each sentence with the awareness of the global context of the document in both source and target languages. We specifically design the model to be able to deal with documents containing any number of sentences, including single sentences. This unified approach allows our model to be trained elegantly on standard datasets without needing to train on sentence and document level data separately. Experimental results demonstrate that our model outperforms Transformer baselines and previous document-level NMT models with substantial margins of up to 2.1 BLEU on state-of-the-art baselines. We also provide analyses which show the benefit of context far beyond the neighboring two or three sentences, which previous studies have typically incorporated.

Enhancing Lexical Translation Consistency for Document-Level Neural Machine Translation

2022 ◽  
Vol 21 (3) ◽  
pp. 1-21
Author(s):  
Xiaomian Kang ◽  
Yang Zhao ◽  
Jiajun Zhang ◽  
Chengqing Zong
Keyword(s):  
Machine Translation ◽  
English Translation ◽  
Test Set ◽  
Neural Machine Translation ◽  
Global Context ◽  
Translation Quality ◽  
Sentence Level ◽  
Document Level

Document-level neural machine translation (DocNMT) has yielded attractive improvements. In this article, we systematically analyze the discourse phenomena in Chinese-to-English translation, and focus on the most obvious ones, namely lexical translation consistency. To alleviate the lexical inconsistency, we propose an effective approach that is aware of the words which need to be translated consistently and constrains the model to produce more consistent translations. Specifically, we first introduce a global context extractor to extract the document context and consistency context, respectively. Then, the two types of global context are integrated into a encoder enhancer and a decoder enhancer to improve the lexical translation consistency. We create a test set to evaluate the lexical consistency automatically. Experiments demonstrate that our approach can significantly alleviate the lexical translation inconsistency. In addition, our approach can also substantially improve the translation quality compared to sentence-level Transformer.

scholarly journals Correct-and-Memorize: Learning to Translate from Interactive Revisions

2019 ◽  
Author(s):  
Rongxiang Weng ◽  
Hao Zhou ◽  
Shujian Huang ◽  
Lei Li ◽  
Yifan Xia ◽  
...  
Keyword(s):  
Machine Translation ◽  
State Of The Art ◽  
Translation Process ◽  
Neural Machine Translation ◽  
Human Interactions ◽  
Interactive Environment ◽  
Novel Method ◽  
Critical Revision ◽  
Target Languages ◽  
Translation Errors

State-of-the-art machine translation models are still not on a par with human translators. Previous work takes human interactions into the neural machine translation process to obtain improved results in target languages. However, not all model--translation errors are equal -- some are critical while others are minor. In the meanwhile, same translation mistakes occur repeatedly in similar context. To solve both issues, we propose CAMIT, a novel method for translating in an interactive environment. Our proposed method works with critical revision instructions, therefore allows human to correct arbitrary words in model-translated sentences. In addition, CAMIT learns from and softly memorizes revision actions based on the context, alleviating the issue of repeating mistakes. Experiments in both ideal and real interactive translation settings demonstrate that our proposed CAMIT enhances machine translation results significantly while requires fewer revision instructions from human compared to previous methods. 

scholarly journals A Survey on Low-Resource Neural Machine Translation

2021 ◽  
Author(s):  
Rui Wang ◽  
Xu Tan ◽  
Renqian Luo ◽  
Tao Qin ◽  
Tie-Yan Liu
Keyword(s):  
Machine Translation ◽  
Large Scale ◽  
State Of The Art ◽  
Neural Machine Translation ◽  
Modal Data ◽  
Low Resource ◽  
Resource Setting ◽  
Low Resource Setting ◽  
Parallel Data ◽  
Target Languages

Neural approaches have achieved state-of-the-art accuracy on machine translation but suffer from the high cost of collecting large scale parallel data. Thus, a lot of research has been conducted for neural machine translation (NMT) with very limited parallel data, i.e., the low-resource setting. In this paper, we provide a survey for low-resource NMT and classify related works into three categories according to the auxiliary data they used: (1) exploiting monolingual data of source and/or target languages, (2) exploiting data from auxiliary languages, and (3) exploiting multi-modal data. We hope that our survey can help researchers to better understand this field and inspire them to design better algorithms, and help industry practitioners to choose appropriate algorithms for their applications.

scholarly journals Joint Training for Pivot-based Neural Machine Translation

2017 ◽  
Author(s):  
Yong Cheng ◽  
Qian Yang ◽  
Yang Liu ◽  
Maosong Sun ◽  
Wei Xu
Keyword(s):  
Machine Translation ◽  
State Of The Art ◽  
Training Algorithm ◽  
Neural Machine Translation ◽  
Data Scarcity ◽  
Joint Training ◽  
Art Performance ◽  
Target Languages

While recent neural machine translation approaches have delivered state-of-the-art performance for resource-rich language pairs, they suffer from the data scarcity problem for resource-scarce language pairs. Although this problem can be alleviated by exploiting a pivot language to bridge the source and target languages, the source-to-pivot and pivot-to-target translation models are usually independently trained. In this work, we introduce a joint training algorithm for pivot-based neural machine translation. We propose three methods to connect the two models and enable them to interact with each other during training. Experiments on Europarl and WMT corpora show that joint training of source-to-pivot and pivot-to-target models leads to significant improvements over independent training across various languages.

scholarly journals Multilingual Denoising Pre-training for Neural Machine Translation

2020 ◽  
Vol 8 ◽  
pp. 726-742
Author(s):  
Yinhan Liu ◽  
Jiatao Gu ◽  
Naman Goyal ◽  
Xian Li ◽  
Sergey Edunov ◽  
...  
Keyword(s):  
Machine Translation ◽  
Large Scale ◽  
Complete Sequence ◽  
Neural Machine Translation ◽  
Training Corpus ◽  
Sentence Level ◽  
Significant Performance ◽  
Performance Gains ◽  
Multiple Languages ◽  
Document Level

This paper demonstrates that multilingual denoising pre-training produces significant performance gains across a wide variety of machine translation (MT) tasks. We present mBART—a sequence-to-sequence denoising auto-encoder pre-trained on large-scale monolingual corpora in many languages using the BART objective (Lewis et al., 2019 ). mBART is the first method for pre-training a complete sequence-to-sequence model by denoising full texts in multiple languages, whereas previous approaches have focused only on the encoder, decoder, or reconstructing parts of the text. Pre-training a complete model allows it to be directly fine-tuned for supervised (both sentence-level and document-level) and unsupervised machine translation, with no task- specific modifications. We demonstrate that adding mBART initialization produces performance gains in all but the highest-resource settings, including up to 12 BLEU points for low resource MT and over 5 BLEU points for many document-level and unsupervised models. We also show that it enables transfer to language pairs with no bi-text or that were not in the pre-training corpus, and present extensive analysis of which factors contribute the most to effective pre-training. 1

scholarly journals Context-Aware Neural Machine Translation for Korean Honorific Expressions

Electronics ◽  
2021 ◽  
Vol 10 (13) ◽  
pp. 1589
Author(s):  
Yongkeun Hwang ◽  
Yanghoon Kim ◽  
Kyomin Jung
Keyword(s):  
Machine Translation ◽  
Deep Neural Networks ◽  
Contextual Information ◽  
Context Aware ◽  
Neural Machine Translation ◽  
Translation Quality ◽  
Sentence Level ◽  
Proposed Model ◽  
The Given ◽  
The Relationship

Neural machine translation (NMT) is one of the text generation tasks which has achieved significant improvement with the rise of deep neural networks. However, language-specific problems such as handling the translation of honorifics received little attention. In this paper, we propose a context-aware NMT to promote translation improvements of Korean honorifics. By exploiting the information such as the relationship between speakers from the surrounding sentences, our proposed model effectively manages the use of honorific expressions. Specifically, we utilize a novel encoder architecture that can represent the contextual information of the given input sentences. Furthermore, a context-aware post-editing (CAPE) technique is adopted to refine a set of inconsistent sentence-level honorific translations. To demonstrate the efficacy of the proposed method, honorific-labeled test data is required. Thus, we also design a heuristic that labels Korean sentences to distinguish between honorific and non-honorific styles. Experimental results show that our proposed method outperforms sentence-level NMT baselines both in overall translation quality and honorific translations.

Analyzing Subword Techniques to Improve English to Sinhala Neural Machine Translation

2021 ◽  
pp. 2050017
Author(s):  
Rashmini Naranpanawa ◽  
Ravinga Perera ◽  
Thilakshi Fonseka ◽  
Uthayasanker Thayasivam
Keyword(s):  
Machine Translation ◽  
State Of The Art ◽  
Statistical Machine Translation ◽  
Translation System ◽  
Rare Word ◽  
Neural Machine Translation ◽  
Parallel Corpus ◽  
Low Resource ◽  
Word Level ◽  
Morphologically Rich Languages

Neural machine translation (NMT) is a remarkable approach which performs much better than the Statistical machine translation (SMT) models when there is an abundance of parallel corpus. However, vanilla NMT is primarily based upon word-level with a fixed vocabulary. Therefore, low resource morphologically rich languages such as Sinhala are mostly affected by the out of vocabulary (OOV) and Rare word problems. Recent advancements in subword techniques have opened up opportunities for low resource communities by enabling open vocabulary translation. In this paper, we extend our recently published state-of-the-art EN-SI translation system using the transformer and explore standard subword techniques on top of it to identify which subword approach has a greater effect on English Sinhala language pair. Our models demonstrate that subword segmentation strategies along with the state-of-the-art NMT can perform remarkably when translating English sentences into a rich morphology language regardless of a large parallel corpus.

scholarly journals Using Word Embeddings to Enforce Document-Level Lexical Consistency in Machine Translation

2017 ◽  
Vol 108 (1) ◽  
pp. 85-96 ◽  
Author(s):  
Eva Martínez Garcia ◽  
Carles Creus ◽  
Cristina España-Bonet ◽  
Lluís Màrquez
Keyword(s):  
Machine Translation ◽  
Evaluation Metrics ◽  
Automatic Evaluation ◽  
Word Embeddings ◽  
Standard Document ◽  
Sentence Level ◽  
Word Translation ◽  
Stochastic Mechanism ◽  
Document Level

Abstract We integrate new mechanisms in a document-level machine translation decoder to improve the lexical consistency of document translations. First, we develop a document-level feature designed to score the lexical consistency of a translation. This feature, which applies to words that have been translated into different forms within the document, uses word embeddings to measure the adequacy of each word translation given its context. Second, we extend the decoder with a new stochastic mechanism that, at translation time, allows to introduce changes in the translation oriented to improve its lexical consistency. We evaluate our system on English–Spanish document translation, and we conduct automatic and manual assessments of its quality. The automatic evaluation metrics, applied mainly at sentence level, do not reflect significant variations. On the contrary, the manual evaluation shows that the system dealing with lexical consistency is preferred over both a standard sentence-level and a standard document-level phrase-based MT systems.

scholarly journals Efficient Context-Aware Neural Machine Translation with Layer-Wise Weighting and Input-Aware Gating

2020 ◽  
Author(s):  
Hongfei Xu ◽  
Deyi Xiong ◽  
Josef van Genabith ◽  
Qiuhui Liu
Keyword(s):  
Machine Translation ◽  
Contextual Information ◽  
Computational Cost ◽  
Representation Learning ◽  
Vital Role ◽  
Context Aware ◽  
Neural Machine Translation ◽  
Gating Mechanism ◽  
Sentence Level ◽  
Parallel Data

Existing Neural Machine Translation (NMT) systems are generally trained on a large amount of sentence-level parallel data, and during prediction sentences are independently translated, ignoring cross-sentence contextual information. This leads to inconsistency between translated sentences. In order to address this issue, context-aware models have been proposed. However, document-level parallel data constitutes only a small part of the parallel data available, and many approaches build context-aware models based on a pre-trained frozen sentence-level translation model in a two-step training manner. The computational cost of these approaches is usually high. In this paper, we propose to make the most of layers pre-trained on sentence-level data in contextual representation learning, reusing representations from the sentence-level Transformer and significantly reducing the cost of incorporating contexts in translation. We find that representations from shallow layers of a pre-trained sentence-level encoder play a vital role in source context encoding, and propose to perform source context encoding upon weighted combinations of pre-trained encoder layers' outputs. Instead of separately performing source context and input encoding, we propose to iteratively and jointly encode the source input and its contexts and to generate input-aware context representations with a cross-attention layer and a gating mechanism, which resets irrelevant information in context encoding. Our context-aware Transformer model outperforms the recent CADec [Voita et al., 2019c] on the English-Russian subtitle data and is about twice as fast in training and decoding.

Sign in / Sign up

Export Citation Format

Share Document