scholarly journals Effective Use of Target-side Context for Neural Machine Translation

2021 ◽  
Vol 28 (4) ◽  
pp. 1162-1183
Author(s):  
Hideya Mino ◽  
Hitoshi Ito ◽  
Isao Goto ◽  
Ichiro Yamada ◽  
Takenobu Tokunaga
Keyword(s):  
Machine Translation ◽  
Neural Machine Translation ◽  
Effective Use ◽  
Target Side

scholarly journals Effective Use of Target-side Context for Neural Machine Translation

2020 ◽  
Author(s):  
Hideya Mino ◽  
Hitoshi Ito ◽  
Isao Goto ◽  
Ichiro Yamada ◽  
Takenobu Tokunaga
Keyword(s):  
Machine Translation ◽  
Neural Machine Translation ◽  
Effective Use ◽  
Target Side

scholarly journals Tied Transformers: Neural Machine Translation with Shared Encoder and Decoder

2019 ◽  
Vol 33 ◽  
pp. 5466-5473 ◽  
Author(s):  
Yingce Xia ◽  
Tianyu He ◽  
Xu Tan ◽  
Fei Tian ◽  
Di He ◽  
...  
Keyword(s):  
Machine Translation ◽  
English Translation ◽  
State Of The Art ◽  
Compact Model ◽  
Word Embeddings ◽  
Simple Method ◽  
Neural Machine Translation ◽  
German Translation ◽  
One Step ◽  
Target Side

Sharing source and target side vocabularies and word embeddings has been a popular practice in neural machine translation (briefly, NMT) for similar languages (e.g., English to French or German translation). The success of such wordlevel sharing motivates us to move one step further: we consider model-level sharing and tie the whole parts of the encoder and decoder of an NMT model. We share the encoder and decoder of Transformer (Vaswani et al. 2017), the state-of-the-art NMT model, and obtain a compact model named Tied Transformer. Experimental results demonstrate that such a simple method works well for both similar and dissimilar language pairs. We empirically verify our framework for both supervised NMT and unsupervised NMT: we achieve a 35.52 BLEU score on IWSLT 2014 German to English translation, 28.98/29.89 BLEU scores on WMT 2014 English to German translation without/with monolingual data, and a 22.05 BLEU score on WMT 2016 unsupervised German to English translation.

scholarly journals Synchronous Bidirectional Neural Machine Translation

2019 ◽  
Vol 7 ◽  
pp. 91-105 ◽  
Author(s):  
Long Zhou ◽  
Jiajun Zhang ◽  
Chengqing Zong
Keyword(s):  
Machine Translation ◽  
Large Scale ◽  
State Of The Art ◽  
Target Language ◽  
Single Model ◽  
Neural Machine Translation ◽  
German Translation ◽  
Transformer Model ◽  
Target Side ◽  
Future Information

Existing approaches to neural machine translation (NMT) generate the target language sequence token-by-token from left to right. However, this kind of unidirectional decoding framework cannot make full use of the target-side future contexts which can be produced in a right-to-left decoding direction, and thus suffers from the issue of unbalanced outputs. In this paper, we introduce a synchronous bidirectional–neural machine translation (SB-NMT) that predicts its outputs using left-to-right and right-to-left decoding simultaneously and interactively, in order to leverage both of the history and future information at the same time. Specifically, we first propose a new algorithm that enables synchronous bidirectional decoding in a single model. Then, we present an interactive decoding model in which left-to-right (right-to-left) generation does not only depend on its previously generated outputs, but also relies on future contexts predicted by right-to-left (left-to-right) decoding. We extensively evaluate the proposed SB-NMT model on large-scale NIST Chinese-English, WMT14 English-German, and WMT18 Russian-English translation tasks. Experimental results demonstrate that our model achieves significant improvements over the strong Transformer model by 3.92, 1.49, and 1.04 BLEU points, respectively, and obtains the state-of-the-art per- formance on Chinese-English and English- German translation tasks. 1

scholarly journals Tag-less Back-Translation

2021 ◽  
Author(s):  
Idris Abdulmumin ◽  
Bashir Shehu Galadanci ◽  
Aliyu Garba
Keyword(s):  
Machine Translation ◽  
Domain Adaptation ◽  
Fine Tuning ◽  
Huge Amount ◽  
Neural Machine Translation ◽  
Translation Model ◽  
Parallel Data ◽  
Back Translation ◽  
Authentic Data ◽  
Target Side

Abstract An effective method to generate a large number of parallel sentences for training improved neural machine translation (NMT) systems is the use of the back-translations of the target-side monolingual data. The standard back-translation method has been shown to be unable to efficiently utilize the available huge amount of existing monolingual data because of the inability of translation models to differentiate between the authentic and synthetic parallel data during training. Tagging, or using gates, has been used to enable translation models to distinguish between synthetic and authentic data, improving standard back-translation and also enabling the use of iterative back-translation on language pairs that underperformed using standard back-translation. In this work, we approach back-translation as a domain adaptation problem, eliminating the need for explicit tagging. In the approach - tag-less back-translation - the synthetic and authentic parallel data are treated as out-of-domain and in-domain data respectively and, through pre-training and fine-tuning, the translation model is shown to be able to learn more efficiently from them during training. Experimental results have shown that the approach outperforms the standard and tagged back-translation approaches on low resource English-Vietnamese and English-German neural machine translation.

scholarly journals Minimizing the Bag-of-Ngrams Difference for Non-Autoregressive Neural Machine Translation

2020 ◽  
Vol 34 (01) ◽  
pp. 198-205
Author(s):  
Chenze Shao ◽  
Jinchao Zhang ◽  
Yang Feng ◽  
Fandong Meng ◽  
Jie Zhou
Keyword(s):  
Machine Translation ◽  
Cross Entropy ◽  
Sequential Dependency ◽  
Weak Correlation ◽  
Neural Machine Translation ◽  
Translation Quality ◽  
Word Level ◽  
Translation Errors ◽  
Training Objective ◽  
Target Side

Non-Autoregressive Neural Machine Translation (NAT) achieves significant decoding speedup through generating target words independently and simultaneously. However, in the context of non-autoregressive translation, the word-level cross-entropy loss cannot model the target-side sequential dependency properly, leading to its weak correlation with the translation quality. As a result, NAT tends to generate influent translations with over-translation and under-translation errors. In this paper, we propose to train NAT to minimize the Bag-of-Ngrams (BoN) difference between the model output and the reference sentence. The bag-of-ngrams training objective is differentiable and can be efficiently calculated, which encourages NAT to capture the target-side sequential dependency and correlates well with the translation quality. We validate our approach on three translation tasks and show that our approach largely outperforms the NAT baseline by about 5.0 BLEU scores on WMT14 En↔De and about 2.5 BLEU scores on WMT16 En↔Ro.

scholarly journals Pre-Reordering for Neural Machine Translation: Helpful or Harmful?

2017 ◽  
Vol 108 (1) ◽  
pp. 171-182 ◽  
Author(s):  
Jinhua Du ◽  
Andy Way
Keyword(s):  
Machine Translation ◽  
Statistical Machine Translation ◽  
Word Class ◽  
Word Embeddings ◽  
Neural Machine Translation ◽  
Parts Of Speech ◽  
Translation Quality ◽  
The Impact ◽  
Japanese English ◽  
Target Side

AbstractPre-reordering, a preprocessing to make the source-side word orders close to those of the target side, has been proven very helpful for statistical machine translation (SMT) in improving translation quality. However, is it the case in neural machine translation (NMT)? In this paper, we firstly investigate the impact of pre-reordered source-side data on NMT, and then propose to incorporate features for the pre-reordering model in SMT as input factors into NMT (factored NMT). The features, namely parts-of-speech (POS), word class and reordered index, are encoded as feature vectors and concatenated to the word embeddings to provide extra knowledge for NMT. Pre-reordering experiments conducted on Japanese↔English and Chinese↔English show that pre-reordering the source-side data for NMT is redundant and NMT models trained on pre-reordered data deteriorate translation performance. However, factored NMT using SMT-based pre-reordering features on Japanese→English and Chinese→English is beneficial and can further improve by 4.48 and 5.89 relative BLEU points, respectively, compared to the baseline NMT system.

Sign in / Sign up

Export Citation Format

Share Document