scholarly journals Towards a Better Integration of Fuzzy Matches in Neural Machine Translation through Data Augmentation

Informatics ◽  
2021 ◽  
Vol 8 (1) ◽  
pp. 7
Author(s):  
Arda Tezcan ◽  
Bram Bulté ◽  
Bram Vanroy
Keyword(s):  
Machine Translation ◽  
Data Augmentation ◽  
Sentence Length ◽  
Added Value ◽  
Neural Machine Translation ◽  
Combination Technique ◽  
Translation Quality ◽  
Fuzzy Match ◽  
The Impact ◽  
Matching Techniques

We identify a number of aspects that can boost the performance of Neural Fuzzy Repair (NFR), an easy-to-implement method to integrate translation memory matches and neural machine translation (NMT). We explore various ways of maximising the added value of retrieved matches within the NFR paradigm for eight language combinations, using Transformer NMT systems. In particular, we test the impact of different fuzzy matching techniques, sub-word-level segmentation methods and alignment-based features on overall translation quality. Furthermore, we propose a fuzzy match combination technique that aims to maximise the coverage of source words. This is supplemented with an analysis of how translation quality is affected by input sentence length and fuzzy match score. The results show that applying a combination of the tested modifications leads to a significant increase in estimated translation quality over all baselines for all language combinations.

scholarly journals Pre-Reordering for Neural Machine Translation: Helpful or Harmful?

2017 ◽  
Vol 108 (1) ◽  
pp. 171-182 ◽  
Author(s):  
Jinhua Du ◽  
Andy Way
Keyword(s):  
Machine Translation ◽  
Statistical Machine Translation ◽  
Word Class ◽  
Word Embeddings ◽  
Neural Machine Translation ◽  
Parts Of Speech ◽  
Translation Quality ◽  
The Impact ◽  
Japanese English ◽  
Target Side

AbstractPre-reordering, a preprocessing to make the source-side word orders close to those of the target side, has been proven very helpful for statistical machine translation (SMT) in improving translation quality. However, is it the case in neural machine translation (NMT)? In this paper, we firstly investigate the impact of pre-reordered source-side data on NMT, and then propose to incorporate features for the pre-reordering model in SMT as input factors into NMT (factored NMT). The features, namely parts-of-speech (POS), word class and reordered index, are encoded as feature vectors and concatenated to the word embeddings to provide extra knowledge for NMT. Pre-reordering experiments conducted on Japanese↔English and Chinese↔English show that pre-reordering the source-side data for NMT is redundant and NMT models trained on pre-reordered data deteriorate translation performance. However, factored NMT using SMT-based pre-reordering features on Japanese→English and Chinese→English is beneficial and can further improve by 4.48 and 5.89 relative BLEU points, respectively, compared to the baseline NMT system.

scholarly journals Extremely low-resource neural machine translation for Asian languages

2020 ◽  
Vol 34 (4) ◽  
pp. 347-382
Author(s):  
Raphael Rubino ◽  
Benjamin Marie ◽  
Raj Dabre ◽  
Atushi Fujita ◽  
Masao Utiyama ◽  
...  
Keyword(s):  
Machine Translation ◽  
Data Augmentation ◽  
Statistical Machine Translation ◽  
Synthetic Data ◽  
Parameter Tuning ◽  
Data Generation ◽  
Neural Machine Translation ◽  
Low Resource ◽  
Translation Quality ◽  
Asian Languages

AbstractThis paper presents a set of effective approaches to handle extremely low-resource language pairs for self-attention based neural machine translation (NMT) focusing on English and four Asian languages. Starting from an initial set of parallel sentences used to train bilingual baseline models, we introduce additional monolingual corpora and data processing techniques to improve translation quality. We describe a series of best practices and empirically validate the methods through an evaluation conducted on eight translation directions, based on state-of-the-art NMT approaches such as hyper-parameter search, data augmentation with forward and backward translation in combination with tags and noise, as well as joint multilingual training. Experiments show that the commonly used default architecture of self-attention NMT models does not reach the best results, validating previous work on the importance of hyper-parameter tuning. Additionally, empirical results indicate the amount of synthetic data required to efficiently increase the parameters of the models leading to the best translation quality measured by automatic metrics. We show that the best NMT models trained on large amount of tagged back-translations outperform three other synthetic data generation approaches. Finally, comparison with statistical machine translation (SMT) indicates that extremely low-resource NMT requires a large amount of synthetic parallel data obtained with back-translation in order to close the performance gap with the preceding SMT approach.

scholarly journals A Systematic Study of Inner-Attention-Based Sentence Representations in Multilingual Neural Machine Translation

2020 ◽  
Vol 46 (2) ◽  
pp. 387-424 ◽  
Author(s):  
Raúl Vázquez ◽  
Alessandro Raganato ◽  
Mathias Creutz ◽  
Jörg Tiedemann
Keyword(s):  
Machine Translation ◽  
Improve Performance ◽  
Neural Machine Translation ◽  
Translation Quality ◽  
Intermediate Layers ◽  
Depth Analysis ◽  
Classification Tasks ◽  
The Impact ◽  
Meaning Representation

Neural machine translation has considerably improved the quality of automatic translations by learning good representations of input sentences. In this article, we explore a multilingual translation model capable of producing fixed-size sentence representations by incorporating an intermediate crosslingual shared layer, which we refer to as attention bridge. This layer exploits the semantics from each language and develops into a language-agnostic meaning representation that can be efficiently used for transfer learning. We systematically study the impact of the size of the attention bridge and the effect of including additional languages in the model. In contrast to related previous work, we demonstrate that there is no conflict between translation performance and the use of sentence representations in downstream tasks. In particular, we show that larger intermediate layers not only improve translation quality, especially for long sentences, but also push the accuracy of trainable classification tasks. Nevertheless, shorter representations lead to increased compression that is beneficial in non-trainable similarity tasks. Similarly, we show that trainable downstream tasks benefit from multilingual models, whereas additional language signals do not improve performance in non-trainable benchmarks. This is an important insight that helps to properly design models for specific applications. Finally, we also include an in-depth analysis of the proposed attention bridge and its ability to encode linguistic properties. We carefully analyze the information that is captured by individual attention heads and identify interesting patterns that explain the performance of specific settings in linguistic probing tasks.

scholarly journals Evaluating the Impact of Integrating Similar Translations into Neural Machine Translation

Information ◽  
2022 ◽  
Vol 13 (1) ◽  
pp. 19
Author(s):  
Arda Tezcan ◽  
Bram Bulté
Keyword(s):  
Error Analysis ◽  
Machine Translation ◽  
Data Augmentation ◽  
Training Data ◽  
Quality Improvements ◽  
Translation Quality ◽  
Automated Evaluation ◽  
Translation Errors ◽  
Different Characteristics ◽  
The Impact

Previous research has shown that simple methods of augmenting machine translation training data and input sentences with translations of similar sentences (or fuzzy matches), retrieved from a translation memory or bilingual corpus, lead to considerable improvements in translation quality, as assessed by a limited set of automatic evaluation metrics. In this study, we extend this evaluation by calculating a wider range of automated quality metrics that tap into different aspects of translation quality and by performing manual MT error analysis. Moreover, we investigate in more detail how fuzzy matches influence translations and where potential quality improvements could still be made by carrying out a series of quantitative analyses that focus on different characteristics of the retrieved fuzzy matches. The automated evaluation shows that the quality of NFR translations is higher than the NMT baseline in terms of all metrics. However, the manual error analysis did not reveal a difference between the two systems in terms of total number of translation errors; yet, different profiles emerged when considering the types of errors made. Finally, in our analysis of how fuzzy matches influence NFR translations, we identified a number of features that could be used to improve the selection of fuzzy matches for NFR data augmentation.

scholarly journals Context-Aware Neural Machine Translation for Korean Honorific Expressions

Electronics ◽  
2021 ◽  
Vol 10 (13) ◽  
pp. 1589
Author(s):  
Yongkeun Hwang ◽  
Yanghoon Kim ◽  
Kyomin Jung
Keyword(s):  
Machine Translation ◽  
Deep Neural Networks ◽  
Contextual Information ◽  
Context Aware ◽  
Neural Machine Translation ◽  
Translation Quality ◽  
Sentence Level ◽  
Proposed Model ◽  
The Given ◽  
The Relationship

Neural machine translation (NMT) is one of the text generation tasks which has achieved significant improvement with the rise of deep neural networks. However, language-specific problems such as handling the translation of honorifics received little attention. In this paper, we propose a context-aware NMT to promote translation improvements of Korean honorifics. By exploiting the information such as the relationship between speakers from the surrounding sentences, our proposed model effectively manages the use of honorific expressions. Specifically, we utilize a novel encoder architecture that can represent the contextual information of the given input sentences. Furthermore, a context-aware post-editing (CAPE) technique is adopted to refine a set of inconsistent sentence-level honorific translations. To demonstrate the efficacy of the proposed method, honorific-labeled test data is required. Thus, we also design a heuristic that labels Korean sentences to distinguish between honorific and non-honorific styles. Experimental results show that our proposed method outperforms sentence-level NMT baselines both in overall translation quality and honorific translations.

Improving thai-lao neural machine translation with similarity lexicon

2021 ◽  
pp. 1-10
Author(s):  
Zhiqiang Yu ◽  
Yuxin Huang ◽  
Junjun Guo
Keyword(s):  
Machine Translation ◽  
Semantic Information ◽  
Neural Machine Translation ◽  
Low Resource ◽  
Translation Quality ◽  
Decoder Architecture ◽  
Baseline System ◽  
Input Sentence ◽  
Resource Conditions ◽  
Language Pair

It has been shown that the performance of neural machine translation (NMT) drops starkly in low-resource conditions. Thai-Lao is a typical low-resource language pair of tiny parallel corpus, leading to suboptimal NMT performance on it. However, Thai and Lao have considerable similarities in linguistic morphology and have bilingual lexicon which is relatively easy to obtain. To use this feature, we first build a bilingual similarity lexicon composed of pairs of similar words. Then we propose a novel NMT architecture to leverage the similarity between Thai and Lao. Specifically, besides the prevailing sentence encoder, we introduce an extra similarity lexicon encoder into the conventional encoder-decoder architecture, by which the semantic information carried by the similarity lexicon can be represented. We further provide a simple mechanism in the decoder to balance the information representations delivered from the input sentence and the similarity lexicon. Our approach can fully exploit linguistic similarity carried by the similarity lexicon to improve translation quality. Experimental results demonstrate that our approach achieves significant improvements over the state-of-the-art Transformer baseline system and previous similar works.

Improved neural machine translation for low-resource English–Assamese pair

2021 ◽  
pp. 1-12
Author(s):  
Sahinur Rahman Laskar ◽  
Abdullah Faiz Ur Rahman Khilji ◽  
Partha Pakray ◽  
Sivaji Bandyopadhyay
Keyword(s):  
Machine Translation ◽  
Data Augmentation ◽  
Language Translation ◽  
Linguistically Diverse ◽  
Neural Machine Translation ◽  
Low Resource ◽  
Parallel Data ◽  
The World ◽  
Translation Accuracy ◽  
Vocabulary Problems

Language translation is essential to bring the world closer and plays a significant part in building a community among people of different linguistic backgrounds. Machine translation dramatically helps in removing the language barrier and allows easier communication among linguistically diverse communities. Due to the unavailability of resources, major languages of the world are accounted as low-resource languages. This leads to a challenging task of automating translation among various such languages to benefit indigenous speakers. This article investigates neural machine translation for the English–Assamese resource-poor language pair by tackling insufficient data and out-of-vocabulary problems. We have also proposed an approach of data augmentation-based NMT, which exploits synthetic parallel data and shows significantly improved translation accuracy for English-to-Assamese and Assamese-to-English translation and obtained state-of-the-art results.

scholarly journals Training Tips for the Transformer Model

2018 ◽  
Vol 110 (1) ◽  
pp. 43-70 ◽  
Author(s):  
Martin Popel ◽  
Ondřej Bojar
Keyword(s):  
Sentence Length ◽  
Batch Size ◽  
Critical Parameters ◽  
Neural Machine Translation ◽  
Training Time ◽  
Multiple Gpus ◽  
Translation Quality ◽  
Transformer Model ◽  
Data Constraints ◽  
And Training

Abstract This article describes our experiments in neural machine translation using the recent Tensor2Tensor framework and the Transformer sequence-to-sequence model (Vaswani et al., 2017). We examine some of the critical parameters that affect the final translation quality, memory usage, training stability and training time, concluding each experiment with a set of recommendations for fellow researchers. In addition to confirming the general mantra “more data and larger models”, we address scaling to multiple GPUs and provide practical tips for improved training regarding batch size, learning rate, warmup steps, maximum sentence length and checkpoint averaging. We hope that our observations will allow others to get better results given their particular hardware and data constraints.

scholarly journals Recurrent Stacking of Layers for Compact Neural Machine Translation Models

2019 ◽  
Vol 33 ◽  
pp. 6292-6299 ◽  
Author(s):  
Raj Dabre ◽  
Atsushi Fujita
Keyword(s):  
Machine Translation ◽  
Single Layer ◽  
Training Data ◽  
Neural Machine Translation ◽  
Parallel Corpora ◽  
Translation Quality ◽  
Sequence Generation ◽  
Sequence Modeling ◽  
Back Translation

In encoder-decoder based sequence-to-sequence modeling, the most common practice is to stack a number of recurrent, convolutional, or feed-forward layers in the encoder and decoder. While the addition of each new layer improves the sequence generation quality, this also leads to a significant increase in the number of parameters. In this paper, we propose to share parameters across all layers thereby leading to a recurrently stacked sequence-to-sequence model. We report on an extensive case study on neural machine translation (NMT) using our proposed method, experimenting with a variety of datasets. We empirically show that the translation quality of a model that recurrently stacks a single-layer 6 times, despite its significantly fewer parameters, approaches that of a model that stacks 6 different layers. We also show how our method can benefit from a prevalent way for improving NMT, i.e., extending training data with pseudo-parallel corpora generated by back-translation. We then analyze the effects of recurrently stacked layers by visualizing the attentions of models that use recurrently stacked layers and models that do not. Finally, we explore the limits of parameter sharing where we share even the parameters between the encoder and decoder in addition to recurrent stacking of layers.

Sign in / Sign up

Export Citation Format

Share Document