scholarly journals Corpus Augmentation for Neural Machine Translation with Chinese-Japanese Parallel Corpora

2019 ◽  
Vol 9 (10) ◽  
pp. 2036
Author(s):  
Jinyi Zhang ◽  
Tadahiro Matsumoto
Keyword(s):  
Machine Translation ◽  
Scientific Paper ◽  
Training Data ◽  
Word Alignment ◽  
Sentence Pair ◽  
Neural Machine Translation ◽  
Parallel Corpora ◽  
Translation Quality ◽  
Parallel Data ◽  
Source Sentence

The translation quality of Neural Machine Translation (NMT) systems depends strongly on the training data size. Sufficient amounts of parallel data are, however, not available for many language pairs. This paper presents a corpus augmentation method, which has two variations: one is for all language pairs, and the other is for the Chinese-Japanese language pair. The method uses both source and target sentences of the existing parallel corpus and generates multiple pseudo-parallel sentence pairs from a long parallel sentence pair containing punctuation marks as follows: (1) split the sentence pair into parallel partial sentences; (2) back-translate the target partial sentences; and (3) replace each partial sentence in the source sentence with the back-translated target partial sentence to generate pseudo-source sentences. The word alignment information, which is used to determine the split points, is modified with “shared Chinese character rates” in segments of the sentence pairs. The experiment results of the Japanese-Chinese and Chinese-Japanese translation with ASPEC-JC (Asian Scientific Paper Excerpt Corpus, Japanese-Chinese) show that the method substantially improves translation performance. We also supply the code (see Supplementary Materials) that can reproduce our proposed method.

scholarly journals Recurrent Stacking of Layers for Compact Neural Machine Translation Models

2019 ◽  
Vol 33 ◽  
pp. 6292-6299 ◽  
Author(s):  
Raj Dabre ◽  
Atsushi Fujita
Keyword(s):  
Machine Translation ◽  
Single Layer ◽  
Training Data ◽  
Neural Machine Translation ◽  
Parallel Corpora ◽  
Translation Quality ◽  
Sequence Generation ◽  
Sequence Modeling ◽  
Back Translation

In encoder-decoder based sequence-to-sequence modeling, the most common practice is to stack a number of recurrent, convolutional, or feed-forward layers in the encoder and decoder. While the addition of each new layer improves the sequence generation quality, this also leads to a significant increase in the number of parameters. In this paper, we propose to share parameters across all layers thereby leading to a recurrently stacked sequence-to-sequence model. We report on an extensive case study on neural machine translation (NMT) using our proposed method, experimenting with a variety of datasets. We empirically show that the translation quality of a model that recurrently stacks a single-layer 6 times, despite its significantly fewer parameters, approaches that of a model that stacks 6 different layers. We also show how our method can benefit from a prevalent way for improving NMT, i.e., extending training data with pseudo-parallel corpora generated by back-translation. We then analyze the effects of recurrently stacked layers by visualizing the attentions of models that use recurrently stacked layers and models that do not. Finally, we explore the limits of parameter sharing where we share even the parameters between the encoder and decoder in addition to recurrent stacking of layers.

scholarly journals Neural machine translation system for the Kazakh language based on synthetic corpora

2019 ◽  
Vol 252 ◽  
pp. 03006
Author(s):  
Ualsher Tukeyev ◽  
Aidana Karibayeva ◽  
Balzhan Abduali
Keyword(s):  
Machine Translation ◽  
Training Data ◽  
Translation System ◽  
Natural Languages ◽  
Neural Machine Translation ◽  
Translation Quality ◽  
Parallel Data ◽  
Machine Translation System ◽  
Turkic Languages

The lack of big parallel data is present for the Kazakh language. This problem seriously impairs the quality of machine translation from and into Kazakh. This article considers the neural machine translation of the Kazakh language on the basis of synthetic corpora. The Kazakh language belongs to the Turkic languages, which are characterised by rich morphology. Neural machine translation of natural languages requires large training data. The article will show the model for the creation of synthetic corpora, namely the generation of sentences based on complete suffixes for the Kazakh language. The novelty of this approach of the synthetic corpora generation for the Kazakh language is the generation of sentences on the basis of the complete system of suffixes of the Kazakh language. By using generated synthetic corpora we are improving the translation quality in neural machine translation of Kazakh-English and Kazakh-Russian pairs.

scholarly journals The Development of a Comprehensive Data Set for Systematic Studies of Machine Translation

2021 ◽  
pp. 248-262
Author(s):  
Jörg Tiedemann
Keyword(s):  
Machine Translation ◽  
Training Data ◽  
Sufficient Information ◽  
Data Set ◽  
Neural Machine Translation ◽  
High Resource ◽  
Language Groups ◽  
Comprehensive Data ◽  
Parallel Data ◽  
Initial Results

This paper presents our on-going efforts to develop a comprehensive data set and benchmark for machine translation beyond high-resource languages. The current release includes 500GB of compressed parallel data for almost 3,000 language pairs covering over 500 languages and language variants. We present the structure of the data set and demonstrate its use for systematic studies based on baseline experiments with multilingual neural machine translation between Finno-Ugric languages and other language groups. Our initial results show the capabilities of training effective multilingual translation models with skewed training data but also stress the shortcomings with low-resource settings and the difficulties to obtain sufficient information through straightforward transfer from related languages.

scholarly journals Augmenting Neural Machine Translation through Round-Trip Training Approach

2019 ◽  
Vol 9 (1) ◽  
pp. 268-278 ◽  
Author(s):  
Benyamin Ahmadnia ◽  
Bonnie J. Dorr
Keyword(s):  
Machine Translation ◽  
Training Data ◽  
Training Dataset ◽  
Round Trip ◽  
Neural Machine Translation ◽  
Low Resource ◽  
Translation Quality ◽  
High Resource ◽  
Training Approach ◽  
Language Pair

AbstractThe quality of Neural Machine Translation (NMT), as a data-driven approach, massively depends on quantity, quality and relevance of the training dataset. Such approaches have achieved promising results for bilingually high-resource scenarios but are inadequate for low-resource conditions. Generally, the NMT systems learn from millions of words from bilingual training dataset. However, human labeling process is very costly and time consuming. In this paper, we describe a round-trip training approach to bilingual low-resource NMT that takes advantage of monolingual datasets to address training data bottleneck, thus augmenting translation quality. We conduct detailed experiments on English-Spanish as a high-resource language pair as well as Persian-Spanish as a low-resource language pair. Experimental results show that this competitive approach outperforms the baseline systems and improves translation quality.

scholarly journals Incorporating Source-Side Phrase Structures into Neural Machine Translation

2019 ◽  
Vol 45 (2) ◽  
pp. 267-292 ◽  
Author(s):  
Akiko Eriguchi ◽  
Kazuma Hashimoto ◽  
Yoshimasa Tsuruoka
Keyword(s):  
Machine Translation ◽  
Syntactic Structure ◽  
Statistical Machine Translation ◽  
Training Data ◽  
Great Success ◽  
Data Set ◽  
Neural Machine Translation ◽  
Proposed Model ◽  
Source Sentence

Neural machine translation (NMT) has shown great success as a new alternative to the traditional Statistical Machine Translation model in multiple languages. Early NMT models are based on sequence-to-sequence learning that encodes a sequence of source words into a vector space and generates another sequence of target words from the vector. In those NMT models, sentences are simply treated as sequences of words without any internal structure. In this article, we focus on the role of the syntactic structure of source sentences and propose a novel end-to-end syntactic NMT model, which we call a tree-to-sequence NMT model, extending a sequence-to-sequence model with the source-side phrase structure. Our proposed model has an attention mechanism that enables the decoder to generate a translated word while softly aligning it with phrases as well as words of the source sentence. We have empirically compared the proposed model with sequence-to-sequence models in various settings on Chinese-to-Japanese and English-to-Japanese translation tasks. Our experimental results suggest that the use of syntactic structure can be beneficial when the training data set is small, but is not as effective as using a bi-directional encoder. As the size of training data set increases, the benefits of using a syntactic tree tends to diminish.

scholarly journals Synthesizing Parallel Data of User-Generated Texts with Zero-Shot Neural Machine Translation

2020 ◽  
Vol 8 ◽  
pp. 710-725
Author(s):  
Benjamin Marie ◽  
Atsushi Fujita
Keyword(s):  
Social Media ◽  
Machine Translation ◽  
Language Model ◽  
The Other ◽  
Neural Machine Translation ◽  
Translation Quality ◽  
Online Social Media ◽  
Parallel Data

Neural machine translation (NMT) systems are usually trained on clean parallel data. They can perform very well for translating clean in-domain texts. However, as demonstrated by previous work, the translation quality significantly worsens when translating noisy texts, such as user-generated texts (UGT) from online social media. Given the lack of parallel data of UGT that can be used to train or adapt NMT systems, we synthesize parallel data of UGT, exploiting monolingual data of UGT through crosslingual language model pre-training and zero-shot NMT systems. This paper presents two different but complementary approaches: One alters given clean parallel data into UGT-like parallel data whereas the other generates translations from monolingual data of UGT. On the MTNT translation tasks, we show that our synthesized parallel data can lead to better NMT systems for UGT while making them more robust in translating texts from various domains and styles.

scholarly journals Improving Stylized Neural Machine Translation with Iterative Dual Knowledge Transfer

2021 ◽  
Author(s):  
Xuanxuan Wu ◽  
Jian Liu ◽  
Xinjie Li ◽  
Jinan Xu ◽  
Yufeng Chen ◽  
...  
Keyword(s):  
Knowledge Transfer ◽  
Machine Translation ◽  
Large Scale ◽  
Training Data ◽  
Transfer Model ◽  
Paired Data ◽  
Style Transfer ◽  
Neural Machine Translation ◽  
Translation Model ◽  
Parallel Data

Stylized neural machine translation (NMT) aims to translate sentences of one style into sentences of another style, which is essential for the application of machine translation in a real-world scenario. However, a major challenge in this task is the scarcity of high-quality parallel data which is stylized paired. To address this problem, we propose an iterative dual knowledge transfer framework that utilizes informal training data of machine translation and formality style transfer data to create large-scale stylized paired data, for the training of stylized machine translation model. Specifically, we perform bidirectional knowledge transfer between translation model and text style transfer model iteratively through knowledge distillation. Then, we further propose a data-refinement module to process the noisy synthetic parallel data generated during knowledge transfer. Experiment results demonstrate the effectiveness of our method, achieving an improvement over the existing best model by 5 BLEU points on MTFC dataset. Meanwhile, extensive analyses illustrate our method can also improve the accuracy of formality style transfer.

scholarly journals “Bilingual Expert” Can Find Translation Errors

2019 ◽  
Vol 33 ◽  
pp. 6367-6374 ◽  
Author(s):  
Kai Fan ◽  
Jiayi Wang ◽  
Bo Li ◽  
Fengming Zhou ◽  
Boxing Chen ◽  
...  
Keyword(s):  
Machine Translation ◽  
Statistical Machine Translation ◽  
Language Model ◽  
Target Language ◽  
Quality Estimation ◽  
Parallel Corpora ◽  
Translation Quality ◽  
Expert Model ◽  
Parallel Data ◽  
Translation Errors

The performances of machine translation (MT) systems are usually evaluated by the metric BLEU when the golden references are provided. However, in the case of model inference or production deployment, golden references are usually expensively available, such as human annotation with bilingual expertise. In order to address the issue of translation quality estimation (QE) without reference, we propose a general framework for automatic evaluation of the translation output for the QE task in the Conference on Statistical Machine Translation (WMT). We first build a conditional target language model with a novel bidirectional transformer, named neural bilingual expert model, which is pre-trained on large parallel corpora for feature extraction. For QE inference, the bilingual expert model can simultaneously produce the joint latent representation between the source and the translation, and real-valued measurements of possible erroneous tokens based on the prior knowledge learned from parallel data. Subsequently, the features will further be fed into a simple Bi-LSTM predictive model for quality estimation. The experimental results show that our approach achieves the state-of-the-art performance in most public available datasets of WMT 2017/2018 QE task.

scholarly journals A Diverse Data Augmentation Strategy for Low-Resource Neural Machine Translation

Information ◽  
2020 ◽  
Vol 11 (5) ◽  
pp. 255
Author(s):  
Yu Li ◽  
Xiao Li ◽  
Yating Yang ◽  
Rui Dong
Keyword(s):  
Machine Translation ◽  
English Translation ◽  
Data Augmentation ◽  
Sampling Strategy ◽  
Training Data ◽  
Neural Machine Translation ◽  
Low Resource ◽  
Parallel Data ◽  
Augmentation Strategy ◽  
Diverse Data

One important issue that affects the performance of neural machine translation is the scale of available parallel data. For low-resource languages, the amount of parallel data is not sufficient, which results in poor translation quality. In this paper, we propose a diversity data augmentation method that does not use extra monolingual data. We expand the training data by generating diversity pseudo parallel data on the source and target sides. To generate diversity data, the restricted sampling strategy is employed at the decoding steps. Finally, we filter and merge origin data and synthetic parallel corpus to train the final model. In the experiment, the proposed approach achieved 1.96 BLEU points in the IWSLT2014 German–English translation tasks, which was used to simulate a low-resource language. Our approach also consistently and substantially obtained 1.0 to 2.0 BLEU improvement in three other low-resource translation tasks, including English–Turkish, Nepali–English, and Sinhala–English translation tasks.

Statistical Machine Translation with Scarce Resources Using Morpho-syntactic Information

2004 ◽  
Vol 30 (2) ◽  
pp. 181-204 ◽  
Author(s):  
Sonja Nießen ◽  
Hermann Ney
Keyword(s):  
Machine Translation ◽  
Statistical Machine Translation ◽  
Training Data ◽  
Target Language ◽  
Linguistic Knowledge ◽  
Parallel Corpora ◽  
Scarce Resources ◽  
Translation Quality ◽  
Sentence Level ◽  
Statistical Systems

In statistical machine translation, correspondences between the words in the source and the target language are learned from parallel corpora, and often little or no linguistic knowledge is used to structure the underlying models. In particular, existing statistical systems for machine translation often treat different inflected forms of the same lemma as if they were independent of one another. The bilingual training data can be better exploited by explicitly taking into account the interdependencies of related inflected forms. We propose the construction of hierarchical lexicon models on the basis of equivalence classes of words. In addition, we introduce sentence-level restructuring transformations which aim at the assimilation of word order in related sentences. We have systematically investigated the amount of bilingual training data required to maintain an acceptable quality of machine translation. The combination of the suggested methods for improving translation quality in frameworks with scarce resources has been successfully tested: We were able to reduce the amount of bilingual training data to less than 10% of the original corpus, while losing only 1.6% in translation quality. The improvement of the translation results is demonstrated on two German-English corpora taken from the Verbmobil task and the Nespole! task.

Sign in / Sign up

Export Citation Format

Share Document