scholarly journals Machine Translation in Low-Resource Languages by an Adversarial Neural Network

2021 ◽  
Vol 11 (22) ◽  
pp. 10860
Author(s):  
Mengtao Sun ◽  
Hao Wang ◽  
Mark Pasquine ◽  
Ibrahim A. Hameed
Keyword(s):  
Machine Translation ◽  
Transfer Learning ◽  
Grammatical Structure ◽  
Neural Machine Translation ◽  
Low Resource ◽  
High Resource ◽  
Learning Techniques ◽  
Good Potential ◽  
Target Languages ◽  
Cross Lingual

Existing Sequence-to-Sequence (Seq2Seq) Neural Machine Translation (NMT) shows strong capability with High-Resource Languages (HRLs). However, this approach poses serious challenges when processing Low-Resource Languages (LRLs), because the model expression is limited by the training scale of parallel sentence pairs. This study utilizes adversary and transfer learning techniques to mitigate the lack of sentence pairs in LRL corpora. We propose a new Low resource, Adversarial, Cross-lingual (LAC) model for NMT. In terms of the adversary technique, LAC model consists of a generator and discriminator. The generator is a Seq2Seq model that produces the translations from source to target languages, while the discriminator measures the gap between machine and human translations. In addition, we introduce transfer learning on LAC model to help capture the features in rare resources because some languages share the same subject-verb-object grammatical structure. Rather than using the entire pretrained LAC model, we separately utilize the pretrained generator and discriminator. The pretrained discriminator exhibited better performance in all experiments. Experimental results demonstrate that the LAC model achieves higher Bilingual Evaluation Understudy (BLEU) scores and has good potential to augment LRL translations.

scholarly journals Evaluating the Cross-Lingual Effectiveness of Massively Multilingual Neural Machine Translation

2020 ◽  
Vol 34 (05) ◽  
pp. 8854-8861 ◽  
Author(s):  
Aditya Siddhant ◽  
Melvin Johnson ◽  
Henry Tsai ◽  
Naveen Ari ◽  
Jason Riesa ◽  
...  
Keyword(s):  
Machine Translation ◽  
Transfer Learning ◽  
Single Model ◽  
Neural Machine Translation ◽  
Low Resource ◽  
Sequence Labeling ◽  
Transfer Capability ◽  
Learning Scenarios ◽  
The Cross ◽  
Cross Lingual

The recently proposed massively multilingual neural machine translation (NMT) system has been shown to be capable of translating over 100 languages to and from English within a single model (Aharoni, Johnson, and Firat 2019). Its improved translation performance on low resource languages hints at potential cross-lingual transfer capability for downstream tasks. In this paper, we evaluate the cross-lingual effectiveness of representations from the encoder of a massively multilingual NMT model on 5 downstream classification and sequence labeling tasks covering a diverse set of over 50 languages. We compare against a strong baseline, multilingual BERT (mBERT) (Devlin et al. 2018), in different cross-lingual transfer learning scenarios and show gains in zero-shot transfer in 4 out of these 5 tasks.

scholarly journals A Joint Back-Translation and Transfer Learning Method for Low-Resource Neural Machine Translation

2020 ◽  
Vol 2020 ◽  
pp. 1-11
Author(s):  
Gong-Xu Luo ◽  
Ya-Ting Yang ◽  
Rui Dong ◽  
Yan-Hong Chen ◽  
Wen-Bo Zhang
Keyword(s):  
Machine Translation ◽  
Transfer Learning ◽  
Large Scale ◽  
Data Augmentation ◽  
Training Methods ◽  
Learning Method ◽  
Neural Machine Translation ◽  
Low Resource ◽  
Parallel Data ◽  
Back Translation

Neural machine translation (NMT) for low-resource languages has drawn great attention in recent years. In this paper, we propose a joint back-translation and transfer learning method for low-resource languages. It is widely recognized that data augmentation methods and transfer learning methods are both straight forward and effective ways for low-resource problems. However, existing methods, which utilize one of these methods alone, limit the capacity of NMT models for low-resource problems. In order to make full use of the advantages of existing methods and further improve the translation performance of low-resource languages, we propose a new method to perfectly integrate the back-translation method with mainstream transfer learning architectures, which can not only initialize the NMT model by transferring parameters of the pretrained models, but also generate synthetic parallel data by translating large-scale monolingual data of the target side to boost the fluency of translations. We conduct experiments to explore the effectiveness of the joint method by incorporating back-translation into the parent-child and the hierarchical transfer learning architecture. In addition, different preprocessing and training methods are explored to get better performance. Experimental results on Uygur-Chinese and Turkish-English translation demonstrate the superiority of the proposed method over the baselines that use single methods.

scholarly journals Augmenting Neural Machine Translation through Round-Trip Training Approach

2019 ◽  
Vol 9 (1) ◽  
pp. 268-278 ◽  
Author(s):  
Benyamin Ahmadnia ◽  
Bonnie J. Dorr
Keyword(s):  
Machine Translation ◽  
Training Data ◽  
Training Dataset ◽  
Round Trip ◽  
Neural Machine Translation ◽  
Low Resource ◽  
Translation Quality ◽  
High Resource ◽  
Training Approach ◽  
Language Pair

AbstractThe quality of Neural Machine Translation (NMT), as a data-driven approach, massively depends on quantity, quality and relevance of the training dataset. Such approaches have achieved promising results for bilingually high-resource scenarios but are inadequate for low-resource conditions. Generally, the NMT systems learn from millions of words from bilingual training dataset. However, human labeling process is very costly and time consuming. In this paper, we describe a round-trip training approach to bilingual low-resource NMT that takes advantage of monolingual datasets to address training data bottleneck, thus augmenting translation quality. We conduct detailed experiments on English-Spanish as a high-resource language pair as well as Persian-Spanish as a low-resource language pair. Experimental results show that this competitive approach outperforms the baseline systems and improves translation quality.

scholarly journals Cross-Lingual Pre-Training Based Transfer for Zero-Shot Neural Machine Translation

2020 ◽  
Vol 34 (01) ◽  
pp. 115-122 ◽  
Author(s):  
Baijun Ji ◽  
Zhirui Zhang ◽  
Xiangyu Duan ◽  
Min Zhang ◽  
Boxing Chen ◽  
...  
Keyword(s):  
Machine Translation ◽  
Transfer Learning ◽  
Large Scale ◽  
Feature Space ◽  
Target Language ◽  
Smooth Transition ◽  
Training Methods ◽  
Neural Machine Translation ◽  
Cross Lingual ◽  
Effective Transfer

Transfer learning between different language pairs has shown its effectiveness for Neural Machine Translation (NMT) in low-resource scenario. However, existing transfer methods involving a common target language are far from success in the extreme scenario of zero-shot translation, due to the language space mismatch problem between transferor (the parent model) and transferee (the child model) on the source side. To address this challenge, we propose an effective transfer learning approach based on cross-lingual pre-training. Our key idea is to make all source languages share the same feature space and thus enable a smooth transition for zero-shot translation. To this end, we introduce one monolingual pre-training method and two bilingual pre-training methods to obtain a universal encoder for different languages. Once the universal encoder is constructed, the parent model built on such encoder is trained with large-scale annotated data and then directly applied in zero-shot translation scenario. Experiments on two public datasets show that our approach significantly outperforms strong pivot-based baseline and various multilingual NMT approaches.

scholarly journals Transfer Learning for Low-Resource Neural Machine Translation

2016 ◽  
Author(s):  
Barret Zoph ◽  
Deniz Yuret ◽  
Jonathan May ◽  
Kevin Knight
Keyword(s):  
Machine Translation ◽  
Transfer Learning ◽  
Neural Machine Translation ◽  
Low Resource

scholarly journals Hierarchical Transfer Learning Architecture for Low-Resource Neural Machine Translation

IEEE Access ◽  
2019 ◽  
Vol 7 ◽  
pp. 154157-154166 ◽  
Author(s):  
Gongxu Luo ◽  
Yating Yang ◽  
Yang Yuan ◽  
Zhanheng Chen ◽  
Aizimaiti Ainiwaer
Keyword(s):  
Machine Translation ◽  
Transfer Learning ◽  
Neural Machine Translation ◽  
Low Resource

scholarly journals A Survey on Low-Resource Neural Machine Translation

2021 ◽  
Author(s):  
Rui Wang ◽  
Xu Tan ◽  
Renqian Luo ◽  
Tao Qin ◽  
Tie-Yan Liu
Keyword(s):  
Machine Translation ◽  
Large Scale ◽  
State Of The Art ◽  
Neural Machine Translation ◽  
Modal Data ◽  
Low Resource ◽  
Resource Setting ◽  
Low Resource Setting ◽  
Parallel Data ◽  
Target Languages

Neural approaches have achieved state-of-the-art accuracy on machine translation but suffer from the high cost of collecting large scale parallel data. Thus, a lot of research has been conducted for neural machine translation (NMT) with very limited parallel data, i.e., the low-resource setting. In this paper, we provide a survey for low-resource NMT and classify related works into three categories according to the auxiliary data they used: (1) exploiting monolingual data of source and/or target languages, (2) exploiting data from auxiliary languages, and (3) exploiting multi-modal data. We hope that our survey can help researchers to better understand this field and inspire them to design better algorithms, and help industry practitioners to choose appropriate algorithms for their applications.

Using syntax for improving phrase-based SMT in low-resource languages

2019 ◽  
Author(s):  
Hakimeh Fadaei ◽  
Heshaam Faili
Keyword(s):  
Machine Translation ◽  
Main Idea ◽  
Sources Of Information ◽  
Long Distance ◽  
Neural Machine Translation ◽  
Locality Property ◽  
Low Resource ◽  
High Resource ◽  
Local View ◽  
Syntactic Information

AbstractData driven approaches for machine translation, such as statistical and neural machine translation, suffer from sparsity when dealing with low-resource languages. In these cases, using other sources of information including linguistic information could alleviate the problem. In this article, we focus on the problem of word ordering in translation from a high-resource to a low-resource language and try to improve the quality by using syntactic information from the high-resource side. We propose some syntactic features based on Tree Adjoining Grammar (TAG) to be employed in a phrase-based SMT model in order to improve the word ordering. In this work, a set of synchronous TAG rules is extracted and used to estimate the probability of the phrase orders suggested by the phrase-based model. The main idea of the article is to handle the word ordering by using the extended domain of locality property of TAG and abstracting the long distance dependencies into a local view, which is a TAG elementary tree. The experiments on English–Persian and English–German translation showed that, by combining the proposed TAG-based reordering features with lexical and hierarchical reordering models, we gain significant improvements over the baseline and in comparison with a neural reordering model and a pre-reordering model.

Sign in / Sign up

Export Citation Format

Share Document