Neural machine translation of low-resource languages using SMT phrase pair injection

2020 ◽  
pp. 1-22
Author(s):  
Sukanta Sen ◽  
Mohammed Hasanuzzaman ◽  
Asif Ekbal ◽  
Pushpak Bhattacharyya ◽  
Andy Way
Keyword(s):  
Machine Translation ◽  
Large Scale ◽  
Production Systems ◽  
Statistical Machine Translation ◽  
Training Data ◽  
Original Training ◽  
Neural Machine Translation ◽  
Parallel Corpus ◽  
Low Resource ◽  
Better Than

Abstract Neural machine translation (NMT) has recently shown promising results on publicly available benchmark datasets and is being rapidly adopted in various production systems. However, it requires high-quality large-scale parallel corpus, and it is not always possible to have sufficiently large corpus as it requires time, money, and professionals. Hence, many existing large-scale parallel corpus are limited to the specific languages and domains. In this paper, we propose an effective approach to improve an NMT system in low-resource scenario without using any additional data. Our approach aims at augmenting the original training data by means of parallel phrases extracted from the original training data itself using a statistical machine translation (SMT) system. Our proposed approach is based on the gated recurrent unit (GRU) and transformer networks. We choose the Hindi–English, Hindi–Bengali datasets for Health, Tourism, and Judicial (only for Hindi–English) domains. We train our NMT models for 10 translation directions, each using only 5–23k parallel sentences. Experiments show the improvements in the range of 1.38–15.36 BiLingual Evaluation Understudy points over the baseline systems. Experiments show that transformer models perform better than GRU models in low-resource scenarios. In addition to that, we also find that our proposed method outperforms SMT—which is known to work better than the neural models in low-resource scenarios—for some translation directions. In order to further show the effectiveness of our proposed model, we also employ our approach to another interesting NMT task, for example, old-to-modern English translation, using a tiny parallel corpus of only 2.7K sentences. For this task, we use publicly available old-modern English text which is approximately 1000 years old. Evaluation for this task shows significant improvement over the baseline NMT.

Analyzing Subword Techniques to Improve English to Sinhala Neural Machine Translation

2021 ◽  
pp. 2050017
Author(s):  
Rashmini Naranpanawa ◽  
Ravinga Perera ◽  
Thilakshi Fonseka ◽  
Uthayasanker Thayasivam
Keyword(s):  
Machine Translation ◽  
State Of The Art ◽  
Statistical Machine Translation ◽  
Translation System ◽  
Rare Word ◽  
Neural Machine Translation ◽  
Parallel Corpus ◽  
Low Resource ◽  
Word Level ◽  
Morphologically Rich Languages

Neural machine translation (NMT) is a remarkable approach which performs much better than the Statistical machine translation (SMT) models when there is an abundance of parallel corpus. However, vanilla NMT is primarily based upon word-level with a fixed vocabulary. Therefore, low resource morphologically rich languages such as Sinhala are mostly affected by the out of vocabulary (OOV) and Rare word problems. Recent advancements in subword techniques have opened up opportunities for low resource communities by enabling open vocabulary translation. In this paper, we extend our recently published state-of-the-art EN-SI translation system using the transformer and explore standard subword techniques on top of it to identify which subword approach has a greater effect on English Sinhala language pair. Our models demonstrate that subword segmentation strategies along with the state-of-the-art NMT can perform remarkably when translating English sentences into a rich morphology language regardless of a large parallel corpus.

scholarly journals Comparing Statistical and Neural Machine Translation Performance on Hindi-To-Tamil and English-To-Tamil

Digital ◽  
2021 ◽  
Vol 1 (2) ◽  
pp. 86-102
Author(s):  
Akshai Ramesh ◽  
Venkatesh Balavadhani Parthasarathy ◽  
Rejwanul Haque ◽  
Andy Way
Keyword(s):  
Machine Translation ◽  
State Of The Art ◽  
Statistical Machine Translation ◽  
Training Data ◽  
Research Translation ◽  
Neural Machine Translation ◽  
Low Resource ◽  
Evaluation Scheme ◽  
Dominant Paradigm ◽  
Target Side

Phrase-based statistical machine translation (PB-SMT) has been the dominant paradigm in machine translation (MT) research for more than two decades. Deep neural MT models have been producing state-of-the-art performance across many translation tasks for four to five years. To put it another way, neural MT (NMT) took the place of PB-SMT a few years back and currently represents the state-of-the-art in MT research. Translation to or from under-resourced languages has been historically seen as a challenging task. Despite producing state-of-the-art results in many translation tasks, NMT still poses many problems such as performing poorly for many low-resource language pairs mainly because of its learning task’s data-demanding nature. MT researchers have been trying to address this problem via various techniques, e.g., exploiting source- and/or target-side monolingual data for training, augmenting bilingual training data, and transfer learning. Despite some success, none of the present-day benchmarks have entirely overcome the problem of translation in low-resource scenarios for many languages. In this work, we investigate the performance of PB-SMT and NMT on two rarely tested under-resourced language pairs, English-to-Tamil and Hindi-to-Tamil, taking a specialised data domain into consideration. This paper demonstrates our findings and presents results showing the rankings of our MT systems produced via a social media-based human evaluation scheme.

scholarly journals Pseudotext Injection and Advance Filtering of Low-Resource Corpus for Neural Machine Translation

2021 ◽  
Vol 2021 ◽  
pp. 1-10
Author(s):  
Michael Adjeisah ◽  
Guohua Liu ◽  
Douglas Omwenga Nyabuga ◽  
Richard Nuetey Nortey ◽  
Jinling Song
Keyword(s):  
Machine Translation ◽  
Language Processing ◽  
Training Data ◽  
Target Language ◽  
Similarity Metrics ◽  
Mahalanobis Distances ◽  
Parallel Corpora ◽  
Parallel Corpus ◽  
Low Resource ◽  
Sentence Level

Scaling natural language processing (NLP) to low-resourced languages to improve machine translation (MT) performance remains enigmatic. This research contributes to the domain on a low-resource English-Twi translation based on filtered synthetic-parallel corpora. It is often perplexing to learn and understand what a good-quality corpus looks like in low-resource conditions, mainly where the target corpus is the only sample text of the parallel language. To improve the MT performance in such low-resource language pairs, we propose to expand the training data by injecting synthetic-parallel corpus obtained by translating a monolingual corpus from the target language based on bootstrapping with different parameter settings. Furthermore, we performed unsupervised measurements on each sentence pair engaging squared Mahalanobis distances, a filtering technique that predicts sentence parallelism. Additionally, we extensively use three different sentence-level similarity metrics after round-trip translation. Experimental results on a diverse amount of available parallel corpus demonstrate that injecting pseudoparallel corpus and extensive filtering with sentence-level similarity metrics significantly improves the original out-of-the-box MT systems for low-resource language pairs. Compared with existing improvements on the same original framework under the same structure, our approach exhibits tremendous developments in BLEU and TER scores.

scholarly journals A Joint Back-Translation and Transfer Learning Method for Low-Resource Neural Machine Translation

2020 ◽  
Vol 2020 ◽  
pp. 1-11
Author(s):  
Gong-Xu Luo ◽  
Ya-Ting Yang ◽  
Rui Dong ◽  
Yan-Hong Chen ◽  
Wen-Bo Zhang
Keyword(s):  
Machine Translation ◽  
Transfer Learning ◽  
Large Scale ◽  
Data Augmentation ◽  
Training Methods ◽  
Learning Method ◽  
Neural Machine Translation ◽  
Low Resource ◽  
Parallel Data ◽  
Back Translation

Neural machine translation (NMT) for low-resource languages has drawn great attention in recent years. In this paper, we propose a joint back-translation and transfer learning method for low-resource languages. It is widely recognized that data augmentation methods and transfer learning methods are both straight forward and effective ways for low-resource problems. However, existing methods, which utilize one of these methods alone, limit the capacity of NMT models for low-resource problems. In order to make full use of the advantages of existing methods and further improve the translation performance of low-resource languages, we propose a new method to perfectly integrate the back-translation method with mainstream transfer learning architectures, which can not only initialize the NMT model by transferring parameters of the pretrained models, but also generate synthetic parallel data by translating large-scale monolingual data of the target side to boost the fluency of translations. We conduct experiments to explore the effectiveness of the joint method by incorporating back-translation into the parent-child and the hierarchical transfer learning architecture. In addition, different preprocessing and training methods are explored to get better performance. Experimental results on Uygur-Chinese and Turkish-English translation demonstrate the superiority of the proposed method over the baselines that use single methods.

scholarly journals Augmenting Neural Machine Translation through Round-Trip Training Approach

2019 ◽  
Vol 9 (1) ◽  
pp. 268-278 ◽  
Author(s):  
Benyamin Ahmadnia ◽  
Bonnie J. Dorr
Keyword(s):  
Machine Translation ◽  
Training Data ◽  
Training Dataset ◽  
Round Trip ◽  
Neural Machine Translation ◽  
Low Resource ◽  
Translation Quality ◽  
High Resource ◽  
Training Approach ◽  
Language Pair

AbstractThe quality of Neural Machine Translation (NMT), as a data-driven approach, massively depends on quantity, quality and relevance of the training dataset. Such approaches have achieved promising results for bilingually high-resource scenarios but are inadequate for low-resource conditions. Generally, the NMT systems learn from millions of words from bilingual training dataset. However, human labeling process is very costly and time consuming. In this paper, we describe a round-trip training approach to bilingual low-resource NMT that takes advantage of monolingual datasets to address training data bottleneck, thus augmenting translation quality. We conduct detailed experiments on English-Spanish as a high-resource language pair as well as Persian-Spanish as a low-resource language pair. Experimental results show that this competitive approach outperforms the baseline systems and improves translation quality.

scholarly journals UQuAD1.0: Development of an Urdu Question Answering Training Data for Machine Reading Comprehension

2021 ◽  
Author(s):  
Samreen Ahmed ◽  
shakeel khoja
Keyword(s):  
Reading Comprehension ◽  
Machine Translation ◽  
Large Scale ◽  
Question Answering ◽  
Training Data ◽  
Significant Progress ◽  
Rule Based ◽  
Low Resource ◽  
Machine Reading ◽  
Answer Format

<p>In recent years, low-resource Machine Reading Comprehension (MRC) has made significant progress, with models getting remarkable performance on various language datasets. However, none of these models have been customized for the Urdu language. This work explores the semi-automated creation of the Urdu Question Answering Dataset (UQuAD1.0) by combining machine-translated SQuAD with human-generated samples derived from Wikipedia articles and Urdu RC worksheets from Cambridge O-level books. UQuAD1.0 is a large-scale Urdu dataset intended for extractive machine reading comprehension tasks consisting of 49k question Answers pairs in question, passage, and answer format. In UQuAD1.0, 45000 pairs of QA were generated by machine translation of the original SQuAD1.0 and approximately 4000 pairs via crowdsourcing. In this study, we used two types of MRC models: rule-based baseline and advanced Transformer-based models. However, we have discovered that the latter outperforms the others; thus, we have decided to concentrate solely on Transformer-based architectures. Using XLMRoBERTa and multi-lingual BERT, we acquire an F<sub>1</sub> score of 0.66 and 0.63, respectively.</p>

scholarly journals Optimizing Statistical Machine Translation for Text Simplification

2016 ◽  
Vol 4 ◽  
pp. 401-415 ◽  
Author(s):  
Wei Xu ◽  
Courtney Napoles ◽  
Ellie Pavlick ◽  
Quanze Chen ◽  
Chris Callison-Burch
Keyword(s):  
Machine Translation ◽  
Large Scale ◽  
Statistical Machine Translation ◽  
Parallel Corpus ◽  
Iterative Development ◽  
Text Simplification ◽  
Multiple References

Most recent sentence simplification systems use basic machine translation models to learn lexical and syntactic paraphrases from a manually simplified parallel corpus. These methods are limited by the quality and quantity of manually simplified corpora, which are expensive to build. In this paper, we conduct an in-depth adaptation of statistical machine translation to perform text simplification, taking advantage of large-scale paraphrases learned from bilingual texts and a small amount of manual simplifications with multiple references. Our work is the first to design automatic metrics that are effective for tuning and evaluating simplification systems, which will facilitate iterative development for this task.

scholarly journals Incorporating Source-Side Phrase Structures into Neural Machine Translation

2019 ◽  
Vol 45 (2) ◽  
pp. 267-292 ◽  
Author(s):  
Akiko Eriguchi ◽  
Kazuma Hashimoto ◽  
Yoshimasa Tsuruoka
Keyword(s):  
Machine Translation ◽  
Syntactic Structure ◽  
Statistical Machine Translation ◽  
Training Data ◽  
Great Success ◽  
Data Set ◽  
Neural Machine Translation ◽  
Proposed Model ◽  
Source Sentence

Neural machine translation (NMT) has shown great success as a new alternative to the traditional Statistical Machine Translation model in multiple languages. Early NMT models are based on sequence-to-sequence learning that encodes a sequence of source words into a vector space and generates another sequence of target words from the vector. In those NMT models, sentences are simply treated as sequences of words without any internal structure. In this article, we focus on the role of the syntactic structure of source sentences and propose a novel end-to-end syntactic NMT model, which we call a tree-to-sequence NMT model, extending a sequence-to-sequence model with the source-side phrase structure. Our proposed model has an attention mechanism that enables the decoder to generate a translated word while softly aligning it with phrases as well as words of the source sentence. We have empirically compared the proposed model with sequence-to-sequence models in various settings on Chinese-to-Japanese and English-to-Japanese translation tasks. Our experimental results suggest that the use of syntactic structure can be beneficial when the training data set is small, but is not as effective as using a bi-directional encoder. As the size of training data set increases, the benefits of using a syntactic tree tends to diminish.

Automatic translation of spoken English based on improved machine learning algorithm

2020 ◽  
pp. 1-11
Author(s):  
Lin Lin ◽  
Jie Liu ◽  
Xuebing Zhang ◽  
Xiufang Liang
Keyword(s):  
Machine Translation ◽  
Learning Algorithm ◽  
English Word ◽  
Neural Machine Translation ◽  
Parallel Corpus ◽  
Low Resource ◽  
Spoken English ◽  
Automatic Translation ◽  
Study Designs ◽  
Chinese And English

Due to the complexity of English machine translation technology and its broad application prospects, many experts and scholars have invested more energy to analyze it. In view of the complex and changeable English forms, the large difference between Chinese and English word order, and insufficient Chinese-English parallel corpus resources, this paper uses deep learning to complete the conversion between Chinese and English. The research focus of this paper is how to use language pairs with rich parallel corpus resources to improve the performance of Chinese-English neural machine translation, that is, to use multi-task learning to train neural machine translation models. Moreover, this research proposes a low-resource neural machine translation method based on weight sharing, which uses the weight-sharing method to improve the performance of Chinese-English low-resource neural machine translation. In addition, this study designs a control experiment to analyze the effectiveness of this study model. The research results show that the model proposed in this paper has a certain effect.

Sign in / Sign up

Export Citation Format

Share Document