scholarly journals Graph Based Translation Memory for Neural Machine Translation

2019 ◽  
Vol 33 ◽  
pp. 7297-7304 ◽  
Author(s):  
Mengzhou Xia ◽  
Guoping Huang ◽  
Lemao Liu ◽  
Shuming Shi
Keyword(s):  
Machine Translation ◽  
State Of The Art ◽  
Local Information ◽  
Global Information ◽  
Translation Memory ◽  
Neural Machine Translation ◽  
Efficient Approach ◽  
Compact Graph ◽  
Decoding Efficiency ◽  
Space Occupation

A translation memory (TM) is proved to be helpful to improve neural machine translation (NMT). Existing approaches either pursue the decoding efficiency by merely accessing local information in a TM or encode the global information in a TM yet sacrificing efficiency due to redundancy. We propose an efficient approach to making use of the global information in a TM. The key idea is to pack a redundant TM into a compact graph and perform additional attention mechanisms over the packed graph for integrating the TM representation into the decoding network. We implement the model by extending the state-of-the-art NMT, Transformer. Extensive experiments on three language pairs show that the proposed approach is efficient in terms of running time and space occupation, and particularly it outperforms multiple strong baselines in terms of BLEU scores.

Analyzing Subword Techniques to Improve English to Sinhala Neural Machine Translation

2021 ◽  
pp. 2050017
Author(s):  
Rashmini Naranpanawa ◽  
Ravinga Perera ◽  
Thilakshi Fonseka ◽  
Uthayasanker Thayasivam
Keyword(s):  
Machine Translation ◽  
State Of The Art ◽  
Statistical Machine Translation ◽  
Translation System ◽  
Rare Word ◽  
Neural Machine Translation ◽  
Parallel Corpus ◽  
Low Resource ◽  
Word Level ◽  
Morphologically Rich Languages

Neural machine translation (NMT) is a remarkable approach which performs much better than the Statistical machine translation (SMT) models when there is an abundance of parallel corpus. However, vanilla NMT is primarily based upon word-level with a fixed vocabulary. Therefore, low resource morphologically rich languages such as Sinhala are mostly affected by the out of vocabulary (OOV) and Rare word problems. Recent advancements in subword techniques have opened up opportunities for low resource communities by enabling open vocabulary translation. In this paper, we extend our recently published state-of-the-art EN-SI translation system using the transformer and explore standard subword techniques on top of it to identify which subword approach has a greater effect on English Sinhala language pair. Our models demonstrate that subword segmentation strategies along with the state-of-the-art NMT can perform remarkably when translating English sentences into a rich morphology language regardless of a large parallel corpus.

scholarly journals Tied Transformers: Neural Machine Translation with Shared Encoder and Decoder

2019 ◽  
Vol 33 ◽  
pp. 5466-5473 ◽  
Author(s):  
Yingce Xia ◽  
Tianyu He ◽  
Xu Tan ◽  
Fei Tian ◽  
Di He ◽  
...  
Keyword(s):  
Machine Translation ◽  
English Translation ◽  
State Of The Art ◽  
Compact Model ◽  
Word Embeddings ◽  
Simple Method ◽  
Neural Machine Translation ◽  
German Translation ◽  
One Step ◽  
Target Side

Sharing source and target side vocabularies and word embeddings has been a popular practice in neural machine translation (briefly, NMT) for similar languages (e.g., English to French or German translation). The success of such wordlevel sharing motivates us to move one step further: we consider model-level sharing and tie the whole parts of the encoder and decoder of an NMT model. We share the encoder and decoder of Transformer (Vaswani et al. 2017), the state-of-the-art NMT model, and obtain a compact model named Tied Transformer. Experimental results demonstrate that such a simple method works well for both similar and dissimilar language pairs. We empirically verify our framework for both supervised NMT and unsupervised NMT: we achieve a 35.52 BLEU score on IWSLT 2014 German to English translation, 28.98/29.89 BLEU scores on WMT 2014 English to German translation without/with monolingual data, and a 22.05 BLEU score on WMT 2016 unsupervised German to English translation.

Deep Learning-based Roman-Urdu to Urdu Transliteration

2020 ◽  
pp. 2152001
Author(s):  
Mehreen Alam ◽  
Sibt ul Hussain
Keyword(s):  
Machine Translation ◽  
State Of The Art ◽  
Research Problem ◽  
Attention Mechanism ◽  
Data Driven ◽  
Neural Machine Translation ◽  
Parallel Corpus ◽  
Source Language ◽  
Data Driven Approach ◽  
Modern Machine

Attention-based encoder-decoder models have superseded conventional techniques due to their unmatched performance on many neural machine translation problems. Usually, the encoders and decoders are two recurrent neural networks where the decoder is directed to focus on relevant parts of the source language using attention mechanism. This data-driven approach leads to generic and scalable solutions with no reliance on manual hand-crafted features. To the best of our knowledge, none of the modern machine translation approaches has been applied to address the research problem of Urdu machine transliteration. Ours is the first attempt to apply the deep neural network-based encoder-decoder using attention mechanism to address the aforementioned problem using Roman-Urdu and Urdu parallel corpus. To this end, we present (i) the first ever Roman-Urdu to Urdu parallel corpus of 1.1 million sentences, (ii) three state of the art encoder-decoder models, and (iii) a detailed empirical analysis of these three models on the Roman-Urdu to Urdu parallel corpus. Overall, attention-based model gives state-of-the-art performance with the benchmark of 70 BLEU score. Our qualitative experimental evaluation shows that our models generate coherent transliterations which are grammatically and logically correct.

scholarly journals Synchronous Bidirectional Neural Machine Translation

2019 ◽  
Vol 7 ◽  
pp. 91-105 ◽  
Author(s):  
Long Zhou ◽  
Jiajun Zhang ◽  
Chengqing Zong
Keyword(s):  
Machine Translation ◽  
Large Scale ◽  
State Of The Art ◽  
Target Language ◽  
Single Model ◽  
Neural Machine Translation ◽  
German Translation ◽  
Transformer Model ◽  
Target Side ◽  
Future Information

Existing approaches to neural machine translation (NMT) generate the target language sequence token-by-token from left to right. However, this kind of unidirectional decoding framework cannot make full use of the target-side future contexts which can be produced in a right-to-left decoding direction, and thus suffers from the issue of unbalanced outputs. In this paper, we introduce a synchronous bidirectional–neural machine translation (SB-NMT) that predicts its outputs using left-to-right and right-to-left decoding simultaneously and interactively, in order to leverage both of the history and future information at the same time. Specifically, we first propose a new algorithm that enables synchronous bidirectional decoding in a single model. Then, we present an interactive decoding model in which left-to-right (right-to-left) generation does not only depend on its previously generated outputs, but also relies on future contexts predicted by right-to-left (left-to-right) decoding. We extensively evaluate the proposed SB-NMT model on large-scale NIST Chinese-English, WMT14 English-German, and WMT18 Russian-English translation tasks. Experimental results demonstrate that our model achieves significant improvements over the strong Transformer model by 3.92, 1.49, and 1.04 BLEU points, respectively, and obtains the state-of-the-art per- formance on Chinese-English and English- German translation tasks. 1

scholarly journals Empirical Investigation of Optimization Algorithms in Neural Machine Translation

2017 ◽  
Vol 108 (1) ◽  
pp. 13-25 ◽  
Author(s):  
Parnia Bahar ◽  
Tamer Alkhouli ◽  
Jan-Thorsten Peter ◽  
Christopher Jan-Steffen Brix ◽  
Hermann Ney
Keyword(s):  
Neural Networks ◽  
Machine Translation ◽  
Optimization Problem ◽  
Empirical Investigation ◽  
State Of The Art ◽  
Optimization Techniques ◽  
Neural Machine Translation ◽  
Translation Quality ◽  
And Training ◽  
Dimensional Optimization

AbstractTraining neural networks is a non-convex and a high-dimensional optimization problem. In this paper, we provide a comparative study of the most popular stochastic optimization techniques used to train neural networks. We evaluate the methods in terms of convergence speed, translation quality, and training stability. In addition, we investigate combinations that seek to improve optimization in terms of these aspects. We train state-of-the-art attention-based models and apply them to perform neural machine translation. We demonstrate our results on two tasks: WMT 2016 En→Ro and WMT 2015 De→En.

scholarly journals Neural Machine Translation between Vietnamese and English: an Empirical Study

2019 ◽  
Vol 35 (2) ◽  
pp. 147-166 ◽  
Author(s):  
Hong-Hai Phan-Vu ◽  
Viet Trung Tran ◽  
Van Nam Nguyen ◽  
Hoang Vu Dang ◽  
Phan Thuan Do
Keyword(s):  
Neural Networks ◽  
Empirical Study ◽  
Machine Translation ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
Neural Models ◽  
Neural Machine Translation ◽  
Parallel Corpora ◽  
Parameter Search ◽  
Popular Language

Machine translation is shifting to an end-to-end approach based on deep neural networks. The state of the art achieves impressive results for popular language pairs such as English - French or English - Chinese. However for English - Vietnamese the shortage of parallel corpora and expensive hyper-parameter search present practical challenges to neural-based approaches. This paper highlights our efforts on improving English-Vietnamese translations in two directions: (1) Building the largest open Vietnamese - English corpus to date, and (2) Extensive experiments with the latest neural models to achieve the highest BLEU scores. Our experiments provide practical examples of effectively employing different neural machine translation models with low-resource language pairs.

scholarly journals Neural Machine Translation with Monolingual Translation Memory

2021 ◽  
Author(s):  
Deng Cai ◽  
Yan Wang ◽  
Huayang Li ◽  
Wai Lam ◽  
Lemao Liu
Keyword(s):  
Machine Translation ◽  
Translation Memory ◽  
Neural Machine Translation

scholarly journals Guiding Attention in Sequence-to-Sequence Models for Dialogue Act Prediction

2020 ◽  
Vol 34 (05) ◽  
pp. 7594-7601
Author(s):  
Pierre Colombo ◽  
Emile Chapuis ◽  
Matteo Manica ◽  
Emmanuel Vignon ◽  
Giovanna Varni ◽  
...  
Keyword(s):  
Machine Translation ◽  
Random Fields ◽  
Conditional Random Fields ◽  
State Of The Art ◽  
The State ◽  
Attention Mechanism ◽  
Accuracy Score ◽  
Beam Search ◽  
Conversational Agents ◽  
Neural Machine Translation

The task of predicting dialog acts (DA) based on conversational dialog is a key component in the development of conversational agents. Accurately predicting DAs requires a precise modeling of both the conversation and the global tag dependencies. We leverage seq2seq approaches widely adopted in Neural Machine Translation (NMT) to improve the modelling of tag sequentiality. Seq2seq models are known to learn complex global dependencies while currently proposed approaches using linear conditional random fields (CRF) only model local tag dependencies. In this work, we introduce a seq2seq model tailored for DA classification using: a hierarchical encoder, a novel guided attention mechanism and beam search applied to both training and inference. Compared to the state of the art our model does not require handcrafted features and is trained end-to-end. Furthermore, the proposed approach achieves an unmatched accuracy score of 85% on SwDA, and state-of-the-art accuracy score of 91.6% on MRDA.

scholarly journals Modeling Coherence for Discourse Neural Machine Translation

2019 ◽  
Vol 33 ◽  
pp. 7338-7345 ◽  
Author(s):  
Hao Xiong ◽  
Zhongjun He ◽  
Hua Wu ◽  
Haifeng Wang
Keyword(s):  
Machine Translation ◽  
State Of The Art ◽  
The State ◽  
Neural Machine Translation ◽  
Discourse Context ◽  
Translation Quality ◽  
Discourse Coherence ◽  
Baseline System

Discourse coherence plays an important role in the translation of one text. However, the previous reported models most focus on improving performance over individual sentence while ignoring cross-sentence links and dependencies, which affects the coherence of the text. In this paper, we propose to use discourse context and reward to refine the translation quality from the discourse perspective. In particular, we generate the translation of individual sentences at first. Next, we deliberate the preliminary produced translations, and train the model to learn the policy that produces discourse coherent text by a reward teacher. Practical results on multiple discourse test datasets indicate that our model significantly improves the translation quality over the state-of-the-art baseline system by +1.23 BLEU score. Moreover, our model generates more discourse coherent text and obtains +2.2 BLEU improvements when evaluated by discourse metrics.

scholarly journals Unsupervised Quality Estimation for Neural Machine Translation

2020 ◽  
Vol 8 ◽  
pp. 539-555
Author(s):  
Marina Fomicheva ◽  
Shuo Sun ◽  
Lisa Yankovskaya ◽  
Frédéric Blain ◽  
Francisco Guzmán ◽  
...  
Keyword(s):  
Machine Translation ◽  
Real World ◽  
State Of The Art ◽  
Black Box ◽  
Test Time ◽  
Quality Estimation ◽  
Neural Machine Translation ◽  
Real World Applications ◽  
Unsupervised Approach

Quality Estimation (QE) is an important component in making Machine Translation (MT) useful in real-world applications, as it is aimed to inform the user on the quality of the MT output at test time. Existing approaches require large amounts of expert annotated data, computation, and time for training. As an alternative, we devise an unsupervised approach to QE where no training or access to additional resources besides the MT system itself is required. Different from most of the current work that treats the MT system as a black box, we explore useful information that can be extracted from the MT system as a by-product of translation. By utilizing methods for uncertainty quantification, we achieve very good correlation with human judgments of quality, rivaling state-of-the-art supervised QE models. To evaluate our approach we collect the first dataset that enables work on both black-box and glass-box approaches to QE.

Sign in / Sign up

Export Citation Format

Share Document