scholarly journals Explicit Sentence Compression for Neural Machine Translation

2020 ◽  
Vol 34 (05) ◽  
pp. 8311-8318
Author(s):  
Zuchao Li ◽  
Rui Wang ◽  
Kehai Chen ◽  
Masao Utiyama ◽  
Eiichiro Sumita ◽  
...  
Keyword(s):  
Machine Translation ◽  
State Of The Art ◽  
General Information ◽  
Compression Method ◽  
Neural Machine Translation ◽  
Sentence Compression ◽  
French And English ◽  
Source Sentence ◽  
Empirical Tests ◽  
Target Side

State-of-the-art Transformer-based neural machine translation (NMT) systems still follow a standard encoder-decoder framework, in which source sentence representation can be well done by an encoder with self-attention mechanism. Though Transformer-based encoder may effectively capture general information in its resulting source sentence representation, the backbone information, which stands for the gist of a sentence, is not specifically focused on. In this paper, we propose an explicit sentence compression method to enhance the source sentence representation for NMT. In practice, an explicit sentence compression goal used to learn the backbone information in a sentence. We propose three ways, including backbone source-side fusion, target-side fusion, and both-side fusion, to integrate the compressed sentence into NMT. Our empirical tests on the WMT English-to-French and English-to-German translation tasks show that the proposed sentence compression method significantly improves the translation performances over strong baselines.

scholarly journals Tied Transformers: Neural Machine Translation with Shared Encoder and Decoder

2019 ◽  
Vol 33 ◽  
pp. 5466-5473 ◽  
Author(s):  
Yingce Xia ◽  
Tianyu He ◽  
Xu Tan ◽  
Fei Tian ◽  
Di He ◽  
...  
Keyword(s):  
Machine Translation ◽  
English Translation ◽  
State Of The Art ◽  
Compact Model ◽  
Word Embeddings ◽  
Simple Method ◽  
Neural Machine Translation ◽  
German Translation ◽  
One Step ◽  
Target Side

Sharing source and target side vocabularies and word embeddings has been a popular practice in neural machine translation (briefly, NMT) for similar languages (e.g., English to French or German translation). The success of such wordlevel sharing motivates us to move one step further: we consider model-level sharing and tie the whole parts of the encoder and decoder of an NMT model. We share the encoder and decoder of Transformer (Vaswani et al. 2017), the state-of-the-art NMT model, and obtain a compact model named Tied Transformer. Experimental results demonstrate that such a simple method works well for both similar and dissimilar language pairs. We empirically verify our framework for both supervised NMT and unsupervised NMT: we achieve a 35.52 BLEU score on IWSLT 2014 German to English translation, 28.98/29.89 BLEU scores on WMT 2014 English to German translation without/with monolingual data, and a 22.05 BLEU score on WMT 2016 unsupervised German to English translation.

scholarly journals Synchronous Bidirectional Neural Machine Translation

2019 ◽  
Vol 7 ◽  
pp. 91-105 ◽  
Author(s):  
Long Zhou ◽  
Jiajun Zhang ◽  
Chengqing Zong
Keyword(s):  
Machine Translation ◽  
Large Scale ◽  
State Of The Art ◽  
Target Language ◽  
Single Model ◽  
Neural Machine Translation ◽  
German Translation ◽  
Transformer Model ◽  
Target Side ◽  
Future Information

Existing approaches to neural machine translation (NMT) generate the target language sequence token-by-token from left to right. However, this kind of unidirectional decoding framework cannot make full use of the target-side future contexts which can be produced in a right-to-left decoding direction, and thus suffers from the issue of unbalanced outputs. In this paper, we introduce a synchronous bidirectional–neural machine translation (SB-NMT) that predicts its outputs using left-to-right and right-to-left decoding simultaneously and interactively, in order to leverage both of the history and future information at the same time. Specifically, we first propose a new algorithm that enables synchronous bidirectional decoding in a single model. Then, we present an interactive decoding model in which left-to-right (right-to-left) generation does not only depend on its previously generated outputs, but also relies on future contexts predicted by right-to-left (left-to-right) decoding. We extensively evaluate the proposed SB-NMT model on large-scale NIST Chinese-English, WMT14 English-German, and WMT18 Russian-English translation tasks. Experimental results demonstrate that our model achieves significant improvements over the strong Transformer model by 3.92, 1.49, and 1.04 BLEU points, respectively, and obtains the state-of-the-art per- formance on Chinese-English and English- German translation tasks. 1

scholarly journals Improved Neural Machine Translation with Source Syntax

2017 ◽  
Author(s):  
Shuangzhi Wu ◽  
Ming Zhou ◽  
Dongdong Zhang
Keyword(s):  
Machine Translation ◽  
State Of The Art ◽  
Syntactic Structure ◽  
Phrase Structure ◽  
Long Distance ◽  
Neural Machine Translation ◽  
Attention Model ◽  
Word Level ◽  
Source Sentence ◽  
Dependency Structures

Neural Machine Translation (NMT) based on the encoder-decoder architecture has recently achieved the state-of-the-art performance. Researchers have proven that extending word level attention to phrase level attention by incorporating source-side phrase structure can enhance the attention model and achieve promising improvement. However, word dependencies that can be crucial to correctly understand a source sentence are not always in a consecutive fashion (i.e. phrase structure), sometimes they can be in long distance. Phrase structures are not the best way to explicitly model long distance dependencies. In this paper we propose a simple but effective method to incorporate source-side long distance dependencies into NMT. Our method based on dependency trees enriches each source state with global dependency structures, which can better capture the inherent syntactic structure of source sentences. Experiments on Chinese-English and English-Japanese translation tasks show that our proposed method outperforms state-of-the-art SMT and NMT baselines.

scholarly journals ME-MD: An Effective Framework for Neural Machine Translation with Multiple Encoders and Decoders

2017 ◽  
Author(s):  
Jinchao Zhang ◽  
Qun Liu ◽  
Jie Zhou
Keyword(s):  
Machine Translation ◽  
English Translation ◽  
State Of The Art ◽  
The State ◽  
Neural Machine Translation ◽  
Source Sentence ◽  
Target Words

The encoder-decoder neural framework is widely employed for Neural Machine Translation (NMT) with a single encoder to represent the source sentence and a single decoder to generate target words. The translation performance heavily relies on the representation ability of the encoder and the generation ability of the decoder. To further enhance NMT, we propose to extend the original encoder-decoder framework to a novel one, which has multiple encoders and decoders (ME-MD). Through this way, multiple encoders extract more diverse features to represent the source sequence and multiple decoders capture more complicated translation knowledge. Our proposed ME-MD framework is convenient to integrate heterogeneous encoders and decoders with multiple depths and multiple types. Experiment on Chinese-English translation task shows that our ME-MD system surpasses the state-of-the-art NMT system by 2.1 BLEU points and surpasses the phrase-based Moses by 7.38 BLEU points. Our framework is general and can be applied to other sequence to sequence tasks.

scholarly journals Translating with Bilingual Topic Knowledge for Neural Machine Translation

2019 ◽  
Vol 33 ◽  
pp. 7257-7264
Author(s):  
Xiangpeng Wei ◽  
Yue Hu ◽  
Luxi Xing ◽  
Yipeng Wang ◽  
Li Gao
Keyword(s):  
Machine Translation ◽  
State Of The Art ◽  
Attention Mechanism ◽  
Target Domain ◽  
Neural Machine Translation ◽  
Topic Knowledge ◽  
Proposed Model ◽  
Decoder Architecture ◽  
Source Sentence ◽  
Hidden States

The dominant neural machine translation (NMT) models that based on the encoder-decoder architecture have recently achieved the state-of-the-art performance. Traditionally, the NMT models only depend on the representations learned during training for mapping a source sentence into the target domain. However, the learned representations often suffer from implicit and inadequately informed properties. In this paper, we propose a novel bilingual topic enhanced NMT (BLTNMT) model to improve translation performance by incorporating bilingual topic knowledge into NMT. Specifically, the bilingual topic knowledge is included into the hidden states of both encoder and decoder, as well as the attention mechanism. With this new setting, the proposed BLT-NMT has access to the background knowledge implied in bilingual topics which is beyond the sequential context, and enables the attention mechanism to attend to topic-level attentions for generating accurate target words during translation. Experimental results show that the proposed model consistently outperforms the traditional RNNsearch and the previous topic-informed NMT on Chinese-English and EnglishGerman translation tasks. We also introduce the bilingual topic knowledge into the newly emerged Transformer base model on English-German translation and achieve a notable improvement.

scholarly journals Comparing Statistical and Neural Machine Translation Performance on Hindi-To-Tamil and English-To-Tamil

Digital ◽  
2021 ◽  
Vol 1 (2) ◽  
pp. 86-102
Author(s):  
Akshai Ramesh ◽  
Venkatesh Balavadhani Parthasarathy ◽  
Rejwanul Haque ◽  
Andy Way
Keyword(s):  
Machine Translation ◽  
State Of The Art ◽  
Statistical Machine Translation ◽  
Training Data ◽  
Research Translation ◽  
Neural Machine Translation ◽  
Low Resource ◽  
Evaluation Scheme ◽  
Dominant Paradigm ◽  
Target Side

Phrase-based statistical machine translation (PB-SMT) has been the dominant paradigm in machine translation (MT) research for more than two decades. Deep neural MT models have been producing state-of-the-art performance across many translation tasks for four to five years. To put it another way, neural MT (NMT) took the place of PB-SMT a few years back and currently represents the state-of-the-art in MT research. Translation to or from under-resourced languages has been historically seen as a challenging task. Despite producing state-of-the-art results in many translation tasks, NMT still poses many problems such as performing poorly for many low-resource language pairs mainly because of its learning task’s data-demanding nature. MT researchers have been trying to address this problem via various techniques, e.g., exploiting source- and/or target-side monolingual data for training, augmenting bilingual training data, and transfer learning. Despite some success, none of the present-day benchmarks have entirely overcome the problem of translation in low-resource scenarios for many languages. In this work, we investigate the performance of PB-SMT and NMT on two rarely tested under-resourced language pairs, English-to-Tamil and Hindi-to-Tamil, taking a specialised data domain into consideration. This paper demonstrates our findings and presents results showing the rankings of our MT systems produced via a social media-based human evaluation scheme.

Analyzing Subword Techniques to Improve English to Sinhala Neural Machine Translation

2021 ◽  
pp. 2050017
Author(s):  
Rashmini Naranpanawa ◽  
Ravinga Perera ◽  
Thilakshi Fonseka ◽  
Uthayasanker Thayasivam
Keyword(s):  
Machine Translation ◽  
State Of The Art ◽  
Statistical Machine Translation ◽  
Translation System ◽  
Rare Word ◽  
Neural Machine Translation ◽  
Parallel Corpus ◽  
Low Resource ◽  
Word Level ◽  
Morphologically Rich Languages

Neural machine translation (NMT) is a remarkable approach which performs much better than the Statistical machine translation (SMT) models when there is an abundance of parallel corpus. However, vanilla NMT is primarily based upon word-level with a fixed vocabulary. Therefore, low resource morphologically rich languages such as Sinhala are mostly affected by the out of vocabulary (OOV) and Rare word problems. Recent advancements in subword techniques have opened up opportunities for low resource communities by enabling open vocabulary translation. In this paper, we extend our recently published state-of-the-art EN-SI translation system using the transformer and explore standard subword techniques on top of it to identify which subword approach has a greater effect on English Sinhala language pair. Our models demonstrate that subword segmentation strategies along with the state-of-the-art NMT can perform remarkably when translating English sentences into a rich morphology language regardless of a large parallel corpus.

Translation of Medical Texts using Neural Networks

2016 ◽  
Vol 5 (4) ◽  
pp. 51-66 ◽  
Author(s):  
Krzysztof Wolk ◽  
Krzysztof P. Marasek
Keyword(s):  
Machine Translation ◽  
Statistical Machine Translation ◽  
European Medicines Agency ◽  
Translation System ◽  
Training Methods ◽  
Neural Machine Translation ◽  
Machine Translation System ◽  
Source Sentence ◽  
Parallel Text ◽  
Translation Systems

The quality of machine translation is rapidly evolving. Today one can find several machine translation systems on the web that provide reasonable translations, although the systems are not perfect. In some specific domains, the quality may decrease. A recently proposed approach to this domain is neural machine translation. It aims at building a jointly-tuned single neural network that maximizes translation performance, a very different approach from traditional statistical machine translation. Recently proposed neural machine translation models often belong to the encoder-decoder family in which a source sentence is encoded into a fixed length vector that is, in turn, decoded to generate a translation. The present research examines the effects of different training methods on a Polish-English Machine Translation system used for medical data. The European Medicines Agency parallel text corpus was used as the basis for training of neural and statistical network-based translation systems. A comparison and implementation of a medical translator is the main focus of our experiments.

Deep Learning-based Roman-Urdu to Urdu Transliteration

2020 ◽  
pp. 2152001
Author(s):  
Mehreen Alam ◽  
Sibt ul Hussain
Keyword(s):  
Machine Translation ◽  
State Of The Art ◽  
Research Problem ◽  
Attention Mechanism ◽  
Data Driven ◽  
Neural Machine Translation ◽  
Parallel Corpus ◽  
Source Language ◽  
Data Driven Approach ◽  
Modern Machine

Attention-based encoder-decoder models have superseded conventional techniques due to their unmatched performance on many neural machine translation problems. Usually, the encoders and decoders are two recurrent neural networks where the decoder is directed to focus on relevant parts of the source language using attention mechanism. This data-driven approach leads to generic and scalable solutions with no reliance on manual hand-crafted features. To the best of our knowledge, none of the modern machine translation approaches has been applied to address the research problem of Urdu machine transliteration. Ours is the first attempt to apply the deep neural network-based encoder-decoder using attention mechanism to address the aforementioned problem using Roman-Urdu and Urdu parallel corpus. To this end, we present (i) the first ever Roman-Urdu to Urdu parallel corpus of 1.1 million sentences, (ii) three state of the art encoder-decoder models, and (iii) a detailed empirical analysis of these three models on the Roman-Urdu to Urdu parallel corpus. Overall, attention-based model gives state-of-the-art performance with the benchmark of 70 BLEU score. Our qualitative experimental evaluation shows that our models generate coherent transliterations which are grammatically and logically correct.

Sign in / Sign up

Export Citation Format

Share Document