Neural Machine Translation Approach for Singlish to English Translation

Document-level neural machine translation (DocNMT) has yielded attractive improvements. In this article, we systematically analyze the discourse phenomena in Chinese-to-English translation, and focus on the most obvious ones, namely lexical translation consistency. To alleviate the lexical inconsistency, we propose an effective approach that is aware of the words which need to be translated consistently and constrains the model to produce more consistent translations. Specifically, we first introduce a global context extractor to extract the document context and consistency context, respectively. Then, the two types of global context are integrated into a encoder enhancer and a decoder enhancer to improve the lexical translation consistency. We create a test set to evaluate the lexical consistency automatically. Experiments demonstrate that our approach can significantly alleviate the lexical translation inconsistency. In addition, our approach can also substantially improve the translation quality compared to sentence-level Transformer.

Download Full-text

Tied Transformers: Neural Machine Translation with Shared Encoder and Decoder

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v33i01.33015466 ◽

2019 ◽

Vol 33 ◽

pp. 5466-5473 ◽

Cited By ~ 2

Author(s):

Yingce Xia ◽

Tianyu He ◽

Xu Tan ◽

Fei Tian ◽

Di He ◽

...

Keyword(s):

Machine Translation ◽

English Translation ◽

State Of The Art ◽

Compact Model ◽

Word Embeddings ◽

Simple Method ◽

Neural Machine Translation ◽

German Translation ◽

One Step ◽

Target Side

Sharing source and target side vocabularies and word embeddings has been a popular practice in neural machine translation (briefly, NMT) for similar languages (e.g., English to French or German translation). The success of such wordlevel sharing motivates us to move one step further: we consider model-level sharing and tie the whole parts of the encoder and decoder of an NMT model. We share the encoder and decoder of Transformer (Vaswani et al. 2017), the state-of-the-art NMT model, and obtain a compact model named Tied Transformer. Experimental results demonstrate that such a simple method works well for both similar and dissimilar language pairs. We empirically verify our framework for both supervised NMT and unsupervised NMT: we achieve a 35.52 BLEU score on IWSLT 2014 German to English translation, 28.98/29.89 BLEU scores on WMT 2014 English to German translation without/with monolingual data, and a 22.05 BLEU score on WMT 2016 unsupervised German to English translation.

Download Full-text

Modeling hypotactic structure for Chinese-English neural machine translation of complex sentences

Journal of Intelligent & Fuzzy Systems ◽

10.3233/jifs-210908 ◽

2021 ◽

pp. 1-15

Author(s):

Guoyi Miao ◽

Yufeng Chen ◽

Jian Liu ◽

Jinan Xu ◽

Mingtong Liu ◽

...

Keyword(s):

Machine Translation ◽

English Translation ◽

Morphological Changes ◽

Structure Alignment ◽

Complex Sentence ◽

Neural Machine Translation ◽

Grammatical Form ◽

Complex Sentences ◽

Discourse Coherence ◽

Structure Knowledge

The hypotactic structural relation between clauses plays an important role in improving the discourse coherence of document-level translation. However, the standard neural machine translation (NMT) models do not explicitly model the hypotactic relationship between clauses, which usually leads to structurally incorrect translations of long and complex sentences. This problem is particularly noticeable on Chinese-to-English translation task of complex sentences due to the grammatical form distinction between English and Chinese. English is rich in grammatical form (e.g. verb morphological changes and subordinating conjunctions) while Chinese is poor in grammatical form. These linguistic phenomena make it a challenge for NMT to learn the hypotactic structure knowledge from Chinese as well as the structure alignment between Chinese and English. To address these issues, we propose to model the hypotactic structure for Chinese-to-English complex sentence translation by introducing hypotactic structure knowledge. Specifically, we annotate and build a hypotactic structure aligned parallel corpus that provides rich hypotactic structure knowledge for NMT. Moreover, we further propose a structure-infused neural framework to combine the hypotactic structure knowledge with the NMT model through two integrating strategies. In particular, we introduce a specific structure-aware loss to encourage the NMT model to better learn the structure knowledge. Experimental results on WMT17, WMT18 and WMT19 Chinese-to-English translation tasks demonstrate the effectiveness of the proposed methods.

Download Full-text

Neural Machine Translation with Joint Representation

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v34i05.6344 ◽

2020 ◽

Vol 34 (05) ◽

pp. 8285-8292

Author(s):

Yanyang Li ◽

Qiang Wang ◽

Tong Xiao ◽

Tongran Liu ◽

Jingbo Zhu

Keyword(s):

Machine Translation ◽

English Translation ◽

Large Scale ◽

State Of The Art ◽

Statistical Machine Translation ◽

The State ◽

Small Scale ◽

Neural Machine Translation ◽

Joint Representation

Though early successes of Statistical Machine Translation (SMT) systems are attributed in part to the explicit modelling of the interaction between any two source and target units, e.g., alignment, the recent Neural Machine Translation (NMT) systems resort to the attention which partially encodes the interaction for efficiency. In this paper, we employ Joint Representation that fully accounts for each possible interaction. We sidestep the inefficiency issue by refining representations with the proposed efficient attention operation. The resulting Reformer models offer a new Sequence-to-Sequence modelling paradigm besides the Encoder-Decoder framework and outperform the Transformer baseline in either the small scale IWSLT14 German-English, English-German and IWSLT15 Vietnamese-English or the large scale NIST12 Chinese-English translation tasks by about 1 BLEU point. We also propose a systematic model scaling approach, allowing the Reformer model to beat the state-of-the-art Transformer in IWSLT14 German-English and NIST12 Chinese-English with about 50% fewer parameters. The code is publicly available at https://github.com/lyy1994/reformer.

Download Full-text

ME-MD: An Effective Framework for Neural Machine Translation with Multiple Encoders and Decoders

Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2017/474 ◽

2017 ◽

Cited By ~ 1

Author(s):

Jinchao Zhang ◽

Qun Liu ◽

Jie Zhou

Keyword(s):

Machine Translation ◽

English Translation ◽

State Of The Art ◽

The State ◽

Neural Machine Translation ◽

Source Sentence ◽

Target Words

The encoder-decoder neural framework is widely employed for Neural Machine Translation (NMT) with a single encoder to represent the source sentence and a single decoder to generate target words. The translation performance heavily relies on the representation ability of the encoder and the generation ability of the decoder. To further enhance NMT, we propose to extend the original encoder-decoder framework to a novel one, which has multiple encoders and decoders (ME-MD). Through this way, multiple encoders extract more diverse features to represent the source sequence and multiple decoders capture more complicated translation knowledge. Our proposed ME-MD framework is convenient to integrate heterogeneous encoders and decoders with multiple depths and multiple types. Experiment on Chinese-English translation task shows that our ME-MD system surpasses the state-of-the-art NMT system by 2.1 BLEU points and surpasses the phrase-based Moses by 7.38 BLEU points. Our framework is general and can be applied to other sequence to sequence tasks.

Download Full-text

A Diverse Data Augmentation Strategy for Low-Resource Neural Machine Translation

Information ◽

10.3390/info11050255 ◽

2020 ◽

Vol 11 (5) ◽

pp. 255

Author(s):

Yu Li ◽

Xiao Li ◽

Yating Yang ◽

Rui Dong

Keyword(s):

Machine Translation ◽

English Translation ◽

Data Augmentation ◽

Sampling Strategy ◽

Training Data ◽

Neural Machine Translation ◽

Low Resource ◽

Parallel Data ◽

Augmentation Strategy ◽

Diverse Data

One important issue that affects the performance of neural machine translation is the scale of available parallel data. For low-resource languages, the amount of parallel data is not sufficient, which results in poor translation quality. In this paper, we propose a diversity data augmentation method that does not use extra monolingual data. We expand the training data by generating diversity pseudo parallel data on the source and target sides. To generate diversity data, the restricted sampling strategy is employed at the decoding steps. Finally, we filter and merge origin data and synthetic parallel corpus to train the final model. In the experiment, the proposed approach achieved 1.96 BLEU points in the IWSLT2014 German–English translation tasks, which was used to simulate a low-resource language. Our approach also consistently and substantially obtained 1.0 to 2.0 BLEU improvement in three other low-resource translation tasks, including English–Turkish, Nepali–English, and Sinhala–English translation tasks.

Download Full-text

Neural Machine Translation for Amharic-English Translation

Proceedings of the 13th International Conference on Agents and Artificial Intelligence ◽

10.5220/0010383905260532 ◽

2021 ◽

Author(s):

Andargachew Gezmu ◽

Andreas Nürnberger ◽

Tesfaye Bati

Keyword(s):

Machine Translation ◽

English Translation ◽

Neural Machine Translation

Download Full-text

Sentence Wise Telugu to English Translation of Vemana Sathakam using LSTM

International Journal of Recent Technology and Engineering - 2 ◽

10.35940/ijrte.d4340.118419 ◽

2019 ◽

Vol 8 (4) ◽

pp. 10739-10743

Keyword(s):

Machine Translation ◽

English Translation ◽

Short Term Memory ◽

Language Translation ◽

Statistical Machine Translation ◽

Short Term ◽

Neural Machine Translation ◽

Term Memory ◽

Long Short Term Memory ◽

Bidirectional Lstm

Language translation is a power of humans where machines are lagging and need to acquire. Previous statistical machine translation is used for translation but is applicable for large and similar grammar structure dataset. In this paper neural machine translation with long short term memory (LSTM) is used for addressing the issue. This paper uses a bidirectional LSTM to translate Telugu literary poems of Yogi Vemana to English which exhibited satisfactory translation. The results are compared with existing and proposed methods. NMT with LSTM yields better in language translation.

Download Full-text

Neural Machine Translation with Key-Value Memory-Augmented Attention

Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2018/357 ◽

2018 ◽

Cited By ~ 1

Author(s):

Fandong Meng ◽

Zhaopeng Tu ◽

Yong Cheng ◽

Haiyang Wu ◽

Junjie Zhai ◽

...

Keyword(s):

Target Word ◽

Machine Translation ◽

English Translation ◽

Translation Process ◽

Neural Machine Translation ◽

Attention Model ◽

Proposed Model ◽

Source Sentence ◽

Remarkable Progress ◽

Source Word

Although attention-based Neural Machine Translation (NMT) has achieved remarkable progress in recent years, it still suffers from issues of repeating and dropping translations. To alleviate these issues, we propose a novel key-value memory-augmented attention model for NMT, called KVMEMATT. Specifically, we maintain a timely updated keymemory to keep track of attention history and a fixed value-memory to store the representation of source sentence throughout the whole translation process. Via nontrivial transformations and iterative interactions between the two memories, the decoder focuses on more appropriate source word(s) for predicting the next target word at each decoding step, therefore can improve the adequacy of translations. Experimental results on Chinese)English and WMT17 German,English translation tasks demonstrate the superiority of the proposed model.

Download Full-text