scholarly journals Research on Neural Machine Translation Model

2019 ◽  
Vol 1237 ◽  
pp. 052020
Author(s):  
Mengyao Chen ◽  
Yong Li ◽  
Runqi Li
Keyword(s):  
Machine Translation ◽  
Neural Machine Translation ◽  
Translation Model

PhraseAttn: Dynamic Slot Capsule Networks for phrase representation in Neural Machine Translation

2021 ◽  
pp. 1-8
Author(s):  
Binh Nguyen ◽  
Binh Le ◽  
Long H.B. Nguyen ◽  
Dien Dinh
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Machine Translation ◽  
Language Processing ◽  
Vital Role ◽  
Attention Mechanism ◽  
Neural Machine Translation ◽  
Translation Model ◽  
Word Representation

 Word representation plays a vital role in most Natural Language Processing systems, especially for Neural Machine Translation. It tends to capture semantic and similarity between individual words well, but struggle to represent the meaning of phrases or multi-word expressions. In this paper, we investigate a method to generate and use phrase information in a translation model. To generate phrase representations, a Primary Phrase Capsule network is first employed, then iteratively enhancing with a Slot Attention mechanism. Experiments on the IWSLT English to Vietnamese, French, and German datasets show that our proposed method consistently outperforms the baseline Transformer, and attains competitive results over the scaled Transformer with two times lower parameters.

scholarly journals “Found in Translation”: predicting outcomes of complex organic chemistry reactions using neural sequence-to-sequence models

2018 ◽  
Vol 9 (28) ◽  
pp. 6091-6098 ◽  
Author(s):  
Philippe Schwaller ◽  
Théophile Gaudin ◽  
Dávid Lányi ◽  
Costas Bekas ◽  
Teodoro Laino
Keyword(s):  
Organic Chemistry ◽  
Machine Translation ◽  
Chemical Reactions ◽  
Language Processing ◽  
Neural Machine Translation ◽  
Translation Model ◽  
Complex Organic

Using a text-based representation of molecules, chemical reactions are predicted with a neural machine translation model borrowed from language processing.

scholarly journals Dynamic Layer Aggregation for Neural Machine Translation with Routing-by-Agreement

2019 ◽  
Vol 33 ◽  
pp. 86-93 ◽  
Author(s):  
Zi-Yi Dou ◽  
Zhaopeng Tu ◽  
Xing Wang ◽  
Longyue Wang ◽  
Shuming Shi ◽  
...  
Keyword(s):  
Machine Translation ◽  
Recent Progress ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
Baseline Model ◽  
Individual Layer ◽  
Neural Machine Translation ◽  
Translation Model ◽  
Hidden States ◽  
Dynamic Layer

With the promising progress of deep neural networks, layer aggregation has been used to fuse information across layers in various fields, such as computer vision and machine translation. However, most of the previous methods combine layers in a static fashion in that their aggregation strategy is independent of specific hidden states. Inspired by recent progress on capsule networks, in this paper we propose to use routing-by-agreement strategies to aggregate layers dynamically. Specifically, the algorithm learns the probability of a part (individual layer representations) assigned to a whole (aggregated representations) in an iterative way and combines parts accordingly. We implement our algorithm on top of the state-of-the-art neural machine translation model TRANSFORMER and conduct experiments on the widely-used WMT14 sh⇒German and WMT17 Chinese⇒English translation datasets. Experimental results across language pairs show that the proposed approach consistently outperforms the strong baseline model and a representative static aggregation model.

scholarly journals Tag-less Back-Translation

2021 ◽  
Author(s):  
Idris Abdulmumin ◽  
Bashir Shehu Galadanci ◽  
Aliyu Garba
Keyword(s):  
Machine Translation ◽  
Domain Adaptation ◽  
Fine Tuning ◽  
Huge Amount ◽  
Neural Machine Translation ◽  
Translation Model ◽  
Parallel Data ◽  
Back Translation ◽  
Authentic Data ◽  
Target Side

Abstract An effective method to generate a large number of parallel sentences for training improved neural machine translation (NMT) systems is the use of the back-translations of the target-side monolingual data. The standard back-translation method has been shown to be unable to efficiently utilize the available huge amount of existing monolingual data because of the inability of translation models to differentiate between the authentic and synthetic parallel data during training. Tagging, or using gates, has been used to enable translation models to distinguish between synthetic and authentic data, improving standard back-translation and also enabling the use of iterative back-translation on language pairs that underperformed using standard back-translation. In this work, we approach back-translation as a domain adaptation problem, eliminating the need for explicit tagging. In the approach - tag-less back-translation - the synthetic and authentic parallel data are treated as out-of-domain and in-domain data respectively and, through pre-training and fine-tuning, the translation model is shown to be able to learn more efficiently from them during training. Experimental results have shown that the approach outperforms the standard and tagged back-translation approaches on low resource English-Vietnamese and English-German neural machine translation.

scholarly journals Translation Mechanism of Neural Machine Algorithm for Online English Resources

Complexity ◽  
2021 ◽  
Vol 2021 ◽  
pp. 1-11
Author(s):  
Yanping Ye
Keyword(s):  
Data Processing ◽  
Machine Translation ◽  
Statistical Machine Translation ◽  
Language Model ◽  
Processing Methods ◽  
Neural Machine Translation ◽  
Translation Model ◽  
Sentence Similarity ◽  
Alignment Structure ◽  
Data Processing Methods

At the level of English resource vocabulary, due to the lack of vocabulary alignment structure, the translation of neural machine translation has the problem of unfaithfulness. This paper proposes a framework that integrates vocabulary alignment structure for neural machine translation at the vocabulary level. Under the proposed framework, the neural machine translation decoder receives external vocabulary alignment information during each step of the decoding process to further alleviate the problem of missing vocabulary alignment structure. Specifically, this article uses the word alignment structure of statistical machine translation as the external vocabulary alignment information and introduces it into the decoding step of neural machine translation. The model is mainly based on neural machine translation, and the statistical machine translation vocabulary alignment structure is integrated on the basis of neural networks and continuous expression of words. In the model decoding stage, the statistical machine translation system provides appropriate vocabulary alignment information based on the decoding information of the neural machine translation and recommends vocabulary based on the vocabulary alignment information to guide the neural machine translation decoder to more accurately estimate its vocabulary in the target language. From the aspects of data processing methods and machine translation technology, experiments are carried out to compare the data processing methods based on language model and sentence similarity and the effectiveness of machine translation models based on fusion principles. Comparative experiment results show that the data processing method based on language model and sentence similarity effectively guarantees data quality and indirectly improves the algorithm performance of machine translation model; the translation effect of neural machine translation model integrated with statistical machine translation vocabulary alignment structure is compared with other models.

scholarly journals Maximum Expected Likelihood Estimation for Zero-resource Neural Machine Translation

2017 ◽  
Author(s):  
Hao Zheng ◽  
Yong Cheng ◽  
Yang Liu
Keyword(s):  
Machine Translation ◽  
Likelihood Estimation ◽  
Neural Machine Translation ◽  
Parallel Corpora ◽  
Translation Model ◽  
Parallel Corpus ◽  
Target Model ◽  
High Resource ◽  
Remarkable Progress

While neural machine translation (NMT) has made remarkable progress in translating a handful of high-resource language pairs recently, parallel corpora are not always available for many zero-resource language pairs. To deal with this problem, we propose an approach to zero-resource NMT via maximum expected likelihood estimation. The basic idea is to maximize the expectation with respect to a pivot-to-source translation model for the intended source-to-target model on a pivot-target parallel corpus. To approximate the expectation, we propose two methods to connect the pivot-to-source and source-to-target models. Experiments on two zero-resource language pairs show that the proposed approach yields substantial gains over baseline methods. We also observe that when trained jointly with the source-to-target model, the pivot-to-source translation model also obtains improvements over independent training.

Sign in / Sign up

Export Citation Format

Share Document