A Semantic Concept Based Unknown Words Processing Method in Neural Machine Translation

2018 ◽  
pp. 233-242 ◽  
Author(s):  
Shaotong Li ◽  
Jinan Xu ◽  
Guoyi Miao ◽  
Yujie Zhang ◽  
Yufeng Chen
Keyword(s):  
Machine Translation ◽  
Processing Method ◽  
Semantic Concept ◽  
Neural Machine Translation ◽  
Unknown Words

scholarly journals Deep Recurrent Models with Fast-Forward Connections for Neural Machine Translation

2016 ◽  
Vol 4 ◽  
pp. 371-383 ◽  
Author(s):  
Jie Zhou ◽  
Ying Cao ◽  
Xuguang Wang ◽  
Peng Li ◽  
Wei Xu
Keyword(s):  
Machine Translation ◽  
Short Term Memory ◽  
Short Term ◽  
Neural Machine Translation ◽  
Attention Model ◽  
Linear Connections ◽  
New Type ◽  
Long Short Term Memory ◽  
Unknown Words ◽  
First Time

Neural machine translation (NMT) aims at solving machine translation (MT) problems using neural networks and has exhibited promising results in recent years. However, most of the existing NMT models are shallow and there is still a performance gap between a single NMT model and the best conventional MT system. In this work, we introduce a new type of linear connections, named fast-forward connections, based on deep Long Short-Term Memory (LSTM) networks, and an interleaved bi-directional architecture for stacking the LSTM layers. Fast-forward connections play an essential role in propagating the gradients and building a deep topology of depth 16. On the WMT’14 English-to-French task, we achieve BLEU=37.7 with a single attention model, which outperforms the corresponding single shallow model by 6.2 BLEU points. This is the first time that a single NMT model achieves state-of-the-art performance and outperforms the best conventional model by 0.7 BLEU points. We can still achieve BLEU=36.3 even without using an attention mechanism. After special handling of unknown words and model ensembling, we obtain the best score reported to date on this task with BLEU=40.4. Our models are also validated on the more difficult WMT’14 English-to-German task.

scholarly journals Knowledge Graphs Effectiveness in Neural Machine Translation Improvement

2020 ◽  
Vol 21 (3) ◽  
Author(s):  
Benyamin Ahmadnia ◽  
Bonnie J. Dorr ◽  
Parisa Kordjamshidi
Keyword(s):  
Machine Translation ◽  
Semantic Representation ◽  
Language Translation ◽  
Semantic Relations ◽  
Training Data ◽  
Target Language ◽  
Neural Machine Translation ◽  
Source Language ◽  
Knowledge Graphs ◽  
Unknown Words

Neural Machine Translation (NMT) systems require a massive amount of Maintaining semantic relations between words during the translation process yields more accurate target-language output from Neural Machine Translation (NMT). Although difficult to achieve from training data alone, it is possible to leverage Knowledge Graphs (KGs) to retain source-language semantic relations in the corresponding target-language translation. The core idea is to use KG entity relations as embedding constraints to improve the mapping from source to target. This paper describes two embedding constraints, both of which employ Entity Linking (EL)---assigning a unique identity to entities---to associate words in training sentences with those in the KG: (1) a monolingual embedding constraint that supports an enhanced semantic representation of the source words through access to relations between entities in a KG; and (2) a bilingual embedding constraint that forces entity relations in the source-language to be carried over to the corresponding entities in the target-language translation. The method is evaluated for English-Spanish translation exploiting Freebase as a source of knowledge. Our experimental results show that exploiting KG information not only decreases the number of unknown words in the translation but also improves translation quality.

scholarly journals An Empirical Study on Learning Bug-Fixing Patches in the Wild via Neural Machine Translation

2019 ◽  
Vol 28 (4) ◽  
pp. 1-29 ◽  
Author(s):  
Michele Tufano ◽  
Cody Watson ◽  
Gabriele Bavota ◽  
Massimiliano Di Penta ◽  
Martin White ◽  
...  
Keyword(s):  
Empirical Study ◽  
Machine Translation ◽  
Neural Machine Translation ◽  
Bug Fixing ◽  
In The Wild
Sign in / Sign up

Export Citation Format

Share Document