OpenNMT: Open-Source Toolkit for Neural Machine Translation

AbstractWe present a multilingual preordering component tailored for a commercial Statistical Machine translation platform. In commercial settings, issues such as processing speed as well as the ability to adapt models to the customers’ needs play a significant role and have a big impact on the choice of approaches that are added to the custom pipeline to deal with specific problems such as long-range reorderings.We developed a fast and customisable preordering component, also available as an open-source tool, which comes along with a generic implementation that is restricted neither to the translation platform nor to the Machine Translation paradigm. We test preordering on three language pairs: English →Japanese/German/Chinese for both Statistical Machine Translation (SMT) and Neural Machine Translation (NMT). Our experiments confirm previously reported improvements in the SMT output when the models are trained on preordered data, but they also show that preordering does not improve NMT.

Download Full-text

Bridging the “gApp”: improving neural machine translation systems for multiword expression detection

Yearbook of Phraseology ◽

10.1515/phras-2020-0005 ◽

2020 ◽

Vol 11 (1) ◽

pp. 61-80

Author(s):

Carlos Manuel Hidalgo-Ternero ◽

Gloria Corpas Pastor

Keyword(s):

Open Source ◽

Machine Translation ◽

Automatic Identification ◽

Neural Machine Translation ◽

Multiword Expressions ◽

Continuous Form ◽

Text Preprocessing ◽

Translation Systems ◽

Free Open Source

AbstractThe present research introduces the tool gApp, a Python-based text preprocessing system for the automatic identification and conversion of discontinuous multiword expressions (MWEs) into their continuous form in order to enhance neural machine translation (NMT). To this end, an experiment with semi-fixed verb–noun idiomatic combinations (VNICs) will be carried out in order to evaluate to what extent gApp can optimise the performance of the two main free open-source NMT systems —Google Translate and DeepL— under the challenge of MWE discontinuity in the Spanish into English directionality. In the light of our promising results, the study concludes with suggestions on how to further optimise MWE-aware NMT systems.

Download Full-text

CytonMT: an Efficient Neural Machine Translation Open-source Toolkit Implemented in C++

10.18653/v1/d18-2023 ◽

2018 ◽

Cited By ~ 1

Author(s):

Xiaolin Wang ◽

Masao Utiyama ◽

Eiichiro Sumita

Keyword(s):

Open Source ◽

Machine Translation ◽

Neural Machine Translation

Download Full-text

Google Translate vs. DeepL

MonTi Monografías de Traducción e Interpretación ◽

10.6035/monti.2020.ne6.5 ◽

2021 ◽

pp. 154-177

Author(s):

Carlos Manuel Hidalgo-Ternero

Keyword(s):

Open Source ◽

Machine Translation ◽

Automatic Detection ◽

Neural Machine Translation ◽

Free Open Source

The present research analyses the performance of two free open-source neural machine translation (NMT) systems —Google Translate and DeepL— in the (ES>EN) translation of somatisms such as tomar el pelo and meter la pata, their nominal variants (tomadura/tomada de pelo and metedura/metida de pata), and other lower-frequency variants such as meter la pata hasta el corvejón, meter la gamba and metedura/metida de gamba. The machine translation outcomes will be contrasted and classified depending on whether these idioms are presented in their continuous or discontinuous form (Anastasiou 2010), i.e., whether different n-grams split the idiomatic sequence (or not), which may pose some difficulties for their automatic detection and translation. Overall, the insights gained from this study will prove useful in determining for which of the different scenarios either Google Translate or DeepL delivers a better performance under the challenge of phraseological variation and discontinuity.

Download Full-text

Open-Source Neural Machine Translation API Server

Prague Bulletin of Mathematical Linguistics ◽

10.1515/pralin-2017-0034 ◽

2017 ◽

Vol 109 (1) ◽

pp. 5-14

Author(s):

Sander Tars ◽

Kaspar Papli ◽

Dmytro Chasovskyi ◽

Mark Fishel

Keyword(s):

Open Source ◽

Machine Translation ◽

Software Package ◽

Web Application ◽

Service Providers ◽

Neural Machine Translation ◽

Translation Quality ◽

Open Api ◽

Translation Service

Abstract We introduce an open-source implementation of a machine translation API server. The aim of this software package is to enable anyone to run their own multi-engine translation server with neural machine translation engines, supporting an open API for client applications. Besides the hub with the implementation of the client API and the translation service providers running in the background we also describe an open-source demo web application that uses our software package and implements an online translation tool that supports collecting translation quality comparisons from users.

Download Full-text