scholarly journals Evaluating Machine Translation Quality Using Short Segments Annotations

2015 ◽  
Vol 103 (1) ◽  
pp. 85-110
Author(s):  
Matouš Macháček ◽  
Ondřej Bojar
Keyword(s):  
Machine Translation ◽  
Evaluation Method ◽  
Statistical Machine Translation ◽  
Translation System ◽  
Translation Quality ◽  
Machine Translation System

Abstract We propose a manual evaluation method for machine translation (MT), in which annotators rank only translations of short segments instead of whole sentences. This results in an easier and more efficient annotation. We have conducted an annotation experiment and evaluated a set of MT systems using this method. The obtained results are very close to the official WMT14 evaluation results. We also use the collected database of annotations to automatically evaluate new, unseen systems and to tune parameters of a statistical machine translation system. The evaluation of unseen systems, however, does not work and we analyze the reasons

scholarly journals Exploration of Corpus Augmentation Approach for English-Hindi Bidirectional Statistical Machine Translation System

2016 ◽  
Vol 6 (3) ◽  
pp. 1059 ◽  
Author(s):  
K. Jaya ◽  
Deepa Gupta
Keyword(s):  
Machine Translation ◽  
Statistical Machine Translation ◽  
Research Work ◽  
Translation System ◽  
Indian Languages ◽  
Translation Quality ◽  
Parallel Data ◽  
Machine Translation System ◽  
Best Parameter

Even though lot of Statistical Machine Translation(SMT) research work is happening for English-Hindi language pair, there is no effort done to standardize the dataset. Each of the research work uses different dataset, different parameters and different number of sentences during various phases of translation resulting in varied translation output. So comparing  these models, understand the result of these models, to get insight into corpus behavior for these models, regenerating the result of these research work  becomes tedious. This necessitates the need for standardization of dataset and to identify the common parameter for the development of model.  The main contribution of this paper is to discuss an approach to standardize the dataset and to identify the best parameter which in combination gives best performance. It also investigates a novel corpus augmentation approach to improve the translation quality of English-Hindi bidirectional statistical machine translation system. This model works well for the scarce resource without incorporating the external parallel data corpus of the underlying language.  This experiment is carried out using Open Source phrase-based toolkit Moses. Indian Languages Corpora Initiative (ILCI) Hindi-English tourism corpus is used.  With limited dataset, considerable improvement is achieved using the corpus augmentation approach for the English-Hindi bidirectional SMT system.

Exploration of Corpus Augmentation Approach for English-Hindi Bidirectional Statistical Machine Translation System

2016 ◽  
Vol 6 (3) ◽  
pp. 1059 ◽  
Author(s):  
K. Jaya ◽  
Deepa Gupta
Keyword(s):  
Machine Translation ◽  
Statistical Machine Translation ◽  
Research Work ◽  
Translation System ◽  
Indian Languages ◽  
Translation Quality ◽  
Parallel Data ◽  
Machine Translation System ◽  
Best Parameter

Even though lot of Statistical Machine Translation(SMT) research work is happening for English-Hindi language pair, there is no effort done to standardize the dataset. Each of the research work uses different dataset, different parameters and different number of sentences during various phases of translation resulting in varied translation output. So comparing  these models, understand the result of these models, to get insight into corpus behavior for these models, regenerating the result of these research work  becomes tedious. This necessitates the need for standardization of dataset and to identify the common parameter for the development of model.  The main contribution of this paper is to discuss an approach to standardize the dataset and to identify the best parameter which in combination gives best performance. It also investigates a novel corpus augmentation approach to improve the translation quality of English-Hindi bidirectional statistical machine translation system. This model works well for the scarce resource without incorporating the external parallel data corpus of the underlying language.  This experiment is carried out using Open Source phrase-based toolkit Moses. Indian Languages Corpora Initiative (ILCI) Hindi-English tourism corpus is used.  With limited dataset, considerable improvement is achieved using the corpus augmentation approach for the English-Hindi bidirectional SMT system.

Designing a machine translation system for Canadian weather warnings: A case study

2013 ◽  
Vol 20 (3) ◽  
pp. 399-433 ◽  
Author(s):  
FABRIZIO GOTTI ◽  
PHILIPPE LANGLAIS ◽  
GUY LAPALME
Keyword(s):  
Machine Translation ◽  
Statistical Machine Translation ◽  
Production Quality ◽  
Translation System ◽  
Translation Quality ◽  
French And English ◽  
Machine Translation System ◽  
Text Presentation ◽  
The Many ◽  
Evaluation Metric

In this paper we describe the many steps involved in building a production quality Machine Translation system for translating weather warnings between French and English. Although in principle this task may seem straightforward, the details, especially corpus preparation and final text presentation, involve many difficult aspects that are often glossed over in the literature. On top of the classic Statistical Machine Translation evaluation metric results, four manual evaluations have been performed to assess and improve translation quality. We also show the usefulness of the integration of out-of-domain information sources in a Statistical Machine Translation system to produce high quality translated text.

scholarly journals Sampling Phrase Tables for the Moses Statistical Machine Translation System

2015 ◽  
Vol 104 (1) ◽  
pp. 39-50 ◽  
Author(s):  
Ulrich Germann
Keyword(s):  
Machine Translation ◽  
Statistical Machine Translation ◽  
Practical Implementation ◽  
Translation System ◽  
Translation Quality ◽  
On Demand ◽  
Machine Translation System

Abstract The idea of virtual phrase tables for statistical machine translation (SMT) that construct phrase table entries on demand by sampling a fully indexed bitext was first proposed ten years ago by Callison-Burch et al. (2005). However, until recently (Germann, 2014) no working and practical implementation of this approach was available in the Moses SMT system. We describe and evaluate this implementation in more detail. Sampling phrase tables are much faster to build and are competitive with conventional phrase tables in terms of translation quality and speed.

English-Dogri Translation System using MOSES

2016 ◽  
Vol 1 (1) ◽  
pp. 45-49
Author(s):  
Avinash Singh ◽  
Asmeet Kour ◽  
Shubhnandan S. Jamwal
Keyword(s):  
Natural Language Processing ◽  
Machine Translation ◽  
Language Processing ◽  
Statistical Machine Translation ◽  
Translation System ◽  
Parallel Corpus ◽  
English System ◽  
Machine Translation System ◽  
Translation Machine ◽  
Language Pair

The objective behind this paper is to analyze the English-Dogri parallel corpus translation. Machine translation is the translation from one language into another language. Machine translation is the biggest application of the Natural Language Processing (NLP). Moses is statistical machine translation system allow to train translation models for any language pair. We have developed translation system using Statistical based approach which helps in translating English to Dogri and vice versa. The parallel corpus consists of 98,973 sentences. The system gives accuracy of 80% in translating English to Dogri and the system gives accuracy of 87% in translating Dogri to English system.

scholarly journals An Open-Source Web-Based Tool for Resource-Agnostic Interactive Translation Prediction

2014 ◽  
Vol 102 (1) ◽  
pp. 69-80 ◽  
Author(s):  
Torregrosa Daniel ◽  
Forcada Mikel L. ◽  
Pérez-Ortiz Juan Antonio
Keyword(s):  
Open Source ◽  
Machine Translation ◽  
Web Application ◽  
Statistical Machine Translation ◽  
Black Box ◽  
Translation System ◽  
Web Tool ◽  
Web Based ◽  
Strongly Coupled ◽  
Machine Translation System

Abstract We present a web-based open-source tool for interactive translation prediction (ITP) and describe its underlying architecture. ITP systems assist human translators by making context-based computer-generated suggestions as they type. Most of the ITP systems in literature are strongly coupled with a statistical machine translation system that is conveniently adapted to provide the suggestions. Our system, however, follows a resource-agnostic approach and suggestions are obtained from any unmodified black-box bilingual resource. This paper reviews our ITP method and describes the architecture of Forecat, a web tool, partly based on the recent technology of web components, that eases the use of our ITP approach in any web application requiring this kind of translation assistance. We also evaluate the performance of our method when using an unmodified Moses-based statistical machine translation system as the bilingual resource.

Development of a method for assessing the quality of machine translation based on ensemble methods in machine learning

2021 ◽  
Author(s):  
A.V. Kozina ◽  
Yu.S. Belov
Keyword(s):  
Machine Learning ◽  
Random Forest ◽  
Quality Assessment ◽  
Machine Translation ◽  
Translation System ◽  
Human Judgment ◽  
Translation Quality ◽  
Machine Translation System ◽  
Translation Systems

Automatically assessing the quality of machine translation is an important yet challenging task for machine translation research. Translation quality assessment is understood as predicting translation quality without reference to the source text. Translation quality depends on the specific machine translation system and often requires post-editing. Manual editing is a long and expensive process. Since the need to quickly determine the quality of translation increases, its automation is required. In this paper, we propose a quality assessment method based on ensemble supervised machine learning methods. The bilingual corpus WMT 2019 for the EnglishRussian language pair was used as data. The text data volume is 17089 sentences, 85% of the data was used for training, and 15% for testing the model. Linguistic functions extracted from the text in the source and target languages were used as features for training the system, since it is these characteristics that can most accurately characterize the translation in terms of quality. The following tools were used for feature extraction: a free language modeling tool based on SRILM and a Stanford POS Tagger parts of speech tagger. Before training the system, the text was preprocessed. The model was trained using three regression methods: Bagging, Extra Tree, and Random Forest. The algorithms were implemented in the Python programming language using the Scikit learn library. The parameters of the random forest method have been optimized using a grid search. The performance of the model was assessed by the mean absolute error MAE and the root mean square error RMSE, as well as by the Pearsоn coefficient, which determines the correlation with human judgment. Testing was carried out using three machine translation systems: Google and Bing neural systems, Mouses statistical machine translation systems based on phrases and based on syntax. Based on the results of the work, the method of additional trees showed itself best. In addition, for all categories of indicators under consideration, the best results are achieved using the Google machine translation system. The developed method showed good results close to human judgment. The system can be used for further research in the task of assessing the quality of translation.

Translation of Medical Texts using Neural Networks

2016 ◽  
Vol 5 (4) ◽  
pp. 51-66 ◽  
Author(s):  
Krzysztof Wolk ◽  
Krzysztof P. Marasek
Keyword(s):  
Machine Translation ◽  
Statistical Machine Translation ◽  
European Medicines Agency ◽  
Translation System ◽  
Training Methods ◽  
Neural Machine Translation ◽  
Machine Translation System ◽  
Source Sentence ◽  
Parallel Text ◽  
Translation Systems

The quality of machine translation is rapidly evolving. Today one can find several machine translation systems on the web that provide reasonable translations, although the systems are not perfect. In some specific domains, the quality may decrease. A recently proposed approach to this domain is neural machine translation. It aims at building a jointly-tuned single neural network that maximizes translation performance, a very different approach from traditional statistical machine translation. Recently proposed neural machine translation models often belong to the encoder-decoder family in which a source sentence is encoded into a fixed length vector that is, in turn, decoded to generate a translation. The present research examines the effects of different training methods on a Polish-English Machine Translation system used for medical data. The European Medicines Agency parallel text corpus was used as the basis for training of neural and statistical network-based translation systems. A comparison and implementation of a medical translator is the main focus of our experiments.

Sign in / Sign up

Export Citation Format

Share Document