scholarly journals A Review of Machine Translation for South Asian Low Resource Languages

2021 ◽  
Vol 12 (5) ◽  
pp. 1134-1147
Author(s):  
Syed Abdul Basit Andrabi, Et. al.
Keyword(s):  
Natural Language ◽  
Programming Languages ◽  
Machine Translation ◽  
Language Processing ◽  
Statistical Machine Translation ◽  
Automatic Generation ◽  
Natural Languages ◽  
Native Languages ◽  
Present Information ◽  
Translation Systems

Machine translation is an application of natural language processing. Humans use native languages to communicate with one another, whereas programming languages communicate between humans and computers. NLP is the field that involves a broad set of techniques for analysis, manipulation and automatic generation of human languages or natural languages with the help of computers. It is essential to provide access to information to people for their development in the present information age. It is necessary to put equal emphasis on removing the barrier of language between different divisions of society. The area of NLP strives to fill this gap of the language barrier by applying machine translation. One natural language is transformed into another natural language with the aid of computers. The first few years of this area were dedicated to the development of rule-based systems. Still, later on, due to the increase in computational power, there was a transition towards statistical machine translation. The motive of machine translation is that the meaning of the translated text should be preserved during translation. This research paper aims to analyse the machine translation approaches used for resource-poor languages and determine the needs and challenges the researchers face. This paper also reviews the machine translation systems that are available for poor research languages.

scholarly journals Using machine translation for converting Python 2 to Python 3 code

2015 ◽  
Author(s):  
Karan Aggarwal ◽  
Mohammad Salameh ◽  
Abram Hindle
Keyword(s):  
Pilot Study ◽  
Natural Language ◽  
Programming Languages ◽  
Machine Translation ◽  
Programming Language ◽  
Statistical Machine Translation ◽  
Natural Languages ◽  
The Cross ◽  
Cross Project

In this paper, we have tried to use statistical machine translation in order to convert Python 2 code to Python 3 code. We use data from two projects and achieve a high BLEU score. We also investigate the cross-project training and testing to analyze the errors so as to ascertain differences with previous case. We have described a pilot study on modeling programming languages as natural language to build translation models on the lines of natural languages. This can be further worked on to translate between versions of a programming language or cross-programming-languages code translation.

scholarly journals Using machine translation for converting Python 2 to Python 3 code

2015 ◽  
Author(s):  
Karan Aggarwal ◽  
Mohammad Salameh ◽  
Abram Hindle
Keyword(s):  
Pilot Study ◽  
Natural Language ◽  
Programming Languages ◽  
Machine Translation ◽  
Programming Language ◽  
Statistical Machine Translation ◽  
Natural Languages ◽  
The Cross ◽  
Cross Project

In this paper, we have tried to use statistical machine translation in order to convert Python 2 code to Python 3 code. We use data from two projects and achieve a high BLEU score. We also investigate the cross-project training and testing to analyze the errors so as to ascertain differences with previous case. We have described a pilot study on modeling programming languages as natural language to build translation models on the lines of natural languages. This can be further worked on to translate between versions of a programming language or cross-programming-languages code translation.

Extracting parallel phrases from comparable data for machine translation

2016 ◽  
Vol 22 (4) ◽  
pp. 549-573 ◽  
Author(s):  
SANJIKA HEWAVITHARANA ◽  
STEPHAN VOGEL
Keyword(s):  
Machine Translation ◽  
Language Processing ◽  
Statistical Machine Translation ◽  
Word Alignment ◽  
Data Set ◽  
Comparable Corpora ◽  
Alignment Algorithms ◽  
Extraction Algorithm ◽  
Phrase Alignment ◽  
Translation Systems

AbstractMining parallel data from comparable corpora is a promising approach for overcoming the data sparseness in statistical machine translation and other natural language processing applications. In this paper, we address the task of detecting parallel phrase pairs embedded in comparable sentence pairs. We present a novel phrase alignment approach that is designed to only align parallel sections bypassing non-parallel sections of the sentence. We compare the proposed approach with two other alignment methods: (1) the standard phrase extraction algorithm, which relies on the Viterbi path of the word alignment, (2) a binary classifier to detect parallel phrase pairs when presented with a large collection of phrase pair candidates. We evaluate the accuracy of these approaches using a manually aligned data set, and show that the proposed approach outperforms the other two approaches. Finally, we demonstrate the effectiveness of the extracted phrase pairs by using them in Arabic–English and Urdu–English translation systems, which resulted in improvements upto 1.2 Bleu over the baseline. The main contributions of this paper are two-fold: (1) novel phrase alignment algorithms to extract parallel phrase pairs from comparable sentences, (2) evaluating the utility of the extracted phrases by using them directly in the MT decoder.

scholarly journals A SYSTEMATIC READING IN STATISTICAL TRANSLATION: FROM THE STATISTICAL MACHINE TRANSLATION TO THE NEURAL TRANSLATION MODELS.

2017 ◽  
Author(s):  
Zakaria El Maazouzi ◽  
Badr Eddine EL Mohajir ◽  
Mohammed Al Achhab
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Machine Translation ◽  
Language Processing ◽  
State Of The Art ◽  
Statistical Machine Translation ◽  
High Accuracy ◽  
Neural Machine Translation ◽  
Translation Quality ◽  
Automatic Translation

Achieving high accuracy in automatic translation tasks has been one of the challenging goals for researchers in the area of machine translation since decades. Thus, the eagerness of exploring new possible ways to improve machine translation was always the matter for researchers in the field. Automatic translation as a key application in the natural language processing domain has developed many approaches, namely statistical machine translation and recently neural machine translation that improved largely the translation quality especially for Latin languages. They have even made it possible for the translation of some language pairs to approach human translation quality. In this paper, we present a survey of the state of the art of statistical translation, where we describe the different existing methodologies, and we overview the recent research studies while pointing out the main strengths and limitations of the different approaches.  

On the Statistical Machine Translation Studies

2013 ◽  
Vol 347-350 ◽  
pp. 3262-3266
Author(s):  
Ai Ling Wang
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Machine Translation ◽  
Language Processing ◽  
Large Scale ◽  
Statistical Machine Translation ◽  
Translation Model ◽  
Statistical Natural Language Processing ◽  
Translation Methods ◽  
Important Branch

Machine translation (MT) is one of the core application of natural language processing and an important branch of artificial intelligence research; statistical methods have already become the mainstream of machine translation. This paper explores the comparative analysis on the translation model of statistical natural language processing based on the large-scale corpus; discusses word-based, phrase-based and syntax-based machine translation methods respectively, summarizes the evaluation factors of machine translation and analyzes evaluation methods of machine translation.

scholarly journals Modern Machine Translation Systems: Trends and Prospects

2021 ◽  
pp. 41-52
Author(s):  
Oleg Kuzmin ◽  
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Machine Translation ◽  
Language Processing ◽  
Economic Benefits ◽  
Modern World ◽  
Semantic Features ◽  
Translation Systems ◽  
Modern Machine

The modern world is moving towards global digitalization and accelerated software development with a clear tendency to replace human resources by digital services or programs that imitate the doing of similar tasks. There is no doubt that, long term, the use of such technologies has economic benefits for enterprises and companies. Despite this, however, the quality of the final result is often less than satisfactory, and machine translation systems are no exception, as editing of texts translated by using online translation services is still a demanding task. At the moment, producing high-quality translations using only machine translation systems remains impossible for multiple reasons, the main of which lies in the mysteries of natural language: the existence of sublanguages, abstract words, polysemy, etc. Since improving the quality of machine translation systems is one of the priorities of natural language processing (NLP), this article describes current trends in developing modern machine translation systems as well as the latest advances in the field of natural language processing (NLP) and gives suggestions about software innovations that would minimize the number of errors. Even though recent years have seen a significant breakthrough in the speed of information analysis, in all probability, this will not be a priority issue in the future. The main criteria for evaluating the quality of translated texts will be the semantic coherence of these texts and the semantic accuracy of the lexical material used. To improve machine translation systems, we should introduce elements of data differentiation and personalization of information for individual users and their tasks, employing the method of thematic modeling for determining the subject area of a particular text. Currently, there are algorithms based on deep learning that are able to perform these tasks. However, the process of identifying unique lexical units requires a more detailed linguistic description of their semantic features. The parsing methods that will be used in analyzing texts should also provide for the possibility of clustering by sublanguages. Creating automated electronic dictionaries for specific fields of professional knowledge will help improve the quality of machine translation systems. Notably, to date there have been no successful projects of creating dictionaries for machine translation systems for specific sub-languages. Thus, there is a need to develop such dictionaries and to integrate them into existing online translation systems.

scholarly journals Direct Machine Translation System from Punjabi to Hindi for Newspapers headlines Domain

2013 ◽  
Vol 8 (3) ◽  
pp. 908-912 ◽  
Author(s):  
Sumita Rani ◽  
Dr. Vijay Luxmi
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Machine Translation ◽  
Language Processing ◽  
Translation System ◽  
Natural Languages ◽  
Machine Translation System ◽  
Common Parent

Machine Translation System is an important area in Natural Language Processing. The Direct MT system is based upon the utilization of syntactic and vocabulary similarities between more or few related natural languages. The relation between two or more languages is based upon their common parent language. The similarity between Punjabi and Hindi languages is due to their parent language Sanskrit. Punjabi and Hindi are closely related languages with lots of similarities in syntax and vocabulary. In the present paper, Direct Machine Translation System from Punjabi to Hindi has been developed and its output is evaluated in order to get the suitability of the system.

scholarly journals Extracting Parallel Sentences from Nonparallel Corpora Using Parallel Hierarchical Attention Network

2020 ◽  
Vol 2020 ◽  
pp. 1-9
Author(s):  
Shaolin Zhu ◽  
Yong Yang ◽  
Chun Xu
Keyword(s):  
Neural Network ◽  
Natural Language Processing ◽  
Natural Language ◽  
Machine Translation ◽  
Language Processing ◽  
State Of The Art ◽  
Research Problem ◽  
Shared Task ◽  
Translation Systems

Collecting parallel sentences from nonparallel data is a long-standing natural language processing research problem. In particular, parallel training sentences are very important for the quality of machine translation systems. While many existing methods have shown encouraging results, they cannot learn various alignment weights in parallel sentences. To address this issue, we propose a novel parallel hierarchical attention neural network which encodes monolingual sentences versus bilingual sentences and construct a classifier to extract parallel sentences. In particular, our attention mechanism structure can learn different alignment weights of words in parallel sentences. Experimental results show that our model can obtain state-of-the-art performance on the English-French, English-German, and English-Chinese dataset of BUCC 2017 shared task about parallel sentences’ extraction.

A Review and evaluation of Machine Translation methods for Lumasaaba

2020 ◽  
pp. 3-17
Author(s):  
Peter Nabende
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Machine Translation ◽  
Language Processing ◽  
Research Area ◽  
Data Driven ◽  
East African ◽  
Data Set ◽  
African Languages ◽  
Translation Methods

Natural Language Processing for under-resourced languages is now a mainstream research area. However, there are limited studies on Natural Language Processing applications for many indigenous East African languages. As a contribution to covering the current gap of knowledge, this paper focuses on evaluating the application of well-established machine translation methods for one heavily under-resourced indigenous East African language called Lumasaaba. Specifically, we review the most common machine translation methods in the context of Lumasaaba including both rule-based and data-driven methods. Then we apply a state of the art data-driven machine translation method to learn models for automating translation between Lumasaaba and English using a very limited data set of parallel sentences. Automatic evaluation results show that a transformer-based Neural Machine Translation model architecture leads to consistently better BLEU scores than the recurrent neural network-based models. Moreover, the automatically generated translations can be comprehended to a reasonable extent and are usually associated with the source language input.

A Machine Learning Application for Raising WASH Awareness in the Times of COVID-19 Pandemic (Preprint)

2020 ◽  
Author(s):  
Rohan Pandey ◽  
Vaibhav Gautam ◽  
Ridam Pal ◽  
Harsh Bandhey ◽  
Lovedeep Singh Dhingra ◽  
...  
Keyword(s):  
Machine Learning ◽  
Natural Language Processing ◽  
Natural Language ◽  
Machine Translation ◽  
Language Processing ◽  
User Feedback ◽  
Who Guidelines ◽  
The Times ◽  
The Right ◽  
Local Languages

BACKGROUND The COVID-19 pandemic has uncovered the potential of digital misinformation in shaping the health of nations. The deluge of unverified information that spreads faster than the epidemic itself is an unprecedented phenomenon that has put millions of lives in danger. Mitigating this ‘Infodemic’ requires strong health messaging systems that are engaging, vernacular, scalable, effective and continuously learn the new patterns of misinformation. OBJECTIVE We created WashKaro, a multi-pronged intervention for mitigating misinformation through conversational AI, machine translation and natural language processing. WashKaro provides the right information matched against WHO guidelines through AI, and delivers it in the right format in local languages. METHODS We theorize (i) an NLP based AI engine that could continuously incorporate user feedback to improve relevance of information, (ii) bite sized audio in the local language to improve penetrance in a country with skewed gender literacy ratios, and (iii) conversational but interactive AI engagement with users towards an increased health awareness in the community. RESULTS A total of 5026 people who downloaded the app during the study window, among those 1545 were active users. Our study shows that 3.4 times more females engaged with the App in Hindi as compared to males, the relevance of AI-filtered news content doubled within 45 days of continuous machine learning, and the prudence of integrated AI chatbot “Satya” increased thus proving the usefulness of an mHealth platform to mitigate health misinformation. CONCLUSIONS We conclude that a multi-pronged machine learning application delivering vernacular bite-sized audios and conversational AI is an effective approach to mitigate health misinformation. CLINICALTRIAL Not Applicable

Sign in / Sign up

Export Citation Format

Share Document