scholarly journals Identifying Semantics in Clinical Reports Using Neural Machine Translation

2019 ◽  
Vol 33 ◽  
pp. 9552-9557
Author(s):  
Srikanth Mujjiga ◽  
Vamsi Krishna ◽  
Kalyan Chakravarthi ◽  
Vijayananda J
Keyword(s):  
Machine Translation ◽  
Large Scale ◽  
Language Translation ◽  
Semantic Search ◽  
Neural Machine Translation ◽  
Healthcare Facilities ◽  
Semantic Level ◽  
Concept Space ◽  
Unique Approach ◽  
Retrieval Problem

Clinical documents are vital resources for radiologists when they have to consult or refer while studying similar cases. In large healthcare facilities where millions of reports are generated, searching for relevant documents is quite challenging. With abundant interchangeable words in clinical domain, understanding the semantics of the words in the clinical documents is vital to improve the search results. This paper details an end to end semantic search application to address the large scale information retrieval problem of clinical reports. The paper specifically focuses on the challenge of identifying semantics in the clinical reports to facilitate search at semantic level. The semantic search works by mapping the documents into the concept space and the search is performed in the concept space. A unique approach of framing the concept mapping problem as a language translation problem is proposed in this paper. The concept mapper is modelled using the Neural machine translation model (NMT) based on encoder-decoder with attention architecture. The regular expression based concept mapper takes approximately 3 seconds to extract UMLS concepts from a single document, where as the trained NMT does the same in approximately 30 milliseconds. NMT based model further enables incorporation of negation detection to identify whether a concept is negated or not, facilitating search for negated queries.

Improved neural machine translation for low-resource English–Assamese pair

2021 ◽  
pp. 1-12
Author(s):  
Sahinur Rahman Laskar ◽  
Abdullah Faiz Ur Rahman Khilji ◽  
Partha Pakray ◽  
Sivaji Bandyopadhyay
Keyword(s):  
Machine Translation ◽  
Data Augmentation ◽  
Language Translation ◽  
Linguistically Diverse ◽  
Neural Machine Translation ◽  
Low Resource ◽  
Parallel Data ◽  
The World ◽  
Translation Accuracy ◽  
Vocabulary Problems

Language translation is essential to bring the world closer and plays a significant part in building a community among people of different linguistic backgrounds. Machine translation dramatically helps in removing the language barrier and allows easier communication among linguistically diverse communities. Due to the unavailability of resources, major languages of the world are accounted as low-resource languages. This leads to a challenging task of automating translation among various such languages to benefit indigenous speakers. This article investigates neural machine translation for the English–Assamese resource-poor language pair by tackling insufficient data and out-of-vocabulary problems. We have also proposed an approach of data augmentation-based NMT, which exploits synthetic parallel data and shows significantly improved translation accuracy for English-to-Assamese and Assamese-to-English translation and obtained state-of-the-art results.

scholarly journals A Joint Back-Translation and Transfer Learning Method for Low-Resource Neural Machine Translation

2020 ◽  
Vol 2020 ◽  
pp. 1-11
Author(s):  
Gong-Xu Luo ◽  
Ya-Ting Yang ◽  
Rui Dong ◽  
Yan-Hong Chen ◽  
Wen-Bo Zhang
Keyword(s):  
Machine Translation ◽  
Transfer Learning ◽  
Large Scale ◽  
Data Augmentation ◽  
Training Methods ◽  
Learning Method ◽  
Neural Machine Translation ◽  
Low Resource ◽  
Parallel Data ◽  
Back Translation

Neural machine translation (NMT) for low-resource languages has drawn great attention in recent years. In this paper, we propose a joint back-translation and transfer learning method for low-resource languages. It is widely recognized that data augmentation methods and transfer learning methods are both straight forward and effective ways for low-resource problems. However, existing methods, which utilize one of these methods alone, limit the capacity of NMT models for low-resource problems. In order to make full use of the advantages of existing methods and further improve the translation performance of low-resource languages, we propose a new method to perfectly integrate the back-translation method with mainstream transfer learning architectures, which can not only initialize the NMT model by transferring parameters of the pretrained models, but also generate synthetic parallel data by translating large-scale monolingual data of the target side to boost the fluency of translations. We conduct experiments to explore the effectiveness of the joint method by incorporating back-translation into the parent-child and the hierarchical transfer learning architecture. In addition, different preprocessing and training methods are explored to get better performance. Experimental results on Uygur-Chinese and Turkish-English translation demonstrate the superiority of the proposed method over the baselines that use single methods.

scholarly journals Synchronous Bidirectional Neural Machine Translation

2019 ◽  
Vol 7 ◽  
pp. 91-105 ◽  
Author(s):  
Long Zhou ◽  
Jiajun Zhang ◽  
Chengqing Zong
Keyword(s):  
Machine Translation ◽  
Large Scale ◽  
State Of The Art ◽  
Target Language ◽  
Single Model ◽  
Neural Machine Translation ◽  
German Translation ◽  
Transformer Model ◽  
Target Side ◽  
Future Information

Existing approaches to neural machine translation (NMT) generate the target language sequence token-by-token from left to right. However, this kind of unidirectional decoding framework cannot make full use of the target-side future contexts which can be produced in a right-to-left decoding direction, and thus suffers from the issue of unbalanced outputs. In this paper, we introduce a synchronous bidirectional–neural machine translation (SB-NMT) that predicts its outputs using left-to-right and right-to-left decoding simultaneously and interactively, in order to leverage both of the history and future information at the same time. Specifically, we first propose a new algorithm that enables synchronous bidirectional decoding in a single model. Then, we present an interactive decoding model in which left-to-right (right-to-left) generation does not only depend on its previously generated outputs, but also relies on future contexts predicted by right-to-left (left-to-right) decoding. We extensively evaluate the proposed SB-NMT model on large-scale NIST Chinese-English, WMT14 English-German, and WMT18 Russian-English translation tasks. Experimental results demonstrate that our model achieves significant improvements over the strong Transformer model by 3.92, 1.49, and 1.04 BLEU points, respectively, and obtains the state-of-the-art per- formance on Chinese-English and English- German translation tasks. 1

scholarly journals A Study of Neural Machine Translation from Chinese to Urdu

2020 ◽  
Vol 2 (4) ◽  
pp. 28
Author(s):  
. Zeeshan
Keyword(s):  
Machine Translation ◽  
Chinese Language ◽  
Language Translation ◽  
Target Language ◽  
Foreign Languages ◽  
Neural Machine Translation ◽  
Source Language ◽  
Great Progress ◽  
Score Method ◽  
Translation Methods

Machine Translation (MT) is used for giving a translation from a source language to a target language. Machine translation simply translates text or speech from one language to another language, but this process is not sufficient to give the perfect translation of a text due to the requirement of identification of whole expressions and their direct counterparts. Neural Machine Translation (NMT) is one of the most standard machine translation methods, which has made great progress in the recent years especially in non-universal languages. However, local language translation software for other foreign languages is limited and needs improving. In this paper, the Chinese language is translated to the Urdu language with the help of Open Neural Machine Translation (OpenNMT) in Deep Learning. Firstly, a Chineseto Urdu language sentences datasets were established and supported with Seven million sentences. After that, these datasets were trained by using the Open Neural Machine Translation (OpenNMT) method. At the final stage, the translation was compared to the desired translation with the help of the Bleu Score Method.

scholarly journals Knowledge Graphs Effectiveness in Neural Machine Translation Improvement

2020 ◽  
Vol 21 (3) ◽  
Author(s):  
Benyamin Ahmadnia ◽  
Bonnie J. Dorr ◽  
Parisa Kordjamshidi
Keyword(s):  
Machine Translation ◽  
Semantic Representation ◽  
Language Translation ◽  
Semantic Relations ◽  
Training Data ◽  
Target Language ◽  
Neural Machine Translation ◽  
Source Language ◽  
Knowledge Graphs ◽  
Unknown Words

Neural Machine Translation (NMT) systems require a massive amount of Maintaining semantic relations between words during the translation process yields more accurate target-language output from Neural Machine Translation (NMT). Although difficult to achieve from training data alone, it is possible to leverage Knowledge Graphs (KGs) to retain source-language semantic relations in the corresponding target-language translation. The core idea is to use KG entity relations as embedding constraints to improve the mapping from source to target. This paper describes two embedding constraints, both of which employ Entity Linking (EL)---assigning a unique identity to entities---to associate words in training sentences with those in the KG: (1) a monolingual embedding constraint that supports an enhanced semantic representation of the source words through access to relations between entities in a KG; and (2) a bilingual embedding constraint that forces entity relations in the source-language to be carried over to the corresponding entities in the target-language translation. The method is evaluated for English-Spanish translation exploiting Freebase as a source of knowledge. Our experimental results show that exploiting KG information not only decreases the number of unknown words in the translation but also improves translation quality.

Neural machine translation of low-resource languages using SMT phrase pair injection

2020 ◽  
pp. 1-22
Author(s):  
Sukanta Sen ◽  
Mohammed Hasanuzzaman ◽  
Asif Ekbal ◽  
Pushpak Bhattacharyya ◽  
Andy Way
Keyword(s):  
Machine Translation ◽  
Large Scale ◽  
Production Systems ◽  
Statistical Machine Translation ◽  
Training Data ◽  
Original Training ◽  
Neural Machine Translation ◽  
Parallel Corpus ◽  
Low Resource ◽  
Better Than

Abstract Neural machine translation (NMT) has recently shown promising results on publicly available benchmark datasets and is being rapidly adopted in various production systems. However, it requires high-quality large-scale parallel corpus, and it is not always possible to have sufficiently large corpus as it requires time, money, and professionals. Hence, many existing large-scale parallel corpus are limited to the specific languages and domains. In this paper, we propose an effective approach to improve an NMT system in low-resource scenario without using any additional data. Our approach aims at augmenting the original training data by means of parallel phrases extracted from the original training data itself using a statistical machine translation (SMT) system. Our proposed approach is based on the gated recurrent unit (GRU) and transformer networks. We choose the Hindi–English, Hindi–Bengali datasets for Health, Tourism, and Judicial (only for Hindi–English) domains. We train our NMT models for 10 translation directions, each using only 5–23k parallel sentences. Experiments show the improvements in the range of 1.38–15.36 BiLingual Evaluation Understudy points over the baseline systems. Experiments show that transformer models perform better than GRU models in low-resource scenarios. In addition to that, we also find that our proposed method outperforms SMT—which is known to work better than the neural models in low-resource scenarios—for some translation directions. In order to further show the effectiveness of our proposed model, we also employ our approach to another interesting NMT task, for example, old-to-modern English translation, using a tiny parallel corpus of only 2.7K sentences. For this task, we use publicly available old-modern English text which is approximately 1000 years old. Evaluation for this task shows significant improvement over the baseline NMT.

Translation of Sentence Lampung-Indonesian Languages with Neural Machine Translation Attention Based Approach

2018 ◽  
Vol 6 (02) ◽  
pp. 191-206
Author(s):  
Zaenal Abidin
Keyword(s):  
Machine Translation ◽  
Language Translation ◽  
Training Process ◽  
Neural Machine Translation ◽  
New Approach ◽  
Source Language ◽  
Average Value ◽  
Approach Method ◽  
Stable Vectors ◽  
Network Component

In this research, automatically Lampung language translation into the Indonesian language was using neural machine translation (NMT) attention based approach. NMT, a new approach method in machine translation technology, that has worked by combining the encoder and decoder. The encoder in NMT is a recurrent neural network component that encrypts the source language to several length-stable vectors and the decoder is a recurrent neural networks component that generates translation result comprehensive. NMT Research has begun with creating a pair of 3000 parallel sentences of Lampung language (api dialect) and Indonesian language. Then it continues to decide the NMT parameter model for the data training process. The next step is building NMT model and evaluate it. The testing of this approach has used 25 single sentences without out-of-vocabulary (OOV), 25 single sentences with OOV, 25 plural sentences without OOV, and 25 plural sentences with OOV. The testing translation result using NMT attention shows the bilingual evaluation understudy (BLEU) an average value is 51, 96 %.

scholarly journals Cross-Lingual Pre-Training Based Transfer for Zero-Shot Neural Machine Translation

2020 ◽  
Vol 34 (01) ◽  
pp. 115-122 ◽  
Author(s):  
Baijun Ji ◽  
Zhirui Zhang ◽  
Xiangyu Duan ◽  
Min Zhang ◽  
Boxing Chen ◽  
...  
Keyword(s):  
Machine Translation ◽  
Transfer Learning ◽  
Large Scale ◽  
Feature Space ◽  
Target Language ◽  
Smooth Transition ◽  
Training Methods ◽  
Neural Machine Translation ◽  
Cross Lingual ◽  
Effective Transfer

Transfer learning between different language pairs has shown its effectiveness for Neural Machine Translation (NMT) in low-resource scenario. However, existing transfer methods involving a common target language are far from success in the extreme scenario of zero-shot translation, due to the language space mismatch problem between transferor (the parent model) and transferee (the child model) on the source side. To address this challenge, we propose an effective transfer learning approach based on cross-lingual pre-training. Our key idea is to make all source languages share the same feature space and thus enable a smooth transition for zero-shot translation. To this end, we introduce one monolingual pre-training method and two bilingual pre-training methods to obtain a universal encoder for different languages. Once the universal encoder is constructed, the parent model built on such encoder is trained with large-scale annotated data and then directly applied in zero-shot translation scenario. Experiments on two public datasets show that our approach significantly outperforms strong pivot-based baseline and various multilingual NMT approaches.

scholarly journals Improving Stylized Neural Machine Translation with Iterative Dual Knowledge Transfer

2021 ◽  
Author(s):  
Xuanxuan Wu ◽  
Jian Liu ◽  
Xinjie Li ◽  
Jinan Xu ◽  
Yufeng Chen ◽  
...  
Keyword(s):  
Knowledge Transfer ◽  
Machine Translation ◽  
Large Scale ◽  
Training Data ◽  
Transfer Model ◽  
Paired Data ◽  
Style Transfer ◽  
Neural Machine Translation ◽  
Translation Model ◽  
Parallel Data

Stylized neural machine translation (NMT) aims to translate sentences of one style into sentences of another style, which is essential for the application of machine translation in a real-world scenario. However, a major challenge in this task is the scarcity of high-quality parallel data which is stylized paired. To address this problem, we propose an iterative dual knowledge transfer framework that utilizes informal training data of machine translation and formality style transfer data to create large-scale stylized paired data, for the training of stylized machine translation model. Specifically, we perform bidirectional knowledge transfer between translation model and text style transfer model iteratively through knowledge distillation. Then, we further propose a data-refinement module to process the noisy synthetic parallel data generated during knowledge transfer. Experiment results demonstrate the effectiveness of our method, achieving an improvement over the existing best model by 5 BLEU points on MTFC dataset. Meanwhile, extensive analyses illustrate our method can also improve the accuracy of formality style transfer.

scholarly journals Polygon-Net: A General Framework for Jointly Boosting Multiple Unsupervised Neural Machine Translation Models

2019 ◽  
Author(s):  
Chang Xu ◽  
Tao Qin ◽  
Gang Wang ◽  
Tie-Yan Liu
Keyword(s):  
Machine Translation ◽  
Loss Function ◽  
General Framework ◽  
Large Scale ◽  
Great Success ◽  
Neural Machine Translation ◽  
Low Resource ◽  
Parallel Data ◽  
Benchmark Datasets ◽  
First Time

Neural machine translation (NMT) has achieved great success. However, collecting large-scale parallel data for training is costly and laborious.  Recently, unsupervised neural machine translation has attracted more and more attention, due to its demand for monolingual corpus only, which is common and easy to obtain, and its great potentials for the low-resource or even zero-resource machine translation. In this work, we propose a general framework called Polygon-Net, which leverages multi auxiliary languages for jointly boosting unsupervised neural machine translation models. Specifically, we design a novel loss function for multi-language unsupervised neural machine translation. In addition, different from the literature that just updating one or two models individually, Polygon-Net enables multiple unsupervised models in the framework to update in turn and enhance each other for the first time. In this way, multiple unsupervised translation models are associated with each other for training to achieve better performance. Experiments on the benchmark datasets including UN Corpus and WMT show that our approach significantly improves over the two-language based methods, and achieves better performance with more languages introduced to the framework. 

Sign in / Sign up

Export Citation Format

Share Document