scholarly journals Deep Recurrent Models with Fast-Forward Connections for Neural Machine Translation

2016 ◽  
Vol 4 ◽  
pp. 371-383 ◽  
Author(s):  
Jie Zhou ◽  
Ying Cao ◽  
Xuguang Wang ◽  
Peng Li ◽  
Wei Xu
Keyword(s):  
Machine Translation ◽  
Short Term Memory ◽  
Short Term ◽  
Neural Machine Translation ◽  
Attention Model ◽  
Linear Connections ◽  
New Type ◽  
Long Short Term Memory ◽  
Unknown Words ◽  
First Time

Neural machine translation (NMT) aims at solving machine translation (MT) problems using neural networks and has exhibited promising results in recent years. However, most of the existing NMT models are shallow and there is still a performance gap between a single NMT model and the best conventional MT system. In this work, we introduce a new type of linear connections, named fast-forward connections, based on deep Long Short-Term Memory (LSTM) networks, and an interleaved bi-directional architecture for stacking the LSTM layers. Fast-forward connections play an essential role in propagating the gradients and building a deep topology of depth 16. On the WMT’14 English-to-French task, we achieve BLEU=37.7 with a single attention model, which outperforms the corresponding single shallow model by 6.2 BLEU points. This is the first time that a single NMT model achieves state-of-the-art performance and outperforms the best conventional model by 0.7 BLEU points. We can still achieve BLEU=36.3 even without using an attention mechanism. After special handling of unknown words and model ensembling, we obtain the best score reported to date on this task with BLEU=40.4. Our models are also validated on the more difficult WMT’14 English-to-German task.

scholarly journals Sentence Wise Telugu to English Translation of Vemana Sathakam using LSTM

2019 ◽  
Vol 8 (4) ◽  
pp. 10739-10743
Keyword(s):  
Machine Translation ◽  
English Translation ◽  
Short Term Memory ◽  
Language Translation ◽  
Statistical Machine Translation ◽  
Short Term ◽  
Neural Machine Translation ◽  
Term Memory ◽  
Long Short Term Memory ◽  
Bidirectional Lstm

Language translation is a power of humans where machines are lagging and need to acquire. Previous statistical machine translation is used for translation but is applicable for large and similar grammar structure dataset. In this paper neural machine translation with long short term memory (LSTM) is used for addressing the issue. This paper uses a bidirectional LSTM to translate Telugu literary poems of Yogi Vemana to English which exhibited satisfactory translation. The results are compared with existing and proposed methods. NMT with LSTM yields better in language translation.

scholarly journals Prediction of Head Movement in 360-Degree Videos Using Attention Model

Sensors ◽  
2021 ◽  
Vol 21 (11) ◽  
pp. 3678
Author(s):  
Dongwon Lee ◽  
Minji Choi ◽  
Joohyun Lee
Keyword(s):  
Machine Learning ◽  
Short Term Memory ◽  
Moving Average ◽  
The Other ◽  
Learning Models ◽  
Short Term ◽  
Term Memory ◽  
Attention Model ◽  
Long Short Term Memory ◽  
Machine Learning Models

In this paper, we propose a prediction algorithm, the combination of Long Short-Term Memory (LSTM) and attention model, based on machine learning models to predict the vision coordinates when watching 360-degree videos in a Virtual Reality (VR) or Augmented Reality (AR) system. Predicting the vision coordinates while video streaming is important when the network condition is degraded. However, the traditional prediction models such as Moving Average (MA) and Autoregression Moving Average (ARMA) are linear so they cannot consider the nonlinear relationship. Therefore, machine learning models based on deep learning are recently used for nonlinear predictions. We use the Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) neural network methods, originated in Recurrent Neural Networks (RNN), and predict the head position in the 360-degree videos. Therefore, we adopt the attention model to LSTM to make more accurate results. We also compare the performance of the proposed model with the other machine learning models such as Multi-Layer Perceptron (MLP) and RNN using the root mean squared error (RMSE) of predicted and real coordinates. We demonstrate that our model can predict the vision coordinates more accurately than the other models in various videos.

scholarly journals The use of machine translation algorithm based on residual and LSTM neural network in translation teaching

PLoS ONE ◽  
2020 ◽  
Vol 15 (11) ◽  
pp. e0240663
Author(s):  
Beibei Ren
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Machine Translation ◽  
Short Term Memory ◽  
Short Term ◽  
Term Memory ◽  
Translation Model ◽  
Translation Quality ◽  
Long Short Term Memory ◽  
Deep Learning Neural Network

With the rapid development of big data and deep learning, breakthroughs have been made in phonetic and textual research, the two fundamental attributes of language. Language is an essential medium of information exchange in teaching activity. The aim is to promote the transformation of the training mode and content of translation major and the application of the translation service industry in various fields. Based on previous research, the SCN-LSTM (Skip Convolutional Network and Long Short Term Memory) translation model of deep learning neural network is constructed by learning and training the real dataset and the public PTB (Penn Treebank Dataset). The feasibility of the model’s performance, translation quality, and adaptability in practical teaching is analyzed to provide a theoretical basis for the research and application of the SCN-LSTM translation model in English teaching. The results show that the capability of the neural network for translation teaching is nearly one times higher than that of the traditional N-tuple translation model, and the fusion model performs much better than the single model, translation quality, and teaching effect. To be specific, the accuracy of the SCN-LSTM translation model based on deep learning neural network is 95.21%, the degree of translation confusion is reduced by 39.21% compared with that of the LSTM (Long Short Term Memory) model, and the adaptability is 0.4 times that of the N-tuple model. With the highest level of satisfaction in practical teaching evaluation, the SCN-LSTM translation model has achieved a favorable effect on the translation teaching of the English major. In summary, the performance and quality of the translation model are improved significantly by learning the language characteristics in translations by teachers and students, providing ideas for applying machine translation in professional translation teaching.

An Approach to NMT Re-Ranking Using Sequence-Labeling for Grammatical Error Correction

2020 ◽  
Vol 24 (4) ◽  
pp. 557-567
Author(s):  
Bo Wang ◽  
◽  
Kaoru Hirota ◽  
Chang Liu ◽  
Yaping Dai ◽  
...  
Keyword(s):  
Error Correction ◽  
Short Term Memory ◽  
Conditional Random Field ◽  
Short Term ◽  
Neural Machine Translation ◽  
Term Memory ◽  
Sequence Labeling ◽  
Grammatical Error ◽  
Small Set ◽  
Long Short Term Memory

An approach to N-best hypotheses re-ranking using a sequence-labeling model is applied to resolve the data deficiency problem in Grammatical Error Correction (GEC). Multiple candidate sentences are generated using a Neural Machine Translation (NMT) model; thereafter, these sentences are re-ranked via a stacked Transformer following a Bidirectional Long Short-Term Memory (BiLSTM) with Conditional Random Field (CRF). Correlations within the sentences are extracted using the sequence-labeling model based on the Transformer, which is particularly suitable for long sentences. Meanwhile, the knowledge from a large amount of unlabeled data is acquired through the pre-trained structure. Thus, completely revised sentences are adopted instead of partially modified sentences. Compared with conventional NMT, experiments on the NUCLE and FCE datasets demonstrate that the model improves the F0.5 score by 8.22% and 2.09%, respectively. As an advantage, the proposed re-ranking method has the advantage of only requires a small set of easily computed features that do not need linguistic inputs.

Semantic Segmentation of QRS Complex in Single Channel ECG with Bidirectional LSTM Networks

2020 ◽  
Vol 10 (3) ◽  
pp. 758-762 ◽  
Author(s):  
Lingfeng Liu ◽  
Baodan Bai ◽  
Xinrong Chen ◽  
Qin Xia
Keyword(s):  
Short Term Memory ◽  
Single Channel ◽  
Semantic Segmentation ◽  
Time Interval ◽  
Qrs Complex ◽  
Short Term ◽  
Attention Model ◽  
Interval Prediction ◽  
Long Short Term Memory ◽  
Electrocardiogram Ecg

In this paper, bidirectional Long Short-Term Memory (BiLSTM) networks are designed to realize the semantic segmentation of QRS complex in single channel electrocardiogram (ECG) for the tasks of R peak detection and heart rate estimation. Three types of seq2seq BiLSTM networks are introduced, including the densely connected BiLSTM with attention model, the BiLSTM U-Net, and the BiLSTM U-Net++. To alleviate the sparse problem of the QRS labels, symmetric label expansion is applied by extending the single R peak into a time interval of fixed length. Linear ensemble method is introduced that averages the outputs of different BiLSTM networks. The cross-validation results show significant increase of the accuracy and decrease of discontinuous gaps in the QRS interval prediction by the ensembling over singular neural networks.

scholarly journals Recognizing Continuous and Discontinuous Adverse Drug Reaction Mentions from Social Media Using LSTM-CRF

2018 ◽  
Vol 2018 ◽  
pp. 1-8 ◽  
Author(s):  
Buzhou Tang ◽  
Jianglu Hu ◽  
Xiaolong Wang ◽  
Qingcai Chen
Keyword(s):  
Neural Networks ◽  
Social Media ◽  
Medical Information ◽  
Conditional Random Fields ◽  
Short Term Memory ◽  
Knowledge Bases ◽  
Short Term ◽  
External Knowledge ◽  
Long Short Term Memory ◽  
First Time

Social media in medicine, where patients can express their personal treatment experiences by personal computers and mobile devices, usually contains plenty of useful medical information, such as adverse drug reactions (ADRs); mining this useful medical information from social media has attracted more and more attention from researchers. In this study, we propose a deep neural network (called LSTM-CRF) combining long short-term memory (LSTM) neural networks (a type of recurrent neural networks) and conditional random fields (CRFs) to recognize ADR mentions from social media in medicine and investigate the effects of three factors on ADR mention recognition. The three factors are as follows: (1) representation for continuous and discontinuous ADR mentions: two novel representations, that is, “BIOHD” and “Multilabel,” are compared; (2) subject of posts: each post has a subject (i.e., drug here); and (3) external knowledge bases. Experiments conducted on a benchmark corpus, that is, CADEC, show that LSTM-CRF achieves better F-score than CRF; “Multilabel” is better in representing continuous and discontinuous ADR mentions than “BIOHD”; both subjects of comments and external knowledge bases are individually beneficial to ADR mention recognition. To the best of our knowledge, this is the first time to investigate deep neural networks to mine continuous and discontinuous ADRs from social media.

Neural Machine Translation Using Sequence Modeling

2021 ◽  
Vol 9 (8) ◽  
pp. 2153-2169
Author(s):  
N Revathi
Keyword(s):  
Neural Network ◽  
Machine Translation ◽  
Language Processing ◽  
Recurrent Neural Network ◽  
Deep Neural Network ◽  
Short Term Memory ◽  
Short Term ◽  
Neural Machine Translation ◽  
Term Memory ◽  
Sequence Modeling

Abstract: Language is a main mode of communication, and translation is a critical tool for understanding information in a foreign language. Without the help of human translators, machine translation allows users to absorb unfamiliar linguistic material. The main goal of this project is to create a practical language translation from English to Hindi. Given its relevance and potential in the English-Hindi translation, machine translation is an efficient way to turn content into a new language without employing people. Among all available translation machines, Neural Machine Translation (NMT) is one of the most efficient ways. So, in this case, we're employing Sequence to Sequence Modeling, which includes the Recurrent Neural Network (RNN), Long and Short Term Memory (LSTM), and Encoder-Decoder methods. Deep Neural Network (DNN) comprehension and principles of deep learning, i.e. machine translation, are disclosed in the field of Natural Language Processing (NLP). In machine reclining techniques, DNN plays a crucial role. Keywords: Sequence to Sequence, Encoder-Decoder, Recurrent Neural Network, Long & Short term Memory, Deep Neural Network.

Comparative Analysis of Language Translation and Detection System Using Machine Learning

2021 ◽  
Vol 9 (8) ◽  
pp. 1200-1211
Author(s):  
Aishwarya R. Verma
Keyword(s):  
Machine Learning ◽  
Natural Language Processing ◽  
Machine Translation ◽  
Language Processing ◽  
Short Term Memory ◽  
Detection System ◽  
English Text ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory

Abstract: Words are the meaty component which can be expressed through speech, writing or signals. It is important that the actual message or meaning of the words sent must conveys the same meaning to the one receives. The evolution from manual language translator to the digital machine translation have helped us a lot for finding the exact meaning such that each word must give at least close to exact actual meaning. To make machine translator more human-friendly feeling, natural language processing (NLP) with machine learning (ML) can make the best combination. The main challenges in machine translated sentence can involve ambiguities, lexical divergence, syntactic, lexical mismatches, semantic issues, etc. which can be seen in grammar, spellings, punctuations, spaces, etc. After analysis on different algorithms, we have implemented a two different machine translator using two different Long Short-Term Memory (LSTM) approaches and performed the comparative study of the quality of the translated text based on their respective accuracy. We have used two different training approaches of encodingdecoding techniques using same datasets, which translates the source English text to the target Hindi text. To detect the text entered is English or Hindi language, we have used Sequential LSTM training model for which the analysis has been performed based on its accuracy. As the result, the first LSTM trained model is 84% accurate and the second LSTM trained model is 71% accurate in its translation from English to Hindi text, while the detection LSTM trained model is 78% accurate in detecting English text and 81% accurate in detecting Hindi text. This study has helped us to analyze the appropriate machine translation based on its accuracy. Keywords: Accuracy, Decoding, Machine Learning (ML), Detection System, Encoding, Long Short-Term Memory (LSTM), Machine Translation, Natural Language Processing (NLP), Sequential

Sign in / Sign up

Export Citation Format

Share Document