Efficient Embedded Decoding of Neural Network Language Models in a Machine Translation System

Neural Network Language Models (NNLMs) are a successful approach to Natural Language Processing tasks, such as Machine Translation. We introduce in this work a Statistical Machine Translation (SMT) system which fully integrates NNLMs in the decoding stage, breaking the traditional approach based on [Formula: see text]-best list rescoring. The neural net models (both language models (LMs) and translation models) are fully coupled in the decoding stage, allowing to more strongly influence the translation quality. Computational issues were solved by using a novel idea based on memorization and smoothing of the softmax constants to avoid their computation, which introduces a trade-off between LM quality and computational cost. These ideas were studied in a machine translation task with different combinations of neural networks used both as translation models and as target LMs, comparing phrase-based and [Formula: see text]-gram-based systems, showing that the integrated approach seems more promising for [Formula: see text]-gram-based systems, even with nonfull-quality NNLMs.

Download Full-text

English-Dogri Translation System using MOSES

Circulation in Computer Science ◽

10.22632/ccs-2016-251-25 ◽

2016 ◽

Vol 1 (1) ◽

pp. 45-49

Author(s):

Avinash Singh ◽

Asmeet Kour ◽

Shubhnandan S. Jamwal

Keyword(s):

Natural Language Processing ◽

Machine Translation ◽

Language Processing ◽

Statistical Machine Translation ◽

Translation System ◽

Parallel Corpus ◽

English System ◽

Machine Translation System ◽

Translation Machine ◽

Language Pair

The objective behind this paper is to analyze the English-Dogri parallel corpus translation. Machine translation is the translation from one language into another language. Machine translation is the biggest application of the Natural Language Processing (NLP). Moses is statistical machine translation system allow to train translation models for any language pair. We have developed translation system using Statistical based approach which helps in translating English to Dogri and vice versa. The parallel corpus consists of 98,973 sentences. The system gives accuracy of 80% in translating English to Dogri and the system gives accuracy of 87% in translating Dogri to English system.

Download Full-text

Factorised Hidden Layer Based Domain Adaptation for Recurrent Neural Network Language Models

2018 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) ◽

10.23919/apsipa.2018.8659473 ◽

2018 ◽

Cited By ~ 1

Author(s):

Michael Hentschel ◽

Marc Delcroix ◽

Atsunori Ogawa ◽

Tomoharu Iwata ◽

Tomohiro Nakatani

Keyword(s):

Neural Network ◽

Recurrent Neural Network ◽

Domain Adaptation ◽

Language Models ◽

Hidden Layer ◽

Network Language

Download Full-text

Efficient Transfer Learning for Neural Network Language Models

2018 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM) ◽

10.1109/asonam.2018.8508304 ◽

2018 ◽

Author(s):

Jacek Skryzalin ◽

Hamilton Link ◽

Jeremy Wendt ◽

Richard Field ◽

Samuel N. Richter

Keyword(s):

Neural Network ◽

Transfer Learning ◽

Language Models ◽

Network Language ◽

Efficient Transfer

Download Full-text

Limited-Memory BFGS Optimization of Recurrent Neural Network Language Models for Speech Recognition

2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) ◽

10.1109/icassp.2018.8461550 ◽

2018 ◽

Cited By ~ 4

Author(s):

Xunying Liu ◽

Shansong Liu ◽

Jinze Sha ◽

Jianwei Yu ◽

Zhiyuan Xu ◽

...

Keyword(s):

Neural Network ◽

Speech Recognition ◽

Recurrent Neural Network ◽

Language Models ◽

Limited Memory ◽

Network Language

Download Full-text

Discriminative method for recurrent neural network language models

2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) ◽

10.1109/icassp.2015.7179000 ◽

2015 ◽

Cited By ~ 4

Author(s):

Yuuki Tachioka ◽

Shinji Watanabe

Keyword(s):

Neural Network ◽

Recurrent Neural Network ◽

Language Models ◽

Discriminative Method ◽

Network Language

Download Full-text

Tagging terms in text

Terminology ◽

10.1075/term.21010.rig ◽

2022 ◽

Author(s):

Ayla Rigouts Terryn ◽

Véronique Hoste ◽

Els Lefever

Keyword(s):

Neural Network ◽

Machine Learning ◽

Language Processing ◽

Conditional Random Fields ◽

Traditional Approach ◽

Learning Approaches ◽

Term Extraction ◽

The Neural Network ◽

Predicted Probability ◽

Automatic Term Extraction

Abstract As with many tasks in natural language processing, automatic term extraction (ATE) is increasingly approached as a machine learning problem. So far, most machine learning approaches to ATE broadly follow the traditional hybrid methodology, by first extracting a list of unique candidate terms, and classifying these candidates based on the predicted probability that they are valid terms. However, with the rise of neural networks and word embeddings, the next development in ATE might be towards sequential approaches, i.e., classifying each occurrence of each token within its original context. To test the validity of such approaches for ATE, two sequential methodologies were developed, evaluated, and compared: one feature-based conditional random fields classifier and one embedding-based recurrent neural network. An additional comparison was added with a machine learning interpretation of the traditional approach. All systems were trained and evaluated on identical data in multiple languages and domains to identify their respective strengths and weaknesses. The sequential methodologies were proven to be valid approaches to ATE, and the neural network even outperformed the more traditional approach. Interestingly, a combination of multiple approaches can outperform all of them separately, showing new ways to push the state-of-the-art in ATE.

Download Full-text