Using Noisy Word-Level Labels to Train a Phoneme Recognizer based on Neural Networks by Expectation Maximization

2019 ◽  
Author(s):  
Chen Li ◽  
Bo Zhang ◽  
Shan Huang ◽  
Zhenhuan Liu
Keyword(s):  
Neural Networks ◽  
Expectation Maximization ◽  
Word Level

scholarly journals Explaining Character-Aware Neural Networks for Word-Level Prediction: Do They Discover Linguistic Rules?

2018 ◽  
Author(s):  
Fréderic Godin ◽  
Kris Demuynck ◽  
Joni Dambre ◽  
Wesley De Neve ◽  
Thomas Demeester
Keyword(s):  
Neural Networks ◽  
Word Level ◽  
Linguistic Rules

scholarly journals RTHN: A RNN-Transformer Hierarchical Network for Emotion Cause Extraction

2019 ◽  
Author(s):  
Rui Xia ◽  
Mengran Zhang ◽  
Zixiang Ding
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Relative Position ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
Hierarchical Network ◽  
Classification Problems ◽  
Rule Based ◽  
Machine Learning Methods ◽  
Word Level

The emotion cause extraction (ECE) task aims at discovering the potential causes behind a certain emotion expression in a document. Techniques including rule-based methods, traditional machine learning methods and deep neural networks have been proposed to solve this task. However, most of the previous work considered ECE as a set of independent clause classification problems and ignored the relations between multiple clauses in a document. In this work, we propose a joint emotion cause extraction framework, named RNN-Transformer Hierarchical Network (RTHN), to encode and classify multiple clauses synchronously. RTHN is composed of a lower word-level encoder based on RNNs to encode multiple words in each clause, and an upper clause-level encoder based on Transformer to learn the correlation between multiple clauses in a document. We furthermore propose ways to encode the relative position and global predication information into Transformer that can capture the causality between clauses and make RTHN more efficient. We finally achieve the best performance among 12 compared systems and improve the F1 score of the state-of-the-art from 72.69% to 76.77%.

scholarly journals Expectation pooling: an effective and interpretable pooling method for predicting DNA–protein binding

2019 ◽  
Author(s):  
Xiao Luo ◽  
Xinming Tu ◽  
Yang Ding ◽  
Ge Gao ◽  
Minghua Deng
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Protein Binding ◽  
Expectation Maximization ◽  
Probabilistic Models ◽  
Expectation Maximization Algorithm ◽  
Prediction Performance ◽  
Supplementary Information ◽  
Sequence Specificity ◽  
Future Studies

Abstract Motivation Convolutional neural networks (CNNs) have outperformed conventional methods in modeling the sequence specificity of DNA–protein binding. While previous studies have built a connection between CNNs and probabilistic models, simple models of CNNs cannot achieve sufficient accuracy on this problem. Recently, some methods of neural networks have increased performance using complex neural networks whose results cannot be directly interpreted. However, it is difficult to combine probabilistic models and CNNs effectively to improve DNA–protein binding predictions. Results In this article, we present a novel global pooling method: expectation pooling for predicting DNA–protein binding. Our pooling method stems naturally from the expectation maximization algorithm, and its benefits can be interpreted both statistically and via deep learning theory. Through experiments, we demonstrate that our pooling method improves the prediction performance DNA–protein binding. Our interpretable pooling method combines probabilistic ideas with global pooling by taking the expectations of inputs without increasing the number of parameters. We also analyze the hyperparameters in our method and propose optional structures to help fit different datasets. We explore how to effectively utilize these novel pooling methods and show that combining statistical methods with deep learning is highly beneficial, which is promising and meaningful for future studies in this field. Availability and implementation All code is public in https://github.com/gao-lab/ePooling. Supplementary information Supplementary data are available at Bioinformatics online.

scholarly journals Linguistic knowledge-based vocabularies for Neural Machine Translation

2020 ◽  
pp. 1-22
Author(s):  
Noe Casas ◽  
Marta R. Costa-jussà ◽  
José A. R. Fonollosa ◽  
Juan A. Alonso ◽  
Ramón Fanlo
Keyword(s):  
Neural Networks ◽  
Machine Translation ◽  
Word Formation ◽  
Linguistic Knowledge ◽  
Learning Approaches ◽  
Neural Machine Translation ◽  
Knowledge Based ◽  
Word Level ◽  
The Neural Network ◽  
Cross Lingual

Abstract Neural Networks applied to Machine Translation need a finite vocabulary to express textual information as a sequence of discrete tokens. The currently dominant subword vocabularies exploit statistically-discovered common parts of words to achieve the flexibility of character-based vocabularies without delegating the whole learning of word formation to the neural network. However, they trade this for the inability to apply word-level token associations, which limits their use in semantically-rich areas and prevents some transfer learning approaches e.g. cross-lingual pretrained embeddings, and reduces their interpretability. In this work, we propose new hybrid linguistically-grounded vocabulary definition strategies that keep both the advantages of subword vocabularies and the word-level associations, enabling neural networks to profit from the derived benefits. We test the proposed approaches in both morphologically rich and poor languages, showing that, for the former, the quality in the translation of out-of-domain texts is improved with respect to a strong subword baseline.

Sign in / Sign up

Export Citation Format

Share Document