scholarly journals Distillation of weighted automata from recurrent neural networks using a spectral approach

2021 ◽  
Author(s):  
Rémi Eyraud ◽  
Stéphane Ayache
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Spectral Approach ◽  
Weighted Automata

scholarly journals Weighted Automata Extraction from Recurrent Neural Networks via Regression on State Spaces

2020 ◽  
Vol 34 (04) ◽  
pp. 5306-5314
Author(s):  
Takamasa Okudono ◽  
Masaki Waga ◽  
Taro Sekiyama ◽  
Ichiro Hasuo
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Recurrent Neural Network ◽  
Recurrent Neural Networks ◽  
Learning Algorithm ◽  
Internal State ◽  
State Spaces ◽  
Regression Methods ◽  
Weighted Automata ◽  
Equivalence Queries

We present a method to extract a weighted finite automaton (WFA) from a recurrent neural network (RNN). Our method is based on the WFA learning algorithm by Balle and Mohri, which is in turn an extension of Angluin's classic L* algorithm. Our technical novelty is in the use of regression methods for the so-called equivalence queries, thus exploiting the internal state space of an RNN to prioritize counterexample candidates. This way we achieve a quantitative/weighted extension of the recent work by Weiss, Goldberg and Yahav that extracts DFAs. We experimentally evaluate the accuracy, expressivity and efficiency of the extracted WFAs.

Spike timing-dependent plasticity in sparse recurrent neural networks

2014 ◽  
Vol 1 ◽  
pp. 485-488
Author(s):  
Hideyuki Kato ◽  
Tohru Ikeguchi
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Spike Timing ◽  
Spike Timing Dependent Plasticity ◽  
Dependent Plasticity

Direct Adaptive Control of Process Systems Using Recurrent Neural Networks

1992 ◽  
Author(s):  
Sanjay Parthasarathy ◽  
Alexander G. Parlos ◽  
Amir F. Atiya
Keyword(s):  
Neural Networks ◽  
Adaptive Control ◽  
Recurrent Neural Networks ◽  
Process Systems ◽  
Direct Adaptive Control

L2 approximation properties of recurrent neural networks

1997 ◽  
Author(s):  
A. Ruiz ◽  
D.H. Owens ◽  
S. Townley
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Approximation Properties

Levenshtein Augmentation Improves Performance of SMILES Based Deep-Learning Synthesis Prediction

2020 ◽  
Author(s):  
Dean Sumner ◽  
Jiazhen He ◽  
Amol Thakkar ◽  
Ola Engkvist ◽  
Esben Jannik Bjerrum
Keyword(s):  
Neural Networks ◽  
Pattern Recognition ◽  
Deep Learning ◽  
Recurrent Neural Networks ◽  
Data Augmentation ◽  
State Of The Art ◽  
Sequence Similarity ◽  
Learning Models ◽  
Underlying Network

<p>SMILES randomization, a form of data augmentation, has previously been shown to increase the performance of deep learning models compared to non-augmented baselines. Here, we propose a novel data augmentation method we call “Levenshtein augmentation” which considers local SMILES sub-sequence similarity between reactants and their respective products when creating training pairs. The performance of Levenshtein augmentation was tested using two state of the art models - transformer and sequence-to-sequence based recurrent neural networks with attention. Levenshtein augmentation demonstrated an increase performance over non-augmented, and conventionally SMILES randomization augmented data when used for training of baseline models. Furthermore, Levenshtein augmentation seemingly results in what we define as <i>attentional gain </i>– an enhancement in the pattern recognition capabilities of the underlying network to molecular motifs.</p>

LatticeRnn: Recurrent Neural Networks Over Lattices

2016 ◽  
Author(s):  
Faisal Ladhak ◽  
Ankur Gandhe ◽  
Markus Dreyer ◽  
Lambert Mathias ◽  
Ariya Rastrow ◽  
...  
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks

Automatic Measurement of Voice Onset Time and Prevoicing Using Recurrent Neural Networks

2016 ◽  
Author(s):  
Yossi Adi ◽  
Joseph Keshet ◽  
Olga Dmitrieva ◽  
Matt Goldrick
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Voice Onset Time ◽  
Onset Time ◽  
Automatic Measurement
Sign in / Sign up

Export Citation Format

Share Document