Unification of MAP Estimation and Marginal Inference in Recurrent Neural Networks

2018 ◽  
Vol 29 (11) ◽  
pp. 5761-5766 ◽  
Author(s):  
Zhaofei Yu ◽  
Feng Chen ◽  
Fei Deng
2018 ◽  
Vol 16 (04) ◽  
pp. 1850009
Author(s):  
Chi-Kan Chen

Background: The inference of genetic regulatory networks (GRNs) provides insight into the cellular responses to signals. A class of recurrent neural networks (RNNs) capturing the dynamics of GRN has been used as a basis for inferring small-scale GRNs from gene expression time series. The Bayesian framework facilitates incorporating the hypothesis of GRN into the model estimation to improve the accuracy of GRN inference. Results: We present new methods for inferring small-scale GRNs based on RNNs. The weights of wires of RNN represent the strengths of gene-to-gene regulatory interactions. We use a class of automatic relevance determination (ARD) priors to enforce the sparsity in the maximum a posteriori (MAP) estimates of wire weights of RNN. A particle swarm optimization (PSO) is integrated as an optimization engine into the MAP estimation process. Likely networks of genes generated based on estimated wire weights are combined using the majority rule to determine a final estimated GRN. As an alternative, a class of [Formula: see text]-norm ([Formula: see text]) priors is used for attaining the sparse MAP estimates of wire weights of RNN. We also infer the GRN using the maximum likelihood (ML) estimates of wire weights of RNN. The RNN-based GRN inference algorithms, ARD-RNN, [Formula: see text]-RNN, and ML-RNN are tested on simulated and experimental E. coli and yeast time series containing 6–11 genes and 7–19 data points. Published GRN inference algorithms based on regressions and mutual information networks are performed on the benchmark datasets to compare performances. Conclusion: ARD and [Formula: see text]-norm priors are used for the estimation of wire weights of RNN. Results of GRN inference experiments show that ARD-RNN, [Formula: see text]-RNN have similar best accuracies on the simulated time series. The ARD-RNN is more accurate than [Formula: see text]-RNN, ML-RNN, and mostly more accurate than the reference algorithms on the experimental time series. The effectiveness of ARD-RNN for inferring small-scale GRNs using gene expression time series of limited length is empirically verified.


2020 ◽  
Author(s):  
Dean Sumner ◽  
Jiazhen He ◽  
Amol Thakkar ◽  
Ola Engkvist ◽  
Esben Jannik Bjerrum

<p>SMILES randomization, a form of data augmentation, has previously been shown to increase the performance of deep learning models compared to non-augmented baselines. Here, we propose a novel data augmentation method we call “Levenshtein augmentation” which considers local SMILES sub-sequence similarity between reactants and their respective products when creating training pairs. The performance of Levenshtein augmentation was tested using two state of the art models - transformer and sequence-to-sequence based recurrent neural networks with attention. Levenshtein augmentation demonstrated an increase performance over non-augmented, and conventionally SMILES randomization augmented data when used for training of baseline models. Furthermore, Levenshtein augmentation seemingly results in what we define as <i>attentional gain </i>– an enhancement in the pattern recognition capabilities of the underlying network to molecular motifs.</p>


Author(s):  
Faisal Ladhak ◽  
Ankur Gandhe ◽  
Markus Dreyer ◽  
Lambert Mathias ◽  
Ariya Rastrow ◽  
...  

Sign in / Sign up

Export Citation Format

Share Document