Two-line-element propagation improvement and uncertainty estimation using recurrent neural networks

Predictive monitoring—making predictions about future states and monitoring if the predicted states satisfy requirements—offers a promising paradigm in supporting the decision making of Cyber-Physical Systems (CPS). Existing works of predictive monitoring mostly focus on monitoring individual predictions rather than sequential predictions. We develop a novel approach for monitoring sequential predictions generated from Bayesian Recurrent Neural Networks (RNNs) that can capture the inherent uncertainty in CPS, drawing on insights from our study of real-world CPS datasets. We propose a new logic named Signal Temporal Logic with Uncertainty (STL-U) to monitor a flowpipe containing an infinite set of uncertain sequences predicted by Bayesian RNNs. We define STL-U strong and weak satisfaction semantics based on whether all or some sequences contained in a flowpipe satisfy the requirement. We also develop methods to compute the range of confidence levels under which a flowpipe is guaranteed to strongly (weakly) satisfy an STL-U formula. Furthermore, we develop novel criteria that leverage STL-U monitoring results to calibrate the uncertainty estimation in Bayesian RNNs. Finally, we evaluate the proposed approach via experiments with real-world CPS datasets and a simulated smart city case study, which show very encouraging results of STL-U based predictive monitoring approach outperforming baselines.

Download Full-text

Spike timing-dependent plasticity in sparse recurrent neural networks

IEICE Proceeding Series ◽

10.15248/proc.1.485 ◽

2014 ◽

Vol 1 ◽

pp. 485-488

Author(s):

Hideyuki Kato ◽

Tohru Ikeguchi

Keyword(s):

Neural Networks ◽

Recurrent Neural Networks ◽

Spike Timing ◽

Spike Timing Dependent Plasticity ◽

Dependent Plasticity

Download Full-text

Direct Adaptive Control of Process Systems Using Recurrent Neural Networks

1992 American Control Conference ◽

10.23919/acc.1992.4792020 ◽

1992 ◽

Author(s):

Sanjay Parthasarathy ◽

Alexander G. Parlos ◽

Amir F. Atiya

Keyword(s):

Neural Networks ◽

Adaptive Control ◽

Recurrent Neural Networks ◽

Process Systems ◽

Direct Adaptive Control

Download Full-text

L2 approximation properties of recurrent neural networks

1997 European Control Conference (ECC) ◽

10.23919/ecc.1997.7082360 ◽

1997 ◽

Cited By ~ 1

Author(s):

A. Ruiz ◽

D.H. Owens ◽

S. Townley

Keyword(s):

Neural Networks ◽

Recurrent Neural Networks ◽

Approximation Properties

Download Full-text

Levenshtein Augmentation Improves Performance of SMILES Based Deep-Learning Synthesis Prediction

10.26434/chemrxiv.12562121 ◽

2020 ◽

Author(s):

Dean Sumner ◽

Jiazhen He ◽

Amol Thakkar ◽

Ola Engkvist ◽

Esben Jannik Bjerrum

Keyword(s):

Neural Networks ◽

Pattern Recognition ◽

Deep Learning ◽

Recurrent Neural Networks ◽

Data Augmentation ◽

State Of The Art ◽

Sequence Similarity ◽

Learning Models ◽

Underlying Network

<p>SMILES randomization, a form of data augmentation, has previously been shown to increase the performance of deep learning models compared to non-augmented baselines. Here, we propose a novel data augmentation method we call “Levenshtein augmentation” which considers local SMILES sub-sequence similarity between reactants and their respective products when creating training pairs. The performance of Levenshtein augmentation was tested using two state of the art models - transformer and sequence-to-sequence based recurrent neural networks with attention. Levenshtein augmentation demonstrated an increase performance over non-augmented, and conventionally SMILES randomization augmented data when used for training of baseline models. Furthermore, Levenshtein augmentation seemingly results in what we define as <i>attentional gain </i>– an enhancement in the pattern recognition capabilities of the underlying network to molecular motifs.</p>

Download Full-text