TIME-DEPENDENT PROPERTIES FOR NEURAL NETWORKS WITH CONTINUOUS SPIN VARIABLES

1995 ◽  
Vol 09 (10) ◽  
pp. 1159-1169 ◽  
Author(s):  
VARSHA BANERJEE ◽  
SANJAY PURI
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Relaxation Time ◽  
Continuous Time ◽  
Storage Capacity ◽  
Time Dependent ◽  
Input Output ◽  
Computationally Efficient ◽  
Continuous Spin ◽  
Target Pattern

We present a continuous-time neural network model which consists of neurons with a continuous input-output relation. We use a computationally efficient discrete-time equivalent of this model to study its time-dependent properties. Detailed numerical results are presented for the behavior of the relaxation time to a target pattern as a function of the storage capacity of the network.

scholarly journals Control on the Manifolds of Mappings with a View to the Deep Learning

2021 ◽  
Author(s):  
Andrei Agrachev ◽  
Andrey Sarychev
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Control System ◽  
Continuous Time ◽  
Time Variable ◽  
Input Output ◽  
Training Set ◽  
Time Control ◽  
Interpolation Problems

AbstractDeep learning of the artificial neural networks (ANN) can be treated as a particular class of interpolation problems. The goal is to find a neural network whose input-output map approximates well the desired map on a finite or an infinite training set. Our idea consists of taking as an approximant the input-output map, which arises from a nonlinear continuous-time control system. In the limit such control system can be seen as a network with a continuum of layers, each one labelled by the time variable. The values of the controls at each instant of time are the parameters of the layer.

A Deep Neural Network Model for Learning Runtime Frequency Response Function Using Sensor Measurements

2021 ◽  
Author(s):  
Yongzhi Qu ◽  
Gregory W. Vogl ◽  
Zechao Wang
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Fourier Transform ◽  
Time Domain ◽  
Frequency Response Function ◽  
Frequency Component ◽  
Operating Conditions ◽  
Input Output ◽  
The Fourier Transform ◽  
The Time Domain

Abstract The frequency response function (FRF), defined as the ratio between the Fourier transform of the time-domain output and the Fourier transform of the time-domain input, is a common tool to analyze the relationships between inputs and outputs of a mechanical system. Learning the FRF for mechanical systems can facilitate system identification, condition-based health monitoring, and improve performance metrics, by providing an input-output model that describes the system dynamics. Existing FRF identification assumes there is a one-to-one mapping between each input frequency component and output frequency component. However, during dynamic operations, the FRF can present complex dependencies with frequency cross-correlations due to modulation effects, nonlinearities, and mechanical noise. Furthermore, existing FRFs assume linearity between input-output spectrums with varying mechanical loads, while in practice FRFs can depend on the operating conditions and show high nonlinearities. Outputs of existing neural networks are typically low-dimensional labels rather than real-time high-dimensional measurements. This paper proposes a vector regression method based on deep neural networks for the learning of runtime FRFs from measurement data under different operating conditions. More specifically, a neural network based on an encoder-decoder with a symmetric compression structure is proposed. The deep encoder-decoder network features simultaneous learning of the regression relationship between input and output embeddings, as well as a discriminative model for output spectrum classification under different operating conditions. The learning model is validated using experimental data from a high-pressure hydraulic test rig. The results show that the proposed model can learn the FRF between sensor measurements under different operating conditions with high accuracy and denoising capability. The learned FRF model provides an estimation for sensor measurements when a physical sensor is not feasible and can be used for operating condition recognition.

Autonomous Specialization in a Multi-Robot System using Evolving Neural Networks

2011 ◽  
pp. 941-955
Author(s):  
Masanori Goka ◽  
Kazuhiro Ohkura
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Artificial Neural Networks ◽  
Continuous Time ◽  
Collective Behaviour ◽  
Artificial Evolution ◽  
Autonomous Mobile Robot ◽  
Robot System ◽  
Artificial Neural ◽  
Multi Robot

Artificial evolution has been considered as a promising approach for coordinating the controller of an autonomous mobile robot. However, it is not yet established whether artificial evolution is also effective in generating collective behaviour in a multi-robot system (MRS). In this study, two types of evolving artificial neural networks are utilized in an MRS. The first is the evolving continuous time recurrent neural network, which is used in the most conventional method, and the second is the topology and weight evolving artificial neural networks, which is used in the noble method. Several computer simulations are conducted in order to examine how the artificial evolution can be used to coordinate the collective behaviour in an MRS.

Autonomous Specialization in a Multi-Robot System using Evolving Neural Networks

2010 ◽  
pp. 156-173
Author(s):  
Masanori Goka ◽  
Kazuhiro Ohkura
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Artificial Neural Networks ◽  
Continuous Time ◽  
Collective Behaviour ◽  
Artificial Evolution ◽  
Autonomous Mobile Robot ◽  
Robot System ◽  
Artificial Neural ◽  
Multi Robot

Artificial evolution has been considered as a promising approach for coordinating the controller of an autonomous mobile robot. However, it is not yet established whether artificial evolution is also effective in generating collective behaviour in a multi-robot system (MRS). In this study, two types of evolving artificial neural networks are utilized in an MRS. The first is the evolving continuous time recurrent neural network, which is used in the most conventional method, and the second is the topology and weight evolving artificial neural networks, which is used in the noble method. Several computer simulations are conducted in order to examine how the artificial evolution can be used to coordinate the collective behaviour in an MRS.

scholarly journals Discrete-time Cohen-Grossberg neural networks with transmission delays and impulses

2009 ◽  
Vol 43 (1) ◽  
pp. 145-161 ◽  
Author(s):  
Sannay Mohamad ◽  
Haydar Akça ◽  
Valéry Covachev
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Exponential Stability ◽  
Equilibrium Point ◽  
Discrete Time ◽  
Continuous Time ◽  
Global Exponential Stability ◽  
Exponential Convergence ◽  
Unique Equilibrium ◽  
Transmission Delays

Abstract A discrete-time analogue is formulated for an impulsive Cohen- -Grossberg neural network with transmission delay in a manner in which the global exponential stability characterisitics of a unique equilibrium point of the network are preserved. The formulation is based on extending the existing semidiscretization method that has been implemented for computer simulations of neural networks with linear stabilizing feedback terms. The exponential convergence in the p-norm of the analogue towards the unique equilibrium point is analysed by exploiting an appropriate Lyapunov sequence and properties of an M-matrix. The main result yields a Lyapunov exponent that involves the magnitude and frequency of the impulses. One can use the result for deriving the exponential stability of non-impulsive discrete-time neural networks, and also for simulating the exponential stability of impulsive and non-impulsive continuous-time networks.

Using Neural Networks to Model Conditional Multivariate Densities

1996 ◽  
Vol 8 (4) ◽  
pp. 843-854 ◽  
Author(s):  
Peter M. Williams
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Multivariate Data ◽  
Time Dependent ◽  
Local Error ◽  
Statistical Distributions ◽  
Conditional Distributions ◽  
Conditional Correlation ◽  
Univariate Case ◽  
Error Bars

Neural network outputs are interpreted as parameters of statistical distributions. This allows us to fit conditional distributions in which the parameters depend on the inputs to the network. We exploit this in modeling multivariate data, including the univariate case, in which there may be input-dependent (e.g., time-dependent) correlations between output components. This provides a novel way of modeling conditional correlation that extends existing techniques for determining input-dependent (local) error bars.

scholarly journals APPLICATION OF ARTIFICIAL NEURAL NETWORKS IN CONTROL ALGORITHMS OF ROBOTIC SYSTEMS

2020 ◽  
Vol 3 (156) ◽  
pp. 46-48
Author(s):  
D. Zubenko
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Stability Analysis ◽  
Discrete Time ◽  
Continuous Time ◽  
Control Design ◽  
Stability And Convergence ◽  
Switching Systems ◽  
Language Recognition ◽  
Artificial Neural

The problem of stability analysis for the general class of random pulsed and switching neural networks is presented in this paper, which is to be investigated both continuous dynamics and impulsive jumps of random disturbances. Two numerical examples are used to explain and highlight the effectiveness of the results developed.The purpose of this article is to provide a comprehensive overview of studies, including continuous time and discrete time models for solving various problems, and their application in motion planning and superfluous manipulator management, chaotic system tracking, or even population control in mathematical biological sciences. Considering the fact that real-time performance is in demand for time-varying problems in practice, analysis of the stability and convergence of various models with continuous time is considered in a unified form in detail. In the case of solving the problems of discrete time, procedures are summarized for how to discriminate a continuous model and methods for obtaining an accuracy decision. Due to its strong ability to extract features and autonomous learning, neural networks are rooted in many industries, for example. neuroscience, mathematics, informatics and engineering, transport, etc. Despite their widespread use in various fields, such as artificial intelligence, language recognition, and computer simulation, the issue of neural network stability analysis is the most primary and fundamental that has attracted intense attention in recent decades.and references therein. It is well known that pulse and switching systems are formulated by combining pulse systems with switching systems, which is a more complex model of nonlinear systems. With their increasing use in network management, power systems, and the like, impulse control theory and switching systems have been a hot topic of research for the past decade. The fruitful results of research on stability analysis and control design of pulse and switching systemssuch as input stability, time-limited, controllability and observation and feedback control design, etc. On the other hand, it is also noteworthy. Keywords: artificial neural network, electric transport, numerical algorithms, control reliability

scholarly journals Harnessing Deep Neural Networks to Solve Inverse Problems in Quantum Dynamics: Machine-Learned Predictions of Time-Dependent Optimal Control Fields

2020 ◽  
Author(s):  
Xian Wang ◽  
Anshuman Kumar ◽  
Christian Shelton ◽  
Bryan Wong
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Neural Networks ◽  
Optimal Control ◽  
Quantum Dynamics ◽  
Deep Neural Networks ◽  
Time Dependent ◽  
Learning Approaches ◽  
Quantum Dynamical ◽  
Quantum Dynamical Systems

Inverse problems continue to garner immense interest in the physical sciences, particularly in the context of controlling desired phenomena in non-equilibrium systems. In this work, we utilize a series of deep neural networks for predicting time-dependent optimal control fields, <i>E(t)</i>, that enable desired electronic transitions in reduced-dimensional quantum dynamical systems. To solve this inverse problem, we investigated two independent machine learning approaches: (1) a feedforward neural network for predicting the frequency and amplitude content of the power spectrum in the frequency domain (i.e., the Fourier transform of <i>E(t)</i>), and (2) a cross-correlation neural network approach for directly predicting <i>E(t)</i> in the time domain. Both of these machine learning methods give complementary approaches for probing the underlying quantum dynamics and also exhibit impressive performance in accurately predicting both the frequency and strength of the optimal control field. We provide detailed architectures and hyperparameters for these deep neural networks as well as performance metrics for each of our machine-learned models. From these results, we show that machine learning approaches, particularly deep neural networks, can be employed as a cost-effective statistical approach for designing electromagnetic fields to enable desired transitions in these quantum dynamical systems.

scholarly journals Stably Accelerating Stiff Quantitative Systems Pharmacology Models: Continuous-Time Echo State Networks as Implicit Machine Learning

2021 ◽  
Author(s):  
Ranjan Anantharaman ◽  
Anas Abdelrehim ◽  
Anand Jain ◽  
Avik Pal ◽  
Danny Sharp ◽  
...  
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Neural Networks ◽  
Continuous Time ◽  
Reaction System ◽  
Reaction Diffusion ◽  
Potential Method ◽  
Mechanistic Modeling ◽  
Systems Pharmacology ◽  
Quantitative Systems Pharmacology

AbstractQuantitative systems pharmacology (QsP) may need to change in order to accommodate machine learning (ML), but ML may need to change to work for QsP. Here we investigate the use of neural network surrogates of stiff QsP models. This technique reduces and accelerates QsP models by training ML approximations on simulations. We describe how common neural network methodologies, such as residual neural networks, recurrent neural networks, and physics/biologically-informed neural networks, are fundamentally related to explicit solvers of ordinary differential equations (ODEs). Similar to how explicit ODE solvers are unstable on stiff QsP models, we demonstrate how these ML architectures see similar training instabilities. To address this issue, we showcase methods from scientific machine learning (SciML) which combine techniques from mechanistic modeling with traditional deep learning. We describe the continuous-time echo state network (CTESN) as the implicit analogue of ML architectures and showcase its ability to accurately train and predict on these stiff models where other methods fail. We demonstrate the CTESN’s ability to surrogatize a production QsP model, a >1,000 ODE chemical reaction system from the SBML Biomodels repository, and a reaction-diffusion partial differential equation. We showcase the ability to accelerate QsP simulations by up to 56x against the optimized DifferentialEquations.jl solvers while achieving <5% relative error in all of the examples. This shows how incorporating the numerical properties of QsP methods into ML can improve the intersection, and thus presents a potential method for accelerating repeated calculations such as global sensitivity analysis and virtual populations.

Sign in / Sign up

Export Citation Format

Share Document