Connectivity, Dynamics, and Memory in Reservoir Computing with Binary and Analog Neurons

2010 ◽  
Vol 22 (5) ◽  
pp. 1272-1311 ◽  
Author(s):  
Lars Büsing ◽  
Benjamin Schrauwen ◽  
Robert Legenstein
Keyword(s):  
Analog Circuits ◽  
Memory Function ◽  
Mean Field ◽  
Network Connectivity ◽  
Network Models ◽  
Reservoir Computing ◽  
Computational Performance ◽  
Kernel Quality ◽  
The One ◽  
Long Timescales

Reservoir computing (RC) systems are powerful models for online computations on input sequences. They consist of a memoryless readout neuron that is trained on top of a randomly connected recurrent neural network. RC systems are commonly used in two flavors: with analog or binary (spiking) neurons in the recurrent circuits. Previous work indicated a fundamental difference in the behavior of these two implementations of the RC idea. The performance of an RC system built from binary neurons seems to depend strongly on the network connectivity structure. In networks of analog neurons, such clear dependency has not been observed. In this letter, we address this apparent dichotomy by investigating the influence of the network connectivity (parameterized by the neuron in-degree) on a family of network models that interpolates between analog and binary networks. Our analyses are based on a novel estimation of the Lyapunov exponent of the network dynamics with the help of branching process theory, rank measures that estimate the kernel quality and generalization capabilities of recurrent networks, and a novel mean field predictor for computational performance. These analyses reveal that the phase transition between ordered and chaotic network behavior of binary circuits qualitatively differs from the one in analog circuits, leading to differences in the integration of information over short and long timescales. This explains the decreased computational performance observed in binary circuits that are densely connected. The mean field predictor is also used to bound the memory function of recurrent circuits of binary neurons.

scholarly journals Step-like dependence of memory function on pulse width in spintronics reservoir computing

2020 ◽  
Vol 10 (1) ◽  
Author(s):  
Terufumi Yamaguchi ◽  
Nozomi Akashi ◽  
Kohei Nakajima ◽  
Hitoshi Kubota ◽  
Sumito Tsunegi ◽  
...  
Keyword(s):  
Vortex Core ◽  
Pulse Width ◽  
Short Term Memory ◽  
Memory Function ◽  
Magnetic Vortex ◽  
Physical Parameters ◽  
Dynamical Response ◽  
Reservoir Computing ◽  
Relaxation Dynamics ◽  
Computational Performance

Abstract Physical reservoir computing is a type of recurrent neural network that applies the dynamical response from physical systems to information processing. However, the relation between computation performance and physical parameters/phenomena still remains unclear. This study reports our progress regarding the role of current-dependent magnetic damping in the computational performance of reservoir computing. The current-dependent relaxation dynamics of a magnetic vortex core results in an asymmetric memory function with respect to binary inputs. A fast relaxation caused by a large input leads to a fast fading of the input memory, whereas a slow relaxation by a small input enables the reservoir to keep the input memory for a relatively long time. As a result, a step-like dependence is found for the short-term memory and parity-check capacities on the pulse width of input data, where the capacities remain at 1.5 for a certain range of the pulse width, and drop to 1.0 for a long pulse-width limit. Both analytical and numerical analyses clarify that the step-like behavior can be attributed to the current-dependent relaxation time of the vortex core to a limit-cycle state.

scholarly journals Modelling community structure and temporal spreading on complex networks

2021 ◽  
Vol 8 (1) ◽  
Author(s):  
Vesa Kuikka
Keyword(s):  
Community Structure ◽  
Complex Networks ◽  
Community Detection ◽  
Network Structure ◽  
Network Connectivity ◽  
Network Models ◽  
Building Blocks ◽  
Detection Algorithm ◽  
Research Area ◽  
Detection Methods

AbstractWe present methods for analysing hierarchical and overlapping community structure and spreading phenomena on complex networks. Different models can be developed for describing static connectivity or dynamical processes on a network topology. In this study, classical network connectivity and influence spreading models are used as examples for network models. Analysis of results is based on a probability matrix describing interactions between all pairs of nodes in the network. One popular research area has been detecting communities and their structure in complex networks. The community detection method of this study is based on optimising a quality function calculated from the probability matrix. The same method is proposed for detecting underlying groups of nodes that are building blocks of different sub-communities in the network structure. We present different quantitative measures for comparing and ranking solutions of the community detection algorithm. These measures describe properties of sub-communities: strength of a community, probability of formation and robustness of composition. The main contribution of this study is proposing a common methodology for analysing network structure and dynamics on complex networks. We illustrate the community detection methods with two small network topologies. In the case of network spreading models, time development of spreading in the network can be studied. Two different temporal spreading distributions demonstrate the methods with three real-world social networks of different sizes. The Poisson distribution describes a random response time and the e-mail forwarding distribution describes a process of receiving and forwarding messages.

scholarly journals Linear Response of General Observables in Spiking Neuronal Network Models

Entropy ◽  
2021 ◽  
Vol 23 (2) ◽  
pp. 155
Author(s):  
Bruno Cessac ◽  
Ignacio Ampuero ◽  
Rodrigo Cofré
Keyword(s):  
Linear Response ◽  
Network Connectivity ◽  
Network Models ◽  
General Context ◽  
Collective Effect ◽  
Neuronal Dynamics ◽  
The Past ◽  
Spatio Temporal ◽  
External Stimuli ◽  
Spike Dynamics

We establish a general linear response relation for spiking neuronal networks, based on chains with unbounded memory. This relation allow us to predict the influence of a weak amplitude time dependent external stimuli on spatio-temporal spike correlations, from the spontaneous statistics (without stimulus) in a general context where the memory in spike dynamics can extend arbitrarily far in the past. Using this approach, we show how the linear response is explicitly related to the collective effect of the stimuli, intrinsic neuronal dynamics, and network connectivity on spike train statistics. We illustrate our results with numerical simulations performed over a discrete time integrate and fire model.

scholarly journals Two-term expansion of the ground state one-body density matrix of a mean-field Bose gas

2021 ◽  
Vol 60 (3) ◽  
Author(s):  
Phan Thành Nam ◽  
Marcin Napiórkowski
Keyword(s):  
Density Matrix ◽  
Ground State ◽  
Mean Field ◽  
Interaction Strength ◽  
Bose Gas ◽  
Body Density ◽  
The Mean ◽  
The One ◽  
Mean Field Regime ◽  
Number Of Particles

AbstractWe consider the homogeneous Bose gas on a unit torus in the mean-field regime when the interaction strength is proportional to the inverse of the particle number. In the limit when the number of particles becomes large, we derive a two-term expansion of the one-body density matrix of the ground state. The proof is based on a cubic correction to Bogoliubov’s approximation of the ground state energy and the ground state.

scholarly journals Friendship paradox in growth networks: analytical and empirical analysis

2021 ◽  
Vol 6 (1) ◽  
Author(s):  
Sergei P. Sidorov ◽  
Sergei V. Mironov ◽  
Alexey A. Grigoriev
Keyword(s):  
Real World ◽  
Preferential Attachment ◽  
Empirical Studies ◽  
Mean Field ◽  
Network Models ◽  
Average Degree ◽  
Closure Model ◽  
Triadic Closure ◽  
Attachment Mechanism ◽  
Over Time

AbstractMany empirical studies have shown that in social, citation, collaboration, and other types of networks in real world, the degree of almost every node is less than the average degree of its neighbors. This imbalance is well known in sociology as the friendship paradox and states that your friends are more popular than you on average. If we introduce a value equal to the ratio of the average degree of the neighbors for a certain node to the degree of this node (which is called the ‘friendship index’, FI), then the FI value of more than 1 for most nodes indicates the presence of the friendship paradox in the network. In this paper, we study the behavior of the FI over time for networks generated by growth network models. We will focus our analysis on two models based on the use of the preferential attachment mechanism: the Barabási–Albert model and the triadic closure model. Using the mean-field approach, we obtain differential equations describing the dynamics of changes in the FI over time, and accordingly, after obtaining their solutions, we find the expected values of this index over iterations. The results show that the values of FI are decreasing over time for all nodes in both models. However, for networks constructed in accordance with the triadic closure model, this decrease occurs at a much slower rate than for the Barabási–Albert graphs. In addition, we analyze several real-world networks and show that their FI distributions follow a power law. We show that both the Barabási–Albert and the triadic closure networks exhibit the same behavior. However, for networks based on the triadic closure model, the distributions of FI are more heavy-tailed and, in this sense, are closer to the distributions for real networks.

scholarly journals A MEAN FIELD GAME ANALYSIS OF SIR DYNAMICS WITH VACCINATION

2020 ◽  
pp. 1-18
Author(s):  
Josu Doncel ◽  
Nicolas Gast ◽  
Bruno Gaujal
Keyword(s):  
Mean Field ◽  
Vaccination Strategy ◽  
Game Analysis ◽  
Total Cost ◽  
Mean Field Game ◽  
Vaccination Behavior ◽  
Switching Curve ◽  
The One ◽  
The Cost ◽  
Vaccination Period

We analyze a mean field game model of SIR dynamics (Susceptible, Infected, and Recovered) where players choose when to vaccinate. We show that this game admits a unique mean field equilibrium (MFE) that consists in vaccinating at a maximal rate until a given time and then not vaccinating. The vaccination strategy that minimizes the total cost has the same structure as the MFE. We prove that the vaccination period of the MFE is always smaller than the one minimizing the total cost. This implies that, to encourage optimal vaccination behavior, vaccination should always be subsidized. Finally, we provide numerical experiments to study the convergence of the equilibrium when the system is composed by a finite number of agents ( $N$ ) to the MFE. These experiments show that the convergence rate of the cost is $1/N$ and the convergence of the switching curve is monotone.

Properties of Relativistic Mean-Field Theory

2007 ◽  
Author(s):  
Kenneth G. Dyall ◽  
Knut Faegri
Keyword(s):  
Mean Field Theory ◽  
Mean Field ◽  
Field Method ◽  
Basis Sets ◽  
Electron Interaction ◽  
Relativistic Mean Field ◽  
Dirac Equations ◽  
Electrons And Positrons ◽  
The One ◽  
Atomic Structure Calculations

We have previously seen how the Dirac equation for one particle requires some rather special consideration and interpretation in order to arrive at a form that is able to treat electrons and positrons on an equal footing. These problems persist also when we go to systems with more than one electron. One might think that the extension to several electrons should not introduce dramatic changes. After all, we noted that even the one-electron problem must be viewed as a many-electron (and -positron) system in order to arrive at a consistent description. The problem with introducing more electrons is that electron–electron interactions that were previously small—for the one-electron case typically arising from vacuum polarization and self-interaction—now occur to the same order as the kinetic energy and the interaction with the potential. So while a perturbative approach such as QED can use the solutions of the one-electron Dirac equations as a very good starting approximation to a more accurate description of the full system, the same would not work for a system with more electrons because it would mean neglecting interactions of the same magnitude as the zeroth-order energy. For applications to quantum chemistry, the treatment of the entire electron–electron interaction as a perturbation would be hopelessly impractical, as it is even in manyelectron relativistic atomic structure calculations. The technique for dealing with this problem is well known from nonrelativistic calculations on many-electron systems. One-particle basis sets are developed by considering the behavior of the single electron in the mean field of all the other electrons, and while this neglects a smaller part of the interaction energy, the electron correlation, it provides a suitable starting point for further variational or perturbational treatments to recover more of the electron–electron interaction. It is only natural to pursue the same approach for the relativistic case. Thus one may proceed to construct a mean-field method that can be used as a basis for the perturbation theory of QED.

Neural Network Models

2016 ◽  
pp. 368-395
Author(s):  
Eliano Pessa
Keyword(s):  
Neural Network ◽  
Statistical Physics ◽  
Optimization Theory ◽  
Network Models ◽  
Cooperative Interaction ◽  
New Paradigm ◽  
Neural Network Models ◽  
Ann Models ◽  
Artificial Neural Network Ann ◽  
The One

The Artificial Neural Network (ANN) models gained a wide popularity owing to a number of claimed advantages such as biological plausibility, tolerance with respect to errors or noise in the input data, learning ability allowing an adaptability to environmental constraints. Notwithstanding the fact that most of these advantages are not typical only of ANNs, engineers, psychologists and neuroscientists made an extended use of ANN models in a large number of scientific investigations. In most cases, however, these models have been introduced in order to provide optimization tools more useful than the ones commonly used by traditional Optimization Theory. Unfortunately, just the successful performance of ANN models in optimization tasks produced a widespread neglect of the true – and important – objectives pursued by the first promoters of these models. These objectives can be shortly summarized by the manifesto of connectionist psychology, stating that mental processes are nothing but macroscopic phenomena, emergent from the cooperative interaction of a large number of microscopic knowledge units. This statement – wholly in line with the goal of statistical mechanics – can be readily extended to other processes, beyond the mental ones, including social, economic, and, in general, organizational ones. Therefore this chapter has been designed in order to answer a number of related questions, such as: are the ANN models able to grant for the occurrence of this sort of emergence? How can the occurrence of this emergence be empirically detected? How can the emergence produced by ANN models be controlled? In which sense the ANN emergence could offer a new paradigm for the explanation of macroscopic phenomena? Answering these questions induces to focus the chapter on less popular ANNs, such as the recurrent ones, while neglecting more popular models, such as perceptrons, and on less used units, such as spiking neurons, rather than on McCulloch-Pitts neurons. Moreover, the chapter must mention a number of strategies of emergence detection, useful for researchers performing computer simulations of ANN behaviours. Among these strategies it is possible to quote the reduction of ANN models to continuous models, such as the neural field models or the neural mass models, the recourse to the methods of Network Theory and the employment of techniques borrowed by Statistical Physics, like the one based on the Renormalization Group. Of course, owing to space (and mathematical expertise) requirements, most mathematical details of the proposed arguments are neglected, and, to gain more information, the reader is deferred to the quoted literature.

Sign in / Sign up

Export Citation Format

Share Document