scholarly journals Hidden Hypergraphs, Error-Correcting Codes, and Critical Learning in Hopfield Networks

Entropy ◽  
2021 ◽  
Vol 23 (11) ◽  
pp. 1494
Author(s):  
Christopher Hillar ◽  
Tenzin Chan ◽  
Rachel Taubman ◽  
David Rolnick
Keyword(s):  
Finite Automata ◽  
Minimum Energy ◽  
Error Correcting Codes ◽  
Critical Ratio ◽  
Computer Design ◽  
Hopfield Networks ◽  
Coding Schemes ◽  
Training Samples ◽  
Fixed Point Attractor ◽  
Point Attractor

In 1943, McCulloch and Pitts introduced a discrete recurrent neural network as a model for computation in brains. The work inspired breakthroughs such as the first computer design and the theory of finite automata. We focus on learning in Hopfield networks, a special case with symmetric weights and fixed-point attractor dynamics. Specifically, we explore minimum energy flow (MEF) as a scalable convex objective for determining network parameters. We catalog various properties of MEF, such as biological plausibility, and them compare to classical approaches in the theory of learning. Trained Hopfield networks can perform unsupervised clustering and define novel error-correcting coding schemes. They also efficiently find hidden structures (cliques) in graph theory. We extend this known connection from graphs to hypergraphs and discover n-node networks with robust storage of 2Ω(n1−ϵ) memories for any ϵ>0. In the case of graphs, we also determine a critical ratio of training samples at which networks generalize completely.

scholarly journals Equivalent Electronic Circuit of a System of Oscillators Connected With Periodically Variable Stiffness

2022 ◽  
Author(s):  
Soumyajit Seth ◽  
Grzegorz Kudra ◽  
Krzysztof Witkowski ◽  
Jan Awrejcewicz
Keyword(s):  
Mechanical System ◽  
Electronic Circuit ◽  
Mechanical Design ◽  
Variable Stiffness ◽  
State Variables ◽  
Fixed Point Attractor ◽  
Point Attractor ◽  
Magnetic Resistance ◽  
Nonlinear Behaviors ◽  
Resistance Forces

In this paper, we have shown the electronic circuit equivalence of a mechanical system consists of two oscillators coupled with each other. The mechanical design has the effects of the magnetic, resistance forces and the spring constant of the system is periodically varying. We have shown that the system’s state variables, such as the displacements and the velocities, under the effects of different forces, lead to some nonlinear behaviors, like a transition from the fixed point attractor to the chaotic attractor through the periodic and quasi-periodic attractors. We have constructed the equivalent electronic circuit of this mechanical system and have verified the numerically obtained behaviors using the electronic circuit.

The First Computational Theory of Cognition

2020 ◽  
pp. 107-127
Author(s):  
Gualtiero Piccinini
Keyword(s):  
Neural Networks ◽  
Body Problem ◽  
Digital Circuits ◽  
Finite Automata ◽  
Cognitive Activity ◽  
Logic Gates ◽  
Computer Design ◽  
Computational Theory ◽  
Modern Computer ◽  
The Mind

McCulloch and Pitts were the first to use and Alan Turing’s notion of computation to understand neural, and thus cognitive, activity. McCulloch and Pitts’s contributions included (i) a formalism whose refinement and generalization led to the notion of finite automata, which is an important formalism in computability theory, (ii) a technique that inspired the notion of logic design, which is a fundamental part of modern computer design, (iii) the first use of computation to address the mind–body problem, and (iv) the first modern computational theory of cognition, which posits that neurons are equivalent to logic gates and neural networks are digital circuits.

EFFECT OF DISSIPATION ON THE HAMILTONIAN CHAOS IN COUPLED OSCILLATOR SYSTEMS

2002 ◽  
Vol 12 (04) ◽  
pp. 859-867 ◽  
Author(s):  
V. SHEEJA ◽  
M. SABIR
Keyword(s):  
Degrees Of Freedom ◽  
Chaotic Behavior ◽  
Period Doubling ◽  
Dissipation Function ◽  
Hamiltonian Chaos ◽  
Fixed Point Attractor ◽  
Point Attractor ◽  
Route To Chaos ◽  
Period Doubling Bifurcations ◽  
External Driving

We study the effect of linear dissipative forces on the chaotic behavior of coupled quartic oscillators with two degrees of freedom. The effect of quadratic Rayleigh dissipation functions, one with diagonal coefficients only and the other with nondiagonal coefficients as well are studied. It is found that the effect of Rayleigh Dissipation function with diagonal coefficients is to suppress chaos in the system and to lead the system to its equilibrium state. However, with a dissipation function with nondiagonal elements, other types of behaviors — including fixed point attractor, periodic attractors and even chaotic attractors — are possible even when there is no external driving. In such a system the route to chaos is through period-doubling bifurcations. This result contradicts the view that linear dissipation always causes decay of oscillations in oscillator models.

scholarly journals Permitted and Forbidden Sets in Symmetric Threshold-Linear Networks

2003 ◽  
Vol 15 (3) ◽  
pp. 621-638 ◽  
Author(s):  
Richard H. R. Hahnloser ◽  
H. Sebastian Seung ◽  
Jean-Jacques Slotine
Keyword(s):  
Fixed Points ◽  
Positive Semidefinite ◽  
Volterra Equations ◽  
Stable Steady State ◽  
Long Term Memory ◽  
Linear Networks ◽  
Fixed Point Attractor ◽  
Point Attractor ◽  
High Level ◽  
Biological Computation

The richness and complexity of recurrent cortical circuits is an inexhaustible source of inspiration for thinking about high-level biological computation. In past theoretical studies, constraints on the synaptic connection patterns of threshold-linear networks were found that guaranteed bounded network dynamics, convergence to attractive fixed points, and multistability, all fundamental aspects of cortical information processing. However, these conditions were only sufficient, and it remained unclear which were the minimal (necessary) conditions for convergence and multistability. We show that symmetric threshold-linear networks converge to a set of attractive fixed points if and only if the network matrix is copositive. Furthermore, the set of attractive fixed points is nonconnected (the network is multiattractive) if and only if the network matrix is not positive semidefinite. There are permitted sets of neurons that can be coactive at a stable steady state and forbidden sets that cannot. Permitted sets are clustered in the sense that subsets of permitted sets are permitted and supersets of forbidden sets are forbidden. By viewing permitted sets as memories stored in the synaptic connections, we provide a formulation of long-term memory that is more general than the traditional perspective of fixed-point attractor networks. There is a close correspondence between threshold-linear networks and networks defined by the generalized Lotka-Volterra equations.

scholarly journals Solitonic Fixed Point Attractors in the Complex Ginzburg–Landau Equation for Associative Memories

Symmetry ◽  
2019 ◽  
Vol 12 (1) ◽  
pp. 24
Author(s):  
Alexey N. Pyrkov ◽  
Tim Byrnes ◽  
Valentin V. Cherny
Keyword(s):  
Fixed Point ◽  
Landau Equation ◽  
Photon Absorption ◽  
Ginzburg Landau ◽  
Two Photon Absorption ◽  
Two Photon ◽  
Solitonic Solution ◽  
Fixed Point Attractor ◽  
Point Attractor ◽  
Gain Loss

It was recently shown that the nonlinear Schrodinger equation with a simplified dissipative perturbation features a zero-velocity solitonic solution of non-zero amplitude which can be used in analogy to attractors of Hopfield’s associative memory. In this work, we consider a more complex dissipative perturbation adding the effect of two-photon absorption and the quintic gain/loss effects that yields the complex Ginzburg–Landau equation (CGLE). We construct a perturbation theory for the CGLE with a small dissipative perturbation, define the behavior of the solitonic solutions with parameters of the system and compare the solution with numerical simulations of the CGLE. We show, in a similar way to the nonlinear Schrodinger equation with a simplified dissipation term, a zero-velocity solitonic solution of non-zero amplitude appears as an attractor for the CGLE. In this case, the amplitude and velocity of the solitonic fixed point attractor does not depend on the quintic gain/loss effects. Furthermore, the effect of two-photon absorption leads to an increase in the strength of the solitonic fixed point attractor.

Sequential Triangle Strip Generator Based on Hopfield Networks

2009 ◽  
Vol 21 (2) ◽  
pp. 583-617
Author(s):  
Jiří Šíma ◽  
Radim Lněnička
Keyword(s):  
Simulated Annealing ◽  
Linear Time ◽  
Minimum Energy ◽  
Global Optimum ◽  
Neural Nets ◽  
Hopfield Network ◽  
Surface Model ◽  
Hopfield Networks ◽  
Initial State ◽  
Triangulated Surface

The important task of generating the minimum number of sequential triangle strips (tristrips) for a given triangulated surface model is motivated by applications in computer graphics. This hard combinatorial optimization problem is reduced to the minimum energy problem in Hopfield nets by a linear-size construction. In particular, the classes of equivalent optimal stripifications are mapped one to one to the minimum energy states reached by a Hopfield network during sequential computation starting at the zero initial state. Thus, the underlying Hopfield network powered by simulated annealing (i.e., Boltzmann machine), which is implemented in the program HTGEN, can be used for computing the semioptimal stripifications. Practical experiments confirm that one can obtain much better results using HTGEN than by a leading conventional stripification program FTSG (a reference stripification method not based on neural nets), although the running time of simulated annealing grows rapidly near the global optimum. Nevertheless, HTGEN exhibits empirical linear time complexity when the parameters of simulated annealing (i.e., the initial temperature and the stopping criterion) are fixed and thus provides the semioptimal offline solutions, even for huge models of hundreds of thousands of triangles, within a reasonable time.

The Dynamics of Self-Evaluation

2002 ◽  
Vol 6 (4) ◽  
pp. 370-379 ◽  
Author(s):  
Robin R. Vallacher ◽  
Andrzej Nowak ◽  
Michael Froehlich ◽  
Matthew Rockloff
Keyword(s):  
Dynamic Properties ◽  
Self Esteem ◽  
Self Concept ◽  
Self Reflection ◽  
Self Evaluation ◽  
Computer Mouse ◽  
Fixed Point Attractor ◽  
Point Attractor ◽  
The Moment ◽  
Rate Of Movement

We conceptualize self-concept as a self-organizing dynamical system and investigate implications of this perspective for the dynamic and fixed-point attractor tendencies of self-evaluative thought. Participants who differed in self-concept valence (self-esteem) and coherence (self-certainty, self-stability) engaged in verbal self-reflection for several minutes, then used a computer mouse to track the moment-to-moment self-evaluation expressed in their recorded narrative. Prior to self-reflection, participants recalled positive or negative past actions (positive vs. negative priming), or did not recall past actions (no priming). Priming affected overall self-evaluation (i.e., greatest positivity under positive priming), but only early in the narrative. The effects of self-concept, in contrast, became stronger over time. Self-esteem affected overall self-evaluation, whereas self-certainty and self-stability affected the dynamic properties (e.g., rate of movement between self-evaluative states) and attractor tendencies of self-evaluation. Discussion centers on the interplay between structure and dynamics in the self-system.

Fixed-point attractor

2012 ◽  
Vol 6 (6) ◽  
pp. 342-342
Author(s):  
Rachel Won
Keyword(s):  
Fixed Point ◽  
Fixed Point Attractor ◽  
Point Attractor

scholarly journals Improving MIMO-OFDM decision-directed channel estimation by utilizing error-correcting codes

2009 ◽  
Vol 7 ◽  
pp. 83-88
Author(s):  
P. Beinschob ◽  
M. Lieberei ◽  
U. Zölzer
Keyword(s):  
Channel Estimation ◽  
Turbo Codes ◽  
Adaptive Filters ◽  
Multiple Input Multiple Output ◽  
Error Correcting Codes ◽  
Mimo Channel ◽  
Time Frequency ◽  
Coding Schemes ◽  
Channel Estimate ◽  
Input Multiple Output

Abstract. In this paper a decision-directed Multiple-Input Multiple-Output (MIMO) channel tracking algorithm is enhanced to raise the channel estimate accuracy. While DDCE is prone to error propagation the enhancement employs channel decoding in the tracking process. Therefore, a quantized block of symbols is checked on consistency via the channel decoder, possibly corrected and then used. This yields a more robust tracking of the channel in terms of bit error rate and improves the channel estimate under certain conditions. Equalization is performed to prove the feasibility of the obtained channel estimate. Therefore a combined signal consisting of data and pilot symbols is sent. Adaptive filters are applied to exploit correlations in time, frequency and spatial domain. By using good error-correcting coding schemes like Turbo Codes or Low Density Parity Check (LDPC) codes, adequate channel estimates can be acquired even at low signal to noise ratios (SNR). The proposed algorithm among two others is applied for channel estimation and equalization and results are compared.

scholarly journals A mathematical model relates intracellular TLR4 oscillations to sepsis progression

2017 ◽  
Author(s):  
Razvan C. Stan ◽  
Francisco G. Soriano ◽  
Maristela M. de Camargo
Keyword(s):  
Sepsis Syndrome ◽  
Signaling Cascades ◽  
Hospital Data ◽  
Cell Organelles ◽  
Toll Like Receptor 4 ◽  
Downstream Signaling ◽  
Fixed Point Attractor ◽  
Point Attractor ◽  
Bacterial Stimulation ◽  
Concentration Changes

AbstractOscillations drive many biological processes and their modulation is determinant for various pathologies. In sepsis syndrome, Toll-like receptor 4 (TLR4) is a key sensor for signaling the presence of Gram-negative bacteria. Its expression and activity, along with its intracellular trafficking rates shift the equilibrium between the pro‐ and anti-inflammatory downstream signaling cascades, leading to either the physiological resolution of the bacterial stimulation or to sepsis. We hypothesize that the initial tlr4 expression in patients diagnosed with sepsis and TLR4 dynamic concentration changes on the cell membrane or intracellularly, dictates how the sepsis syndrome is initiated. Using a set of three differential equations, we defined the TLR4 flux between relevant cell organelles. We obtained three different regions in the phase space: 1. a limit-cycle describing unstimulated physiological oscillations, 2. a fixed-point attractor resulting from moderate LPS stimulation that is resolved and 3. a double-attractor resulting from sustained LPS stimulation that leads to sepsis. We tested the models against hospital data of sepsis patients and we correctly evaluate the clinical outcome of these patients.

Sign in / Sign up

Export Citation Format

Share Document