Echo State Networks with Self-Normalizing Activations on the Hyper-Sphere

Among the various architectures of Recurrent Neural Networks, Echo State Networks (ESNs) emerged due to their simplified and inexpensive training procedure. These networks are known to be sensitive to the setting of hyper-parameters, which critically affect their behavior. Results show that their performance is usually maximized in a narrow region of hyper-parameter space called edge of criticality. Finding such a region requires searching in hyper-parameter space in a sensible way: hyper-parameter configurations marginally outside such a region might yield networks exhibiting fully developed chaos, hence producing unreliable computations. The performance gain due to optimizing hyper-parameters can be studied by considering the memory–nonlinearity trade-off, i.e., the fact that increasing the nonlinear behavior of the network degrades its ability to remember past inputs, and vice-versa. In this paper, we propose a model of ESNs that eliminates critical dependence on hyper-parameters, resulting in networks that provably cannot enter a chaotic regime and, at the same time, denotes nonlinear behavior in phase space characterized by a large memory of past inputs, comparable to the one of linear networks. Our contribution is supported by experiments corroborating our theoretical findings, showing that the proposed model displays dynamics that are rich-enough to approximate many common nonlinear systems used for benchmarking.

Statistics of Finite Scale Local Lyapunov Exponents in Fully Developed Homogeneous Isotropic Turbulence

The present work analyzes the statistics of finite scale local Lyapunov exponents of pairs of fluid particles trajectories in fully developed incompressible homogeneous isotropic turbulence. According to the hypothesis of fully developed chaos, this statistics is here analyzed assuming that the entropy associated with the fluid kinematic state is maximum. The distribution of the local Lyapunov exponents results in an unsymmetrical uniform function in a proper interval of variation. From this PDF, we determine the relationship between average and maximum Lyapunov exponents and the longitudinal velocity correlation function. This link, which in turn leads to the closure of von Kármán–Howarth and Corrsin equations, agrees with results of previous works, supporting the proposed PDF calculation, at least for the purposes of the energy cascade main effect estimation. Furthermore, through the property that the Lyapunov vectors tend to align the direction of the maximum growth rate of trajectories distance, we obtain the link between maximum and average Lyapunov exponents in line with the previous results. To validate the proposed theoretical results, we present different numerical simulations whose results justify the hypotheses of the present analysis.

Fast physical and pseudo random number generation based on a nonlinear optoelectronic oscillator

High speed random number generation (RNG) utilizing a nonlinear optoelectronic oscillator (OEO) is explored experimentally. It has been found that by simply adjusting either the injected optical power or the gain of the modulator driver, low complexity dynamics such as square wave, and more complex dynamics including fully developed chaos can be experimentally achieved. More importantly, physical RNG based on high-speed-oscilloscope measurements and pseudo RNG based on post-processing are implemented in this paper. The generated bit sequences pass all the standard statistical random tests, indicating that fast physical and pseudo RNG could be achieved based on the same OEO entropy source. Our results could provide further insight into the implementation of RNG based on chaotic optical systems.

AN EXPERIMENTAL STUDY OF NONLINEAR DYNAMICS AND CHAOS IN A NONAUTONOMOUS ELECTRONIC CIRCUIT

In this work we investigate experimentally the dynamics of a piecewise-linear nonautonomous electronic circuit proposed by Lacy [1996]. In particular, we examine the effect of a dc offset in the input signal. The circuit is first driven by a sine wave with a dc offset. When the amplitude and frequency of the sine wave are suitably chosen and the dc offset is varied, the output of the circuit is found to undergo a rich variety of bifurcations including chaos. A nonzero dc offset can also suppress a chaotic response which persists when the dc offset is zero. Furthermore, for certain amplitude and frequency of the sine wave input with no dc offset, an intermittent response is observed. The intermittent response may be altered by introducing a small amount of dc offset, resulting in a regular periodic motion or fully developed chaos. In addition, by varying the dc offset we have observed mode locking phenomena and a devil's staircase pattern. Finally the response of the circuit to a burst wave input is explored. We find that depending on the parameter values of the burst wave, the circuit may exhibit complex dynamics consisting of intermittency, chaos and mixed-mode oscillations.

CIRCULAR CHAOS GAME REPRESENTATION OF 1-D CHAOS AND ITS RELATION TO THE COMPLEX WEIERSTRASS FUNCTION

Barnsley's chaos game which is originally played on a triangular field is generalized to a circular field and is used to visualize one-dimensional (1-D) fully developed chaos. It is shown that when the 1-D map used is the shift map or its extension (the r-adic map) the game point is exactly represented by the complex Weierstrass function.

A Turning Point Analysis of the Ergodic Dynamics of Iterative Maps

The dynamics of one-dimensional iterative maps in the regime of fully developed chaos is studied in detail. Motivated by the observation of dynamical structures around the unstable fixed point we introduce the geometrical concept of a turning point which represents a local minimum or maximum of the trajectory. Following we investigate the highly organized and structured distribution of turning points. The turning point dynamics is discussed and the corresponding turning point map which possesses an appealing asymptotic scaling property is investigated. Strong correlations are shown to exist for the turning point trajectories which contain the information of the fixed points as well as the stability coefficients of the dynamical system. For the more specialized case of symmetric maps which possess a symmetric density we derive universal statistical properties of the corresponding turning point dynamics. Using the turning point concept we finally develop a method for the analysis of (one-dimensional) time series.

Strange Attractors in an Antiferromagnetic Ising Model

The three-site antiferromagnetic Ising model on Husimi tree is investigated in an external magnetic field. The full bifurcation diagram, including chaos, of the magnetization is exhibited. With the "thermodynamic formalism", we investigate the antiferromagnetic Ising model in the case of fully developed chaos and describe the chaotic properties of this statistical mechanical system via the invariants characterizing a strange attractor. It is shown that this system displays in the chaotic region a phase transition at a positive "temperature" whereas in a class of maps close to x→ 4x(1-x), the phase transitions occur at negative "temperatures". The Frobenius-Perron recursion equation is numerically solved and the density of the invariant measure is obtained.