Determining the Information Flow of Dynamical Systems from Continuous Probability Distributions

1997 ◽  
Vol 78 (12) ◽  
pp. 2345-2348 ◽  
Author(s):  
Gustavo Deco ◽  
Christian Schittenkopf ◽  
Bernd Schürmann
2022 ◽  
pp. 1-24
Author(s):  
Kohei Ichikawa ◽  
Asaki Kataoka

Abstract Animals make efficient probabilistic inferences based on uncertain and noisy information from the outside environment. It is known that probabilistic population codes, which have been proposed as a neural basis for encoding probability distributions, allow general neural networks (NNs) to perform near-optimal point estimation. However, the mechanism of sampling-based probabilistic inference has not been clarified. In this study, we trained two types of artificial NNs, feedforward NN (FFNN) and recurrent NN (RNN), to perform sampling-based probabilistic inference. Then we analyzed and compared their mechanisms of sampling. We found that sampling in RNN was performed by a mechanism that efficiently uses the properties of dynamical systems, unlike FFNN. In addition, we found that sampling in RNNs acted as an inductive bias, enabling a more accurate estimation than in maximum a posteriori estimation. These results provide important arguments for discussing the relationship between dynamical systems and information processing in NNs.


2012 ◽  
Vol 9 (1) ◽  
pp. 78-79 ◽  
Author(s):  
Jakob Heinzle ◽  
Carsten Allefeld ◽  
John-Dylan Haynes

Author(s):  
Ross P. Anderson ◽  
Maurizio Porfiri

Information-theoretical notions of causality provide a model-free approach to identification of the magnitude and direction of influence among sub-components of a stochastic dynamical system. In addition to detecting causal influences, any effective test should also report the level of statistical significance of the finding. Here, we focus on transfer entropy, which has recently been considered for causality detection in a variety of fields based on statistical significance tests that are valid only in the asymptotic regime, that is, with enormous amounts of data. In the interest of applications with limited available data, we develop a non-asymptotic theory for the probability distribution of the difference between the empirically-estimated transfer entropy and the true transfer entropy. Based on this result, we additionally demonstrate an approach for statistical hypothesis testing for directed information flow in dynamical systems with a given number of observed time steps.


Author(s):  
P. Platzer ◽  
P. Yiou ◽  
P. Naveau ◽  
J-F. Filipot ◽  
M. Thiébaut ◽  
...  

AbstractSome properties of chaotic dynamical systems can be probed through features of recurrences, also called analogs. In practice, analogs are nearest neighbours of the state of a system, taken from a large database called the catalog. Analogs have been used in many atmospheric applications including forecasts, downscaling, predictability estimation, and attribution of extreme events. The distances of the analogs to the target state usually condition the performances of analog applications. These distances can be viewed as random variables, and their probability distributions can be related to the catalog size and properties of the system at stake. A few studies have focused on the first moments of return time statistics for the closest analog, fixing an objective of maximum distance from this analog to the target state. However, for practical use and to reduce estimation variance, applications usually require not just one, but many analogs. In this paper, we evaluate from a theoretical standpoint and with numerical experiments the probability distributions of the K shortest analog-to-target distances. We show that dimensionality plays a role on the size of the catalog needed to find good analogs, and also on the relative means and variances of the K closest analogs. Our results are based on recently developed tools from dynamical systems theory. These findings are illustrated with numerical simulations of well-known chaotic dynamical systems and on 10m-wind reanalysis data in north-west France. Practical applications of our derivations are shown for forecasts of an idealized chaotic dynamical system and for objective-based dimension reduction using the 10m-wind reanalysis data.


2011 ◽  
Vol 196 (1) ◽  
pp. 182-189 ◽  
Author(s):  
Linda Sommerlade ◽  
Florian Amtage ◽  
Olga Lapp ◽  
Bernhard Hellwig ◽  
Carl Hermann Lücking ◽  
...  

Sign in / Sign up

Export Citation Format

Share Document