scholarly journals Canonical neural networks perform active inference

2020 ◽  
Author(s):  
Takuya Isomura ◽  
Hideaki Shimazaki ◽  
Karl Friston
Keyword(s):  
Neural Networks ◽  
Process Model ◽  
Active Inference ◽  
Hebbian Plasticity ◽  
Neuronal Mechanisms ◽  
Partially Observed ◽  
Markov Decision ◽  
Rate Coding ◽  
Mathematical Analyses ◽  
Future Outcomes

AbstractThis work considers a class of canonical neural networks comprising rate coding models, wherein neural activity and plasticity minimise a common cost function—and plasticity is modulated with a certain delay. We show that such neural networks implicitly perform active inference and learning to minimise the risk associated with future outcomes. Mathematical analyses demonstrate that this biological optimisation can be cast as maximisation of model evidence, or equivalently minimisation of variational free energy, under the well-known form of a partially observed Markov decision process model. This equivalence indicates that the delayed modulation of Hebbian plasticity—accompanied with adaptation of firing thresholds—is a sufficient neuronal substrate to attain Bayes optimal control. We corroborated this proposition using numerical analyses of maze tasks. This theory offers a universal characterisation of canonical neural networks in terms of Bayesian belief updating and provides insight into the neuronal mechanisms underlying planning and adaptive behavioural control.

scholarly journals Canonical neural networks perform active inference

2022 ◽  
Vol 5 (1) ◽  
Author(s):  
Takuya Isomura ◽  
Hideaki Shimazaki ◽  
Karl J. Friston
Keyword(s):  
Neural Networks ◽  
Process Model ◽  
Active Inference ◽  
Hebbian Plasticity ◽  
Neuronal Mechanisms ◽  
Markov Decision ◽  
Rate Coding ◽  
Mathematical Analyses ◽  
Future Outcomes ◽  
And Control

AbstractThis work considers a class of canonical neural networks comprising rate coding models, wherein neural activity and plasticity minimise a common cost function—and plasticity is modulated with a certain delay. We show that such neural networks implicitly perform active inference and learning to minimise the risk associated with future outcomes. Mathematical analyses demonstrate that this biological optimisation can be cast as maximisation of model evidence, or equivalently minimisation of variational free energy, under the well-known form of a partially observed Markov decision process model. This equivalence indicates that the delayed modulation of Hebbian plasticity—accompanied with adaptation of firing thresholds—is a sufficient neuronal substrate to attain Bayes optimal inference and control. We corroborated this proposition using numerical analyses of maze tasks. This theory offers a universal characterisation of canonical neural networks in terms of Bayesian belief updating and provides insight into the neuronal mechanisms underlying planning and adaptive behavioural control.

scholarly journals An Active Inference Approach to Dissecting Reasons for Non-Adherence to Antidepressants

2019 ◽  
Author(s):  
Ryan Smith ◽  
Sahib Khalsa ◽  
Martin Paulus
Keyword(s):  
Decision Making ◽  
Process Model ◽  
Time Course ◽  
Antidepressant Medication ◽  
Influential Factors ◽  
Active Inference ◽  
Individual Level ◽  
Markov Decision ◽  
Decision Making Processes ◽  
Partially Observable

AbstractBackgroundAntidepressant medication adherence is among the most important problems in health care worldwide. Interventions designed to increase adherence have largely failed, pointing towards a critical need to better understand the underlying decision-making processes that contribute to adherence. A computational decision-making model that integrates empirical data with a fundamental action selection principle could be pragmatically useful in 1) making individual level predictions about adherence, and 2) providing an explanatory framework that improves our understanding of non-adherence.MethodsHere we formulate a partially observable Markov decision process model based on the active inference framework that can simulate several processes that plausibly influence adherence decisions.ResultsUsing model simulations of the day-to-day decisions to take a prescribed selective serotonin reuptake inhibitor (SSRI), we show that several distinct parameters in the model can influence adherence decisions in predictable ways. These parameters include differences in policy depth (i.e., how far into the future one considers when deciding), decision uncertainty, beliefs about the predictability (stochasticity) of symptoms, beliefs about the magnitude and time course of symptom reductions and side effects, and the strength of medication-taking habits that one has acquired.ConclusionsClarifying these influential factors will be an important first step toward empirically determining which are contributing to non-adherence to antidepressants in individual patients. The model can also be seamlessly extended to simulate adherence to other medications (by incorporating the known symptom reduction and side effect trajectories of those medications), with the potential promise of identifying which medications may be best suited for different patients.

scholarly journals Generalised free energy and active inference: can the future cause the past?

2018 ◽  
Author(s):  
Thomas Parr ◽  
Karl J Friston
Keyword(s):  
Free Energy ◽  
Energy Functional ◽  
Generative Model ◽  
Free Energy Function ◽  
Active Inference ◽  
Future Expectations ◽  
Energy Functionals ◽  
The Future ◽  
Markov Decision ◽  
Future Outcomes

AbstractWe compare two free energy functionals for active inference under Markov decision processes. One of these is a functional of beliefs about states and policies, but a function of observations, while the second is a functional of beliefs about all three. In the former (expected free energy), prior beliefs about outcomes are not part of the generative model (because they are absorbed into the prior over policies). Conversely, in the second (generalised free energy); priors over outcomes become an explicit component of the generative model. When using the free energy function, which is blind to counterfactual (i.e., future) observations, we equip the generative model with a prior over policies that ensure preferred (i.e., priors over) outcomes are realised. In other words, selected policies minimise uncertainty about future outcomes by minimising the free energy expected in the future. When using the free energy functional – that effectively treats counterfactual observations as hidden states – we show that policies are inferred or selected that realise prior preferences by minimising the free energy of future expectations. Interestingly, the form of posterior beliefs about policies (and associated belief updating) turns out to be identical under both formulations, but the quantities used to compute them are not.

scholarly journals Impulsivity and Active Inference

2019 ◽  
Vol 31 (2) ◽  
pp. 202-220 ◽  
Author(s):  
M. Berk Mirza ◽  
Rick A. Adams ◽  
Thomas Parr ◽  
Karl Friston
Keyword(s):  
Markov Decision Process ◽  
Decision Process ◽  
Message Passing ◽  
Process Model ◽  
Impulsive Behavior ◽  
Active Inference ◽  
Prior Beliefs ◽  
Optimal Behavior ◽  
Markov Decision ◽  
Change Beliefs

This paper characterizes impulsive behavior using a patch-leaving paradigm and active inference—a framework for describing Bayes optimal behavior. This paradigm comprises different environments (patches) with limited resources that decline over time at different rates. The challenge is to decide when to leave the current patch for another to maximize reward. We chose this task because it offers an operational characterization of impulsive behavior, namely, maximizing proximal reward at the expense of future gain. We use a Markov decision process formulation of active inference to simulate behavioral and electrophysiological responses under different models and prior beliefs. Our main finding is that there are at least three distinct causes of impulsive behavior, which we demonstrate by manipulating three different components of the Markov decision process model. These components comprise (i) the depth of planning, (ii) the capacity to maintain and process information, and (iii) the perceived value of immediate (relative to delayed) rewards. We show how these manipulations change beliefs and subsequent choices through variational message passing. Furthermore, we appeal to the process theories associated with this message passing to simulate neuronal correlates. In future work, we will use this scheme to identify the prior beliefs that underlie different sorts of impulsive behavior—and ask whether different causes of impulsivity can be inferred from the electrophysiological correlates of choice behavior.

scholarly journals On the complexity of partially observed Markov decision processes

1996 ◽  
Vol 157 (2) ◽  
pp. 161-183 ◽  
Author(s):  
Dima Burago ◽  
Michel de Rougemont ◽  
Anatol Slissenko
Keyword(s):  
Markov Decision Processes ◽  
Decision Processes ◽  
Partially Observed ◽  
Markov Decision

scholarly journals Hebbian plasticity in parallel synaptic pathways: A circuit mechanism for systems memory consolidation

2021 ◽  
Vol 17 (12) ◽  
pp. e1009681
Author(s):  
Michiel W. H. Remme ◽  
Urs Bergmann ◽  
Denis Alevi ◽  
Susanne Schreiber ◽  
Henning Sprekeler ◽  
...  
Keyword(s):  
Memory Consolidation ◽  
Computational Models ◽  
Hippocampal Formation ◽  
Spatial Scales ◽  
Single Cells ◽  
Brain Regions ◽  
Quantitative Agreement ◽  
Long Term Memory ◽  
Hebbian Plasticity ◽  
Mathematical Analyses

Systems memory consolidation involves the transfer of memories across brain regions and the transformation of memory content. For example, declarative memories that transiently depend on the hippocampal formation are transformed into long-term memory traces in neocortical networks, and procedural memories are transformed within cortico-striatal networks. These consolidation processes are thought to rely on replay and repetition of recently acquired memories, but the cellular and network mechanisms that mediate the changes of memories are poorly understood. Here, we suggest that systems memory consolidation could arise from Hebbian plasticity in networks with parallel synaptic pathways—two ubiquitous features of neural circuits in the brain. We explore this hypothesis in the context of hippocampus-dependent memories. Using computational models and mathematical analyses, we illustrate how memories are transferred across circuits and discuss why their representations could change. The analyses suggest that Hebbian plasticity mediates consolidation by transferring a linear approximation of a previously acquired memory into a parallel pathway. Our modelling results are further in quantitative agreement with lesion studies in rodents. Moreover, a hierarchical iteration of the mechanism yields power-law forgetting—as observed in psychophysical studies in humans. The predicted circuit mechanism thus bridges spatial scales from single cells to cortical areas and time scales from milliseconds to years.

Sign in / Sign up

Export Citation Format

Share Document