Canonical neural networks perform active inference

AbstractThis work considers a class of canonical neural networks comprising rate coding models, wherein neural activity and plasticity minimise a common cost function—and plasticity is modulated with a certain delay. We show that such neural networks implicitly perform active inference and learning to minimise the risk associated with future outcomes. Mathematical analyses demonstrate that this biological optimisation can be cast as maximisation of model evidence, or equivalently minimisation of variational free energy, under the well-known form of a partially observed Markov decision process model. This equivalence indicates that the delayed modulation of Hebbian plasticity—accompanied with adaptation of firing thresholds—is a sufficient neuronal substrate to attain Bayes optimal inference and control. We corroborated this proposition using numerical analyses of maze tasks. This theory offers a universal characterisation of canonical neural networks in terms of Bayesian belief updating and provides insight into the neuronal mechanisms underlying planning and adaptive behavioural control.

Download Full-text

Canonical neural networks perform active inference

10.1101/2020.12.10.420547 ◽

2020 ◽

Author(s):

Takuya Isomura ◽

Hideaki Shimazaki ◽

Karl Friston

Keyword(s):

Neural Networks ◽

Process Model ◽

Active Inference ◽

Hebbian Plasticity ◽

Neuronal Mechanisms ◽

Partially Observed ◽

Markov Decision ◽

Rate Coding ◽

Mathematical Analyses ◽

Future Outcomes

AbstractThis work considers a class of canonical neural networks comprising rate coding models, wherein neural activity and plasticity minimise a common cost function—and plasticity is modulated with a certain delay. We show that such neural networks implicitly perform active inference and learning to minimise the risk associated with future outcomes. Mathematical analyses demonstrate that this biological optimisation can be cast as maximisation of model evidence, or equivalently minimisation of variational free energy, under the well-known form of a partially observed Markov decision process model. This equivalence indicates that the delayed modulation of Hebbian plasticity—accompanied with adaptation of firing thresholds—is a sufficient neuronal substrate to attain Bayes optimal control. We corroborated this proposition using numerical analyses of maze tasks. This theory offers a universal characterisation of canonical neural networks in terms of Bayesian belief updating and provides insight into the neuronal mechanisms underlying planning and adaptive behavioural control.

Download Full-text

Modeling and Control of the Lithographic Offset Color Printing Process Using Artificial Neural Networks

Journal of Engineering for Industry ◽

10.1115/1.2901942 ◽

1994 ◽

Vol 116 (2) ◽

pp. 274-276 ◽

Cited By ~ 5

Author(s):

Ming-Shong Lan ◽

P. Lin ◽

J. Bain

Keyword(s):

Neural Networks ◽

Artificial Neural Networks ◽

Process Model ◽

Spatial Relationship ◽

Printing Process ◽

Modeling And Control ◽

Color Printing ◽

Artificial Neural ◽

Flow Effects ◽

And Control

This paper investigates the use of artificial neural networks (ANNs) for modeling and control of the lithographic offset color printing process. The color controller consists of two ANNs; the controller network, which learns an inverse model of the process, takes a set of desired colors as input and generates a set of ink key settings, while the model network learns a forward model of the process through which the controller network can be adapted by using the error backpropagation method. We use three-layer networks with “local” connections between neurons of adjacent layers for the process model as well as for the controller; the architectures address the spatial relationship of multiple inking zones and consider the crosswise ink flow effects existing in the printing process.

Download Full-text

An Active Inference Approach to Dissecting Reasons for Non-Adherence to Antidepressants

10.1101/743542 ◽

2019 ◽

Author(s):

Ryan Smith ◽

Sahib Khalsa ◽

Martin Paulus

Keyword(s):

Decision Making ◽

Process Model ◽

Time Course ◽

Antidepressant Medication ◽

Influential Factors ◽

Active Inference ◽

Individual Level ◽

Markov Decision ◽

Decision Making Processes ◽

Partially Observable

AbstractBackgroundAntidepressant medication adherence is among the most important problems in health care worldwide. Interventions designed to increase adherence have largely failed, pointing towards a critical need to better understand the underlying decision-making processes that contribute to adherence. A computational decision-making model that integrates empirical data with a fundamental action selection principle could be pragmatically useful in 1) making individual level predictions about adherence, and 2) providing an explanatory framework that improves our understanding of non-adherence.MethodsHere we formulate a partially observable Markov decision process model based on the active inference framework that can simulate several processes that plausibly influence adherence decisions.ResultsUsing model simulations of the day-to-day decisions to take a prescribed selective serotonin reuptake inhibitor (SSRI), we show that several distinct parameters in the model can influence adherence decisions in predictable ways. These parameters include differences in policy depth (i.e., how far into the future one considers when deciding), decision uncertainty, beliefs about the predictability (stochasticity) of symptoms, beliefs about the magnitude and time course of symptom reductions and side effects, and the strength of medication-taking habits that one has acquired.ConclusionsClarifying these influential factors will be an important first step toward empirically determining which are contributing to non-adherence to antidepressants in individual patients. The model can also be seamlessly extended to simulate adherence to other medications (by incorporating the known symptom reduction and side effect trajectories of those medications), with the potential promise of identifying which medications may be best suited for different patients.

Download Full-text

Generalised free energy and active inference: can the future cause the past?

10.1101/304782 ◽

2018 ◽

Cited By ~ 6

Author(s):

Thomas Parr ◽

Karl J Friston

Keyword(s):

Free Energy ◽

Energy Functional ◽

Generative Model ◽

Free Energy Function ◽

Active Inference ◽

Future Expectations ◽

Energy Functionals ◽

The Future ◽

Markov Decision ◽

Future Outcomes

AbstractWe compare two free energy functionals for active inference under Markov decision processes. One of these is a functional of beliefs about states and policies, but a function of observations, while the second is a functional of beliefs about all three. In the former (expected free energy), prior beliefs about outcomes are not part of the generative model (because they are absorbed into the prior over policies). Conversely, in the second (generalised free energy); priors over outcomes become an explicit component of the generative model. When using the free energy function, which is blind to counterfactual (i.e., future) observations, we equip the generative model with a prior over policies that ensure preferred (i.e., priors over) outcomes are realised. In other words, selected policies minimise uncertainty about future outcomes by minimising the free energy expected in the future. When using the free energy functional – that effectively treats counterfactual observations as hidden states – we show that policies are inferred or selected that realise prior preferences by minimising the free energy of future expectations. Interestingly, the form of posterior beliefs about policies (and associated belief updating) turns out to be identical under both formulations, but the quantities used to compute them are not.

Download Full-text

Impulsivity and Active Inference

Journal of Cognitive Neuroscience ◽

10.1162/jocn_a_01352 ◽

2019 ◽

Vol 31 (2) ◽

pp. 202-220 ◽

Cited By ~ 7

Author(s):

M. Berk Mirza ◽

Rick A. Adams ◽

Thomas Parr ◽

Karl Friston

Keyword(s):

Markov Decision Process ◽

Decision Process ◽

Message Passing ◽

Process Model ◽

Impulsive Behavior ◽

Active Inference ◽

Prior Beliefs ◽

Optimal Behavior ◽

Markov Decision ◽

Change Beliefs

This paper characterizes impulsive behavior using a patch-leaving paradigm and active inference—a framework for describing Bayes optimal behavior. This paradigm comprises different environments (patches) with limited resources that decline over time at different rates. The challenge is to decide when to leave the current patch for another to maximize reward. We chose this task because it offers an operational characterization of impulsive behavior, namely, maximizing proximal reward at the expense of future gain. We use a Markov decision process formulation of active inference to simulate behavioral and electrophysiological responses under different models and prior beliefs. Our main finding is that there are at least three distinct causes of impulsive behavior, which we demonstrate by manipulating three different components of the Markov decision process model. These components comprise (i) the depth of planning, (ii) the capacity to maintain and process information, and (iii) the perceived value of immediate (relative to delayed) rewards. We show how these manipulations change beliefs and subsequent choices through variational message passing. Furthermore, we appeal to the process theories associated with this message passing to simulate neuronal correlates. In future work, we will use this scheme to identify the prior beliefs that underlie different sorts of impulsive behavior—and ask whether different causes of impulsivity can be inferred from the electrophysiological correlates of choice behavior.

Download Full-text

An Integerated Estimation and Control Scheme for Piece-Wise Linear Systems using Neural Networks

10.23919/acc.1990.4791269 ◽

1990 ◽

Author(s):

A. M. Elramsisi ◽

M. A. Zohdy ◽

M. S. Fadali

Keyword(s):

Neural Networks ◽

Linear Systems ◽

Control Scheme ◽

Estimation And Control ◽

And Control

Download Full-text

Modelling and control of an alkaline water electrolysis process

International Journal of Computational Physics Series ◽

10.29167/a1i2p9-14 ◽

2018 ◽

Vol 1 (2) ◽

pp. 9-14

Author(s):

Marisol Cervantes-Bobadilla ◽

Ricardo Fabricio Escobar Jiménez ◽

José Francisco Gómez Aguilar ◽

Tomas Emmanuel Higareda Pliego ◽

Alberto Armando Alvares Gallegos

Keyword(s):

Process Model ◽

Water Electrolysis ◽

Alkaline Water Electrolysis ◽

Alkaline Water ◽

Electrolysis Process ◽

Linear Dynamic ◽

Energy Current ◽

Identification Technique ◽

Nonlinear Static ◽

And Control

In this research, an alkaline water electrolysis process is modelled. The electrochemical electrolysis is carried out in an electrolyzer composed of 12 series-connected steel cells with a solution 30% wt of potassium hydroxide. The electrolysis process model was developed using a nonlinear identification technique based on the Hammerstein structure. This structure consists of a nonlinear static block and a linear dynamic block. In this work, the nonlinear static function is modelled by a polynomial approximation equation, and the linear dynamic is modelled using the ARX structure. To control the current feed to the electrolyzer an unconstraint predictive controller was implemented, once the unconstrained MPC was simulated, some restrictions are proposed to design a constrained MPC (CMPC). The CMPC aim is to reduce the electrolyzer's energy consumption (power supply current). Simulation results showed the advantages of using the CMPC since the energy (current) overshoots are avoided.

Download Full-text

The Frustration Regulation Journal: A Reflective Framework for Educators

Relay Journal ◽

10.37237/relay/020205 ◽

2019 ◽

Author(s):

Sam Morris

Keyword(s):

Emotion Regulation ◽

Process Model ◽

Negative Impact ◽

Language Education ◽

Short Article ◽

Language Classroom ◽

As Stress ◽

And Control ◽

Important System ◽

Behavioural Strategies

Teachers and advisors involved in the emotional business of language education feel frustrated from time to time, and if such emotions are not managed healthily, they may lead to negative outcomes such as stress and burnout. One important system for taking control of frustration is emotion regulation, the cognitive and behavioural strategies through which individuals manage their emotions. In this short article, I define frustration and discuss its negative impact on the language classroom. I then introduce a structured reflective journaling tool, built upon Gross’s Process model of emotion regulation (Gross, 2014, 2015) which may help teachers and advisors develop greater awareness and control over experiences of frustration.

Download Full-text