scholarly journals Recent advances in the application of predictive coding and active inference models within clinical neuroscience

2020 ◽  
Vol 75 (1) ◽  
pp. 3-13
Author(s):  
Ryan Smith ◽  
Paul Badcock ◽  
Karl J. Friston
Keyword(s):  
Predictive Coding ◽  
Active Inference ◽  
Clinical Neuroscience ◽  
Inference Models ◽  
Recent Advances

scholarly journals Implementing Predictive Processing and Active Inference: Preliminary Steps and Results

2019 ◽  
Author(s):  
Beren Millidge
Keyword(s):  
Predictive Coding ◽  
Predictive Processing ◽  
Active Inference ◽  
Inference Models ◽  
Perceptual Model ◽  
Challenging Tasks ◽  
Generalised Coordinates

Initial and preliminary implementations of predictive processing and active inference models are presented. These include the baseline hierarchical predictive coding models of (Friston 2003, 2005), and dynamical predictive coding models using generalised coordinates (Friston 2008, 2010, Buckley 2017). Additionally, we re-implement and experiment with the active inference thermostat presented in (Buckley 2017) and also implement an active inference agent with a hierarchical predictive coding perceptual model on the more challenging cart-pole task from OpanAI gym. We discuss the initial performance, capabilities, and limitations of these models in their preliminary stages and consider how they might be further scaled up to tackle more challenging tasks.

scholarly journals Active inference models do not contradict folk psychology

2021 ◽  
Author(s):  
Ryan Smith ◽  
maxwell ramstead ◽  
Alex Kiefer
Keyword(s):  
Decision Making ◽  
Folk Psychology ◽  
Predictive Coding ◽  
Decision Processes ◽  
Unified Theory ◽  
Active Inference ◽  
Energy Principle ◽  
Inference Models ◽  
Free Energy Principle ◽  
Dark Room

Active inference offers a unified theory of perception, learning, and decision-making at computational and neural levels of description. In this article, we address the worry that active inference may be in tension with folk psychology because it does not include terms for desires (or other conative constructs) at the mathematical level of description. To resolve this concern, we first provide a brief review of the historical progression from predictive coding to active inference, enabling us to distinguish between active inference formulations of motor control (which need not have desires under folk psychology) and active inference formulations of decision processes (which do have desires within folk psychology). We then show that, despite a superficial tension when viewed at the mathematical level, the active inference formalism contains terms that are readily identifiable as encoding both the objects of desire and the strength of desire at the psychological level. We demonstrate this with simple simulations of an active inference agent motivated to leave a dark room for different reasons. Despite their consistency, we further show how active inference may increase the granularity of folk-psychological descriptions by highlighting distinctions between drives to seek information vs. reward – and how it may also offer more precise, quantitative folk-psychological predictions. Finally, we consider how the implicitly conative components of active inference may have partial analogues (i.e., “as if” desires) in other systems describable by the broader free energy principle to which it conforms.

scholarly journals Variability in Action Selection Relates to Striatal Dopamine 2/3 Receptor Availability in Humans: A PET Neuroimaging Study Using Reinforcement Learning and Active Inference Models

2020 ◽  
Vol 30 (6) ◽  
pp. 3573-3589 ◽  
Author(s):  
Rick A Adams ◽  
Michael Moutoussis ◽  
Matthew M Nour ◽  
Tarik Dahoun ◽  
Declan Lewis ◽  
...  
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Model Comparison ◽  
Behavioral Model ◽  
Action Selection ◽  
Striatal Dopamine ◽  
Active Inference ◽  
Inference Models ◽  
Positron Emission ◽  
Dopamine Signaling

Abstract Choosing actions that result in advantageous outcomes is a fundamental function of nervous systems. All computational decision-making models contain a mechanism that controls the variability of (or confidence in) action selection, but its neural implementation is unclear—especially in humans. We investigated this mechanism using two influential decision-making frameworks: active inference (AI) and reinforcement learning (RL). In AI, the precision (inverse variance) of beliefs about policies controls action selection variability—similar to decision ‘noise’ parameters in RL—and is thought to be encoded by striatal dopamine signaling. We tested this hypothesis by administering a ‘go/no-go’ task to 75 healthy participants, and measuring striatal dopamine 2/3 receptor (D2/3R) availability in a subset (n = 25) using [11C]-(+)-PHNO positron emission tomography. In behavioral model comparison, RL performed best across the whole group but AI performed best in participants performing above chance levels. Limbic striatal D2/3R availability had linear relationships with AI policy precision (P = 0.029) as well as with RL irreducible decision ‘noise’ (P = 0.020), and this relationship with D2/3R availability was confirmed with a ‘decision stochasticity’ factor that aggregated across both models (P = 0.0006). These findings are consistent with occupancy of inhibitory striatal D2/3Rs decreasing the variability of action selection in humans.

scholarly journals ‘Through others we become ourselves’: The dialectics of predictive coding and active inference

2019 ◽  
Author(s):  
Dimitris Bolis ◽  
Leonhard Schilbach
Keyword(s):  
Free Energy ◽  
Social Interaction ◽  
Predictive Coding ◽  
Real Life ◽  
Other Minds ◽  
Active Inference ◽  
Energy Principle ◽  
Free Energy Principle ◽  
Multiscale Dynamics ◽  
Cultural Processes

Thinking Through Other Minds (TTOM) creatively situates the free energy principle within real-life cultural processes, thereby enriching both sociocultural theories and Bayesian accounts of cognition. Here, shifting the attention from thinking to becoming, we suggest complementing such an account by focusing on the empirical, computational and conceptual investigation of the multiscale dynamics of social interaction.

scholarly journals The cutaneous borders of interoception: Active and social inference of pain and pleasure on the skin

2018 ◽  
Author(s):  
Mariana von Mohr ◽  
Aikaterini Fotopoulou
Keyword(s):  
Social Factors ◽  
Predictive Coding ◽  
Active Inference ◽  
Top Down ◽  
Bottom Up ◽  
Social Inference ◽  
Brain Levels ◽  
The Brain

Pain and pleasant touch have been recently classified as interoceptive modalities. This reclassification lies at the heart of long-standing debates questioning whether these modalities should be defined as sensations on their basis of neurophysiological specificity at the periphery or as homeostatic emotions on the basis of top-down convergence and modulation at the spinal and brain levels. Here, we outline the literature on the peripheral and central neurophysiology of pain and pleasant touch. We next recast this literature within a recent Bayesian predictive coding framework, namely active inference. This recasting puts forward a unifying model of bottom-up and top-down determinants of pain and pleasant touch and the role of social factors in modulating the salience of peripheral signals reaching the brain.

scholarly journals Combining Active Inference and Hierarchical Predictive Coding: A Tutorial Introduction and Case Study

2019 ◽  
Author(s):  
Beren Millidge
Keyword(s):  
Free Energy ◽  
Reinforcement Learning ◽  
Predictive Coding ◽  
Action Selection ◽  
Proof Of Concept ◽  
Active Inference ◽  
Energy Principle ◽  
Free Energy Principle ◽  
Perceptual Model

This paper combines the active inference formulation of action (Friston, 2009) with hierarchical predictive coding models (Friston, 2003) to provide a proof-of-concept implementation of an active inference agent able to solve a common reinforcement learning baseline -- the cart-pole environment in OpenAI gym. It demonstrates empirically that predictive coding and active inference approaches can be successfully scaled up to tasks more challenging than the mountain car (Friston 2009, 2012). We show that hierarchical predictive coding models can be learned from scratch during the task, and can successfully drive action selection via active inference. To our knowledge, it is the first implemented active inference agent to combine active inference with a hierarchical predictive coding perceptual model. We also provide a tutorial walk-through of the free-energy principle, hierarchical predictive coding, and active inference, including an in-depth derivation of our agent.

scholarly journals Neural Underpinning of Japanese Particle Processing in Non-native Speakers

2021 ◽  
Author(s):  
Chise Kasai ◽  
Motofumi Sumiya ◽  
Takahiko Koike ◽  
Takaaki Yoshimoto ◽  
Hideki Maki ◽  
...  
Keyword(s):  
Working Memory ◽  
Native Speakers ◽  
Inferior Frontal Gyrus ◽  
Predictive Coding ◽  
Primary Source ◽  
Verbal Working Memory ◽  
Forward Model ◽  
Neural Representation ◽  
Middle Temporal Gyrus ◽  
Active Inference

Abstract Grammar acquisition by non-native learners (L2) is typically less successful and may produce fundamentally different grammatical systems than that by native speakers (L1). The neural representation of grammatical processing between L1 and L2 speakers remains controversial. We hypothesized that working memory is the primary source of L1/L2 differences, and operationalized working memory is an active inference within the predictive coding account, which models grammatical processes as higher-level neuronal representations of cortical hierarchies, generating predictions (forward model) of lower-level representations. A functional MRI study was conducted with L1 Japanese speakers and highly proficient Japanese learners requiring oral production of grammatically correct Japanese particles. Selecting proper particles requires forward model-dependent active inference as their functions are highly context-dependent. As a control, participants read out a visually designated mora indicated by underlining. Particle selection by L1/L2 groups commonly activated the bilateral inferior frontal gyrus/insula, pre-supplementary motor area, left caudate, middle temporal gyrus, and right cerebellum, which constituted the core linguistic production system. In contrast, the left inferior frontal sulcus, known as the neural substrate of verbal working memory, showed more prominent activation in L2 than in L1. Thus, the active inference process causes L1/L2 differences even in highly proficient L2 learners.

scholarly journals Goal-Directed Planning for Habituated Agents by Active Inference Using a Variational Recurrent Neural Network

Entropy ◽  
2020 ◽  
Vol 22 (5) ◽  
pp. 564
Author(s):  
Takazumi Matsumoto ◽  
Jun Tani
Keyword(s):  
Lower Bound ◽  
Latent Variables ◽  
Predictive Coding ◽  
Forward Model ◽  
Training Data ◽  
Active Inference ◽  
Model Framework ◽  
Model Learning ◽  
Proposed Model ◽  
Sensory Motor

It is crucial to ask how agents can achieve goals by generating action plans using only partial models of the world acquired through habituated sensory-motor experiences. Although many existing robotics studies use a forward model framework, there are generalization issues with high degrees of freedom. The current study shows that the predictive coding (PC) and active inference (AIF) frameworks, which employ a generative model, can develop better generalization by learning a prior distribution in a low dimensional latent state space representing probabilistic structures extracted from well habituated sensory-motor trajectories. In our proposed model, learning is carried out by inferring optimal latent variables as well as synaptic weights for maximizing the evidence lower bound, while goal-directed planning is accomplished by inferring latent variables for maximizing the estimated lower bound. Our proposed model was evaluated with both simple and complex robotic tasks in simulation, which demonstrated sufficient generalization in learning with limited training data by setting an intermediate value for a regularization coefficient. Furthermore, comparative simulation results show that the proposed model outperforms a conventional forward model in goal-directed planning, due to the learned prior confining the search of motor plans within the range of habituated trajectories.

scholarly journals Interoceptive accuracy moderates the response to a glucose load: a test of the predictive coding framework

2019 ◽  
Vol 286 (1898) ◽  
pp. 20190244 ◽  
Author(s):  
Hayley A. Young ◽  
Chantelle M. Gaylor ◽  
Danielle de Kerckhove ◽  
Heather Watkins ◽  
David Benton
Keyword(s):  
Heart Rate ◽  
Blood Glucose ◽  
Emotional Health ◽  
Predictive Coding ◽  
The Body ◽  
Active Inference ◽  
Glucose Load ◽  
Postprandial Period ◽  
Interoceptive Accuracy

Recently, interoception and homeostasis have been described in terms of predictive coding and active inference. Afferent signals update prior predictions about the state of the body, and stimulate the autonomic mediation of homeostasis. Performance on tests of interoceptive accuracy (IAc) may indicate an individual's ability to assign precision to interoceptive signals, thus determining the relative influence of ascending signals and the descending prior predictions. Accordingly, individuals with high IAc should be better able to regulate during the postprandial period. One hundred females were allocated to consume glucose, an artificially sweetened drink, water or no drink. Before, and 30 min after a drink, IAc, heart rate (HR) and blood glucose (BG) were measured, and participants rated their hunger, thirst and mood. A higher IAc was related to lower BG levels, a decline in anxiety and a higher HR, after consuming glucose. A higher IAc also resulted in a larger decline in hunger if they consumed either glucose or sucralose. These data support the role of active inference in interoception and homeostasis, and suggest that the ability to attend to interoceptive signals may be critical to the maintenance of physical and emotional health.

Sign in / Sign up

Export Citation Format

Share Document