scholarly journals Generative Models for Active Vision

2021 ◽  
Vol 15 ◽  
Author(s):  
Thomas Parr ◽  
Noor Sajid ◽  
Lancelot Da Costa ◽  
M. Berk Mirza ◽  
Karl J. Friston
Keyword(s):  
Information Gain ◽  
Cell Activity ◽  
Active Vision ◽  
Oculomotor System ◽  
Generative Models ◽  
Generative Model ◽  
Retinal Cell ◽  
Proprioceptive Information ◽  
Active Inference ◽  
The Brain

The active visual system comprises the visual cortices, cerebral attention networks, and oculomotor system. While fascinating in its own right, it is also an important model for sensorimotor networks in general. A prominent approach to studying this system is active inference—which assumes the brain makes use of an internal (generative) model to predict proprioceptive and visual input. This approach treats action as ensuring sensations conform to predictions (i.e., by moving the eyes) and posits that visual percepts are the consequence of updating predictions to conform to sensations. Under active inference, the challenge is to identify the form of the generative model that makes these predictions—and thus directs behavior. In this paper, we provide an overview of the generative models that the brain must employ to engage in active vision. This means specifying the processes that explain retinal cell activity and proprioceptive information from oculomotor muscle fibers. In addition to the mechanics of the eyes and retina, these processes include our choices about where to move our eyes. These decisions rest upon beliefs about salient locations, or the potential for information gain and belief-updating. A key theme of this paper is the relationship between “looking” and “seeing” under the brain's implicit generative model of the visual world.

scholarly journals Generalised free energy and active inference

2019 ◽  
Vol 113 (5-6) ◽  
pp. 495-513 ◽  
Author(s):  
Thomas Parr ◽  
Karl J. Friston
Keyword(s):  
Free Energy ◽  
Generative Model ◽  
Free Energy Function ◽  
Active Inference ◽  
Future Expectations ◽  
Perceptual Inference ◽  
Energy Functionals ◽  
Sensory Data ◽  
The World ◽  
The Brain

Abstract Active inference is an approach to understanding behaviour that rests upon the idea that the brain uses an internal generative model to predict incoming sensory data. The fit between this model and data may be improved in two ways. The brain could optimise probabilistic beliefs about the variables in the generative model (i.e. perceptual inference). Alternatively, by acting on the world, it could change the sensory data, such that they are more consistent with the model. This implies a common objective function (variational free energy) for action and perception that scores the fit between an internal model and the world. We compare two free energy functionals for active inference in the framework of Markov decision processes. One of these is a functional of beliefs (i.e. probability distributions) about states and policies, but a function of observations, while the second is a functional of beliefs about all three. In the former (expected free energy), prior beliefs about outcomes are not part of the generative model (because they are absorbed into the prior over policies). Conversely, in the second (generalised free energy), priors over outcomes become an explicit component of the generative model. When using the free energy function, which is blind to future observations, we equip the generative model with a prior over policies that ensure preferred (i.e. priors over) outcomes are realised. In other words, if we expect to encounter a particular kind of outcome, this lends plausibility to those policies for which this outcome is a consequence. In addition, this formulation ensures that selected policies minimise uncertainty about future outcomes by minimising the free energy expected in the future. When using the free energy functional—that effectively treats future observations as hidden states—we show that policies are inferred or selected that realise prior preferences by minimising the free energy of future expectations. Interestingly, the form of posterior beliefs about policies (and associated belief updating) turns out to be identical under both formulations, but the quantities used to compute them are not.

scholarly journals A tale of two densities: active inference is enactive inference

2019 ◽  
Vol 28 (4) ◽  
pp. 225-239 ◽  
Author(s):  
Maxwell JD Ramstead ◽  
Michael D Kirchhoff ◽  
Karl J Friston
Keyword(s):  
Predictive Coding ◽  
Generative Models ◽  
Predictive Processing ◽  
Active Inference ◽  
Energy Principle ◽  
Free Energy Principle ◽  
The One ◽  
And Control ◽  
The Brain ◽  
Selection Of

The aim of this article is to clarify how best to interpret some of the central constructs that underwrite the free-energy principle (FEP) – and its corollary, active inference – in theoretical neuroscience and biology: namely, the role that generative models and variational densities play in this theory. We argue that these constructs have been systematically misrepresented in the literature, because of the conflation between the FEP and active inference, on the one hand, and distinct (albeit closely related) Bayesian formulations, centred on the brain – variously known as predictive processing, predictive coding or the prediction error minimisation framework. More specifically, we examine two contrasting interpretations of these models: a structural representationalist interpretation and an enactive interpretation. We argue that the structural representationalist interpretation of generative and recognition models does not do justice to the role that these constructs play in active inference under the FEP. We propose an enactive interpretation of active inference – what might be called enactive inference. In active inference under the FEP, the generative and recognition models are best cast as realising inference and control – the self-organising, belief-guided selection of action policies – and do not have the properties ascribed by structural representationalists.

scholarly journals Generative models, language and active inference

2020 ◽  
Author(s):  
Karl Friston ◽  
Thomas Parr ◽  
Yan Yufik ◽  
Noor Sajid ◽  
Cathy J. Price ◽  
...  
Keyword(s):  
Language Processing ◽  
Hierarchical Models ◽  
Message Passing ◽  
Generative Models ◽  
Generative Model ◽  
Active Inference ◽  
Factorial Structure ◽  
The Rich ◽  
The Right

This paper presents a biologically plausible generative model and inference scheme that is capable of simulating the generation and comprehension of language, when synthetic subjects talk to each other. Building on active inference formulations of dyadic interactions, we simulate linguistic exchange to explore generative models that support dialogues. These models employ high-order interactions among abstract (discrete) states in deep (hierarchical) models. The sequential nature of language processing mandates generative models with a particular factorial structure—necessary to accommodate the rich combinatorics of language. We illustrate this by simulating a synthetic subject who can play the ‘Twenty Questions’ game. In this game, synthetic subjects take the role of the questioner or answerer, using the same generative model. This simulation setup is used to illustrate some key architectural points and demonstrate that many behavioural and neurophysiological correlates of language processing emerge under variational (marginal) message passing, given the right kind of generative model. For example, we show that theta-gamma coupling is an emergent property of belief updating, when listening to another.

scholarly journals The computational neurology of movement under active inference

Brain ◽  
2021 ◽  
Author(s):  
Thomas Parr ◽  
Jakub Limanowski ◽  
Vishal Rawji ◽  
Karl Friston
Keyword(s):  
Movement Control ◽  
Generative Model ◽  
Clinical Neurology ◽  
Active Inference ◽  
Significant Development ◽  
Prior Probabilities ◽  
Sensory Data ◽  
Limb Coordination ◽  
The Right ◽  
The Brain

Abstract We propose a computational neurology of movement based on the convergence of theoretical neurobiology and clinical neurology. A significant development in the former is the idea that we can frame brain function as a process of (active) inference, in which the nervous system makes predictions about its sensory data. These predictions depend upon an implicit predictive (generative) model used by the brain. This means neural dynamics can be framed as generating actions to ensure sensations are consistent with these predictions—and adjusting predictions when they are not. We illustrate the significance of this formulation for clinical neurology through simulating a clinical examination of the motor system; i.e. an upper limb coordination task. Specifically, we show how tendon reflexes emerge naturally under the right kind of generative model. Through simulated perturbations, pertaining to prior probabilities of this model’s variables, we illustrate the emergence of hyperreflexia and pendular reflexes, reminiscent of neurological lesions in the corticospinal tract and cerebellum. We then turn to the computational lesions causing hypokinesia and deficits of coordination. This in silico lesion-deficit analysis provides an opportunity to revisit classic neurological dichotomies (e.g. pyramidal versus extrapyramidal systems) from the perspective of modern approaches to theoretical neurobiology—and our understanding of the neurocomputational architecture of movement control based on first principles.

Consciousness

2018 ◽  
Author(s):  
Anil K. Seth
Keyword(s):  
Conscious Experience ◽  
Real Problem ◽  
Active Inference ◽  
Biological Mechanisms ◽  
Specific Content ◽  
Brain Responses ◽  
Biomimetic Approach ◽  
Multiple Levels ◽  
In Virtue Of ◽  
The Brain

Consciousness is perhaps the most familiar aspect of our existence, yet we still do not know its biological basis. This chapter outlines a biomimetic approach to consciousness science, identifying three principles linking properties of conscious experience to potential biological mechanisms. First, conscious experiences generate large quantities of information in virtue of being simultaneously integrated and differentiated. Second, the brain continuously generates predictions about the world and self, which account for the specific content of conscious scenes. Third, the conscious self depends on active inference of self-related signals at multiple levels. Research following these principles helps move from establishing correlations between brain responses and consciousness towards explanations which account for phenomenological properties—addressing what can be called the “real problem” of consciousness. The picture that emerges is one in which consciousness, mind, and life, are tightly bound together—with implications for any possible future “conscious machines.”

Equivariant Adversarial Network for Image-to-image Translation

2021 ◽  
Vol 17 (2s) ◽  
pp. 1-14
Author(s):  
Masoumeh Zareapoor ◽  
Jie Yang
Keyword(s):  
State Of The Art ◽  
Generative Models ◽  
Generative Model ◽  
Target Domain ◽  
Adversarial Network ◽  
Proposed Model ◽  
Image Translation ◽  
Great Performance ◽  
Representative Model ◽  
The Ideal

Image-to-Image translation aims to learn an image from a source domain to a target domain. However, there are three main challenges, such as lack of paired datasets, multimodality, and diversity, that are associated with these problems and need to be dealt with. Convolutional neural networks (CNNs), despite of having great performance in many computer vision tasks, they fail to detect the hierarchy of spatial relationships between different parts of an object and thus do not form the ideal representative model we look for. This article presents a new variation of generative models that aims to remedy this problem. We use a trainable transformer, which explicitly allows the spatial manipulation of data within training. This differentiable module can be augmented into the convolutional layers in the generative model, and it allows to freely alter the generated distributions for image-to-image translation. To reap the benefits of proposed module into generative model, our architecture incorporates a new loss function to facilitate an effective end-to-end generative learning for image-to-image translation. The proposed model is evaluated through comprehensive experiments on image synthesizing and image-to-image translation, along with comparisons with several state-of-the-art algorithms.

scholarly journals Active Inference: Applicability to Different Types of Social Organization Explained through Reference to Industrial Engineering and Quality Management

Entropy ◽  
2021 ◽  
Vol 23 (2) ◽  
pp. 198
Author(s):  
Stephen Fox
Keyword(s):  
Artificial Intelligence ◽  
Quality Management ◽  
Social Organization ◽  
Generative Models ◽  
Industrial Engineering ◽  
Prediction Errors ◽  
Active Inference ◽  
For Profit ◽  
Different Types ◽  
Inference Theory

Active inference is a physics of life process theory of perception, action and learning that is applicable to natural and artificial agents. In this paper, active inference theory is related to different types of practice in social organization. Here, the term social organization is used to clarify that this paper does not encompass organization in biological systems. Rather, the paper addresses active inference in social organization that utilizes industrial engineering, quality management, and artificial intelligence alongside human intelligence. Social organization referred to in this paper can be in private companies, public institutions, other for-profit or not-for-profit organizations, and any combination of them. The relevance of active inference theory is explained in terms of variational free energy, prediction errors, generative models, and Markov blankets. Active inference theory is most relevant to the social organization of work that is highly repetitive. By contrast, there are more challenges involved in applying active inference theory for social organization of less repetitive endeavors such as one-of-a-kind projects. These challenges need to be addressed in order for active inference to provide a unifying framework for different types of social organization employing human and artificial intelligence.

scholarly journals The brain timewise: how timing shapes and supports brain function

2015 ◽  
Vol 370 (1668) ◽  
pp. 20140170 ◽  
Author(s):  
Riitta Hari ◽  
Lauri Parkkonen
Keyword(s):  
Human Brain ◽  
Brain Imaging ◽  
Brain Function ◽  
Temporal Dynamics ◽  
Generative Models ◽  
Network Activity ◽  
Brain Diseases ◽  
Specific Information ◽  
Non Invasive ◽  
The Brain

We discuss the importance of timing in brain function: how temporal dynamics of the world has left its traces in the brain during evolution and how we can monitor the dynamics of the human brain with non-invasive measurements. Accurate timing is important for the interplay of neurons, neuronal circuitries, brain areas and human individuals. In the human brain, multiple temporal integration windows are hierarchically organized, with temporal scales ranging from microseconds to tens and hundreds of milliseconds for perceptual, motor and cognitive functions, and up to minutes, hours and even months for hormonal and mood changes. Accurate timing is impaired in several brain diseases. From the current repertoire of non-invasive brain imaging methods, only magnetoencephalography (MEG) and scalp electroencephalography (EEG) provide millisecond time-resolution; our focus in this paper is on MEG. Since the introduction of high-density whole-scalp MEG/EEG coverage in the 1990s, the instrumentation has not changed drastically; yet, novel data analyses are advancing the field rapidly by shifting the focus from the mere pinpointing of activity hotspots to seeking stimulus- or task-specific information and to characterizing functional networks. During the next decades, we can expect increased spatial resolution and accuracy of the time-resolved brain imaging and better understanding of brain function, especially its temporal constraints, with the development of novel instrumentation and finer-grained, physiologically inspired generative models of local and network activity. Merging both spatial and temporal information with increasing accuracy and carrying out recordings in naturalistic conditions, including social interaction, will bring much new information about human brain function.

scholarly journals Efficient generation of reciprocal signals by inhibition

2012 ◽  
Vol 107 (9) ◽  
pp. 2453-2462 ◽  
Author(s):  
Sung-min Park ◽  
Esra Tara ◽  
Kamran Khodakhah
Keyword(s):  
Purkinje Cells ◽  
Granule Cell ◽  
Granule Cells ◽  
Mossy Fiber ◽  
Cell Activity ◽  
Common Mechanism ◽  
Input Output ◽  
Firing Rates ◽  
Neuronal Computation ◽  
The Brain

Reciprocal activity between populations of neurons has been widely observed in the brain and is essential for neuronal computation. The different mechanisms by which reciprocal neuronal activity is generated remain to be established. A common motif in neuronal circuits is the presence of afferents that provide excitation to one set of principal neurons and, via interneurons, inhibition to a second set of principal neurons. This circuitry can be the substrate for generation of reciprocal signals. Here we demonstrate that this equivalent circuit in the cerebellar cortex enables the reciprocal firing rates of Purkinje cells to be efficiently generated from a common set of mossy fiber inputs. The activity of a mossy fiber is relayed to Purkinje cells positioned immediately above it by excitatory granule cells. The firing rates of these Purkinje cells increase as a linear function of mossy fiber, and thus granule cell, activity. In addition to exciting Purkinje cells positioned immediately above it, the activity of a mossy fiber is relayed to laterally positioned Purkinje cells by a disynaptic granule cell → molecular layer interneuron pathway. Here we show in acutely prepared cerebellar slices that the input-output relationship of these laterally positioned Purkinje cells is linear and reciprocal to the first set. A similar linear input-output relationship between decreases in Purkinje cell firing and strength of stimulation of laterally positioned granule cells was also observed in vivo. Use of interneurons to generate reciprocal firing rates may be a common mechanism by which the brain generates reciprocal signals.

scholarly journals Monte Carlo and Reconstruction Membership Inference Attacks against Generative Models

2019 ◽  
Vol 2019 (4) ◽  
pp. 232-249 ◽  
Author(s):  
Benjamin Hilprecht ◽  
Martin Härterich ◽  
Daniel Bernau
Keyword(s):  
Data Privacy ◽  
Information Leakage ◽  
Generative Models ◽  
Generative Model ◽  
Training Data ◽  
Generative Adversarial Networks ◽  
Data Sets ◽  
Success Rates ◽  
Model Quality ◽  
Type Formalization

Abstract We present two information leakage attacks that outperform previous work on membership inference against generative models. The first attack allows membership inference without assumptions on the type of the generative model. Contrary to previous evaluation metrics for generative models, like Kernel Density Estimation, it only considers samples of the model which are close to training data records. The second attack specifically targets Variational Autoencoders, achieving high membership inference accuracy. Furthermore, previous work mostly considers membership inference adversaries who perform single record membership inference. We argue for considering regulatory actors who perform set membership inference to identify the use of specific datasets for training. The attacks are evaluated on two generative model architectures, Generative Adversarial Networks (GANs) and Variational Autoen-coders (VAEs), trained on standard image datasets. Our results show that the two attacks yield success rates superior to previous work on most data sets while at the same time having only very mild assumptions. We envision the two attacks in combination with the membership inference attack type formalization as especially useful. For example, to enforce data privacy standards and automatically assessing model quality in machine learning as a service setups. In practice, our work motivates the use of GANs since they prove less vulnerable against information leakage attacks while producing detailed samples.

Sign in / Sign up

Export Citation Format

Share Document