scholarly journals Quantification of Effective Connectivity in the Brain Using a Measure of Directed Information

2012 ◽  
Vol 2012 ◽  
pp. 1-16 ◽  
Author(s):  
Ying Liu ◽  
Selin Aviyente
Keyword(s):  
Granger Causality ◽  
Linear Models ◽  
Effective Connectivity ◽  
Causal Modeling ◽  
Detection Methods ◽  
Information Theoretic ◽  
Model Based ◽  
Model Free ◽  
Directed Information ◽  
The Brain

Effective connectivity refers to the influence one neural system exerts on another and corresponds to the parameter of a model that tries to explain the observed dependencies. In this sense, effective connectivity corresponds to the intuitive notion of coupling or directed causal influence. Traditional measures to quantify the effective connectivity include model-based methods, such as dynamic causal modeling (DCM), Granger causality (GC), and information-theoretic methods. Directed information (DI) has been a recently proposed information-theoretic measure that captures the causality between two time series. Compared to traditional causality detection methods based on linear models, directed information is a model-free measure and can detect both linear and nonlinear causality relationships. However, the effectiveness of using DI for capturing the causality in different models and neurophysiological data has not been thoroughly illustrated to date. In addition, the advantage of DI compared to model-based measures, especially those used to implement Granger causality, has not been fully investigated. In this paper, we address these issues by evaluating the performance of directed information on both simulated data sets and electroencephalogram (EEG) data to illustrate its effectiveness for quantifying the effective connectivity in the brain.

scholarly journals Prefrontal solution to the bias-variance tradeoff during reinforcement learning

2020 ◽  
Author(s):  
Dongjae Kim ◽  
Jaeseung Jeong ◽  
Sang Wan Lee
Keyword(s):  
Adaptive Control ◽  
Reinforcement Learning ◽  
Prediction Error ◽  
Brain Regions ◽  
Decision Task ◽  
Prediction Errors ◽  
Model Based ◽  
Model Free ◽  
Bias Variance ◽  
The Brain

AbstractThe goal of learning is to maximize future rewards by minimizing prediction errors. Evidence have shown that the brain achieves this by combining model-based and model-free learning. However, the prediction error minimization is challenged by a bias-variance tradeoff, which imposes constraints on each strategy’s performance. We provide new theoretical insight into how this tradeoff can be resolved through the adaptive control of model-based and model-free learning. The theory predicts the baseline correction for prediction error reduces the lower bound of the bias–variance error by factoring out irreducible noise. Using a Markov decision task with context changes, we showed behavioral evidence of adaptive control. Model-based behavioral analyses show that the prediction error baseline signals context changes to improve adaptability. Critically, the neural results support this view, demonstrating multiplexed representations of prediction error baseline within the ventrolateral and ventromedial prefrontal cortex, key brain regions known to guide model-based and model-free learning.One sentence summaryA theoretical, behavioral, computational, and neural account of how the brain resolves the bias-variance tradeoff during reinforcement learning is described.

scholarly journals Neurostimulation Reveals Context-Dependent Arbitration Between Model-Based and Model-Free Reinforcement Learning

2019 ◽  
Vol 29 (11) ◽  
pp. 4850-4862 ◽  
Author(s):  
Sebastian Weissengruber ◽  
Sang Wan Lee ◽  
John P O’Doherty ◽  
Christian C Ruff
Keyword(s):  
Reinforcement Learning ◽  
Neural Mechanism ◽  
Ventrolateral Prefrontal Cortex ◽  
Control Behavior ◽  
Model Based ◽  
Model Free ◽  
Context Dependent ◽  
The Brain ◽  
Over Time ◽  
Environmental Demands

Abstract While it is established that humans use model-based (MB) and model-free (MF) reinforcement learning in a complementary fashion, much less is known about how the brain determines which of these systems should control behavior at any given moment. Here we provide causal evidence for a neural mechanism that acts as a context-dependent arbitrator between both systems. We applied excitatory and inhibitory transcranial direct current stimulation over a region of the left ventrolateral prefrontal cortex previously found to encode the reliability of both learning systems. The opposing neural interventions resulted in a bidirectional shift of control between MB and MF learning. Stimulation also affected the sensitivity of the arbitration mechanism itself, as it changed how often subjects switched between the dominant system over time. Both of these effects depended on varying task contexts that either favored MB or MF control, indicating that this arbitration mechanism is not context-invariant but flexibly incorporates information about current environmental demands.

scholarly journals MOU-EC: model-based whole-brain effective connectivity to extract biomarkers for brain dynamics from fMRI data and study distributed cognition

2019 ◽  
Author(s):  
M Gilson ◽  
G Zamora-López ◽  
V Pallarés ◽  
MH Adhikari ◽  
M Senden ◽  
...  
Keyword(s):  
Distributed Cognition ◽  
Dynamic Models ◽  
Effective Connectivity ◽  
Statistical Testing ◽  
Fmri Data ◽  
Brain Areas ◽  
Whole Brain ◽  
Model Based ◽  
The Brain ◽  
Connectivity Measure

AbstractNeuroimaging techniques are increasingly used to study brain cognition in humans. Beyond their individual activation, the functional associations between brain areas have become a standard proxy to describe how information is distributed across the brain network. Among the many analysis tools available, dynamic models of brain activity have been developed to overcome the limitations of original connectivity measures such as functional connectivity. In particular, much effort has been devoted to the assessment of directional interactions between brain areas from their observed activity. This paper summarizes our recent approach to analyze fMRI data based on our whole-brain effective connectivity referred to as MOU-EC, while discussing the pros and cons of its underlying assumptions with respect to other established approaches. Once tuned, the model provides a connectivity measure that reflects the dynamical state of BOLD activity obtained using fMRI, which can be used to explore the brain cognition. We focus on two important applications. First, as a connectivity measure, MOU-EC can be used to extract biomarkers for task-specific brain coordination, understood as the patterns of areas exchanging information. The multivariate nature of connectivity measures raises several challenges for whole-brain analysis, for which machine-learning tools presents some advantages over statistical testing. Second, we show how to interpret changes in MOU-EC connections in a collective and model-based manner, bridging with network analysis. To illustrate our framework, we use a dataset where subjects were recorded in two conditions, watching a movie and a black screen (referred to as rest). Our framework provides a comprehensive set of tools that open exciting perspectives for the study of distributed cognition, as well as neuropathologies.

scholarly journals Habituation and goal-directed arbitration mechanisms and failures under partial observability

2020 ◽  
Author(s):  
Martí Sánchez-Fibla
Keyword(s):  
Compulsive Behavior ◽  
Partial Observability ◽  
Model Based ◽  
Model Free ◽  
Different Types ◽  
Compulsive Checking ◽  
The Right ◽  
New Treatments ◽  
Choice Tasks ◽  
The Brain

AbstractWe often need to make decisions under incomplete information (partial observability) and the brain manages to add the right minimal context to the decision-making. Partial observability may also be handled by other mechanisms than adding contextual experience / memory. We propose that parallel and sequential arbitration of Habituation (Model-Free, MF) and Goal-Directed (Model-Based, MB) behavior may be at play to deal with partial observability “on-the-fly”, and that MB may be of different types (going beyond the MF/MB dichotomy [4]). To illustrate this, we identify, describe and model with Reinforcement Learning (RL) a behavioral anomaly (an habituation failure) occurring during the so-called Hotel Elevators Rows (HER, for short) task: a prototypical partial observation situation that can be reduced to the well studied Two and One Sequence Choice Tasks. The following hypothesis are supported by RL simulation results: (1) a parallel (semi)model-based successor representation mechanism is operative while learning to habituate which detects model-based mismatches and serves as an habituation surveillance, (2) a retrospective inference is triggered to identify the source of the habituation failure (3) a model-free mechanism can trigger model-based mechanisms in states in which habituation failed. The “failures” in the title refer to: the habituation failures that need to be monitored and surveilled (1) and to the failures that we identified in prototypical state of the art Model-Based algorithms (like DynaQ) when facing partial observability. As other research on MF/MB arbitration shows, the identification of these new mechanisms could shine light into new treatments for addiction, compulsive behavior (like compulsive checking) and understand better accidents caused by habituation behaviors.

Reinforcement Learning and Causal Models

2017 ◽  
Author(s):  
Samuel J. Gershman
Keyword(s):  
Reinforcement Learning ◽  
Causal Knowledge ◽  
Free System ◽  
Sensory Data ◽  
Model Based ◽  
Model Free ◽  
Two Systems ◽  
Hidden States ◽  
State Inference ◽  
The Brain

This chapter reviews the diverse roles that causal knowledge plays in reinforcement learning. The first half of the chapter contrasts a “model-free” system that learns to repeat actions that lead to reward with a “model-based” system that learns a probabilistic causal model of the environment, which it then uses to plan action sequences. Evidence suggests that these two systems coexist in the brain, both competing and cooperating with each other. The interplay of two systems allows the brain to negotiate a balance between cognitively cheap but inaccurate model-free algorithms and accurate but expensive model-based algorithms. The second half of the chapter reviews research on hidden state inference in reinforcement learning. The problem of inferring hidden states can be construed in terms of inferring the latent causes that give rise to sensory data and rewards. Because hidden state inference affects both model-based and model-free reinforcement learning, causal knowledge impinges upon both systems.

scholarly journals Task complexity interacts with state-space uncertainty in the arbitration process between model-based and model-free reinforcement-learning at both behavioral and neural levels

2018 ◽  
Author(s):  
Dongjae Kim ◽  
Geon Yeong Park ◽  
John P. O’Doherty ◽  
Sang Wan Lee
Keyword(s):  
Prefrontal Cortex ◽  
Reinforcement Learning ◽  
State Space ◽  
Task Complexity ◽  
Model Based ◽  
Model Free ◽  
Arbitration Process ◽  
Two Systems ◽  
The Brain

SUMMARYA major open question concerns how the brain governs the allocation of control between two distinct strategies for learning from reinforcement: model-based and model-free reinforcement learning. While there is evidence to suggest that the reliability of the predictions of the two systems is a key variable responsible for the arbitration process, another key variable has remained relatively unexplored: the role of task complexity. By using a combination of novel task design, computational modeling, and model-based fMRI analysis, we examined the role of task complexity alongside state-space uncertainty in the arbitration process between model-based and model-free RL. We found evidence to suggest that task complexity plays a role in influencing the arbitration process alongside state-space uncertainty. Participants tended to increase model-based RL control in response to increasing task complexity. However, they resorted to model-free RL when both uncertainty and task complexity were high, suggesting that these two variables interact during the arbitration process. Computational fMRI revealed that task complexity interacts with neural representations of the reliability of the two systems in the inferior prefrontal cortex bilaterally. These findings provide insight into how the inferior prefrontal cortex negotiates the trade-off between model-based and model-free RL in the presence of uncertainty and complexity, and more generally, illustrates how the brain resolves uncertainty and complexity in dynamically changing environments.SUMMARY OF FINDINGS- Elucidated the role of state-space uncertainty and complexity in model-based and model-free RL.- Found behavioral and neural evidence for complexity-sensitive prefrontal arbitration.- High task complexity induces explorative model-based RL.

THE EFFECTIVE CONNECTIVITY BETWEEN THE TWO PRIMARY MOTOR AREAS IN THE BRAIN DURING BILATERAL TAPPING OF HAND FINGERS

2012 ◽  
Vol 09 ◽  
pp. 398-405
Author(s):  
A. N. YUSOFF ◽  
K. A. HAMID
Keyword(s):  
Causal Model ◽  
Right Hemisphere ◽  
Effective Connectivity ◽  
Causal Modeling ◽  
Dynamic Causal Modeling ◽  
Male And Female ◽  
Linear Behavior ◽  
Left And Right ◽  
Dynamic Causal Model ◽  
The Brain

Dynamic causal modeling (DCM) was implemented on datasets obtained from an externally-triggered finger tapping functional MRI experiment performed by 5 male and female subjects. The objective was to model the effective connectivity between two significantly activated primary motor regions (M1). The left and right hemisphere M1s are found to be effectively and bidirectionally connected to each other. Both connections are modulated by the stimulus-free contextual input. These connectivities are however not gated (influenced) by any of the two M1s, ruling out the possibility of the non-linear behavior of connections between both M1s. A dynamic causal model was finally suggested.

scholarly journals Attentional reorientation along the meridians of the visual field: are there different neural mechanisms at play?

2019 ◽  
Author(s):  
Simon R. Steinkamp ◽  
Simone Vossel ◽  
Gereon R. Fink ◽  
Ralph Weidner
Keyword(s):  
Effective Connectivity ◽  
Target Position ◽  
Causal Modeling ◽  
Orientation Discrimination ◽  
Behavioral Measures ◽  
Directional Tuning ◽  
Vertical Meridian ◽  
Attention Networks ◽  
Predictive Approaches ◽  
The Brain

AbstractHemispatial neglect, after unilateral lesions to parietal brain areas, is characterized by an inability to respond to unexpected stimuli in contralesional space. As the visual field’s horizontal meridian is most severely affected, the brain networks controlling visuospatial processes might be tuned explicitly to this axis. We investigated such a potential directional tuning in the dorsal and ventral frontoparietal attention networks, with a particular focus on attentional reorientation. We used an orientation-discrimination task where a spatial pre-cue indicated the target position with 80% validity. Healthy participants (n = 29) performed this task in two runs and were required to (re-)orient attention either only along the horizontal or the vertical meridian, while fMRI and behavioral measures were recorded. By using a General Linear Model for behavioral and fMRI data, Dynamic Causal Modeling for effective connectivity, and other predictive approaches, we found strong statistical evidence for a reorientation effect for horizontal and vertical runs. However, neither neural nor behavioral measures differed between vertical and horizontal reorienting. Moreover, models from one run successfully predicted the cueing condition in the respective other run. Our results suggest that activations in the dorsal and ventral attention networks represent higher-order cognitive processes related to spatial attentional (re-)orientating that are independent of directional tuning.

Sign in / Sign up

Export Citation Format

Share Document