Optimal Decision Making in the Cortico-Basal-Ganglia Circuit

2015 ◽  
pp. 291-302
Author(s):  
Rafal Bogacz
Keyword(s):  
Decision Making ◽  
Basal Ganglia ◽  
Optimal Decision ◽  
Optimal Decision Making

The Basal Ganglia and Cortex Implement Optimal Decision Making Between Alternative Actions

2007 ◽  
Vol 19 (2) ◽  
pp. 442-477 ◽  
Author(s):  
Rafal Bogacz ◽  
Kevin Gurney
Keyword(s):  
Decision Making ◽  
Basal Ganglia ◽  
Brain Regions ◽  
Optimal Decision ◽  
Detailed Knowledge ◽  
Ratio Test ◽  
Anatomy And Physiology ◽  
Neurophysiological Studies ◽  
Sequential Probability ◽  
Optimal Decision Making

Neurophysiological studies have identified a number of brain regions critically involved in solving the problem of action selection or decision making. In the case of highly practiced tasks, these regions include cortical areas hypothesized to integrate evidence supporting alternative actions and the basal ganglia, hypothesized to act as a central switch in gating behavioral requests. However, despite our relatively detailed knowledge of basal ganglia biology and its connectivity with the cortex and numerical simulation studies demonstrating selective function, no formal theoretical framework exists that supplies an algorithmic description of these circuits. This article shows how many aspects of the anatomy and physiology of the circuit involving the cortex and basal ganglia are exactly those required to implement the computation defined by an asymptotically optimal statistical test for decision making: the multihypothesis sequential probability ratio test (MSPRT). The resulting model of basal ganglia provides a new framework for understanding the computation in the basal ganglia during decision making in highly practiced tasks. The predictions of the theory concerning the properties of particular neuronal populations are validated in existing experimental data. Further, we show that this neurobiologically grounded implementation of MSPRT outperforms other candidates for neural decision making, that it is structurally and parametrically robust, and that it can accommodate cortical mechanisms for decision making in a way that complements those in basal ganglia.

The Basal Ganglia Optimize Decision Making over General Perceptual Hypotheses

2012 ◽  
Vol 24 (11) ◽  
pp. 2924-2945 ◽  
Author(s):  
Nathan F. Lepora ◽  
Kevin N. Gurney
Keyword(s):  
Decision Making ◽  
Basal Ganglia ◽  
Sequential Analysis ◽  
Probability Distributions ◽  
Optimal Decision ◽  
Likelihood Ratios ◽  
Recent Proposal ◽  
Sensory Data ◽  
Log Odds ◽  
Optimal Decision Making

The basal ganglia are a subcortical group of interconnected nuclei involved in mediating action selection within cortex. A recent proposal is that this selection leads to optimal decision making over multiple alternatives because the basal ganglia anatomy maps onto a network implementation of an optimal statistical method for hypothesis testing, assuming that cortical activity encodes evidence for constrained gaussian-distributed alternatives. This letter demonstrates that this model of the basal ganglia extends naturally to encompass general Bayesian sequential analysis over arbitrary probability distributions, which raises the proposal to a practically realizable theory over generic perceptual hypotheses. We also show that the evidence in this model can represent either log likelihoods, log-likelihood ratios, or log odds, all leading proposals for the cortical processing of sensory data. For these reasons, we claim that the basal ganglia optimize decision making over general perceptual hypotheses represented in cortex. The relation of this theory to cortical encoding, cortico-basal ganglia anatomy, and reinforcement learning is discussed.

Integration of Reinforcement Learning and Optimal Decision-Making Theories of the Basal Ganglia

2011 ◽  
Vol 23 (4) ◽  
pp. 817-851 ◽  
Author(s):  
Rafal Bogacz ◽  
Tobias Larsen
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Basal Ganglia ◽  
Reaction Times ◽  
Learning Theories ◽  
Optimal Decision ◽  
Neural Responses ◽  
Sensory Evidence ◽  
Optimal Decision Making ◽  
Reinforcement Learning Models

This article seeks to integrate two sets of theories describing action selection in the basal ganglia: reinforcement learning theories describing learning which actions to select to maximize reward and decision-making theories proposing that the basal ganglia selects actions on the basis of sensory evidence accumulated in the cortex. In particular, we present a model that integrates the actor-critic model of reinforcement learning and a model assuming that the cortico-basal-ganglia circuit implements a statistically optimal decision-making procedure. The values of corico-striatal weights required for optimal decision making in our model differ from those provided by standard reinforcement learning models. Nevertheless, we show that an actor-critic model converges to the weights required for optimal decision making when biologically realistic limits on synaptic weights are introduced. We also describe the model's predictions concerning reaction times and neural responses during learning, and we discuss directions required for further integration of reinforcement learning and optimal decision-making theories.

scholarly journals GEAR: On Optimal Decision Making with Auxiliary Data

Stat ◽  
2021 ◽  
Author(s):  
Hengrui Cai ◽  
Rui Song ◽  
Wenbin Lu
Keyword(s):  
Decision Making ◽  
Optimal Decision ◽  
Auxiliary Data ◽  
Optimal Decision Making

Operation decision of logistics service supply chain: considering interruption probability in emergencies

Kybernetes ◽  
2021 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Guangsheng Zhang ◽  
Xiao Wang ◽  
Zhiqing Meng ◽  
Qirui Zhang ◽  
Kexin Wu
Keyword(s):  
Decision Making ◽  
Supply Chain ◽  
Evolutionary Game ◽  
Optimal Strategies ◽  
Optimal Decision ◽  
Content Type ◽  
Logistics Service ◽  
Social Reputation ◽  
Punishment Intensity ◽  
Optimal Decision Making

PurposeTo remedy the inherent defect in current research that focuses only on a single type of participants, this paper endeavors to look into the situation as an evolutionary game between a representative Logistics Service Integrator (LSI) and a representative Functional Logistics Service Provider (FLSP) in an environment with sudden crisis and tries to analyze how LSI supervises FLSP's operations and how FLSP responds in a recurrent pattern with different interruption probabilities.Design/methodology/approachRegarding the risks of supply chain interruption in emergencies, this paper develops a two-level model of single LSI and single FLSP, using Evolutionary Game theory to analyze their optimal decision-making, as well as their strategic behaviors on different risk levels regarding the interruption probability to achieve the optimal return with bounded rationality.FindingsThe results show that on a low-risk level, if LSI increases the degree of punishment, it will fail to enhance FLSP's operational activeness in the long term; when the risk rises to an intermediate level, a circular game occurs between LSI and FLSP; and on a high level of risk, FLSP will actively take actions, and its functional probability further impacts LSI's strategic choices. Finally, this paper analyzes the moderating impact of punishment intensity and social reputation loss on the evolutionary model in emergencies and provides relevant managerial implications.Originality/valueFirst, by taking both interruption probability and emergencies into consideration, this paper explores the interactions among the factors relevant to LSI's and FLSP's optimal decision-making. Second, this paper analyzes the optimal evolutionary game strategies of LSI and FLSP with different interruption probability and the range of their optimal strategies. Third, the findings of this paper provide valuable implications for relevant practices, such that the punishment intensity and social reputation loss determine the optimal strategies of LSI and FLSP, and thus it is an effective vehicle for LSSC system administrator to achieve the maximum efficiency of the system.

Sign in / Sign up

Export Citation Format

Share Document