scholarly journals Reward value revealed by auction in rhesus monkeys

2019 ◽  
Author(s):  
Alaa Al-Mohammad ◽  
Wolfram Schultz
Keyword(s):  
Water Budget ◽  
Rhesus Monkeys ◽  
Single Trial ◽  
Economic Choice ◽  
Subjective Value ◽  
Reward Value ◽  
Binary Choices ◽  
Procedural Invariance ◽  
Individual Choices

AbstractEconomic choice is thought to involve the elicitation of the private and subjective values of various choice options. Thus far, the estimation of subjective values in animals has relied upon repeated choices and was expressed as an average from dozens of stochastic decisions. However, decisions are made moment-to-moment, and their consequences are usually felt immediately. Here we describe a Becker-DeGroot-Marschak (BDM) auction-like mechanism that encourages animals to truthfully reveal their subjective value in individual choices. The monkeys reliably placed well-ranked BDM bids for up to five juice volumes while paying from a water budget. The bids closely approximated the average subjective values estimated with conventional binary choices, thus demonstrating procedural invariance and aligning with the wealth of knowledge acquired with these less direct estimates. The feasibility of BDM bidding in monkeys encourages single-trial neuronal studies and bridges the gap to the widely used BDM method in human neuroeconomics.

scholarly journals Corticostriatal Circuits Encode the Subjective Value of Perceived Control

2019 ◽  
Vol 29 (12) ◽  
pp. 5049-5060 ◽  
Author(s):  
Kainan S Wang ◽  
Mauricio R Delgado
Keyword(s):  
Perceived Control ◽  
Well Being ◽  
Control Option ◽  
Subjective Value ◽  
Binary Choices ◽  
Mere Presence ◽  
The Difference ◽  
Neural Underpinnings ◽  
Evoked Activity

AbstractThe ability to perceive and exercise control over an outcome is both desirable and beneficial to our well-being. It has been shown that animals and humans alike exhibit behavioral bias towards seeking control and that such bias recruits the ventromedial prefrontal cortex (vmPFC) and striatum. Yet, this bias remains to be quantitatively captured and studied neurally. Here, we employed a behavioral task to measure the preference for control and characterize its neural underpinnings. Participants made a series of binary choices between having control and no-control over a game for monetary reward. The mere presence of the control option evoked activity in the ventral striatum. Importantly, we manipulated the expected value (EV) of each choice pair to extract the pairing where participants were equally likely to choose either option. The difference in EV between the options at this point of equivalence was inferred as the subjective value of control. Strikingly, perceiving control inflated the reward value of the associated option by 30% and this value inflation was tracked by the vmPFC. Altogether, these results capture the subjective value of perceived control inherent in decision making and highlight the role of corticostriatal circuitry in the perception of control.

scholarly journals The foraging perspective on economic choice

2017 ◽  
Author(s):  
Benjamin Y. Hayden
Keyword(s):  
Self Control ◽  
Binary Choice ◽  
List Type ◽  
Common Basis ◽  
Economic Choice ◽  
Binary Choices ◽  
Foraging Models ◽  
The Relationship ◽  
Economic Choices ◽  
Special Case

AbstractForaging theory offers an alternative foundation for understanding economic choice, one that sees economic choices as the outcome of psychological processes that evolved to help our ancestors search for food. Most of the choices encountered by foragers are between pursuing an encountered prey (accept) or ignoring it in favor of continued search (reject). Binary choices, which typically occur between simultaneously presented items, are special case, and are resolved through paired alternating accept-reject decisions limited by the narrow focus of attention. The foraging approach also holds out promise for helping to understand self-control and invites a reconceptualization of the mechanisms of binary choice, the relationship between choosing and stopping, and of the meaning of reward value.HighlightsForaging provides a basis for modeling economic choice based on adaptivenessForaging choices are accept-reject; foraging models interpret binary choice accordinglyThe foraging view offers a different perspective on self-control decisionsEconomic and stopping decisions may have a common basis

scholarly journals Reward-specific satiety affects subjective value signals in orbitofrontal cortex during multicomponent economic choice

2021 ◽  
Vol 118 (30) ◽  
pp. e2022650118
Author(s):  
Alexandre Pastor-Bernier ◽  
Arkadiusz Stasiak ◽  
Wolfram Schultz
Keyword(s):  
Orbitofrontal Cortex ◽  
Economic Value ◽  
Revealed Preference ◽  
Value Change ◽  
Neuronal Coding ◽  
Preference Theory ◽  
Subjective Value ◽  
Reward Value ◽  
Value Loss ◽  
Reward Functions

Sensitivity to satiety constitutes a basic requirement for neuronal coding of subjective reward value. Satiety from natural ongoing consumption affects reward functions in learning and approach behavior. More specifically, satiety reduces the subjective economic value of individual rewards during choice between options that typically contain multiple reward components. The unconfounded assessment of economic reward value requires tests at choice indifference between two options, which is difficult to achieve with sated rewards. By conceptualizing choices between options with multiple reward components (“bundles”), Revealed Preference Theory may offer a solution. Despite satiety, choices against an unaltered reference bundle may remain indifferent when the reduced value of a sated bundle reward is compensated by larger amounts of an unsated reward of the same bundle, and then the value loss of the sated reward is indicated by the amount of the added unsated reward. Here, we show psychophysically titrated choice indifference in monkeys between bundles of differently sated rewards. Neuronal chosen value signals in the orbitofrontal cortex (OFC) followed closely the subjective value change within recording periods of individual neurons. A neuronal classifier distinguishing the bundles and predicting choice substantiated the subjective value change. The choice between conventional single rewards confirmed the neuronal changes seen with two-reward bundles. Thus, reward-specific satiety reduces subjective reward value signals in OFC. With satiety being an important factor of subjective reward value, these results extend the notion of subjective economic reward value coding in OFC neurons.

scholarly journals Visual fixation patterns during economic choice reflect covert valuation processes that emerge with learning

2019 ◽  
Vol 116 (45) ◽  
pp. 22795-22801 ◽  
Author(s):  
Sean E. Cavanagh ◽  
W. M. Nishantha Malalasekera ◽  
Bruno Miranda ◽  
Laurence T. Hunt ◽  
Steven W. Kennerley
Keyword(s):  
Decision Making ◽  
Vital Role ◽  
The Novel ◽  
Final Choice ◽  
Decision Systems ◽  
Economic Choice ◽  
Binary Choices ◽  
Visual Fixations ◽  
Economic Choices

Visual fixations play a vital role in decision making. Recent studies have demonstrated that the longer subjects fixate an option, the more likely they are to choose it. However, the role of evaluating stimuli covertly (i.e., without fixating them), and how covert evaluations determine where to subsequently fixate, remains relatively unexplored. Here, we trained monkeys to perform a decision-making task where they made binary choices between reward-predictive stimuli which were well-learned (“overtrained”), recently learned (“novel”), or a combination of both (“mixed”). Subjects were free to saccade around the screen and make a choice (via joystick response) at any time. Subjects rarely fixated both options, yet choice behavior was better explained by assuming the values of both stimuli governed choices. The first fixation latency was fast (∼150 ms) but, surprisingly, its direction was value-driven. This suggests covert evaluation of stimulus values prior to first saccade. This was particularly evident for overtrained stimuli. For novel stimuli, first fixations became increasingly value-driven throughout a behavioral session. However, this improvement lagged behind learning of accurate economic choices, suggesting separate processes governed their learning. Finally, mixed trials revealed a strong bias toward fixating the novel stimulus first but no bias toward choosing it. Our results suggest that the primate brain contains fast covert evaluation mechanisms for guiding fixations toward highly valuable and novel information. By employing such covert mechanisms, fixation behavior becomes dissociable from the value comparison processes that drive final choice. This implies that primates use separable decision systems for value-guided fixations and value-guided choice.

Oculomotor capture is influenced by expected reward value but (maybe) not predictiveness

2018 ◽  
Vol 72 (2) ◽  
pp. 168-181 ◽  
Author(s):  
Mike E Le Pelley ◽  
Daniel Pearson ◽  
Alexis Porter ◽  
Hannah Yee ◽  
David Luque
Keyword(s):  
Visual Search ◽  
Eye Movements ◽  
Eye Tracking ◽  
Large Body ◽  
Search Procedure ◽  
Oculomotor Capture ◽  
Critical Variable ◽  
Incentive Value ◽  
Subjective Value ◽  
Reward Value

A large body of research has shown that learning about relationships between neutral stimuli and events of significance – rewards or punishments – influences the extent to which people attend to those stimuli in the future. However, different accounts of this influence differ in terms of the critical variable that is proposed to determine learned changes in attention. We describe two experiments using eye-tracking with a rewarded visual search procedure to investigate whether attentional capture is influenced by the predictiveness of stimuli (i.e., the extent to which they provide information about upcoming events) or by their absolute associative value (i.e., the expected incentive value of the outcome that a stimulus predicts). Results demonstrated a clear influence of associative value on the likelihood that stimuli will capture eye-movements, but the evidence for a distinct influence of predictiveness was less compelling. The results of these experiments can be reconciled within a simple account under which attentional prioritization is a monotonic function of the expected, subjective value of the reward that is signalled by a stimulus.

scholarly journals Single-trial dynamics of hippocampal spatial representations are modulated by reward value

2021 ◽  
Author(s):  
Frédéric Michon ◽  
Esther Krul ◽  
Jyh-Jang Sun ◽  
Fabian Kloosterman
Keyword(s):  
Spatial Representations ◽  
Single Trial ◽  
Reward Value

scholarly journals Neuronal Reward and Decision Signals: From Theories to Data

2015 ◽  
Vol 95 (3) ◽  
pp. 853-951 ◽  
Author(s):  
Wolfram Schultz
Keyword(s):  
Frontal Cortex ◽  
Prediction Error ◽  
Choice Theory ◽  
Dopamine Neurons ◽  
Decision Variable ◽  
Neuronal Populations ◽  
Economic Choice ◽  
Subjective Value ◽  
Decision Mechanisms ◽  
Behavior Choices

Rewards are crucial objects that induce learning, approach behavior, choices, and emotions. Whereas emotions are difficult to investigate in animals, the learning function is mediated by neuronal reward prediction error signals which implement basic constructs of reinforcement learning theory. These signals are found in dopamine neurons, which emit a global reward signal to striatum and frontal cortex, and in specific neurons in striatum, amygdala, and frontal cortex projecting to select neuronal populations. The approach and choice functions involve subjective value, which is objectively assessed by behavioral choices eliciting internal, subjective reward preferences. Utility is the formal mathematical characterization of subjective value and a prime decision variable in economic choice theory. It is coded as utility prediction error by phasic dopamine responses. Utility can incorporate various influences, including risk, delay, effort, and social interaction. Appropriate for formal decision mechanisms, rewards are coded as object value, action value, difference value, and chosen value by specific neurons. Although all reward, reinforcement, and decision variables are theoretical constructs, their neuronal signals constitute measurable physical implementations and as such confirm the validity of these concepts. The neuronal reward signals provide guidance for behavior while constraining the free will to act.

scholarly journals Orbitofrontal cortex neurons code utility changes during natural reward consumption as correlates of relative reward-specific satiety

2020 ◽  
Author(s):  
Alexandre Pastor-Bernier ◽  
Arkadiusz Stasiak ◽  
Wolfram Schultz
Keyword(s):  
Orbitofrontal Cortex ◽  
Revealed Preference ◽  
Two Dimensional ◽  
Independent Manner ◽  
Preference Theory ◽  
Economic Choice ◽  
Subjective Value ◽  
Differential Reward ◽  
Specific Alteration ◽  
Differential Utility

AbstractNatural, on-going reward consumption can differentially reduce the subjective value (‘utility’) of specific rewards, which indicates relative, reward-specific satiety. Two-dimensional choice indifference curves (IC) represent the utility of choice options with two distinct reward components (‘bundles’) according to Revealed Preference Theory. We estimated two-dimensional ICs from stochastic choices and found that natural on-going consumption of two bundle rewards induced specific IC distortions that indicated differential reduction of reward utility indicative of relative reward-specific satiety. Licking changes confirmed satiety in a mechanism-independent manner. Neuronal signals in orbitofrontal cortex (OFC) that coded the value of the chosen option followed closely the consumption-induced IC distortions within recording periods of individual neurons. A neuronal classifier predicted well the changed utility inferred from the altered behavioral choices. Neuronal signals for more conventional single-reward choice options showed similar relationships to utility alterations from on-going consumption. These results demonstrate a neuronal substrate for the differential, reward-specific alteration of utility by on-going reward consumption reflecting reward-specific satiety.SignificanceRepeated delivery reduces the subjective value (‘utility’) of rewards to different degrees depending on their individual properties, a phenomenon commonly referred to as sensory-specific satiety. We tested monkeys during economic choice of two-component options. On-going consumption differentially reduced reward utility in a way that suggested relative reward-specific satiety between the two components. Neurons in the orbitofrontal cortex (OFC) changed their responses in close correspondence to the differential utility reduction, thus representing a neuronal correlate of relative reward-specific satiety. Control experiments with conventional single-component choice showed similar satiety-induced differential response reductions. These results are compatible with the notion of OFC neurons coding crucial decision variables robustly across different satiety levels.

scholarly journals Reward Value Revealed by Auction in Rhesus Monkeys

2020 ◽  
Author(s):  
Alaa Al-Mohammad ◽  
Wolfram Schultz
Keyword(s):  
Rhesus Monkeys ◽  
Reward Value
Sign in / Sign up

Export Citation Format

Share Document