The orbitofrontal cortex and the computation of subjective value: consolidated concepts and new perspectives

ABSTRACTNeural representations of value underlie many behaviors that are crucial for survival. Previously, we found that value representations in primate orbitofrontal cortex (OFC) are modulated by attention, specifically, by overt shifts of gaze towards or away from reward-associated visual cues (McGinty et al., 2016). Here, we investigate the influence of overt attention on behavior, by asking how gaze shifts correlate with reward anticipatory responses, and whether activity in OFC mediates this correlation. Macaque monkeys viewed Pavlovian-conditioned appetitive cues on a visual display, while the fraction of time they spent looking towards or away from the cues was measured using an eye tracker. Also measured during cue presentation were the monkeys’ reward anticipation, indicated by conditioned licking responses (CRs), and single neuron activity in OFC. In general, gaze allocation predicted subsequent licking responses: the longer the monkeys spent looking at a cue at a given time point in a trial, the more likely they were to produce an anticipatory CR later in that trial, as if the subjective value of the cue were increased. To address neural mechanisms, mediation analysis measured the extent to which the gaze-CR correlation could be statistically explained by the concurrently recorded firing of single OFC neurons. The resulting mediation effects were indistinguishable from chance. Therefore, while overt attention may increase the subjective value of reward-associated cues (as revealed by anticipatory behaviors), the underlying mechanism remains unknown, as does the functional significance of gaze-driven modulation of OFC value signals.

Download Full-text

Reward-specific satiety affects subjective value signals in orbitofrontal cortex during multicomponent economic choice

Proceedings of the National Academy of Sciences ◽

10.1073/pnas.2022650118 ◽

2021 ◽

Vol 118 (30) ◽

pp. e2022650118

Author(s):

Alexandre Pastor-Bernier ◽

Arkadiusz Stasiak ◽

Wolfram Schultz

Keyword(s):

Orbitofrontal Cortex ◽

Economic Value ◽

Revealed Preference ◽

Value Change ◽

Neuronal Coding ◽

Preference Theory ◽

Subjective Value ◽

Reward Value ◽

Value Loss ◽

Reward Functions

Sensitivity to satiety constitutes a basic requirement for neuronal coding of subjective reward value. Satiety from natural ongoing consumption affects reward functions in learning and approach behavior. More specifically, satiety reduces the subjective economic value of individual rewards during choice between options that typically contain multiple reward components. The unconfounded assessment of economic reward value requires tests at choice indifference between two options, which is difficult to achieve with sated rewards. By conceptualizing choices between options with multiple reward components (“bundles”), Revealed Preference Theory may offer a solution. Despite satiety, choices against an unaltered reference bundle may remain indifferent when the reduced value of a sated bundle reward is compensated by larger amounts of an unsated reward of the same bundle, and then the value loss of the sated reward is indicated by the amount of the added unsated reward. Here, we show psychophysically titrated choice indifference in monkeys between bundles of differently sated rewards. Neuronal chosen value signals in the orbitofrontal cortex (OFC) followed closely the subjective value change within recording periods of individual neurons. A neuronal classifier distinguishing the bundles and predicting choice substantiated the subjective value change. The choice between conventional single rewards confirmed the neuronal changes seen with two-reward bundles. Thus, reward-specific satiety reduces subjective reward value signals in OFC. With satiety being an important factor of subjective reward value, these results extend the notion of subjective economic reward value coding in OFC neurons.

Download Full-text

The case against economic values in the orbitofrontal cortex (or anywhere else in the brain)

10.31234/osf.io/7hgup ◽

2020 ◽

Author(s):

Benjamin Hayden ◽

Yael Niv

Keyword(s):

Orbitofrontal Cortex ◽

Economic Value ◽

Brain Regions ◽

Process Models ◽

Neural Basis ◽

Economic Concept ◽

Free Process ◽

Subjective Value ◽

Epistemic Constraints ◽

The Brain

Much of traditional neuroeconomics proceeds from the hypothesis that value is reified in the brain, that is, that there are neurons or brain regions whose responses serve the discrete purpose of encoding value. This hypothesis is supported by the finding that the activity of many neurons covaries with subjective value as estimated in specific tasks and has led to the idea that the primary function of the orbitofrontal cortex is to compute and signal economic value. Here we consider an alternative: that economic value, in the cardinal, common-currency sense, is not represented in the brain and used for choice by default. This idea is motivated by consideration of the economic concept of value, which places important epistemic constraints on our ability to identify its neural basis. It is also motivated by the behavioral economics literature, especially work on heuristics, which proposes value-free process models for much if not all of choice. Finally, it is buoyed by recent neural and behavioral findings regarding how animals and humans learn to choose between options. In light of our hypothesis, we critically reevaluate putative neural evidence for the representation of value and explore an alternative: direct learning of action policies. We delineate how this alternative can provide a robust account of behavior that concords with existing empirical data.

Download Full-text

Orbitofrontal cortex neurons code utility changes during natural reward consumption as correlates of relative reward-specific satiety

10.1101/2020.07.04.187518 ◽

2020 ◽

Author(s):

Alexandre Pastor-Bernier ◽

Arkadiusz Stasiak ◽

Wolfram Schultz

Keyword(s):

Orbitofrontal Cortex ◽

Revealed Preference ◽

Two Dimensional ◽

Independent Manner ◽

Preference Theory ◽

Economic Choice ◽

Subjective Value ◽

Differential Reward ◽

Specific Alteration ◽

Differential Utility

AbstractNatural, on-going reward consumption can differentially reduce the subjective value (‘utility’) of specific rewards, which indicates relative, reward-specific satiety. Two-dimensional choice indifference curves (IC) represent the utility of choice options with two distinct reward components (‘bundles’) according to Revealed Preference Theory. We estimated two-dimensional ICs from stochastic choices and found that natural on-going consumption of two bundle rewards induced specific IC distortions that indicated differential reduction of reward utility indicative of relative reward-specific satiety. Licking changes confirmed satiety in a mechanism-independent manner. Neuronal signals in orbitofrontal cortex (OFC) that coded the value of the chosen option followed closely the consumption-induced IC distortions within recording periods of individual neurons. A neuronal classifier predicted well the changed utility inferred from the altered behavioral choices. Neuronal signals for more conventional single-reward choice options showed similar relationships to utility alterations from on-going consumption. These results demonstrate a neuronal substrate for the differential, reward-specific alteration of utility by on-going reward consumption reflecting reward-specific satiety.SignificanceRepeated delivery reduces the subjective value (‘utility’) of rewards to different degrees depending on their individual properties, a phenomenon commonly referred to as sensory-specific satiety. We tested monkeys during economic choice of two-component options. On-going consumption differentially reduced reward utility in a way that suggested relative reward-specific satiety between the two components. Neurons in the orbitofrontal cortex (OFC) changed their responses in close correspondence to the differential utility reduction, thus representing a neuronal correlate of relative reward-specific satiety. Control experiments with conventional single-component choice showed similar satiety-induced differential response reductions. These results are compatible with the notion of OFC neurons coding crucial decision variables robustly across different satiety levels.

Download Full-text

Neurons in the Human Left Amygdala Automatically Encode Subjective Value Irrespective of Task

Cerebral Cortex ◽

10.1093/cercor/bhx330 ◽

2017 ◽

Vol 29 (1) ◽

pp. 265-272 ◽

Cited By ~ 3

Author(s):

F Mormann ◽

M Bausch ◽

S Knieling ◽

I Fried

Keyword(s):

Entorhinal Cortex ◽

Orbitofrontal Cortex ◽

Food Preferences ◽

Exact Nature ◽

Parahippocampal Cortex ◽

Subjective Value ◽

Neurosurgical Patients ◽

Valuation Task ◽

Amygdala Activity ◽

Current Value

Abstract The amygdala plays an important role in the computation of internal reward signals. In animals it has been shown to enable a stimulus to indicate the current value of a reinforcer. However, the exact nature of the current value representations in humans remains unknown. Specifically, do neurons of the human amygdala represent current value signals only in tasks requiring valuation? We recorded from 406 neurons in the amygdala, orbitofrontal cortex, parahippocampal cortex, entorhinal cortex, and hippocampus of 6 neurosurgical patients while subjects repeatedly viewed 40 different pictures of sweet or salty “junk food” items in 2 different tasks. Neural activity during stimulus inspection in a valuation task reflected food preferences in the amygdala, orbitofrontal cortex, hippocampus, and entorhinal cortex. Notably, only left amygdala activity represented these food preferences even in a sweet–salty classification task. Valuation signals of the left amygdala thus appear to be stimulus-, not-task driven.

Download Full-text

Single-dimensional human brain signals for two-dimensional economic choice options

10.1101/2020.04.06.028001 ◽

2020 ◽

Author(s):

Leo Chi U Seak ◽

Konstantin Volkmann ◽

Alexandre Pastor-Bernier ◽

Fabian Grabenhorst ◽

Wolfram Schultz

Keyword(s):

Orbitofrontal Cortex ◽

Brain Structures ◽

Two Dimensional ◽

Neural Signals ◽

Economic Choice ◽

Multiple Components ◽

Brain Responses ◽

Subjective Value ◽

Behavioral Preferences ◽

Value Coding

AbstractRewarding choice options typically contain multiple components, but neural signals in single brain voxels are scalar and primarily vary up or down. In a previous study, we had designed reward bundles that contained the same two milkshakes with independently set amounts; we had used psychophysics and rigorous economic concepts to estimate two-dimensional choice indifference curves (IC) that represented revealed stochastic preferences for these bundles in a systematic, integrated manner. All bundles on the same ICs were equally revealed preferred (and thus had same utility, as inferred from choice indifference); bundles on higher ICs (higher utility) were preferred to bundles on lower ICs (lower utility). In the current study, we used the established behavior for testing with functional magnetic resonance imaging (fMRI). We now demonstrate neural responses in reward-related brain structures of human female and male participants, including striatum, midbrain and medial orbitofrontal cortex that followed the characteristic pattern of ICs: similar responses along ICs (same utility despite different bundle composition), but monotonic change across ICs (different utility). Thus, these brain structures integrated multiple reward components into a scalar signal, well beyond the known subjective value coding of single-component rewards.Significance StatementRewards have several components, like the taste and size of an apple, but it is unclear how each component contributes to the overall value of the reward. While choice indifference curves of economic theory provide behavioural approaches to this question, it is unclear whether brain responses capture the preference and utility integrated from multiple components. We report activations in striatum, midbrain and orbitofrontal cortex that follow choice indifference curves representing behavioral preferences over and above variations of individual reward components. In addition, the concept-driven approach encourages future studies on natural, multi-component rewards that are prone to irrational choice of normal and brain-damaged individuals.

Download Full-text

Neural correlates of economic value and valuation context: an event-related potential study

Journal of Neurophysiology ◽

10.1152/jn.00524.2017 ◽

2018 ◽

Vol 119 (5) ◽

pp. 1924-1933 ◽

Cited By ~ 5

Author(s):

John Tyson-Carr ◽

Katerina Kokmotou ◽

Vicente Soto ◽

Stephanie Cook ◽

Nicholas Fallon ◽

...

Keyword(s):

Willingness To Pay ◽

Orbitofrontal Cortex ◽

Economic Value ◽

Event Related Potentials ◽

Anterior Insula ◽

Event Related Potential ◽

Parahippocampal Gyrus ◽

Subjective Value ◽

Valuation Process ◽

Related Potentials

The value of environmental cues and internal states is continuously evaluated by the human brain, and it is this subjective value that largely guides decision making. The present study aimed to investigate the initial value attribution process, specifically the spatiotemporal activation patterns associated with values and valuation context, using electroencephalographic event-related potentials (ERPs). Participants completed a stimulus rating task in which everyday household items marketed up to a price of £4 were evaluated with respect to their desirability or material properties. The subjective values of items were evaluated as willingness to pay (WTP) in a Becker-DeGroot-Marschak auction. On the basis of the individual’s subjective WTP values, the stimuli were divided into high- and low-value items. Source dipole modeling was applied to estimate the cortical sources underlying ERP components modulated by subjective values (high vs. low WTP) and the evaluation condition (value-relevant vs. value-irrelevant judgments). Low-WTP items and value-relevant judgments both led to a more pronounced N2 visual evoked potential at right frontal scalp electrodes. Source activity in right anterior insula and left orbitofrontal cortex was larger for low vs. high WTP at ∼200 ms. At a similar latency, source activity in right anterior insula and right parahippocampal gyrus was larger for value-relevant vs. value-irrelevant judgments. A stronger response for low- than high-value items in anterior insula and orbitofrontal cortex appears to reflect aversion to low-valued item acquisition, which in an auction experiment would be perceived as a relative loss. This initial low-value bias occurs automatically irrespective of the valuation context. NEW & NOTEWORTHY We demonstrate the spatiotemporal characteristics of the brain valuation process using event-related potentials and willingness to pay as a measure of subjective value. The N2 component resolves values of objects with a bias toward low-value items. The value-related changes of the N2 component are part of an automatic valuation process.

Download Full-text

Neuronal Adaptation to the Value Range in the Macaque Orbitofrontal Cortex

10.1101/399071 ◽

2018 ◽

Cited By ~ 1

Author(s):

Katherine E. Conen ◽

Camillo Padoa-Schioppa

Keyword(s):

Orbitofrontal Cortex ◽

Behavioral Context ◽

Minimum Value ◽

Economic Choice ◽

Maximum Value ◽

Baseline Activity ◽

Subjective Value ◽

Baseline Response ◽

Value Range ◽

Decision Circuit

AbstractEconomic choice involves computing and comparing the subjective values of different options. The magnitude of these values can vary immensely in different situations. To compensate for this variability, decision-making neural circuits adapt to the current behavioral context. In orbitofrontal cortex (OFC), neurons encode the subjective value of offered and chosen goods in a quasi-linear way. Previous work found that the gain of the encoding is lower when the value range is wider. However, previous studies did not disambiguate between neurons adapting to the value range or to the maximum value. Furthermore, they did not examine changes in baseline activity. Here we investigated how neurons in the macaque OFC adapt to changes in the value distribution. We found that neurons adapt to both the maximum and the minimum value, but only partially. Concurrently, the baseline response is higher when the minimum value is larger. Using a simulated decision circuit, we showed that higher baseline activity increases choice variability, and thus lowers the expected payoff in high value contexts.

Download Full-text

Base-rate neglect and neural computations for subjective weight in probabilistic inference

10.1101/671396 ◽

2019 ◽

Author(s):

Yun-Yen Yang ◽

Shih-Wei Wu

Keyword(s):

Prefrontal Cortex ◽

Medial Prefrontal Cortex ◽

Orbitofrontal Cortex ◽

Base Rate ◽

Policy Implications ◽

Sources Of Information ◽

Base Rate Neglect ◽

Subjective Weight ◽

Subjective Value ◽

The Brain

AbstractHumans show systematic biases when estimating probability of uncertain events. Base-rate neglect is a well-known bias that describes the tendency to underweight information from the past relative to the present. In this study, we characterized base-rate neglect at the computational and neural implementation levels. At the computational level, we established that base-rate neglect arises from insufficient adjustment to weighting prior information in response to changes in prior variability. At the neural implementation level, we found that orbitofrontal cortex (OFC) and medial prefrontal cortex (mPFC) represent subjective weighting of information that reflects base-rate neglect. Critically, both subjective-weight and subjective-value signals that guide choice were found in mPFC. However, subjective-weight signals preceded subjective-value signals. These results indicate that when facing multiple sources of information, estimation bias such as base-rate neglect arises from information weighting computed in OFC and mPFC, which directly contributes to subjective-value computations that guide decisions under uncertainty.Significance StatementFacing uncertainty, estimating the probability of different potential outcomes carries significant weight in affecting how we act and decide. Decades of research show that humans are prone to giving biased estimation but it remains elusive how these biases arise in the brain. We focus on base-rate neglect, a well-known bias in probability estimation and find that it is tightly associated with activity in the medial prefrontal cortex and orbitofrontal cortex. These regions represent the degree to which human participants weigh different sources of information, suggesting that base-rate neglect arises from information-weighting computations in the brain. As technology provides us the opportunity to seek and gather information at an ever-increasing pace, understanding information-weighting and its biases also carry important policy implications.

Download Full-text