scholarly journals Disentangling reward processes underlying payoff maximization from individual differences in gain frequency bias and reinforcement learning

2021 ◽  
Author(s):  
Pragathi Priyadharsini Balasubramani ◽  
Juan Diaz-Delgado ◽  
Gillian Grennan ◽  
Mariam Zafar-Khan ◽  
Fahad Alim ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Individual Differences ◽  
Human Subjects ◽  
Reward Processing ◽  
Anterior Cingulate ◽  
Beta Activity ◽  
Neural Basis ◽  
Healthy Human ◽  
Specific Symptom ◽  
The Right

Humans make choices based on both reward magnitude and reward frequency. Probabilistic decision making is popularly tested using multi-choice gambling paradigms that require participants to maximize task payoff. However, research shows that performance in such paradigms suffers from individual bias towards the frequency of gains as well as individual differences that mediate reinforcement learning, including attention to stimuli, sensitivity to rewards and risks, learning rate, and exploration vs. exploitation based executive policies. Here, we developed a two-choice reward task, implemented in 186 healthy human subjects across the adult lifespan, to understand the cognitive and neural basis of payoff-based performance. We controlled for individual gain frequency biases using experimental block manipulations and modeled individual differences in reinforcement learning parameters. Simultaneously recorded electroencephalography (EEG)-based cortical activations showed that diminished theta activity in the right rostral anterior cingulate cortex (ACC) as well as diminished beta activity in the right parsorbitalis region of the inferior frontal cortex (IFC) during cumulative reward presentation correspond to better payoff performance. These neural activations further associated with specific symptom self-reports for depression (greater ACC theta) and inattention (greater IFC beta), suggestive of reward processing markers of clinical utility.

scholarly journals Rostral Anterior Cingulate Activations Inversely Relate To Reward Payoff Maximation & Predict Depressed Mood

2021 ◽  
Author(s):  
Pragathi Priyadharsini Balasubramani ◽  
Juan Diaz-Delgado ◽  
Gillian Grennan ◽  
Fahad Alim ◽  
Mariam Zafar-Khan ◽  
...  
Keyword(s):  
Depressed Mood ◽  
Reward Processing ◽  
Anterior Cingulate ◽  
Healthy Human ◽  
Adult Lifespan ◽  
Selection Strategies ◽  
Potential Clinical Utility ◽  
Task Trial ◽  
The Right ◽  
Reward Information

Abstract Choice selection strategies and decision making are typically investigated using multiple-choice gambling paradigms that require participants to maximize reward payoff. However, research shows that performance in such paradigms suffers from individual biases towards the frequency of gains to choose smaller local gains over larger longer term gain, also referred to as melioration. Here, we developed a simple two-choice reward task, implemented in 186 healthy human adult subjects across the adult lifespan to understand the behavioral, computational, and neural bases of payoff maximization versus melioration. The observed reward choice behavior on this task was best explained by a reinforcement learning model of differential future reward prediction. Simultaneously recorded and source-localized electroencephalography (EEG) showed that diminished theta-band activations in the right rostral anterior cingulate cortex (rACC) correspond to greater reward payoff maximization, specifically during the presentation of cumulative reward information at the end of each task trial. Notably, these activations (greater rACC theta) predicted self-reported depressed mood symptoms, thereby showcasing a reward processing marker of potential clinical utility.

scholarly journals The neural representation of facial-emotion categories reflects conceptual structure

2019 ◽  
Vol 116 (32) ◽  
pp. 15861-15870 ◽  
Author(s):  
Jeffrey A. Brooks ◽  
Junichi Chikazoe ◽  
Norihiro Sadato ◽  
Jonathan B. Freeman
Keyword(s):  
Individual Differences ◽  
Emotion Perception ◽  
Conceptual Structure ◽  
Neural Representation ◽  
Facial Emotion ◽  
Neural Basis ◽  
Concept Knowledge ◽  
Cross Cultural Differences ◽  
Specific Emotion ◽  
The Right

Humans reliably categorize configurations of facial actions into specific emotion categories, leading some to argue that this process is invariant between individuals and cultures. However, growing behavioral evidence suggests that factors such as emotion-concept knowledge may shape the way emotions are visually perceived, leading to variability—rather than universality—in facial-emotion perception. Understanding variability in emotion perception is only emerging, and the neural basis of any impact from the structure of emotion-concept knowledge remains unknown. In a neuroimaging study, we used a representational similarity analysis (RSA) approach to measure the correspondence between the conceptual, perceptual, and neural representational structures of the six emotion categories Anger, Disgust, Fear, Happiness, Sadness, and Surprise. We found that subjects exhibited individual differences in their conceptual structure of emotions, which predicted their own unique perceptual structure. When viewing faces, the representational structure of multivoxel patterns in the right fusiform gyrus was significantly predicted by a subject’s unique conceptual structure, even when controlling for potential physical similarity in the faces themselves. Finally, cross-cultural differences in emotion perception were also observed, which could be explained by individual differences in conceptual structure. Our results suggest that the representational structure of emotion expressions in visual face-processing regions may be shaped by idiosyncratic conceptual understanding of emotion categories.

scholarly journals Intelligence moderates reinforcement learning: a mini-review of the neural evidence

2015 ◽  
Vol 113 (10) ◽  
pp. 3459-3461 ◽  
Author(s):  
Chong Chen
Keyword(s):  
Reinforcement Learning ◽  
Prediction Error ◽  
Dorsolateral Prefrontal Cortex ◽  
Anterior Cingulate ◽  
Neural Responses ◽  
Key Factors ◽  
Neural Basis ◽  
Dorsal Anterior Cingulate Cortex ◽  
Neural Signal ◽  
Dorsolateral Prefrontal

Our understanding of the neural basis of reinforcement learning and intelligence, two key factors contributing to human strivings, has progressed significantly recently. However, the overlap of these two lines of research, namely, how intelligence affects neural responses during reinforcement learning, remains uninvestigated. A mini-review of three existing studies suggests that higher IQ (especially fluid IQ) may enhance the neural signal of positive prediction error in dorsolateral prefrontal cortex, dorsal anterior cingulate cortex, and striatum, several brain substrates of reinforcement learning or intelligence.

scholarly journals Shared and unique neural circuitry underlying temporally unpredictable threat and reward processing

2021 ◽  
Author(s):  
Milena Radoman ◽  
Lynne Lieberman ◽  
Jagan Jimmy ◽  
Stephanie M Gorka
Keyword(s):  
Inferior Frontal Gyrus ◽  
Neural Circuitry ◽  
Reward Processing ◽  
Anterior Cingulate ◽  
Neural Systems ◽  
Imaging Data ◽  
Dorsal Anterior Cingulate Cortex ◽  
Posterior Insula ◽  
Middle Occipital Gyrus ◽  
The Right

Abstract Temporally unpredictable stimuli influence behavior across species, as previously demonstrated for sequences of simple threats and rewards with fixed or variable onset. Neuroimaging studies have identified a specific frontolimbic circuit that may become engaged during the anticipation of temporally unpredictable threat (U-threat). However, the neural mechanisms underlying processing of temporally unpredictable reward (U-reward) are incompletely understood. It is also unclear whether these processes are mediated by overlapping or distinct neural systems. These knowledge gaps are noteworthy given that disruptions within these neural systems may lead to maladaptive response to uncertainty. Here, using functional magnetic resonance imaging data from a sample of 159 young adults, we showed that anticipation of both U-threat and U-reward elicited activation in the right anterior insula, right ventral anterior nucleus of the thalamus and right inferior frontal gyrus. U-threat also activated the right posterior insula and dorsal anterior cingulate cortex, relative to U-reward. In contrast, U-reward elicited activation in the right fusiform and left middle occipital gyrus, relative to U-threat. Although there is some overlap in the neural circuitry underlying anticipation of U-threat and U-reward, these processes appear to be largely mediated by distinct circuits. Future studies are needed to corroborate and extend these preliminary findings.

scholarly journals Three-Dimensional Binocular Kinematics of Torsional Vestibular Nystagmus During Convergence on Head-Fixed Targets in Humans

2001 ◽  
Vol 86 (1) ◽  
pp. 113-122 ◽  
Author(s):  
O. Bergamin ◽  
D. Straumann
Keyword(s):  
Human Subjects ◽  
Three Dimensional ◽  
Vertical Component ◽  
Vestibular Stimulation ◽  
Fixed Target ◽  
Healthy Human ◽  
Retinal Slip ◽  
Gain Reduction ◽  
The Right ◽  
Eye Velocity

When a human subject is oscillated about the nasooccipital axis and fixes upon targets along the horizontal head-fixed meridian, angular eye velocity includes a vertical component that increases with the horizontal eccentricity of the line-of-sight. This vertical eye movement component is necessary to prevent retinal slip. We asked whether fixation on a near head-fixed target during the same torsional vestibular stimulation would lead to differences of vertical eye movements between the right and the left eye, as the directions of the two lines-of-sight are not parallel during convergence. Healthy human subjects ( n = 6) were oscillated (0.3 Hz, ±30°) about the nasooccipital axis on a three-dimensional motor-driven turntable. Binocular movements were recorded using the dual search coil technique. A head-fixed laser dot was presented 1.4 m (far head-fixed target) or 0.25 m (near head-fixed target) in front of the right eye. We found highly significant ( P < 0.01) correlations (R binocular = 0.8, monocular = 0.59) between the convergence angle and the difference of the vertical eye velocity between the two eyes. The slope of the fitted linear regression between the two parameters ( s = 0.45) was close to the theoretical slope necessary to prevent vertical retinal slippage (predicted s = 0.5). Covering the left eye did not significantly change the slope ( s = 0.52). In addition, there was a marked gain reduction (∼35%) of the torsional vestibuloocular reflex (VOR) between viewing the far and the near targets, confirming earlier results by others. There was no difference in torsional gain reduction between the two eyes. Lenses of +3 dpt positioned in front of both eyes to decrease the amount of accommodation did not further change the gain of the torsional VOR. In conclusion, ocular convergence on a near head-fixed target during torsional vestibular stimulation leads to deviations in vertical angular velocity between the two eyes necessary to prevent vertical double vision. The vertical deviation velocity is mainly linked to the amount of convergence, since it also occurs during monocular viewing of the near head-fixed target. This suggests that convergence during vestibular stimulation automatically leads to an alignment of binocular rotation axes with the visual axes independent of retinal slip.

scholarly journals Fractionation of muscle activity in rapid responses to startling cues

2017 ◽  
Vol 117 (4) ◽  
pp. 1713-1719 ◽  
Author(s):  
Lauren R. Dean ◽  
Stuart N. Baker
Keyword(s):  
Muscle Activity ◽  
Human Subjects ◽  
Activity Patterns ◽  
Reaction Times ◽  
Principal Component ◽  
Similar Proportion ◽  
Descending Pathways ◽  
Healthy Human ◽  
Wide Range ◽  
The Right

Movements in response to acoustically startling cues have shorter reaction times than those following less intense sounds; this is known as the StartReact effect. The neural underpinnings for StartReact are unclear. One possibility is that startling cues preferentially invoke the reticulospinal tract to convey motor commands to spinal motoneurons. Reticulospinal outputs are highly divergent, controlling large groups of muscles in synergistic patterns. By contrast the dominant pathway in primate voluntary movement is the corticospinal tract, which can access small groups of muscles selectively. We therefore hypothesized that StartReact responses would be less fractionated than standard voluntary reactions. Electromyogram recordings were made from 15 muscles in 10 healthy human subjects as they carried out 32 varied movements with the right forelimb in response to startling and nonstartling auditory cues. Movements were chosen to elicit a wide range of muscle activations. Multidimensional muscle activity patterns were calculated at delays from 0 to 100 ms after the onset of muscle activity and subjected to principal component analysis to assess fractionation. In all cases, a similar proportion of the total variance could be explained by a reduced number of principal components for the startling and the nonstartling cue. Muscle activity patterns for a given task were very similar in response to startling and nonstartling cues. This suggests that movements produced in the StartReact paradigm rely on similar contributions from different descending pathways as those following voluntary responses to nonstartling cues. NEW & NOTEWORTHY We demonstrate that the ability to activate muscles selectively is preserved during the very rapid reactions produced following a startling cue. This suggests that the contributions from different descending pathways are comparable between these rapid reactions and more typical voluntary movements.

scholarly journals Taste, olfactory and food texture reward processing in the brain and the control of appetite

2012 ◽  
Vol 71 (4) ◽  
pp. 488-501 ◽  
Author(s):  
Edmund T. Rolls
Keyword(s):  
Food Reward ◽  
Functional Neuroimaging ◽  
Human Subjects ◽  
Reward Processing ◽  
Anterior Cingulate ◽  
Sensory Inputs ◽  
Word Level ◽  
Food Texture ◽  
Visual Inputs ◽  
Reward Value

Complementary neuronal recordings and functional neuroimaging in human subjects show that the primary taste cortex in the anterior insula provides separate and combined representations of the taste, temperature and texture (including fat texture) of food in the mouth independently of hunger and thus of reward value and pleasantness. One synapse on, in the orbitofrontal cortex (OFC), these sensory inputs are for some neurons combined by learning with olfactory and visual inputs, and these neurons encode food reward in that they only respond to food when hungry, and in that activations correlate with subjective pleasantness. Cognitive factors, including word-level descriptions, and attention modulate the representation of the reward value of food in the OFC and a region to which it projects, the anterior cingulate cortex. Further, there are individual differences in the representation of the reward value of food in the OFC. It is argued that over-eating and obesity are related in many cases to an increased reward value of the sensory inputs produced by foods, and their modulation by cognition and attention that over-ride existing satiety signals. It is proposed that control of all rather than one or several of these factors that influence food reward and eating may be important in the prevention and treatment of overeating and obesity.

Neural Network Configuration and Efficiency Underlies Individual Differences in Spatial Orientation Ability

2014 ◽  
Vol 26 (2) ◽  
pp. 380-394 ◽  
Author(s):  
Aiden E. G. F. Arnold ◽  
Andrea B. Protzner ◽  
Signe Bray ◽  
Richard M. Levy ◽  
Giuseppe Iaria
Keyword(s):  
Individual Differences ◽  
Spatial Orientation ◽  
Primary Motor Cortex ◽  
Brain Regions ◽  
Network Configuration ◽  
Supramarginal Gyrus ◽  
Neural Basis ◽  
Left Hippocampus ◽  
The Right

Spatial orientation is a complex cognitive process requiring the integration of information processed in a distributed system of brain regions. Current models on the neural basis of spatial orientation are based primarily on the functional role of single brain regions, with limited understanding of how interaction among these brain regions relates to behavior. In this study, we investigated two sources of variability in the neural networks that support spatial orientation—network configuration and efficiency—and assessed whether variability in these topological properties relates to individual differences in orientation accuracy. Participants with higher accuracy were shown to express greater activity in the right supramarginal gyrus, the right precentral cortex, and the left hippocampus, over and above a core network engaged by the whole group. Additionally, high-performing individuals had increased levels of global efficiency within a resting-state network composed of brain regions engaged during orientation and increased levels of node centrality in the right supramarginal gyrus, the right primary motor cortex, and the left hippocampus. These results indicate that individual differences in the configuration of task-related networks and their efficiency measured at rest relate to the ability to spatially orient. Our findings advance systems neuroscience models of orientation and navigation by providing insight into the role of functional integration in shaping orientation behavior.

Reductions in phenomenological, physiological and attentional indices of depressive mood after 2 Hz rTMS over the right parietal cortex in healthy human subjects

2003 ◽  
Vol 120 (1) ◽  
pp. 95-101 ◽  
Author(s):  
Jack van Honk ◽  
Dennis J.L.G. Schutter ◽  
Peter Putman ◽  
Edward H.F de Haan ◽  
Alfredo A.L d'Alfonso
Keyword(s):  
Parietal Cortex ◽  
Human Subjects ◽  
Depressive Mood ◽  
Healthy Human ◽  
Healthy Human Subjects ◽  
The Right

scholarly journals Disruption in Surface-Based Functional Connectivity in the Right Posterior Hippocampal CA3 Subfield: A Probable Neural Basis of Visuospatial Working Memory Impairment in Patients With Right Temporal Lobe Epilepsy

2021 ◽  
Vol 12 ◽  
Author(s):  
Zongxia Lv ◽  
Zirong Chen ◽  
Wei Ye ◽  
Xiaomin Pang ◽  
Liluo Nie ◽  
...  
Keyword(s):  
Working Memory ◽  
Functional Connectivity ◽  
Temporal Lobe Epilepsy ◽  
Temporal Lobe ◽  
Right Hemisphere ◽  
Anterior Cingulate ◽  
Visuospatial Working Memory ◽  
Neural Basis ◽  
Right Temporal Lobe Epilepsy ◽  
The Right

Visuospatial working memory (VSWM) impairment is common in patients with right temporal lobe epilepsy (rTLE). The posterior hippocampus is critical for spatial memory, but the contributions of the different subfields to VSWM deficits remain unclear. Forty-six rTLE patients and 42 healthy controls (HCs) were recruited. Resting-state fMRI (rsfMRI) and structural MRI scans were administered, followed by a VSWM_Nback test. The right posterior hippocampus was automatically segmented, and the surface-based functional connectivity (SBFC) of the subiculum (Sub), CA1, CA3, dentate gyrus (DG), hippocampal tail, and right entorhinal cortex (EC) were compared between groups. Correlation analysis was performed between the altered SBFC and VSWM_Nback scores for rTLE patients. The results showed that rTLE patients underperformed in the VSWM_Nback test, with longer mean reaction time of accurate response (ACCmeanRT) in 0back and 2back condition, lower hit rate (HR) and higher false alarm rate (FAR) in 2back condition. Compared with HCs, the rCA3 in the rTLE group exhibited decreased SBFC with inferior parietal cortex (IPC), temporal lateral cortex (TLC), and posterior visual cortex (PVC) in the right hemisphere as well as the bilateral dorsolateral prefrontal cortex (DLPFC). The SBFC of the rEC and right anterior cingulate cortex (rACC) increased in the rTLE group. Within the rTLE group, the decreased SBFC of the rCA3-rIPC and rCA3-rLTC were correlated with worse VSWM performance. Therefore, the decreased SBFC of the rCA3-rIPC and rCA3-rLTC might be the critical aberrant FC pattern reflecting VSWM impairment in rTLE patients. The mechanism might involve functional disruption between the core subsystem and the medial temporal subsystem of the default mode network (DMN).

Sign in / Sign up

Export Citation Format

Share Document