Brain dynamics for confidence-weighted learning

Learning in a changing and uncertain environment is a difficult problem. A popular solution is to predict future observations and then use surprising outcomes to update those predictions. However, humans also have a sense of confidence that characterizes the precision of their predictions. Bayesian models use this confidence to regulate learning: for a given surprise, the update is smaller when confidence is higher. We explored the human brain dynamics sub-tending such a confidence-weighting using magneto-encephalography. During our volatile probability learning task, subjects’ confidence reports conformed with Bayesian inference. Several stimulus-evoked brain responses reflected surprise, and some of them were indeed further modulated by confidence. Confidence about predictions also modulated pupil-linked arousal and beta-range (15-30 Hz) oscillations, which in turn modulated specific stimulus-evoked surprise responses. Our results suggest thus that confidence about predictions modulates intrinsic properties of the brain state to amplify or dampen surprise responses evoked by discrepant observations.

Exploration and recency as the main proximate causes of probability matching: a reinforcement learning analysis

ABSTRACTResearch has not yet reached a consensus on why humans match probabilities instead of maximise in a probability learning task. The most influential explanation is that they search for patterns in the random sequence of outcomes. Other explanations, such as expectation matching, are plausible, but do not consider how reinforcement learning shapes people’s choices.We aimed to quantify how human performance in a probability learning task is affected by pattern search and reinforcement learning. We collected behavioural data from 84 young adult participants who performed a probability learning task wherein the majority outcome was rewarded with 0.7 probability, and analysed the data using a reinforcement learning model that searches for patterns. Model simulations indicated that pattern search, exploration, recency (discounting early experiences), and forgetting may impair performance.Our analysis estimated that 85% (95% HDI [76, 94]) of participants searched for patterns and believed that each trial outcome depended on one or two previous ones. The estimated impact of pattern search on performance was, however, only 6%, while those of exploration and recency were 19% and 13% respectively. This suggests that probability matching is caused by uncertainty about how outcomes are generated, which leads to pattern search, exploration, and recency.

Learning of Negative Cue Validities as a Function of Statistical Sophistication and Instruction

The effects of cue-criterion instructions on subjects' achievement, consistency, and matching were examined. The probability-learning task involved two cues which were negatively related to the criterion. Subjects varied in their degree of mathematical training prior to the experiment. On all measures, mathematical sophistication enhanced rate of performance. Increasingly detailed information about cue-criterion relationships and negative linear functions greatly improved level of achievement, demonstrating that subjects can immediately utilize a negative rule if given thorough instruction. Results are discussed with respect to their implications concerning theoretical probability-learning processes and suggestions for improving human decision-making in probabilistic environments.

Probability Matching through Vicarious and Direct Experience

Vicarious and direct exposure were compared on a binary probability learning task, using 204 volunteers. Control subjects performed 150 trials alone. Experimentals were run in dyads: the first subject performed 75 trials while his partner observed; then the observer performed individually on 75 trials. Observation established responding closely comparable to direct participation, and all groups were able to estimate closely the relative event probabilities. An associative view of probability learning was questioned because subjects approached the task with a strong problem-solving set, seeking order in the random sequences despite task directions to the contrary.