The probability of reinforcement per trial affects post-trial responding and subsequent extinction, but not within-trial responding

During magazine approach conditioning, rats do not discriminate between a conditioned stimulus (CS) that is consistently reinforced with food and a CS that is occasionally (partially) reinforced, as long as the CSs have the same overall reinforcement rate per second. This implies that rats are indifferent to the probability of reinforcement per trial. However, in the same rats, the per-trial reinforcement rate will affect subsequent extinction—responding extinguishes more rapidly for a CS that was consistently reinforced than for a partially reinforced CS. Here, we trained rats with consistently and partially reinforced CSs that were matched for overall reinforcement rate per second. We measured conditioned responding both during and immediately after the CSs. Differences in the per-trial probability of reinforcement did not affect the acquisition of responding during the CS, but did affect subsequent extinction of that responding, and also affected the post-CS response rates during conditioning. Indeed, CSs with the same probability of reinforcement per trial evoked the same amount of post-CS responding even when they differed in overall reinforcement rate and thus evoked different amounts of responding during the CS. We conclude that reinforcement rate per second controls rats’ acquisition of responding during the CS, but at the same time rats also learn specifically about the probability of reinforcement per trial. The latter learning affects the rats’ expectation of reinforcement as an outcome of the trial, which influences their ability to detect retrospectively that an opportunity for reinforcement was missed, and in turn drives extinction.

The Partial Reinforcement Extinction Effect Depends on Learning about Non-Reinforced Trials Rather than Reinforcement Rate

Conditioned responding extinguishes more slowly after partial (inconsistent) reinforcement than after consistent reinforcement. This Partial Reinforcement Extinction Effect (PREE) is usually attributed to learning about nonreinforcement during the partial schedule. An alternative explanation attributes it to any difference in the rate of reinforcement, arguing that animals can detect the change to nonreinforcement more quickly after a denser schedule than a leaner schedule. Experiments 1a and 1b compared extinction of magazine responding to a conditioned stimulus (CS) reinforced with one food pellet per trial and a CS reinforced with two pellets per trial. Despite the difference in reinforcement rate, there was no reliable difference in extinction. Both experiments did demonstrate the conventional PREE comparing a partial CS (50% reinforced) with a consistent CS. Experiments 2 and 3 tested whether the PREE depends specifically on learning about nonreinforced trials during partial reinforcement. Rats were trained with two CS configurations, A and AX. One was partially reinforced, the other consistently reinforced. When AX was partial and A consistent, responding to AX extinguished more slowly than to A. When AX was consistent and A was partial, there was no difference in their extinction. Therefore, pairing X with partial reinforcement allowed rats to show a PREE to AX that did not generalise to A. Pairing A with partial reinforcement meant that rats showed a PREE to A that generalised to AX. Thus, the PREE depends on learning about nonreinforced trials during partial reinforcement and is not due to any difference in per-trial probability of reinforcement

The Partial Reinforcement Extinction Effect: The Proportion of Trials Reinforced During Conditioning Predicts the Number of Trials to Extinction

Four experiments compared the extinction of responding to a continuously reinforced (CRf) conditioned stimulus (CS) consistently reinforced on every trial, with extinction of responding to a partially reinforced (PRf) CS that had been inconsistently reinforced. To equate the acquisition of responding between the two CSs, the average duration of the CRf CS was extended so that it scheduled the same overall rate of reinforcement per unit time as the PRf CS. Experiment 1 used a within-subjects design to compare the rates of extinction for a 10-s PRf CS reinforced on 33% of trials versus a 30-s CRf CS. Experiment 2 made the same comparison but using a between-subjects design. Experiment 3 compared extinction in a group trained with a 10-s PRf CS reinforced on 20% of trials and a group trained with a 50-s CRf CS. Experiment 4 compared the rates of extinction following two partial reinforcement schedules, a 10-s PRf CS reinforced on 33% of trial versus a 20-s CRf CS reinforced on 66% of trials. In each experiment, responding took longer to extinguish for the CS that scheduled a lower per-trial probability of reinforcement. Modelling of individual extinction curves using Weibull functions indicated that the latency to initiate extinction was directly related to the per-trial probability of reinforcement learned during acquisition. For example, compared to training with a CRf CS, rats reinforced on 33% of trials took approximately three times as many trials to initiate extinction, and rats reinforced on 20% of trials took five times as many trials to initiate extinction. These results provide support for trial-based accounts of extinction (e.g. Capaldi, 1967), whereby rats learn about the expected number of trials per reinforcer, and extinction depends on the number of expected reinforcers that have been omitted rather than on the number of extinction trials per se.

Universal Sample Size Invariant Measures for Uncertainty Quantification in Density Estimation

Previously, we developed a high throughput non-parametric maximum entropy method (PLOS ONE, 13(5): e0196937, 2018) that employs a log-likelihood scoring function to characterize uncertainty in trial probability density estimates through a scaled quantile residual (SQR). The SQR for the true probability density has universal sample size invariant properties equivalent to sampled uniform random data (SURD). Alternative scoring functions are considered that include the Anderson-Darling test. Scoring function effectiveness is evaluated using receiver operator characteristics to quantify efficacy in discriminating SURD from decoy-SURD, and by comparing overall performance characteristics during density estimation across a diverse test set of known probability distributions.

Supergalactic Structure of Multiplets with the Telescope Array Surface Detector

Evidence of supergalactic structure of multiplets has been found for ultra-high energy cosmic rays (UHECR) with energies above 1019 eV using 7 years of data from the Telescope Array (TA) surface detector. The tested hypothesis is that UHECR sources, and intervening magnetic fields, may be correlated with the supergalactic plane, as it is a fit to the average matter density within the GZK horizon. This structure is measured by the average behavior of the strength of intermediate-scale correlations between event energy and position (multiplets). These multiplets are measured in wedge-like shapes on the spherical surface of the fieldof-view to account for uniform and random magnetic fields. The evident structure found is consistent with toy-model simulations of a supergalactic magnetic sheet and the previously published Hot/Coldspot results of TA. The post-trial probability of this feature appearing by chance, on an isotropic sky, is found by Monte Carlo simulation to be ~4.5σ.

Reaction Time of Erroneous Responses in the Go/No-Go Paradigm

Reaction time (RT) of erroneous responses in go/no-go tasks tends to be shorter than RT of correct responses. An opposite difference has been reported ( Halperin, Wolf, Greenblatt, & Young, 1991 ) which could be attributed to differences in go trial probability, or to high memory demand. Two experiments aimed here to test these two explanations, a simultaneous matching task with low memory load (Experiment 1), and a sequential matching task with high memory load (Experiment 2). Go trial probability was also manipulated. Short false positive RT was obtained only in the sequential matching task with high go trial probability, while long false positive RT was obtained in the other three conditions. Low go trial probability and high memory load were both found to be sufficient, by themselves, to create long false positives attributable to confusion. Short false positives in the high go trial probability/low memory load condition were attributed to failure of response inhibition.