Asymmetric and adaptive reward coding via normalized reinforcement learning

Learning is widely modeled in psychology, neuroscience, and computer science by prediction error-guided reinforcement learning (RL) algorithms. While standard RL assumes linear reward functions, reward-related neural activity is a saturating, nonlinear function of reward; however, the computational and behavioral implications of nonlinear RL are unknown. Here, we show that nonlinear RL incorporating the canonical divisive normalization computation introduces an intrinsic and tunable asymmetry in prediction error coding. At the behavioral level, this asymmetry explains empirical variability in risk preferences typically attributed to asymmetric learning rates. At the neural level, diversity in asymmetries provides a computational mechanism for recently proposed theories of distributional RL, allowing the brain to learn the full probability distribution of future rewards. This behavioral and computational flexibility argues for an incorporation of biologically valid value functions in computational models of learning and decision-making.

A-245 So What's the Damage? The Effect of the WISC-V Digitization Debacle

Objective The WISC-V can now be administered in paper format or digitally. Though most subtests are comparable, Processing Speed Index (PSI) subtests, Coding and Symbol Search required complete redesign for digital presentation. We initially collected data to assess comparability of paper versus digital PSI tasks for future use. However, in March of 2020, Pearson issued an alert stating that, due to a programming error, Coding scores may be inflated secondary to timing inaccuracy; they advised against further use of digital Coding. We refocused our analyses to assess the degree to which inaccurate digital Coding impacted overall test results. Method Children with neurological disorders (N=104) received both versions of the PSI subtests (order randomized). Correlational analyses assessed relations between versions, t-tests assessed for administration order effects, and Kappa coefficients assessed agreement across platforms. Results Correlations between paper and digital subtests (r=.570 to .853) and composites (r=.848 to .987) were robust. As expected, Coding was higher digitally (difference=1.91, p < .01, d=.52), but Symbol Search, PSI, and FSIQ were comparable (p>.05). Given evident practice effects, subsequent analyses considered “first administered” versions and score range agreement was best when PSI tasks were administered digitally first (Kappa=.452, p < .001) versus paper first (Kappa=.153, p=.023). Agreement was strong for FSIQ regardless of order (Kappa≥.760, p < .001). Importantly, in highest stakes evaluations (i.e., presence versus absence of intellectual disability), agreement was extraordinarily strong (Kappa≥.93, p < .001). Conclusions Digital Coding scores are inflated in comparison to traditional paper version, but the impact of this programming error was minimal at the level of PSI and FSIQ.

Individual differences in successful self-regulation of the dopaminergic midbrain

The dopaminergic midbrain is associated with brain functions, such as reinforcement learning, motivation and decision-making that are often disturbed in neuropsychiatric disease. Previous research has shown that activity in the dopaminergic midbrain can be endogenously modulated via neurofeedback, suggesting potential for non-pharmacological interventions. However, the robustness of endogenous modulation, a requirement for clinical translation, is unclear. Here, we examined how self-modulation capability relates to regulation transfer. Moreover, to elucidate potential mechanisms underlying successful self-regulation, we studied individual prediction error coding, and, during an independent monetary incentive delay (MID) task, individual reward sensitivity. Fifty-nine participants underwent neurofeedback training either in a veridical or inverted feedback group. Successful self-regulation was associated with post-training activity within the cognitive control network and accompanied by decreasing prefrontal prediction error signals and increased prefrontal reward sensitivity in the MID task. The correlative link of dopaminergic self-regulation with individual differences in prefrontal prediction error and reward sensitivity suggests that reinforcement learning contributes to successful self-regulation. Our findings therefore provide new insights in the control of dopaminergic midbrain activity and pave the way to improve neurofeedback training in neuropsychiatric patients.

Zero-Error Coding via Classical and Quantum Channels in Sensor Networks

Today’s sensor networks need robustness, security and efficiency with a high level of assurance. Error correction is an effective communicational technique that plays a critical role in maintaining robustness in informational transmission. The general way to tackle this problem is by using forward error correction (FEC) between two communication parties. However, by applying zero-error coding one can assure information fidelity while signals are transmitted in sensor networks. In this study, we investigate zero-error coding via both classical and quantum channels, which consist of n obfuscated symbols such as Shannon’s zero-error communication. As a contrast to the standard classical zero-error coding, which has a computational complexity of , a general approach is proposed herein to find zero-error codewords in the case of quantum channel. This method is based on a n-symbol obfuscation model and the matrix’s linear transformation, whose complexity dramatically decreases to . According to a comparison with classical zero-error coding, the quantum zero-error capacity of the proposed method has obvious advantages over its classical counterpart, as the zero-error capacity equals the rank of the quantum coefficient matrix. In particular, the channel capacity can reach n when the rank of coefficient matrix is full in the n-symbol multilateral obfuscation quantum channel, which cannot be reached in the classical case. Considering previous methods such as low density parity check code (LDPC), our work can provide a means of error-free communication through some typical channels. Especially in the quantum case, zero-error coding can reach both a high coding efficiency and large channel capacity, which can improve the robustness of communication in sensor networks.