SNR-based Reinforcement Learning Rate Adaptation for Time Critical Wi-Fi Networks: Assessment through a Calibrated Simulator

Cognitive radio (CR) enables unlicensed users (or secondary users, SUs) to sense for and exploit underutilized licensed spectrum owned by the licensed users (or primary users, PUs). Reinforcement learning (RL) is an artificial intelligence approach that enables a node to observe, learn, and make appropriate decisions on action selection in order to maximize network performance. Routing enables a source node to search for a least-cost route to its destination node. While there have been increasing efforts to enhance the traditional RL approach for routing in wireless networks, this research area remains largely unexplored in the domain of routing in CR networks. This paper applies RL in routing and investigates the effects of various features of RL (i.e., reward function, exploitation, and exploration, as well as learning rate) through simulation. New approaches and recommendations are proposed to enhance the features in order to improve the network performance brought about by RL to routing. Simulation results show that the RL parameters of the reward function, exploitation, and exploration, as well as learning rate, must be well regulated, and the new approaches proposed in this paper improves SUs’ network performance without significantly jeopardizing PUs’ network performance, specifically SUs’ interference to PUs.

Download Full-text

Comparing parameterless learning rate adaptation methods

Proceedings of International Conference on Neural Networks (ICNN'97) ◽

10.1109/icnn.1997.616179 ◽

2002 ◽

Author(s):

E. Fiesler ◽

M. Moreira

Keyword(s):

Rate Adaptation ◽

Learning Rate

Download Full-text

Effect of lysergic acid diethylamide (LSD) on reinforcement learning in humans

10.1101/2020.12.04.412189 ◽

2020 ◽

Author(s):

Jonathan W. Kanen ◽

Qiang Luo ◽

Mojtaba R. Kandroodi ◽

Rudolf N. Cardinal ◽

Trevor W. Robbins ◽

...

Keyword(s):

Reinforcement Learning ◽

Computational Modelling ◽

Lysergic Acid ◽

Learning Rate ◽

Lysergic Acid Diethylamide ◽

Immediate Feedback ◽

Healthy Humans ◽

Learning Rates ◽

Psychedelic Drugs ◽

The Impact

AbstractThe non-selective serotonin 2A (5-HT2A) receptor agonist lysergic acid diethylamide (LSD) holds promise as a treatment for some psychiatric disorders. Psychedelic drugs such as LSD have been suggested to have therapeutic actions through their effects on learning. The behavioural effects of LSD in humans, however, remain largely unexplored. Here we examined how LSD affects probabilistic reversal learning in healthy humans. Conventional measures assessing sensitivity to immediate feedback (“win-stay” and “lose-shift” probabilities) were unaffected, whereas LSD increased the impact of the strength of initial learning on perseveration. Computational modelling revealed that the most pronounced effect of LSD was enhancement of the reward learning rate. The punishment learning rate was also elevated. Increased reinforcement learning rates suggest LSD induced a state of heightened plasticity. These results indicate a potential mechanism through which revision of maladaptive associations could occur.

Download Full-text

Modeling changes in probabilistic reinforcement learning during adolescence

PLoS Computational Biology ◽

10.1371/journal.pcbi.1008524 ◽

2021 ◽

Vol 17 (7) ◽

pp. e1008524

Author(s):

Liyu Xia ◽

Sarah L. Master ◽

Maria K. Eckstein ◽

Beth Baribault ◽

Ronald E. Dahl ◽

...

Keyword(s):

Reinforcement Learning ◽

Learning Task ◽

Learning Rate ◽

Integration Time ◽

Daily Experience ◽

Salivary Testosterone ◽

Probabilistic Uncertainty ◽

Probabilistic Reinforcement ◽

Hierarchical Bayesian Methods ◽

Reinforcement Learning Models

In the real world, many relationships between events are uncertain and probabilistic. Uncertainty is also likely to be a more common feature of daily experience for youth because they have less experience to draw from than adults. Some studies suggest probabilistic learning may be inefficient in youths compared to adults, while others suggest it may be more efficient in youths in mid adolescence. Here we used a probabilistic reinforcement learning task to test how youth age 8-17 (N = 187) and adults age 18-30 (N = 110) learn about stable probabilistic contingencies. Performance increased with age through early-twenties, then stabilized. Using hierarchical Bayesian methods to fit computational reinforcement learning models, we show that all participants’ performance was better explained by models in which negative outcomes had minimal to no impact on learning. The performance increase over age was driven by 1) an increase in learning rate (i.e. decrease in integration time scale); 2) a decrease in noisy/exploratory choices. In mid-adolescence age 13-15, salivary testosterone and learning rate were positively related. We discuss our findings in the context of other studies and hypotheses about adolescent brain development.

Download Full-text

Using reinforcement learning models in social neuroscience: frameworks, pitfalls and suggestions of best practices

Social Cognitive and Affective Neuroscience ◽

10.1093/scan/nsaa089 ◽

2020 ◽

Vol 15 (6) ◽

pp. 695-707 ◽

Cited By ~ 4

Author(s):

Lei Zhang ◽

Lukas Lengersdorff ◽

Nace Mikus ◽

Jan Gläscher ◽

Claus Lamm

Keyword(s):

Reinforcement Learning ◽

Prediction Error ◽

Model Comparison ◽

Hierarchical Modeling ◽

Learning Rate ◽

Neural Activation ◽

Affective Neuroscience ◽

Outcome Valence ◽

Practical Guidelines ◽

Reinforcement Learning Models

Abstract The recent years have witnessed a dramatic increase in the use of reinforcement learning (RL) models in social, cognitive and affective neuroscience. This approach, in combination with neuroimaging techniques such as functional magnetic resonance imaging, enables quantitative investigations into latent mechanistic processes. However, increased use of relatively complex computational approaches has led to potential misconceptions and imprecise interpretations. Here, we present a comprehensive framework for the examination of (social) decision-making with the simple Rescorla–Wagner RL model. We discuss common pitfalls in its application and provide practical suggestions. First, with simulation, we unpack the functional role of the learning rate and pinpoint what could easily go wrong when interpreting differences in the learning rate. Then, we discuss the inevitable collinearity between outcome and prediction error in RL models and provide suggestions of how to justify whether the observed neural activation is related to the prediction error rather than outcome valence. Finally, we suggest posterior predictive check is a crucial step after model comparison, and we articulate employing hierarchical modeling for parameter estimation. We aim to provide simple and scalable explanations and practical guidelines for employing RL models to assist both beginners and advanced users in better implementing and interpreting their model-based analyses.

Download Full-text