Deep Reinforcement Learning for Visual Semantic Navigation with Memory

In-network caching is one of the key features of information-centric networks (ICN), where forwarding entities in a network are equipped with memory with which they can temporarily store contents and satisfy en route requests. Exploiting in-network caching, therefore, presents the challenge of efficiently coordinating the forwarding of requests with the volatile cache states at the routers. In this paper, we address information-centric networks and consider in-network caching specifically for Named Data Networking (NDN) architectures. Our proposal departs from the forwarding algorithms which primarily use links that have been selected by the routing protocol for probing and forwarding. We propose a novel adaptive forwarding strategy using reinforcement learning with the random neural network (NDNFS-RLRNN), which leverages the routing information and actively seeks new delivery paths in a controlled way. Our simulations show that NDNFS-RLRNN achieves better delivery performance than a strategy that uses fixed paths from the routing layer and a more efficient performance than a strategy that retrieves contents from the nearest caches by flooding requests.

Download Full-text

Spatial preferences account for inter-animal variability during the continual learning of a dynamic cognitive task

10.1101/808006 ◽

2019 ◽

Author(s):

David B. Kastner ◽

Eric A. Miller ◽

Zhounan Yang ◽

Demetris K. Roumis ◽

Daniel F. Liu ◽

...

Keyword(s):

Reinforcement Learning ◽

Systematic Approach ◽

Cognitive Task ◽

Learning Ability ◽

Spatial Alternation ◽

Behavioral Design ◽

Memory Support ◽

With Memory ◽

Alternation Task ◽

Continual Learning

AbstractIndividual animals perform tasks in different ways, yet the nature and origin of that variability is poorly understood. In the context of spatial memory tasks, variability is often interpreted as resulting from differences in memory ability, but the validity of this interpretation is seldom tested since we lack a systematic approach for identifying and understanding factors that make one animal’s behavior different than another. Here we identify such factors in the context of spatial alternation in rats, a task often described as relying solely on memory of past choices. We combine hypothesis-driven behavioral design and reinforcement learning modeling to identify spatial preferences that, when combined with memory, support learning of a spatial alternation task. Identifying these preferences allows us to capture differences among animals, including differences in overall learning ability. Our results show that to understand the complexity of behavior requires quantitative accounts of the preferences of each animal.

Download Full-text

Statistical mechanics approach to a reinforcement learning model with memory

Physica A Statistical Mechanics and its Applications ◽

10.1016/j.physa.2009.01.028 ◽

2009 ◽

Vol 388 (9) ◽

pp. 1849-1856 ◽

Cited By ~ 7

Author(s):

Adam Lipowski ◽

Krzysztof Gontarek ◽

Marcel Ausloos

Keyword(s):

Reinforcement Learning ◽

Statistical Mechanics ◽

Learning Model ◽

With Memory ◽

Reinforcement Learning Model

Download Full-text

Effects of dopamine on reinforcement learning and consolidation in Parkinson’s disease

eLife ◽

10.7554/elife.26801 ◽

2017 ◽

Vol 6 ◽

Cited By ~ 18

Author(s):

John P Grogan ◽

Demitra Tsivos ◽

Laura Smith ◽

Brogan E Knight ◽

Rafal Bogacz ◽

...

Keyword(s):

Parkinson’S Disease ◽

Parkinson's Disease ◽

Reinforcement Learning ◽

Negative Reinforcement ◽

Control Group ◽

Dopaminergic Medication ◽

Matched Healthy Control ◽

Healthy Control ◽

Within Subjects ◽

With Memory

Emerging evidence suggests that dopamine may modulate learning and memory with important implications for understanding the neurobiology of memory and future therapeutic targeting. An influential hypothesis posits that dopamine biases reinforcement learning. More recent data also suggest an influence during both consolidation and retrieval. Eighteen Parkinson’s disease patients learned through feedback ON or OFF medication, with memory tested 24 hr later ON or OFF medication (4 conditions, within-subjects design with matched healthy control group). Patients OFF medication during learning decreased in memory accuracy over the following 24 hr. In contrast to previous studies, however, dopaminergic medication during learning and testing did not affect expression of positive or negative reinforcement. Two further experiments were run without the 24 hr delay, but they too failed to reproduce effects of dopaminergic medication on reinforcement learning. While supportive of a dopaminergic role in consolidation, this study failed to replicate previous findings on reinforcement learning.

Download Full-text