Distinct hippocampal network states support theta phase precession and theta sequences in CA1

The hippocampus likely uses temporal coding to represent complex memories via mechanisms such as theta phase precession and theta sequences. Theta sequences are rapid sweeps of spikes from multiple place cells, encoding past or planned trajectories or non-spatial information. Phase precession, the correlation between a place cell's theta firing phase and animal position has been suggested to facilitate sequence emergence. We find that CA1 phase precession varies strongly across cells and environmental contingencies. Phase precession depends on the CA1 network state, and is only present when the medium gamma oscillation (60-90 Hz, linked to Entorhinal inputs) dominates. Conversely, theta sequences are most evident for non-precessing cells or with leading slow gamma (20-45 Hz, linked to CA3 inputs). These results challenge the view that phase precession is the mechanism underlying the emergence of theta sequences and point at a 'dual network states' model for hippocampal temporal code, potentially supporting merging of memory and exogenous information in CA1.

Directional Tuning of Phase Precession Properties in the Hippocampus

Running direction in the hippocampus is encoded by rate modulations of place field activity but also by spike timing correlations known as theta sequences. Whether directional rate codes and the directionality of place field correlations are related, however, has so far not been explored and therefore the nature of how directional information is encoded in the cornu ammonis remains unresolved. Here, using a previously published dataset that contains the spike activity of rat hippocampal place cells in the CA1, CA2 and CA3 subregions during free foraging of male Long-Evans rats in a 2D environment, we found that rate and spike timing codes are related. Opposite to a place field's preferred firing rate direction spikes are more likely to undergo theta phase precession and, hence, more strongly impact paired correlations. Furthermore, we identified a subset of field pairs whose theta correlations are intrinsic in that they maintain the same firing order when the running direction is reversed. Both effects are associated with differences in theta phase distributions, and are more prominent in CA3 than CA1. We thus hypothesize that intrinsic spiking is most prominent when the directionally modulated sensory-motor drive of hippocampal firing rates is minimal, suggesting that extrinsic and intrinsic sequences contribute to phase precession as two distinct mechanisms.

Scalable representation of time in the hippocampus

Hippocampal “time cells” encode specific moments of temporally organized experiences that may support hippocampal functions for episodic memory. However, little is known about the reorganization of the temporal representation of time cells during changes in temporal structures of episodes. We investigated CA1 neuronal activity during temporal bisection tasks, in which the sets of time intervals to be discriminated were designed to be extended or contracted across the blocks of trials. Assemblies of neurons encoded elapsed time during the interval, and the representation was scaled when the set of interval times was varied. Theta phase precession and theta sequences of time cells were also scalable, and the fine temporal relationships were preserved between pairs in theta cycles. Moreover, theta sequences reflected the rats’ decisions on the basis of their time estimation. These findings demonstrate that scalable features of time cells may support the capability of flexible temporal representation for memory formation.

Phase precession in the human hippocampus and entorhinal cortex

Knowing where we are, where we have been, and where we are going is critical to many behaviors, including navigation and memory. One potential neuronal mechanism underlying this ability is phase precession, in which spatially tuned neurons represent sequences of positions by activating at progressively earlier phases of local network theta (~5–10 Hz) oscillations. Phase precession may be a general neural pattern for representing sequential events for learning and memory. However, phase precession has never been observed in humans. By recording human single-neuron activity during spatial navigation, we show that spatially tuned neurons in the human hippocampus and entorhinal cortex exhibit phase precession. Furthermore, beyond the neural representation of locations, we show evidence for phase precession related to specific goal-states. Our findings thus extend theta phase precession to humans and suggest that this phenomenon has a broad functional role for the neural representation of both spatial and non-spatial information.

Biological Reinforcement Learning via Predictive Spacetime Encoding

Recent advances in reinforcement learning (RL) have successfully addressed several challenges, such as performance, scalability, or sample efficiency associated with the use of this technology. Although RL algorithms bear relevance to psychology and neuroscience in a broader context, they lack biological plausibility. Motivated by recent neural findings demonstrating the capacity of the hippocampus and prefrontal cortex to gather space and time information from the environment, this study presents a novel RL model, called spacetime Q-Network (STQN), that exploits predictive spatiotemporal encoding to reliably learn highly uncertain environment. The proposed method consists of two primary components. The first component is the successor representation with theta phase precession implements hippocampal spacetime encoding, acting as a rollout prediction. The second component, called Q switch ensemble, implements prefrontal population coding for reliable reward prediction. We also implement a single learning rule to accommodate both hippocampal-prefrontal replay and synaptic homeostasis, which subserves confidence-based metacognitive learning. To demonstrate the capacity of our model, we design a task array simulating various levels of environmental uncertainty and complexity. Results show that our model significantly outperforms a few state-of-the-art RL models. In the subsequent ablation study, we showed unique contributions of each component to resolving task uncertainty and complexity. Our study has two important implications. First, it provides the theoretical groundwork for closely linking unique characteristics of the distinct brain regions in the context of RL. Second, our implementation is performed in a simple matrix form that accommodates expansion into biologically-plausible, highly-scalable, and generalizable neural architectures.

Enhancement of Hippocampal Spatial Decoding Using a Dynamic Q-Learning Method With a Relative Reward Using Theta Phase Precession

Hippocampal place cells and interneurons in mammals have stable place fields and theta phase precession profiles that encode spatial environmental information. Hippocampal CA1 neurons can represent the animal’s location and prospective information about the goal location. Reinforcement learning (RL) algorithms such as Q-learning have been used to build the navigation models. However, the traditional Q-learning ([Formula: see text]Q-learning) limits the reward function once the animals arrive at the goal location, leading to unsatisfactory location accuracy and convergence rates. Therefore, we proposed a revised version of the Q-learning algorithm, dynamical Q-learning ([Formula: see text]Q-learning), which assigns the reward function adaptively to improve the decoding performance. Firing rate was the input of the neural network of [Formula: see text]Q-learning and was used to predict the movement direction. On the other hand, phase precession was the input of the reward function to update the weights of [Formula: see text]Q-learning. Trajectory predictions using [Formula: see text]Q- and [Formula: see text]Q-learning were compared by the root mean squared error (RMSE) between the actual and predicted rat trajectories. Using [Formula: see text]Q-learning, significantly higher prediction accuracy and faster convergence rate were obtained compared with [Formula: see text]Q-learning in all cell types. Moreover, combining place cells and interneurons with theta phase precession improved the convergence rate and prediction accuracy. The proposed [Formula: see text]Q-learning algorithm is a quick and more accurate method to perform trajectory reconstruction and prediction.

Learning long temporal sequences in spiking networks by multiplexing neural oscillations

Many cognitive and behavioral tasks – such as interval timing, spatial navigation, motor control and speech – require the execution of precisely-timed sequences of neural activation that cannot be fully explained by a succession of external stimuli. We show how repeatable and reliable patterns of spatiotemporal activity can be generated in chaotic and noisy spiking recurrent neural networks. We propose a general solution for networks to autonomously produce rich patterns of activity by providing a multi-periodic oscillatory signal as input. We show that the model accurately learns a variety of tasks, including speech generation, motor control and spatial navigation. Further, the model performs temporal rescaling of natural spoken words and exhibits sequential neural activity commonly found in experimental data involving temporal processing. In the context of spatial navigation, the model learns and replays compressed sequences of place cells and captures features of neural activity such as the emergence of ripples and theta phase precession. Together, our findings suggest that combining oscillatory neuronal inputs with different frequencies provides a key mechanism to generate precisely timed sequences of activity in recurrent circuits of the brain.

Spatial cell firing during virtual navigation of open arenas by head-restrained mice

We present a mouse virtual reality (VR) system which restrains head-movements to horizontal rotations, compatible with multi-photon imaging. This system allows expression of the spatial navigation and neuronal firing patterns characteristic of real open arenas (R). Comparing VR to R: place and grid, but not head-direction, cell firing had broader spatial tuning; place, but not grid, cell firing was more directional; theta frequency increased less with running speed, whereas increases in firing rates with running speed and place and grid cells' theta phase precession were similar. These results suggest that the omni-directional place cell firing in R may require local-cues unavailable in VR, and that the scale of grid and place cell firing patterns, and theta frequency, reflect translational motion inferred from both virtual (visual and proprioceptive) and real (vestibular translation and extra-maze) cues. By contrast, firing rates and theta phase precession appear to reflect visual and proprioceptive cues alone.