A model of naturalistic decision making in preference tests

Decisions as to whether to continue with an ongoing activity or to switch to an alternative are a constant in an animal’s natural world, and in particular underlie foraging behavior and performance in food preference tests. Stimuli experienced by the animal both impact the choice and are themselves impacted by the choice, in a dynamic back and forth. Here, we present model neural circuits, based on spiking neurons, in which the choice to switch away from ongoing behavior instantiates this back and forth, arising as a state transition in neural activity. We analyze two classes of circuit, which differ in whether state transitions result from a loss of hedonic input from the stimulus (an “entice to stay” model) or from aversive stimulus-input (a “repel to leave” model). In both classes of model, we find that the mean time spent sampling a stimulus decreases with increasing value of the alternative stimulus, a fact that we linked to the inclusion of depressing synapses in our model. The competitive interaction is much greater in “entice to stay” model networks, which has qualitative features of the marginal value theorem, and thereby provides a framework for optimal foraging behavior. We offer suggestions as to how our models could be discriminatively tested through the analysis of electrophysiological and behavioral data.

An action-oriented perspective on space and affordances

Architects design spaces that offer perceptual cues, affordances, for our various effectivities. Lina Bo Bardi’s São Paulo Museum demonstrates how praxic and contemplative actions are interleaved—space is effective and affective. Navigation often extends beyond wayfinding to support ongoing behavior. Scripts set out the general rules for a particular kind of behavior, and may suggest places that a building must provide. Cognitive maps support wayfinding. Other maps in the brain represent sensory or motor patterns of activity. Juhani Pallasmaa’s reflections on The Thinking Hand lead into a view of how the brain mediates that thinking, modeling hand–eye coordination at two levels. The first coordinates perceptual and motor schemas. The body schema is an adaptable collage of perceptual and motor skills. The second coordinates the ventral “what” pathway that can support planning of actions, and the dorsal “how” pathway that links affordance-related details to motor control. A complementary challenge is understanding how schemas in the head relate to social schemas. Finally, the chapter compares the cognitive challenges in designing a building and in developing a computational brain model of cognitive processes.

Dopamine Neurons Evaluate Natural Fluctuations in Performance Quality

Many motor skills are learned by comparing ongoing behavior to internal performance benchmarks. Dopamine neurons encode performance error in behavioral paradigms where error is externally induced, but it remains unknown if dopamine also signals the quality of natural performance fluctuations. Here we recorded dopamine neurons in singing birds and examined how spontaneous dopamine spiking activity correlated with natural fluctuations in ongoing song. Antidromically identified basal ganglia-projecting dopamine neurons correlated with recent, and not future, song variations, consistent with a role in evaluation, not production. Furthermore, dopamine spiking was suppressed following the production of outlying vocal variations, consistent with a role for active song maintenance. These data show for the first time that spontaneous dopamine spiking can evaluate natural behavioral fluctuations unperturbed by experimental events such as cues or rewards.

Dorsomedial striatal activity tracks completion of behavioral sequences

For proper execution of goal-directed behaviors, individuals require both a general representation of the goal and an ability to monitor their own progress toward that goal. Here, we examine how dorsomedial striatum (DMS), a region pivotal for forming associations among stimuli, actions, and outcomes, encodes the execution of goal-directed action sequences that require self-monitoring of behavior. We trained rats to complete a sequence of at least 5 consecutive lever presses (without visiting the reward port) to obtain a reward and recorded the activity of individual cells in DMS while rats performed the task. We found that the pattern of DMS activity gradually changed during the execution of the sequence, permitting accurate decoding of sequence progress from neural activity at a population level. Moreover, this sequence-related activity was blunted on trials where rats did not complete a sufficient number of presses. Overall, these data suggest a link between DMS activity and the execution of behavioral sequences that require monitoring of ongoing behavior.

Characterizing neural phase-space trajectories via Principal Louvain Clustering

BackgroundWith the growing size and richness of neuroscience datasets in terms of dimension, volume, and resolution, identifying spatiotemporal patterns in those datasets is increasingly important. Multivariate dimension-reduction methods are particularly adept at addressing these challenges.New MethodIn this paper, we propose a novel method, which we refer to as Principal Louvain Clustering (PLC), to identify clusters in a low-dimensional data subspace, based on time-varying trajectories of spectral dynamics across multisite local field potential (LFP) recordings in awake behaving mice. Data were recorded from prefrontal cortex, hippocampus, and parietal cortex in eleven mice while they explored novel and familiar environments.ResultsPLC-identified subspaces and clusters showed high consistency across animals, and were modulated by the animals’ ongoing behavior.ConclusionsPLC adds to an important growing literature on methods for characterizing dynamics in high-dimensional datasets, using a smaller number of parameters. The method is also applicable to other kinds of datasets, such as EEG or MEG.

Multiple time-scales of decision making in the hippocampus and prefrontal cortex

The prefrontal cortex and hippocampus are crucial for memory-guided decision-making. Neural activity in the hippocampus exhibits place-cell sequences at multiple timescales, including slow behavioral sequences (~seconds) and fast theta sequences (~100-200 ms) within theta oscillation cycles. How prefrontal ensembles interact with hippocampal sequences to support decision-making is unclear. Here, we examined simultaneous hippocampal and prefrontal ensemble activity in rats during learning of a spatial working-memory decision task. We found clear theta sequences in prefrontal cortex, nested within its behavioral sequences. In both regions, behavioral sequences maintained representations of current choices during navigation. In contrast, hippocampal theta sequences encoded alternatives for deliberation, and were coordinated with prefrontal theta sequences that predicted upcoming choices. During error trials, these representations were preserved to guide ongoing behavior, whereas replay sequences during inter-trial periods were impaired prior to navigation. These results establish cooperative interaction between hippocampal and prefrontal sequences at multiple timescales for memory-guided decision-making.

Interactions of Medial and Lateral Prefrontal Cortex in Hierarchical Predictive Coding

Cognitive control and decision-making rely on the interplay of medial and lateral prefrontal cortex (mPFC/lPFC), particularly for circumstances in which correct behavior requires integrating and selecting among multiple sources of interrelated information. While the interaction between mPFC and lPFC is generally acknowledged as a crucial circuit in adaptive behavior, the nature of this interaction remains open to debate, with various proposals suggesting complementary roles in (i) signaling the need for and implementing control, (ii) identifying and selecting appropriate behavioral policies from a candidate set, and (iii) constructing behavioral schemata for performance of structured tasks. Although these proposed roles capture salient aspects of conjoint mPFC/lPFC function, none are sufficiently well-specified to provide a detailed account of the continuous interaction of the two regions during ongoing behavior. A recent computational model of mPFC and lPFC, the Hierarchical Error Representation (HER) model, places the regions within the framework of hierarchical predictive coding, and suggests how they interact during behavioral periods preceding and following salient events. In this manuscript, we extend the HER model to incorporate real-time temporal dynamics and demonstrate how the extended model is able to capture single-unit neurophysiological, behavioral, and network effects previously reported in the literature. Our results add to the wide range of results that can be accounted for by the HER model, and provide further evidence for predictive coding as a unifying framework for understanding PFC function and organization.

Neural Networks With Motivation

Animals rely on internal motivational states to make decisions. The role of motivational salience in decision making is in early stages of mathematical understanding. Here, we propose a reinforcement learning framework that relies on neural networks to learn optimal ongoing behavior for dynamically changing motivation values. First, we show that neural networks implementing Q-learning with motivational salience can navigate in environment with dynamic rewards without adjustments in synaptic strengths when the needs of an agent shift. In this setting, our networks may display elements of addictive behaviors. Second, we use a similar framework in hierarchical manager-agent system to implement a reinforcement learning algorithm with motivation that both infers motivational states and behaves. Finally, we show that, when trained in the Pavlovian conditioning setting, the responses of the neurons in our model resemble previously published neuronal recordings in the ventral pallidum, a basal ganglia structure involved in motivated behaviors. We conclude that motivation allows Q-learning networks to quickly adapt their behavior to conditions when expected reward is modulated by agent’s dynamic needs. Our approach addresses the algorithmic rationale of motivation and makes a step toward better interpretability of behavioral data via inference of motivational dynamics in the brain.

Sequence Memory in the Hippocampal–Entorhinal Region

Episodic memories are constructed from sequences of events. When recalling such a memory, we not only recall individual events, but we also retrieve information about how the sequence of events unfolded. Here, we focus on the role of the hippocampal–entorhinal region in processing and remembering sequences of events, which are thought to be stored in relational networks. We summarize evidence that temporal relations are a central organizational principle for memories in the hippocampus. Importantly, we incorporate novel insights from recent studies about the role of the adjacent entorhinal cortex in sequence memory. In rodents, the lateral entorhinal subregion carries temporal information during ongoing behavior. The human homologue is recruited during memory recall where its representations reflect the temporal relationships between events encountered in a sequence. We further introduce the idea that the hippocampal–entorhinal region might enable temporal scaling of sequence representations. Flexible changes of sequence progression speed could underlie the traversal of episodic memories and mental simulations at different paces. In conclusion, we describe how the entorhinal cortex and hippocampus contribute to remembering event sequences—a core component of episodic memory.

Multiple time-scales of decision making in the hippocampus and prefrontal cortex

ABSTRACTThe prefrontal cortex and hippocampus are crucial for memory-guided decision-making. Neural activity in the hippocampus exhibits place-cell sequences at multiple timescales, including slow behavioral sequences (∼seconds) and fast theta sequences (∼100-200 ms) within theta oscillation cycles. How prefrontal ensembles interact with hippocampal sequences to support decision-making is unclear. Here, we examined simultaneous hippocampal and prefrontal ensemble activity during learning of a spatial working-memory decision task. We found clear theta sequences in prefrontal cortex, nested within its behavioral sequences. In both regions, behavioral sequences maintained representations of current goals during navigation. In contrast, hippocampal theta sequences encoded alternatives for deliberation, and were coordinated with prefrontal theta sequences that predicted upcoming choices. During error trials, these representations were preserved to guide ongoing behavior, whereas replay sequences during inter-trial periods were impaired prior to navigation. These results establish cooperative interaction between hippocampal and prefrontal sequences at multiple timescales for memory-guided decision making.