Curiosity-driven exploration: Diversity of mechanisms and functions

Intrinsically motivated information-seeking, also called curiosity-driven exploration, is widely believed to be a key ingredient for autonomous learning in the real world. Such forms of spontaneous exploration have been studied in multiple independent lines of computational research, producing a diverse range of algorithmic models that capture different aspects of these processes. These algorithms resolve some of the limitations of neurocognitive theories by formally describing computational functions and algorithmic implementations of intrinsically motivated learning. Moreover, they reveal a high diversity of effective forms of intrinsically motivated information-seeking that can be characterized along different mechanistic and functional dimensions. This chapter aims at reviewing different classes of algorithms and highlighting several important dimensions of variation among them. Identifying these dimensions provides means for structuring a comprehensive taxonomy of approaches. We believe this exercise to be useful in working towards a general computational account of information-seeking. Such an account should facilitate the proposition of new hypotheses about information-seeking in humans and complement the existing psychological theory of curiosity.

NMDAr BLOCKING BY MK801 ALTERS HIPPOCAMPAL AND PREFRONTAL CORTEX OSCILLATIONS AND IMPAIRS SPATIAL WORKING MEMORY IN MICE.

Abnormal NMDAr function has been linked to rhythmopathies, psychosis, and cognitive dysfunction in schizophrenia. Here, we investigate the role of NMDAr hypofunction in pathological oscillations and behavior. We implanted mice with tetrodes in the dorsal hippocampus and medial prefrontal cortex (mPFC) and administered them with the NMDAr antagonist MK801, recording oscillations during spontaneous exploration in an open field and in the y-maze spatial working memory test. Our results show that NMDAr blockade increased locomotor activity and impaired spatial working memory. The administration of MK801 disrupted the correlation between oscillations and speed of movement, crucial for internal representations of distance. In the hippocampus, MK801 impaired gamma oscillations and theta/gamma coupling, while in the mPFC, it increased the power of theta, gamma, and generated high-frequency oscillations (HFO 155-185 Hz). Spatial working memory tests in the y-maze revealed that theta/gamma coupling was consistently higher in correct trials. Theta/gamma co-modulation mediated by NMDAr function might be key to explain various of schizophrenia's cognitive symptoms. Further investigating NMDAr hypofunction's role in the emergence of aberrant oscillatory activity will improve our understanding of schizophrenia and inspire new treatments of psychiatric disorders.

Thermal modulation of Zebrafish exploratory statistics reveals constraints on individual behavioral variability

Background Variability is a hallmark of animal behavior. It contributes to survival by endowing individuals and populations with the capacity to adapt to ever-changing environmental conditions. Intra-individual variability is thought to reflect both endogenous and exogenous modulations of the neural dynamics of the central nervous system. However, how variability is internally regulated and modulated by external cues remains elusive. Here, we address this question by analyzing the statistics of spontaneous exploration of freely swimming zebrafish larvae and by probing how these locomotor patterns are impacted when changing the water temperatures within an ethologically relevant range. Results We show that, for this simple animal model, five short-term kinematic parameters — interbout interval, turn amplitude, travelled distance, turn probability, and orientational flipping rate — together control the long-term exploratory dynamics. We establish that the bath temperature consistently impacts the means of these parameters, but leave their pairwise covariance unchanged. These results indicate that the temperature merely controls the sampling statistics within a well-defined kinematic space delineated by this robust statistical structure. At a given temperature, individual animals explore the behavioral space over a timescale of tens of minutes, suggestive of a slow internal state modulation that could be externally biased through the bath temperature. By combining these various observations into a minimal stochastic model of navigation, we show that this thermal modulation of locomotor kinematics results in a thermophobic behavior, complementing direct gradient-sensing mechanisms. Conclusions This study establishes the existence of a well-defined locomotor space accessible to zebrafish larvae during spontaneous exploration, and quantifies self-generated modulation of locomotor patterns. Intra-individual variability reflects a slow diffusive-like probing of this space by the animal. The bath temperature in turn restricts the sampling statistics to sub-regions, endowing the animal with basic thermophobicity. This study suggests that in zebrafish, as well as in other ectothermic animals, ambient temperature could be used to efficiently manipulate internal states in a simple and ethological way.

Location and temporal memory of objects declines in aged marmosets (Callithrix jacchus)

Episodic memory decline is an early marker of cognitive aging in human. Although controversial in animals and called “episodic-like memory”, several models have been successfully developed, however they rarely focused on ageing. While marmoset is an emerging primate model in aging science, episodic-like memory has never been tested in this species and importantly in aged marmosets. Here, we examined if the recall of the what-when and what-where building blocks of episodic-like memory declines in ageing marmosets. We developed a naturalistic approach using spontaneous exploration of real objects by young and old marmosets in the home cage. We implemented a three-trial task with 1 week inter-trial interval. Two different sets of identical objects were presented in sample trials 1 and 2, respectively. For the test trial, two objects from each set were presented in a former position and two in a new one. We quantified the exploratory behaviour and calculated discrimination indices in a cohort of 20 marmosets. Young animals presented a preserved memory for combined what-where, and what-when components of the experiment, which declined with aging. These findings lead one to expect episodic-like memory deficits in aged marmosets.

Conspecific Presence Improves Episodic-Like Memory in Rats

A number of studies have provided evidence that animals, including rats, remember past episodes. However, few experiments have addressed episodic-like memory from a social perspective. In the present study, we evaluated Wistar rats in the WWWhen/ELM task as single setups and in dyads, applying a long retention interval. We also investigated behaviors that could subserve the emergence of this type of memory. We found that only rats tested in the social setting were able to recollect an integrated episodic-like memory that lasted 24 h. Additionally, rats in dyads presented higher levels of exploration during the task. When exposed to the testing environment, the dyads exhibited affiliative behavior toward each other and presented fewer anxiety-like responses. Our findings indicate that the presence of a conspecific could act as a facilitating factor in memory evaluations based on spontaneous exploration of objects and provide empirical support for applying more naturalistic settings in investigations of episodic-like memory in rats.

No preference for direct versus averted gaze in autistic adults: a reinforced preferential looking paradigm

Background With the overarching objective to gain better insights into social attention in autistic adults, the present study addresses three outstanding issues about face processing in autism. First, do autistic adults display a preference for mouths over eyes; second, do they avoid direct gaze; third, is atypical visual exploration of faces in autism mediated by gender, social anxiety or alexithymia? Methods We used a novel reinforced preferential looking paradigm with a group of autistic adults (n = 43, 23 women) pairwise matched on age with neurotypical participants (n = 43, 21 women). Participants watched 28 different pairs of 5 s video recordings of a speaking person: the two videos, simultaneously displayed on the screen, were identical except that gaze was directed at the camera in one video and averted in the other. After a 680 ms transition phase, a short reinforcement animation appeared on the side that had displayed the direct gaze. Results None of the groups showed a preference for mouths over eyes. However, neurotypical participants fixated significantly more the stimuli with direct gaze, while no such preference emerged in autistic participants. As the experiment progressed, neurotypical participants also increasingly anticipated the appearance of the reinforcement, based on the location of the stimulus with the direct gaze, while no such anticipation emerged in autistic participants. Limitations Our autistic participants scored higher on the social anxiety and alexithymia questionnaires than neurotypicals. Future studies should match neurotypical and autistic participants on social anxiety and alexithymia and complement questionnaires with physiological measures of anxiety. Conclusions The absence of preference for direct versus averted gaze in the autistic group is probably due to difficulties in distinguishing eye gaze direction, potentially linked to a reduced spontaneous exploration or avoidance of the eye region. Social attention and preference for direct versus averted gaze correlated with alexithymia and social anxiety scores, but not gender.

Lactobacillus johnsonii BS15 improves intestinal environment against fluoride-induced memory impairment in mice—a study based on the gut–brain axis hypothesis

Background Excessive fluoride can lead to chronic neurodegeneration characterized by neuron and myelin loss and memory dysfunction. The gut–brain axis hypothesis suggests that gut microbiota plays a crucial role in regulating brain function. Thus, using probiotics to adjust the gut microenvironment may be a potential therapy for mental diseases. Methods Mice in the prob group were administrated with Lactobacillus johnsonii BS15 for 28 days prior to and throughout a 70-day exposure to sodium fluoride. The drinking water of all groups (F and prob groups) except the control group were replaced by high-fluoride water (100 mg NaF/L) on day 28. Animals in each group were divided into two subsets: one underwent behavioral test, and the other was sacrificed for sampling. The mRNA expression level and protein content related to inflammatory reaction in the ileum and hippocampus were respectively detected by reverse transcription quantitative polymerase chain reaction (RT-qPCR) and enzyme-linked immunosorbent assay (ELISA). The mRNA expression levels of proteins related to myelin structure, apoptosis, and memory in the hippocampus and tight junction proteins in the ileum were determined by RT-qPCR and/or immunohistochemistry. Gut permeability markers (D-lactate and diamine oxidase (DAO)) in the serum were also examined by ELISA. Results The results showed that fluoride exposure induced a lower spontaneous exploration (P < 0.05) in T-maze test, which indicated an impairment of memory. Spontaneous exploration of BS15-treated mice was significantly higher (P < 0.05) than that in F group. Fluoride reduced (P < 0.05) levels of myelin structural protein (proteolipid protein) and neurogenesis-associated proteins (brain-derived neurotrophic factor and cAMP/Ca2+ responsive element-binding protein), induced disordered inflammatory cytokines (TNF-α, IFN-γ, and IL-6; P < 0.05), increased pro-apoptotic genes (caspase-3; P < 0.05), and decreased anti-apoptotic genes (Bcl-2; P < 0.05) in the hippocampus, of which the influences were reversed by BS15. BS15 treatment exerted significant preventive effects on reversing the gut inflammation induced by excessive fluoride intake by reducing (P < 0.05) the levels of pro-inflammatory cytokines (tumor necrosis factor-alpha (TNF-α) and interferon-gamma (IFN-γ)) and remarkably increasing (P < 0.05) the level of anti-inflammatory cytokines (IL-10). Moreover, the serum DAO activity and D-lactate concentration significantly increased by fluoride were also reduced (P < 0.05) by BS15. This result indicated the profitable effect of BS15 on gut permeability. Conclusion L. johnsonii BS15 intake could benefit the neuroinflammation and demyelination in the hippocampus by improving the gut environment and ameliorating fluorine-induced memory dysfunction.

A Systematic Comparison of Perceptual Performance in Softness Discrimination with Different Fingers

In studies investigating haptic softness perception, participants are typically instructed to explore soft objects by indenting them with their index finger. In contrast, performance with other fingers has rarely been investigated. We wondered which fingers are used in spontaneous exploration and if performance differences between fingers can explain spontaneous usage. In Experiment 1 participants discriminated the softness of two rubber stimuli with hardly any constraints on finger movements. Results indicate that humans use successive phases of different fingers and finger combinations during an exploration, preferring index, middle, and (to a lesser extent) ring finger. In Experiment 2 we compared discrimination thresholds between conditions, with participants using one of the four fingers of the dominant hand. Participants compared the softness of rubber stimuli in a two-interval forced choice discrimination task. Performance with index and middle finger was better as compared to ring and little finger, the little finger was the worst. In Experiment 3 we again compared discrimination thresholds, but participants were told to use constant peak force. Performance with the little finger was worst, whereas performance for the other fingers did not differ. We conclude that in spontaneous exploration the preference of combinations of index, middle, and partly ring finger seems to be well chosen, as indicated by improved performance with the spontaneously used fingers. Better performance seems to be based on both different motor abilities to produce force, mainly linked to using index and middle finger, and different sensory sensitivities, mainly linked to avoiding the little finger.

Mechanics of exploration in Drosophila melanogaster

The Drosophila larva executes a stereotypical exploratory routine that appears to consist of stochastic alternation between straight peristaltic crawling and reorientation events through lateral bending. We present a model of larval mechanics for axial and transverse motion over a planar substrate, and use it to develop a simple, reflexive neuromuscular model from physical principles. In the absence of damping and driving, the mechanics of the body produces axial travelling waves, lateral oscillations, and unpredictable, chaotic deformations. The neuromuscular system counteracts friction to recover these motion patterns, giving rise to forward and backward peristalsis in addition to turning. The model produces spontaneous exploration, even though the model nervous system has no intrinsic pattern generating or decision making ability, and neither senses nor drives bending motions. Ultimately, our model suggests a novel view of larval exploration as a deterministic superdiffusion process which is mechanistically grounded in the chaotic mechanics of the body.