End-to-End Autonomous Exploration with Deep Reinforcement Learning and Intrinsic Motivation

Developing artificial intelligence (AI) agents is challenging for efficient exploration in visually rich and complex environments. In this study, we formulate the exploration question as a reinforcement learning problem and rely on intrinsic motivation to guide exploration behavior. Such intrinsic motivation is driven by curiosity and is calculated based on episode memory. To distribute the intrinsic motivation, we use a count-based method and temporal distance to generate it synchronously. We tested our approach in 3D maze-like environments and validated its performance in exploration tasks through extensive experiments. The experimental results show that our agent can learn exploration ability from raw sensory input and accomplish autonomous exploration across different mazes. In addition, the learned policy is not biased by stochastic objects. We also analyze the effects of different training methods and driving forces on exploration policy.

Linking Behavior, Co-infection Patterns, and Viral Infection Risk With the Whole Gastrointestinal Helminth Community Structure in Mastomys natalensis

Infection probability, load, and community structure of helminths varies strongly between and within animal populations. This can be ascribed to environmental stochasticity or due to individual characteristics of the host such as their age or sex. Other, but understudied, factors are the hosts' behavior and co-infection patterns. In this study, we used the multimammate mouse (Mastomys natalensis) as a model system to investigate how the hosts' sex, age, exploration behavior, and viral infection history affects their infection risk, parasitic load, and community structure of gastrointestinal helminths. We hypothesized that the hosts' exploration behavior would play a key role in the risk for infection by different gastrointestinal helminths, whereby highly explorative individuals would have a higher infection risk leading to a wider diversity of helminths and a larger load compared to less explorative individuals. Fieldwork was performed in Morogoro, Tanzania, where we trapped a total of 214 individual mice. Their exploratory behavior was characterized using a hole-board test after which we collected the helminths inside their gastrointestinal tract. During our study, we found helminths belonging to eight different genera: Hymenolepis sp., Protospirura muricola, Syphacia sp., Trichuris mastomysi, Gongylonema sp., Pterygodermatites sp., Raillietina sp., and Inermicapsifer sp. and one family: Trichostrongylidae. Hierarchical modeling of species communities (HMSC) was used to investigate the effect of the different host-related factors on the infection probability, parasite load, and community structure of these helminths. Our results show that species richness was higher in adults and in females compared to juveniles and males, respectively. Contrary to our expectations, we found that less explorative individuals had higher infection probability with different helminths resulting in a higher diversity, which could be due to a higher exposure rate to these helminths and/or behavioral modification due to the infection.

Heterogeneous selection on exploration behavior within and among West European populations of a passerine bird

Heterogeneous selection is often proposed as a key mechanism maintaining repeatable behavioral variation (“animal personality”) in wild populations. Previous studies largely focused on temporal variation in selection within single populations. The relative importance of spatial versus temporal variation remains unexplored, despite these processes having distinct effects on local adaptation. Using data from >3,500 great tits (Parus major) and 35 nest box plots situated within five West-European populations monitored over 4 to 18 y, we show that selection on exploration behavior varies primarily spatially, across populations, and study plots within populations. Exploration was, simultaneously, selectively neutral in the average population and year. These findings imply that spatial variation in selection may represent a primary mechanism maintaining animal personalities, likely promoting the evolution of local adaptation, phenotype-dependent dispersal, and nonrandom settlement. Selection also varied within populations among years, which may counteract local adaptation. Our study underlines the importance of combining multiple spatiotemporal scales in the study of behavioral adaptation.

How Do Peer Awards Motivate Creative Content? Experimental Evidence from Reddit

We theorize peer awards’ effects on the volume and novelty of creative user-generated content (UGC) produced at online platform communities. We then test our hypotheses via a randomized field experiment on Reddit, wherein we randomly and anonymously assigned Reddit’s Gold Award to 905 users’ posts over a two-month period. We find that peer awards induced recipients to make longer, more frequent posts and that these effects were particularly pronounced among newer community members. Further, we show that recipients were causally influenced to engage in greater (lesser) exploitation (exploration) behavior, producing content that exhibited significantly greater textual similarity to their own past (awarded) content. However, because the effects were most pronounced among new community members, who also produce content that, in general, is systematically more novel than that of established members to begin with, this process yields a desirable outcome: larger volumes of generally novel UGC for the community. This paper was accepted by Chris Forman, information systems.