Modelling concrete and abstract concepts using brain-constrained deep neural networks

A neurobiologically constrained deep neural network mimicking cortical areas relevant for sensorimotor, linguistic and conceptual processing was used to investigate the putative biological mechanisms underlying conceptual category formation and semantic feature extraction. Networks were trained to learn neural patterns representing specific objects and actions relevant to semantically ‘ground’ concrete and abstract concepts. Grounding sets consisted of three grounding patterns with neurons representing specific perceptual or action-related features; neurons were either unique to one pattern or shared between patterns of the same set. Concrete categories were modelled as pattern triplets overlapping in their ‘shared neurons’, thus implementing semantic feature sharing of all instances of a category. In contrast, abstract concepts had partially shared feature neurons common to only pairs of category instances, thus, exhibiting family resemblance, but lacking full feature overlap. Stimulation with concrete and abstract conceptual patterns and biologically realistic unsupervised learning caused formation of strongly connected cell assemblies (CAs) specific to individual grounding patterns, whose neurons were spread out across all areas of the deep network. After learning, the shared neurons of the instances of concrete concepts were more prominent in central areas when compared with peripheral sensorimotor ones, whereas for abstract concepts the converse pattern of results was observed, with central areas exhibiting relatively fewer neurons shared between pairs of category members. We interpret these results in light of the current knowledge about the relative difficulty children show when learning abstract words. Implications for future neurocomputational modelling experiments as well as neurobiological theories of semantic representation are discussed.

Locality in the acquisition of object A’-dependencies: insights from French

Children’s difficulties with dependencies involving movement of an object to the left periphery of the clause (object relative clauses/RCs and wh-questions), have been explained in terms of intervention effects arising when the moved object and the intervening subject share a lexical N feature (Friedmann, Belletti & Rizzi 2009). Such an account raises various questions: (1) Do these effects hold in the absence of a lexical N feature when the object and the intervener share other relevant features? (2) Do phi-features with a semantic role modulate such effects? (3) Does the degree of feature overlap determine a gradience in performance? We addressed these in three sentence-picture matching studies with French-speaking children (4;8 to 6;3), by assessing comprehension of (1) subject and object RCs headed by the demonstrative pronouns celui/celle and matching or mismatching in number; (2) object RCs headed by a lexical N and matching or mismatching in animacy; (3) object who- and which-questions. Our results show that mismatches in number, not in animacy, enhance comprehension of object RCs, even in the absence of a lexical N feature, and confirm previous findings that object who-questions yield better comprehension than object which-questions. Comparing across studies, the following gradation emerges with respect to performance accuracy: disjunction > intersection > inclusion. The global interpretation of these findings is that fine-grained phi-features determining movement are both sufficient and necessary for locality, and the degree of overlap of these features can capture the pattern of performance observed in children, namely higher accuracy as featural differences increase.

Training-dependent transfer within a set of nested tasks

Extended practice on a particular cognitive task can boost the performance of other tasks, even though they themselves have not been practised. This transfer of benefits appears to be specific, occurring most when tasks are very similar to those being trained. But what type of similarity is most important for predicting transfer? This question is addressed with a tightly controlled randomised design, with a relatively large sample ( N = 175) and an adaptive control group. We created a hierarchical set of nested assessment tasks. Participants then trained on two of the tasks: one was relatively “low” in the hierarchy requiring just simultaneous judgements of shapes’ spikiness, whereas the other was relatively “high” requiring delayed judgements of shapes’ spikiness or number of spikes in a switching paradigm. Using the full complement of nested tasks before and after training, we could then test whether and how these “low” and “high” training effects cascade through the hierarchy. For both training groups, relative to the control, whether or not an assessment task shared a single specific feature was the best predictor of transfer patterns. For the low-level training group, the overall proportion of feature overlap also significantly predicted transfer, but the same was not true for the high-level training group. Finally, pre-training between-task correlations were not predictive of the pattern of transfer for either group. Together these findings provide an experimental exploration of the specificity of transfer and establish the nature of task overlap that is crucial for the transfer of performance improvements.

Involuntary autobiographical memories and their relation to other forms of spontaneous thoughts

Involuntary autobiographical memories are memories of personal events that come to mind spontaneously—that is, with no conscious initiation of the retrieval process. Such spontaneously arising memories were long ignored in cognitive psychology, which generally has focused on controlled and strategic forms of remembering, studied in laboratory settings. Recent evidence shows that involuntary memories of past events are highly frequent in daily life, and that they represent a context-sensitive, and associative way of recollecting past events that involves little executive control. They operate by constraints that favour recent events and events with a distinct feature overlap to the current situation, which optimizes the probability of functional relevance to the ongoing situation. In addition to adults, they are documented in young children and great apes and may be an ontogenetic and evolutionary forerunner of strategic retrieval of past events. Findings suggest that intrusive involuntary memories observed clinically after traumatic events should be viewed as a dysfunctional subclass of otherwise functional involuntary autobiographical memories. Because of their highly constrained, situation-dependent and automatic nature, involuntary autobiographical memories form a distinct category of spontaneous thought that cannot be equated with mind wandering. This article is part of the theme issue ‘Offline perception: voluntary and spontaneous perceptual experiences without matching external stimulation’.

Task relevance determines binding of effect features in action planning

Action planning can be construed as the temporary binding of features of perceptual action effects. While previous research demonstrated binding for task-relevant, body-related effect features, the role of task-irrelevant or environment-related effect features in action planning is less clear. Here, we studied whether task-relevance or body-relatedness determines feature binding in action planning. Participants planned an action A, but before executing it initiated an intermediate action B. Each action relied on a body-related effect feature (index vs. middle finger movement) and an environment-related effect feature (cursor movement towards vs. away from a reference object). In Experiments 1 and 2, both effects were task-relevant. Performance in action B suffered from partial feature overlap with action A compared to full feature repetition or alternation, which is in line with binding of both features while planning action A. Importantly, this cost disappeared when all features were available but only body-related features were task-relevant (Experiment 3). When only the environment-related effect of action A was known in advance, action B benefitted when it aimed at the same (vs. a different) environment-related effect (Experiment 4). Consequently, the present results support the idea that task relevance determines whether binding of body-related and environment-related effect features takes place while the pre-activation of environment-related features without binding them primes feature-overlapping actions.

High feature overlap reveals the importance of anterior and medial temporal lobe structures for learning by means of fast mapping

Contrary to traditional theories of declarative memory, it has recently been shown that novel, arbitrary associations can be rapidly and directly integrated into cortical memory networks by means of a learning procedure called fast mapping (FM), possibly bypassing time-consuming hippocampal-neocortical consolidation processes. In the typical FM paradigm, a picture of a previously unknown item is presented next to a picture of a previously known item and participants answer a question referring to an unfamiliar label. It is assumed that they thereby incidentally create associations between the unknown item and the label. However, contradictory findings have been reported and factors moderating rapid cortical integration through FM yet need to be identified. In the context of previous behavioral results showing rapid semantic integration through FM especially if the unknown and the known item shared many features, we propose that due to its computational mechanisms during the processing of complex and particularly highly similar objects, the perirhinal cortex might be especially qualified to support the rapid incorporation of these associations into cortical memory networks within the FM paradigm. We therefore expected that a high degree of feature overlap between the unknown and the known item would trigger strong engagement of the perirhinal cortex at encoding, which in turn might enhance rapid cortical integration of the novel picture-label associations. Within an fMRI experiment, we observed stronger activation for subsequent hits than misses during encoding in the perirhinal cortex and an associated anterior temporal network if the items shared many features than if they shared few features, indicating that the perirhinal cortex indeed contributes to the acquisition of novel associations by means of FM if feature overlap is high.

Training-dependent transfer within a set of nested tasks

Extended practice on a particular cognitive task can boost the performance of other tasks, even though they themselves have not been practiced. This transfer of benefits appears to be specific, occurring most when tasks are very similar to those being trained. But what type of similarity is most important for predicting transfer? This question is addressed with a tightly controlled randomised design, with a relatively large sample (N=175) and an adaptive control group. We created a hierarchical set of nested assessment tasks. Participants then trained on two of the tasks: one was relatively ‘low’ in the hierarchy requiring just simultaneous judgments of shapes’ spikiness, whereas the other was relatively ‘high’ requiring delayed judgments of shapes’ spikiness or number of spikes in a switching paradigm. Using the full complement of nested tasks before and after training we could then test whether and how these ‘low’ and ‘high’ training effects cascade through the hierarchy. For both training groups, relative to the control, whether or not an assessment task shared a single specific feature was the best predictor of transfer patterns. For the lower-level training group, the overall proportion of feature overlap also significantly predicted transfer, but the same was not true for the higher-level training group. Finally, pre-training between-task correlations were not predictive of the pattern of transfer for either group. Together these findings provide an experimental exploration of the specificity of transfer, and establish the nature of task overlap that is crucial for the transfer of performance improvements.

Increased perception-action binding in Tourette syndrome

Gilles de la Tourette syndrome is a multifaceted neurodevelopmental disorder characterized by multiple motor and vocal tics. Research in Tourette syndrome has traditionally focused on the motor system. However, there is increasing evidence that perceptual and cognitive processes play a crucial role as well. Against this background it has been reasoned that processes linking perception and action might be particularly affected in these patients with the strength of perception-action binding being increased. However, this has not yet been studied experimentally. Here, we investigated adult Tourette patients within the framework of the ‘Theory of Event Coding’ using an experimental approach allowing us to directly test the strength of perception-action binding. We included 24 adult patients with Tourette syndrome and n = 24 healthy control subjects using a previously established visual-motor event file task with four levels of feature overlap requiring repeating or alternating responses. Concomitant to behavioural testing, EEG was recorded and analysed using temporal signal decomposition and source localization methods. On a behavioural level, perception-action binding was increased in Tourette patients. Tic frequency correlated with performance in conditions where unbinding processes of previously established perception-action bindings were required with higher tic frequency being associated with stronger perception-action binding. This suggests that perception-action binding is intimately related to the occurrence of tics. Analysis of EEG data showed that behavioural changes cannot be explained based on simple perceptual or motor processes. Instead, cognitive processes linking perception to action in inferior parietal cortices are crucial. Our findings suggest that motor or sensory processes alone are less relevant for the understanding of Tourette syndrome than cognitive processes engaged in linking and restructuring of perception-action association. A broader cognitive framework encompassing perception and action appears well suited to opening new routes for the understanding of Tourette syndrome.