Crossmodal Correspondence Between Auditory Timbre and Visual Shape

Crossmodal correspondences are defined as associations between crossmodal stimuli based on seemingly irrelevant stimulus features (i.e., bright shapes being associated with high-pitched sounds). There is a large body of research describing auditory crossmodal correspondences involving pitch and volume, but not so much involving auditory timbre, the character or quality of a sound. Adeli and colleagues (2014, Front. Hum. Neurosci. 8, 352) found evidence of correspondences between timbre and visual shape. The present study aimed to replicate Adeli et al.’s findings, as well as identify novel timbre–shape correspondences. Participants were tested using two computerized tasks: an association task, which involved matching shapes to presented sounds based on best perceived fit, and a semantic task, which involved rating shapes and sounds on a number of scales. The analysis of association matches reveals nonrandom selection, with certain stimulus pairs being selected at a much higher frequency. The harsh/jagged and smooth/soft correspondences observed by Adeli et al. were found to be associated with a high level of consistency. Additionally, high matching frequency of sounds with unstudied timbre characteristics suggests the existence of novel correspondences. Finally, the ability of the semantic task to supplement existing crossmodal correspondence assessments was demonstrated. Convergent analysis of the semantic and association data demonstrates that the two datasets are significantly correlated (−0.36) meaning stimulus pairs associated with a high level of consensus were more likely to hold similar perceived meaning. The results of this study are discussed in both theoretical and applied contexts.

Neuronal congruency effects in macaque frontal cortex

The interplay between task-relevant and task-irrelevant stimulus features induces conflicts which impair human behavioral performance in many perceptual and cognitive tasks, a.k.a. a behavioral congruency effect. The neuronal mechanisms underlying behavioral congruency effects, however, are poorly understood. We recorded single unit activity in monkey frontal cortex using a novel task-switching paradigm and discovered a neuronal congruency effect that is carried by task-relevant and -irrelevant neurons. The former neurons provide more signal, the latter less noise in congruent compared to incongruent conditions. Their relative activity levels determine the neuronal congruency effect and behavioral performance. Although these neuronal congruency signals are sensitive to selective attention, they cannot be entirely explained by selective attention as gauged by response time. We propose that such neuronal congruency effects can explain behavioral congruency effects in general, as well as previous fMRI and EEG results in various conflict paradigms.

EXPRESS: Mapping effects in choice-response and go/nogo variants of the lexical-decision task: A case for polarity correspondence.

Previous research has shown that responses to words are faster and more accurate in the go/nogo version of the lexical-decision task (LDT) than in the choice-response version. This finding has been attributed to reduced response-selection demands in the go/nogo task. Here we test an alternative account assuming similar response-selection demands in the two tasks, but an additional impact of polarity correspondence between stimuli and responses in the go/nogo task. Positive and negative polarities have been described as a frequent characteristic of binary stimulus and response dimensions. Only for the go/nogo version of the LDT an apparent polarity difference between go and nogo responses exists, with go responses having the same polarity as words and nogo responses the same polarity as nonwords. Thus, as compared with the choice-response LDT, in the go/nogo LDT go responses to words should be facilitated by polarity correspondence, and go responses to nonwords should be inhibited by polarity noncorrespondence. In the present study, each participant performed a go/nogo LDT and a choice-response LDT. Responses to words were faster and more accurate than responses to nonwords, and—consistent with the alternative account—this difference was larger in the go/nogo LDT than in the choice-response LDT. An analysis of performance by means of sequential-sampling models, that take into account a decaying influence of irrelevant stimulus features, supported the effect of polarity correspondence in the go/nogo LDT. This analysis suggested an effect in the choice-response LDT as well, though of a smaller size and a faster decay.

Goal-Based Binding of Irrelevant Stimulus Features for Action Slips

Binding between representations of stimuli and actions and later retrieval of these compounds provide efficient shortcuts in action control. Recent observations indicate that these mechanisms are not only effective when action episodes go as planned, but they also seem to be at play when actions go awry. Moreover, the human cognitive system even corrects traces of error commission on the fly because it binds the intended but not actually executed response to concurrent task-relevant stimuli, thus enabling retrieval of a correct, but not actually executed response when encountering the stimulus again. However, a plausible alternative interpretation of this finding is that error commission triggers selective strengthening of the instructed stimulus–response mapping instead, thus promoting its efficient application in the future. The experiment presented here makes an unequivocal case for episodic binding and retrieval in erroneous action episodes by showing binding between task-irrelevant stimuli and correct responses.

Perceptual decisions are biased toward relevant prior choices

Perceptual decisions are biased by recent perceptual history—a phenomenon termed 'serial dependence.' Here, we investigated what aspects of perceptual decisions lead to serial dependence, and disambiguated the influences of low-level sensory information, prior choices and motor actions. Participants discriminated whether a brief visual stimulus lay to left/right of the screen center. Following a series of biased ‘prior’ location discriminations, subsequent ‘test’ location discriminations were biased toward the prior choices, even when these were reported via different motor actions (using different keys), and when the prior and test stimuli differed in color. By contrast, prior discriminations about an irrelevant stimulus feature (color) did not substantially influence subsequent location discriminations, even though these were reported via the same motor actions. Additionally, when color (not location) was discriminated, a bias in prior stimulus locations no longer influenced subsequent location discriminations. Although low-level stimuli and motor actions did not trigger serial-dependence on their own, similarity of these features across discriminations boosted the effect. These findings suggest that relevance across perceptual decisions is a key factor for serial dependence. Accordingly, serial dependence likely reflects a high-level mechanism by which the brain predicts and interprets new incoming sensory information in accordance with relevant prior choices.

Recurrent dynamics of prefrontal cortex during context-dependent decision-making

A key problem in systems neuroscience is to understand how neural populations integrate relevant sensory inputs during decision-making. Here, we address this problem by training a structured recurrent neural network to reproduce both psychophysical behavior and neural responses recorded from monkey prefrontal cortex during a context-dependent per-ceptual decision-making task. Our approach yields a one-to-one mapping of model neurons to recorded neurons, and explicitly incorporates sensory noise governing the animal’s performance as a function of stimulus strength. We then analyze the dynamics of the resulting model in order to understand how the network computes context-dependent decisions. We find that network dynamics preserve both relevant and irrelevant stimulus information, and exhibit a grid of fixed points for different stimulus conditions as opposed to a one-dimensional line attractor. Our work provides new insights into context-dependent decision-making and offers a powerful framework for linking cognitive function with neural activity within an artificial model.

Feature Overlap and Relevance Determine Sequential Modulations in the Simon Task

Subjects usually respond faster and more accurate in trials in which the response location corresponds to a task-irrelevant stimulus location compared to when not. Previous research has shown, that this so-called Simon effect is reduced after non-corresponding compared to after corresponding trials. As of now it is yet unclear what exact mechanisms drive such sequential modulations. One theory assumes a conflict adaptation mechanism that decreases the influence of incongruent information after non-corresponding trials via increased cognitive control. However, other authors claim that feature integration processes may be the underlying mechanism as the amount of feature overlap differs between different correspondence sequences. Unfortunately, this also means that in the standard Simon task the repetition of task features and correspondence sequences are not independent. In order to address this issue, we mapped four stimuli to two responses in the present EEG study. This way, we created a task, which allows for sequences in which the stimulus identity may change without alternating the required response. These sequences may either comprise a change of the stimulus position or not and the contribution of feature integration as well as conflict adaptation processes could thus be observed independently. Our results indicate that the repetition of task features affects performance to a stronger degree than the correspondence sequence and feature integration processes do. Yet, an impact of both could still be observed. The strongest effect of feature repetitions on task performance was observed for task-relevant features, especially for the imperative stimulus identity itself. The EEG results support these findings. The amplitudes of the fronto-central N2 and the parietal P3 decreased with increasing feature overlap from one trial to the next. The posterior lateralization, reflected by the posterior contralateral negativity (PCN), however, appears to reflect mainly changes in the stimulus location and stimulus–response (S–R) correspondence rather than feature repetitions per se.

Neural Dynamics of Serial Dependence in Numerosity Perception

Serial dependence—an attractive perceptual bias whereby a current stimulus is perceived to be similar to previously seen ones—is thought to represent the process that facilitates the stability and continuity of visual perception. Recent results demonstrate a neural signature of serial dependence in numerosity perception, emerging very early in the time course during perceptual processing. However, whether such a perceptual signature is retained after the initial processing remains unknown. Here, we address this question by investigating the neural dynamics of serial dependence using a recently developed technique that allowed a reactivation of hidden memory states. Participants performed a numerosity discrimination task during EEG recording, with task-relevant dot array stimuli preceded by a task-irrelevant stimulus inducing serial dependence. Importantly, the neural network storing the representation of the numerosity stimulus was perturbed (or pinged) so that the hidden states of that representation can be explicitly quantified. The results first show that a neural signature of serial dependence emerges early in the brain signals, starting soon after stimulus onset. Critical to the central question, the pings at a later latency could successfully reactivate the biased representation of the initial stimulus carrying the signature of serial dependence. These results provide one of the first pieces of empirical evidence that the biased neural representation of a stimulus initially induced by serial dependence is preserved throughout a relatively long period.