scholarly journals On the Relative Nature of (Pitch-Based) Crossmodal Correspondences

2019 ◽  
Vol 32 (3) ◽  
pp. 235-265 ◽  
Author(s):  
Charles Spence
Keyword(s):  
Empirical Investigation ◽  
Visual Stimuli ◽  
Crossmodal Correspondences ◽  
Clear Cut ◽  
Specific Focus ◽  
Sensory Modalities ◽  
One To One ◽  
Crossmodal Associations

Abstract This review deals with the question of the relative vs absolute nature of crossmodal correspondences, with a specific focus on those correspondences involving the auditory dimension of pitch. Crossmodal correspondences have been defined as the often-surprising crossmodal associations that people experience between features, attributes, or dimensions of experience in different sensory modalities, when either physically present, or else merely imagined. In the literature, crossmodal correspondences have often been contrasted with synaesthesia in that the former are frequently said to be relative phenomena (e.g., it is the higher-pitched of two sounds that is matched with the smaller of two visual stimuli, say, rather than there being a specific one-to-one crossmodal mapping between a particular pitch of sound and size of object). By contrast, in the case of synaesthesia, the idiosyncratic mapping between inducer and concurrent tends to be absolute (e.g., it is a particular sonic inducer that elicits a specific colour concurrent). However, a closer analysis of the literature soon reveals that the distinction between relative and absolute in the case of crossmodal correspondences may not be as clear-cut as some commentators would have us believe. Furthermore, it is important to note that the relative vs absolute question may receive different answers depending on the particular (class of) correspondence under empirical investigation.

scholarly journals Assessing the Role of Emotional Mediation in Explaining Crossmodal Correspondences Involving Musical Stimuli

2020 ◽  
Vol 33 (1) ◽  
pp. 1-29 ◽  
Author(s):  
Charles Spence
Keyword(s):  
Classical Music ◽  
Visual Stimuli ◽  
Key Factors ◽  
Crossmodal Correspondences ◽  
Sensory Modalities ◽  
Crossmodal Associations ◽  
Simple Features

Abstract A wide variety of crossmodal correspondences, defined as the often surprising connections that people appear to experience between simple features, attributes, or dimensions of experience, either physically present or else merely imagined, in different sensory modalities, have been demonstrated in recent years. However, a number of crossmodal correspondences have also been documented between more complex (i.e., multi-component) stimuli, such as, for example, pieces of music and paintings. In this review, the extensive evidence supporting the emotional mediation account of the crossmodal correspondences between musical stimuli (mostly pre-recorded short classical music excerpts) and visual stimuli, including colour patches through to, on occasion, paintings, is critically evaluated. According to the emotional mediation account, it is the emotional associations that people have with stimuli that constitutes one of the fundamental bases on which crossmodal associations are established. Taken together, the literature that has been published to date supports emotional mediation as one of the key factors underlying the crossmodal correspondences involving emotionally-valenced stimuli, both simple and complex.

Evaluative similarity hypothesis of crossmodal correspondences: A developmental view

2012 ◽  
Vol 25 (0) ◽  
pp. 127 ◽  
Author(s):  
Dragan Jankovic
Keyword(s):  
Multisensory Integration ◽  
Age Groups ◽  
Auditory Stimuli ◽  
Crossmodal Correspondences ◽  
Affective Valence ◽  
Similarity Hypothesis ◽  
Visual Patterns ◽  
Sensory Modalities ◽  
Valence And Arousal ◽  
Crossmodal Associations

Crossmodal correspondences have been widely demonstrated, although mechanisms that stand behind the phenomenon have not been fully established yet. According to the Evaluative similarity hypothesis crossmodal correspondences are influenced by evaluative (affective) similarity of stimuli from different sensory modalities (Jankovic, 2010, Journal of Vision 10(7), 859). From this view, detection of similar evaluative information in stimulation from different sensory modalities facilitates crossmodal correspondences and multisensory integration. The aim of this study was to explore the evaluative similarity hypothesis of crossmodal correspondences in children. In Experiment 1 two groups of participants (nine- and thirteen-year-olds) were asked to make explicit matches between presented auditory stimuli (1 s long sound clips) and abstract visual patterns. In Experiment 2 the same participants judged abstract visual patterns and auditory stimuli on the set of evaluative attributes measuring affective valence and arousal. The results showed that crossmodal correspondences are mostly influenced by evaluative similarity of visual and auditory stimuli in both age groups. The most frequently matched were visual and auditory stimuli congruent in both valence and arousal, followed by stimuli congruent in valence, and finally stimuli congruent in arousal. Evaluatively incongruent stimuli demonstrated low crossmodal associations especially in older group.

Synaesthesia

2018 ◽  
Author(s):  
Polina Dimova
Keyword(s):  
Sensory System ◽  
Emotional Responses ◽  
High Pitch ◽  
Shared Experience ◽  
Perceptual Condition ◽  
Sensory Stimuli ◽  
Sensory Modalities ◽  
One To One ◽  
Over Time ◽  
Stimulation Of

Synaesthesia is the confusion or conflation of sensory modalities, where one sense is experienced or described in terms of another as in Charles Baudelaire’s simile "perfumes sweet as oboes, green as prairies." Synaesthesia captures an already existing tendency in language to blend the senses as in "sweet melody," "velvety voice," or "loud colors," and psychologists have conducted studies that show our shared experience of weak audiovisual associations between low pitch and darker colors, or high pitch and lighter colors. In a strictly neurological sense, synaesthesia is a perceptual condition in which the stimulation of one sensory system (for example, hearing) triggers sensations in another sensory system (for example, vision). Cross-sensory associations form one-to-one correspondences that are stable, delicately nuanced, and highly individual. For instance, a synaesthete may experience the timbre of violins as lime green, or the pitch A as burgundy. Synaesthetic associations occur as involuntary, automatic, and emotional responses to sensory stimuli. They persist throughout life and often aid memory: some synaesthetes reliably remember historical dates thanks to their color-to-number associations. The prevalence of synaesthesia has been contested over time, with varying ratios of synaesthetes to nonsynaesthetes of 1 in 2,000, 1 in 100 for colored letters and numbers in recent studies, and even 1 in 23 for all types of synaesthesia.

scholarly journals The Role of Landscapes and Landmarks in Bee Navigation: A Review

Insects ◽  
2019 ◽  
Vol 10 (10) ◽  
pp. 342 ◽  
Author(s):  
Bahram Kheradmand ◽  
James C. Nieh
Keyword(s):  
Sensory Input ◽  
Future Research ◽  
Motivational State ◽  
Research Directions ◽  
Clear Cut ◽  
Sensory Modalities ◽  
Future Research Directions ◽  
The Rich

The ability of animals to explore landmarks in their environment is essential to their fitness. Landmarks are widely recognized to play a key role in navigation by providing information in multiple sensory modalities. However, what is a landmark? We propose that animals use a hierarchy of information based upon its utility and salience when an animal is in a given motivational state. Focusing on honeybees, we suggest that foragers choose landmarks based upon their relative uniqueness, conspicuousness, stability, and context. We also propose that it is useful to distinguish between landmarks that provide sensory input that changes (“near”) or does not change (“far”) as the receiver uses these landmarks to navigate. However, we recognize that this distinction occurs on a continuum and is not a clear-cut dichotomy. We review the rich literature on landmarks, focusing on recent studies that have illuminated our understanding of the kinds of information that bees use, how they use it, potential mechanisms, and future research directions.

Audiovisual crossmodal correspondences in Autism Spectrum Disorders (ASDs)

2012 ◽  
Vol 25 (0) ◽  
pp. 44
Author(s):  
Valeria Occelli ◽  
Gianluca Esposito ◽  
Paola Venuti ◽  
Peter Walker ◽  
Massimiliano Zampini
Keyword(s):  
Autism Spectrum ◽  
Crossmodal Correspondences ◽  
Association Pattern ◽  
Visual Patterns ◽  
Visual Inputs ◽  
Sensory Modalities ◽  
Crossmodal Correspondence ◽  
Matching Performance ◽  
Spectrum Disorders ◽  
Auditory Features

The label ‘crossmodal correspondences’ has been used to define the nonarbitrary associations that appear to exist between different basic physical stimulus attributes in different sensory modalities. For instance, it has been consistently shown in the neurotypical population that higher pitched sounds are more frequently matched with visual patterns which are brighter, smaller, and sharper than those associated to lower pitched sounds. Some evidence suggests that patients with ASDs tend not to show this crossmodal preferential association pattern (e.g., curvilinear shapes and labial/lingual consonants vs. rectilinear shapes and plosive consonants). In the present study, we compared the performance of children with ASDs (6–15 years) and matched neurotypical controls in a non-verbal crossmodal correspondence task. The participants were asked to indicate which of two bouncing visual patterns was making a centrally located sound. In intermixed trials, the visual patterns varied in either size, surface brightness, or shape, whereas the sound varied in pitch. The results showed that, whereas the neurotypical controls reliably matched the higher pitched sound to a smaller and brighter visual pattern, the performance of participants with ASDs was at chance level. In the condition where the visual patterns differed in shape, no inter-group difference was observed. Children’s matching performance cannot be attributed to intensity matching or difficulties in understanding the instructions, which were controlled. These data suggest that the tendency to associate congruent visual and auditory features vary as a function of the presence of ASDs, possibly pointing to poorer capabilities to integrate auditory and visual inputs in this population.

Simultaneous and Independent Acquisition of Multisensory and Unisensory Associations

Perception ◽  
10.1068/p5843 ◽  
2007 ◽  
Vol 36 (10) ◽  
pp. 1445-1453 ◽  
Author(s):  
Aaron R Seitz ◽  
Robyn Kim ◽  
Virginie van Wassenhove ◽  
Ladan Shams
Keyword(s):  
Statistical Learning ◽  
Visual Stimulus ◽  
Daily Life ◽  
Visual Stimuli ◽  
Forced Choice ◽  
Independent Manner ◽  
Choice Procedure ◽  
Learning Procedure ◽  
Sensory Modalities ◽  
Forced Choice Procedure

Although humans are almost constantly exposed to stimuli from multiple sensory modalities during daily life, the processes by which we learn to integrate information from multiple senses to acquire knowledge of multisensory objects are not well understood. Here, we present results of a novel audio – visual statistical learning procedure where participants are passively exposed to a rapid serial presentation of arbitrary audio — visual pairings (comprised of artificial/synthetic audio and visual stimuli). Following this exposure, participants were tested with a two-interval forced-choice procedure in which their degree of familiarity with the experienced audio-visual pairings was evaluated against novel audio — visual combinations drawn from the same stimulus set. Our results show that subjects acquire knowledge of visual — visual, audio — audio, and audio — visual stimulus associations and that the learning of these types of associations occurs in an independent manner.

scholarly journals Smelling sensations: olfactory crossmodal correspondences

2020 ◽  
Author(s):  
Ryan Joseph Ward ◽  
Sophie Wuerger ◽  
Alan Marshall
Keyword(s):  
Aromatic Compounds ◽  
Explicit Knowledge ◽  
Principal Component ◽  
Multisensory Perception ◽  
Crossmodal Correspondences ◽  
Cross Model ◽  
Sensory Modalities ◽  
Olfactory Stimuli ◽  
Emotional Dimensions ◽  
Musical Genres

Crossmodal correspondences are the associations between apparently distinct stimuli in different sensory modalities . These associations, albeit surprising, are generally shared in most of the population. Olfaction is ingrained in the fabric of our daily life and constitutes an integral part of our perceptual reality, with olfaction being more commonly used in the entertainment and analytical domains, it is crucial to uncover the robust correspondences underlying common aromatic compounds. Towards this end, we investigated an aggregate of crossmodal correspondences between ten olfactory stimuli and other modalities ( angularity of shapes, smoothness of texture, pleasantness, pitch, colours, musical genres and emotional dimensions ) using a large sample of 68 observers. We uncover the correspondences between these modalities and extent of these associations with respect to the explicit knowledge of the respective aromatic compound. The results revealed the robustness of prior studies, as well as, contributions towards olfactory integration between an aggregate of other dimensions. The knowledge of an odour's identity coupled with the multisensory perception of the odours indicates that these associations, for the most part, are relatively robust and do not rely on explicit knowledge of the odour. Through principal component analysis of the perceptual ratings, new cross-model mediations have been uncovered between odours and their intercorrelated sensory dimensions. Our results demonstrate a collective of associations between olfaction and other dimensions, potential cross modal mediations via exploratory factor analysis and the robustness of these correspondence with respect to the explicit knowledge of an odour. We anticipate the findings reported in this paper could be used as a psychophysical framework aiding in a collective of applications ranging from olfaction enhanced multimedia to marketing.

Timing Flickers across Sensory Modalities

Perception ◽  
10.1068/p6362 ◽  
2009 ◽  
Vol 38 (8) ◽  
pp. 1144-1151 ◽  
Author(s):  
Carmelo Mario Vicario ◽  
Gaetano Rappo ◽  
Anna Maria Pepi ◽  
Massimiliano Oliveri
Keyword(s):  
Critical Role ◽  
Visual Stimuli ◽  
Spatial Position ◽  
Auditory Stimuli ◽  
Comparison Task ◽  
Subjective Duration ◽  
Sensory Modalities ◽  
Visual Distractors ◽  
Duration Comparison ◽  
Auditory Distractors

In tasks requiring a comparison of the duration of a reference and a test visual cue, the spatial position of test cue is likely to be implicitly coded, providing a form of a congruency effect or introducing a response bias according to the environmental scale or its vectorial reference. The precise mechanism generating these perceptual shifts in subjective duration is not understood, although several studies suggest that spatial attentional factors may play a critical role. Here we use a duration comparison task within and across sensory modalities to examine if temporal performance is also modulated when people are exposed to spatial distractors involving different sensory modalities. Different groups of healthy participants performed duration comparison tasks in separate sessions: a time comparison task of visual stimuli during exposure to spatially presented auditory distractors; and a time comparison task of auditory stimuli during exposure to spatially presented visual distractors. We found the duration of visual stimuli biased depending on the spatial position of auditory distractors. Observers underestimated the duration of stimuli presented in the left spatial field, while there was an overestimation trend in estimating the duration of stimuli presented in the right spatial field. In contrast, timing of auditory stimuli was unaffected by exposure to visual distractors. These results support the existence of multisensory interactions between space and time showing that, in cross-modal paradigms, the presence of auditory distractors can modify visuo-temporal perception but not vice versa. This asymmetry is discussed in terms of sensory–perceptual differences between the two systems.

Crossmodal Correspondences: Four Challenges

2016 ◽  
Vol 29 (1-3) ◽  
pp. 29-48 ◽  
Author(s):  
Ophelia Deroy ◽  
Charles Spence
Keyword(s):  
Mental Imagery ◽  
Crossmodal Correspondences ◽  
Sensory Modalities ◽  
Behavioral Tasks ◽  
Multisensory Interactions

The renewed interest that has emerged around the topic of crossmodal correspondences in recent years has demonstrated that crossmodal matchings and mappings exist between the majority of sensory dimensions, and across all combinations of sensory modalities. This renewed interest also offers a rapidly-growing list of ways in which correspondences affect — or interact with — metaphorical understanding, feelings of ‘knowing’, behavioral tasks, learning, mental imagery, and perceptual experiences. Here we highlight why, more generally, crossmodal correspondences matter to theories of multisensory interactions.

Sign in / Sign up

Export Citation Format

Share Document