scholarly journals Crossmodal Correspondences in the Sounds of Chinese Instruments

Perception ◽  
2019 ◽  
Vol 49 (1) ◽  
pp. 81-97
Author(s):  
Yuxuan Qi ◽  
Fuxing Huang ◽  
Zeyan Li ◽  
Xiaoang Wan
Keyword(s):  
Color Patch ◽  
Crossmodal Correspondences ◽  
Chinese Music ◽  
Sensory Modalities ◽  
Percussion Instruments ◽  
Shed Light

People tend to associate stimuli from different sensory modalities, a phenomenon known as crossmodal correspondences. We conducted two experiments to investigate how Chinese participants associated musical notes produced by four types of Chinese instruments (bowed strings, plucked strings, winds, and percussion instruments) with different colors, taste terms, and fabric textures. Specifically, the participants were asked to select a sound to match each color patch or taste term in Experiment 1 and to match the experience of touching each fabric in Experiment 2. The results demonstrated some associations between pitch and color, taste term, or the smoothness of fabrics. Moreover, certain types of Chinese instruments were preferentially chosen to match some of the colors, taste terms, and the texture of certain fabrics. These findings therefore provided insights about the perception of Chinese music and shed light on how to apply the multisensory features of sounds to enhance the composition, performance, and appreciation of music.

TAN DUN'S VIOLIN CONCERTO "THE LOVE" AS A REFLECTION OF STYLISTIC DIVERSITY OF MODERN COMPOSITION

2020 ◽  
Vol 1 (3) ◽  
pp. 61-74
Author(s):  
Xin Xing ◽  
Keyword(s):  
Human Life ◽  
Symphony Orchestra ◽  
Violin Concerto ◽  
Avant Garde ◽  
Chinese Music ◽  
Peking Opera ◽  
Tan Dun ◽  
Percussion Instruments ◽  
Solo Violin ◽  
Stylistic Diversity

The article analyzes "The Love" Violin Concerto (2009) by the famous Chinese and American composer Tan Dun in contextual, composing and linguistic aspects. Based on the statements of his contemporaries, the author considers the composer's musical and aesthetic views that determine the originality of his style, organically combining avant-garde techniques with elements of traditional Chinese music. Tan Dun's Violin Concerto exists in three versions with different titles ("Out of Peking Opera" 1987, "The Love" 2009, "Rhapsody and Fantasy" 2018), representing different artistic concepts with a new viewpoint on the same intonational material. Three movements of the Concerto "The Love" reflect the evolution of feelings at different stages of human life. The composer manages to combine the idea of "national" (a tendency going back to the version of the Concerto "Out of Peking Opera" 1987) with a philosophical understanding of the category of love. The paper discusses the originality of the dramaturgy of "The Love" Violin Concerto, which assumes the third part as the main center (based on the development of thematic material of the first and second movements), the consolidation of parts of the attacca cycle and the rondality of the musical form. The most important peculiarity of the composition is the mixture of elements belonging to different cultural and temporal layers of music. The stylistic diversity of Tan Dun's Concerto is reflected in the following details: the composer's introduction of a stylized tune from the Beijing Opera erhuang, speech intonations resembling recitatives of Chinese dramas, borrowings from his own film music (the second movement), the use of methods typical for traditional Chinese music, yaoban and sanban, the timbre of oriental percussion instruments in a classical symphony orchestra, as well as dodecaphone techniques, aleatorics and Hip-hop rhythms. Special attention in the article is paid to interpretation of expressive possibilities of solo violin: methods of classical-romantic technique are synthesized with traditions of performance on Chinese stringed instruments.

Audiovisual crossmodal correspondences in Autism Spectrum Disorders (ASDs)

2012 ◽  
Vol 25 (0) ◽  
pp. 44
Author(s):  
Valeria Occelli ◽  
Gianluca Esposito ◽  
Paola Venuti ◽  
Peter Walker ◽  
Massimiliano Zampini
Keyword(s):  
Autism Spectrum ◽  
Crossmodal Correspondences ◽  
Association Pattern ◽  
Visual Patterns ◽  
Visual Inputs ◽  
Sensory Modalities ◽  
Crossmodal Correspondence ◽  
Matching Performance ◽  
Spectrum Disorders ◽  
Auditory Features

The label ‘crossmodal correspondences’ has been used to define the nonarbitrary associations that appear to exist between different basic physical stimulus attributes in different sensory modalities. For instance, it has been consistently shown in the neurotypical population that higher pitched sounds are more frequently matched with visual patterns which are brighter, smaller, and sharper than those associated to lower pitched sounds. Some evidence suggests that patients with ASDs tend not to show this crossmodal preferential association pattern (e.g., curvilinear shapes and labial/lingual consonants vs. rectilinear shapes and plosive consonants). In the present study, we compared the performance of children with ASDs (6–15 years) and matched neurotypical controls in a non-verbal crossmodal correspondence task. The participants were asked to indicate which of two bouncing visual patterns was making a centrally located sound. In intermixed trials, the visual patterns varied in either size, surface brightness, or shape, whereas the sound varied in pitch. The results showed that, whereas the neurotypical controls reliably matched the higher pitched sound to a smaller and brighter visual pattern, the performance of participants with ASDs was at chance level. In the condition where the visual patterns differed in shape, no inter-group difference was observed. Children’s matching performance cannot be attributed to intensity matching or difficulties in understanding the instructions, which were controlled. These data suggest that the tendency to associate congruent visual and auditory features vary as a function of the presence of ASDs, possibly pointing to poorer capabilities to integrate auditory and visual inputs in this population.

scholarly journals Which words are most iconic?

2017 ◽  
Vol 18 (3) ◽  
pp. 443-464 ◽  
Author(s):  
Bodo Winter ◽  
Marcus Perlman ◽  
Lynn K. Perry ◽  
Gary Lupyan
Keyword(s):  
Native Speaker ◽  
Statistical Regularity ◽  
Content Type ◽  
Part Of Speech ◽  
Sensory Modalities ◽  
Spoken Words ◽  
The Senses ◽  
The Relationship ◽  
Taste And Smell ◽  
Shed Light

Some spoken words are iconic, exhibiting a resemblance between form and meaning. We used native speaker ratings to assess the iconicity of 3001 English words, analyzing their iconicity in relation to part-of-speech differences and differences between the sensory domain they relate to (sight, sound, touch, taste and smell). First, we replicated previous findings showing that onomatopoeia and interjections were highest in iconicity, followed by verbs and adjectives, and then nouns and grammatical words. We further show that words with meanings related to the senses are more iconic than words with abstract meanings. Moreover, iconicity is not distributed equally across sensory modalities: Auditory and tactile words tend to be more iconic than words denoting concepts related to taste, smell and sight. Last, we examined the relationship between iconicity (resemblance between form and meaning) and systematicity (statistical regularity between form and meaning). We find that iconicity in English words is more strongly related to sensory meanings than systematicity. Altogether, our results shed light on the extent and distribution of iconicity in modern English.

scholarly journals On the Relative Nature of (Pitch-Based) Crossmodal Correspondences

2019 ◽  
Vol 32 (3) ◽  
pp. 235-265 ◽  
Author(s):  
Charles Spence
Keyword(s):  
Empirical Investigation ◽  
Visual Stimuli ◽  
Crossmodal Correspondences ◽  
Clear Cut ◽  
Specific Focus ◽  
Sensory Modalities ◽  
One To One ◽  
Crossmodal Associations

Abstract This review deals with the question of the relative vs absolute nature of crossmodal correspondences, with a specific focus on those correspondences involving the auditory dimension of pitch. Crossmodal correspondences have been defined as the often-surprising crossmodal associations that people experience between features, attributes, or dimensions of experience in different sensory modalities, when either physically present, or else merely imagined. In the literature, crossmodal correspondences have often been contrasted with synaesthesia in that the former are frequently said to be relative phenomena (e.g., it is the higher-pitched of two sounds that is matched with the smaller of two visual stimuli, say, rather than there being a specific one-to-one crossmodal mapping between a particular pitch of sound and size of object). By contrast, in the case of synaesthesia, the idiosyncratic mapping between inducer and concurrent tends to be absolute (e.g., it is a particular sonic inducer that elicits a specific colour concurrent). However, a closer analysis of the literature soon reveals that the distinction between relative and absolute in the case of crossmodal correspondences may not be as clear-cut as some commentators would have us believe. Furthermore, it is important to note that the relative vs absolute question may receive different answers depending on the particular (class of) correspondence under empirical investigation.

scholarly journals Smelling sensations: olfactory crossmodal correspondences

2020 ◽  
Author(s):  
Ryan Joseph Ward ◽  
Sophie Wuerger ◽  
Alan Marshall
Keyword(s):  
Aromatic Compounds ◽  
Explicit Knowledge ◽  
Principal Component ◽  
Multisensory Perception ◽  
Crossmodal Correspondences ◽  
Cross Model ◽  
Sensory Modalities ◽  
Olfactory Stimuli ◽  
Emotional Dimensions ◽  
Musical Genres

Crossmodal correspondences are the associations between apparently distinct stimuli in different sensory modalities . These associations, albeit surprising, are generally shared in most of the population. Olfaction is ingrained in the fabric of our daily life and constitutes an integral part of our perceptual reality, with olfaction being more commonly used in the entertainment and analytical domains, it is crucial to uncover the robust correspondences underlying common aromatic compounds. Towards this end, we investigated an aggregate of crossmodal correspondences between ten olfactory stimuli and other modalities ( angularity of shapes, smoothness of texture, pleasantness, pitch, colours, musical genres and emotional dimensions ) using a large sample of 68 observers. We uncover the correspondences between these modalities and extent of these associations with respect to the explicit knowledge of the respective aromatic compound. The results revealed the robustness of prior studies, as well as, contributions towards olfactory integration between an aggregate of other dimensions. The knowledge of an odour's identity coupled with the multisensory perception of the odours indicates that these associations, for the most part, are relatively robust and do not rely on explicit knowledge of the odour. Through principal component analysis of the perceptual ratings, new cross-model mediations have been uncovered between odours and their intercorrelated sensory dimensions. Our results demonstrate a collective of associations between olfaction and other dimensions, potential cross modal mediations via exploratory factor analysis and the robustness of these correspondence with respect to the explicit knowledge of an odour. We anticipate the findings reported in this paper could be used as a psychophysical framework aiding in a collective of applications ranging from olfaction enhanced multimedia to marketing.

Crossmodal Correspondences: Four Challenges

2016 ◽  
Vol 29 (1-3) ◽  
pp. 29-48 ◽  
Author(s):  
Ophelia Deroy ◽  
Charles Spence
Keyword(s):  
Mental Imagery ◽  
Crossmodal Correspondences ◽  
Sensory Modalities ◽  
Behavioral Tasks ◽  
Multisensory Interactions

The renewed interest that has emerged around the topic of crossmodal correspondences in recent years has demonstrated that crossmodal matchings and mappings exist between the majority of sensory dimensions, and across all combinations of sensory modalities. This renewed interest also offers a rapidly-growing list of ways in which correspondences affect — or interact with — metaphorical understanding, feelings of ‘knowing’, behavioral tasks, learning, mental imagery, and perceptual experiences. Here we highlight why, more generally, crossmodal correspondences matter to theories of multisensory interactions.

scholarly journals The Effects of Mandarin Chinese Lexical Tones in Sound–Shape and Sound–Size Correspondences

2021 ◽  
pp. 1-15
Author(s):  
Yen-Han Chang ◽  
Mingxue Zhao ◽  
Yi-Chuan Chen ◽  
Pi-Chun Huang
Keyword(s):  
Mandarin Chinese ◽  
Shape Matching ◽  
Crossmodal Correspondences ◽  
Lexical Tones ◽  
Chinese Speakers ◽  
Sensory Modalities ◽  
Native Mandarin Chinese Speakers

Abstract Crossmodal correspondences refer to when specific domains of features in different sensory modalities are mapped. We investigated how vowels and lexical tones drive sound–shape (rounded or angular) and sound–size (large or small) mappings among native Mandarin Chinese speakers. We used three vowels (/i/, /u/, and /a/), and each vowel was articulated in four lexical tones. In the sound–shape matching, the tendency to match the rounded shape was decreased in the following order: /u/, /i/, and /a/. Tone 2 was more likely to be matched to the rounded pattern, whereas Tone 4 was more likely to be matched to the angular pattern. In the sound–size matching, /a/ was matched to the larger object more than /u/ and /i/, and Tone 2 and Tone 4 correspond to the large–small contrast. The results demonstrated that both vowels and tones play prominent roles in crossmodal correspondences, and sound–shape and sound–size mappings are heterogeneous phenomena.

scholarly journals QoE of cross-modally mapped Mulsemedia: an assessment using eye gaze and heart rate

2020 ◽  
Vol 79 (11-12) ◽  
pp. 7987-8009 ◽  
Author(s):  
Gebremariam Mesfin ◽  
Nadia Hussain ◽  
Elahe Kani-Zabihi ◽  
Alexandra Covaci ◽  
Estêvão B. Saleme ◽  
...  
Keyword(s):  
Heart Rate ◽  
Research Effort ◽  
Eye Gaze ◽  
Video Clip ◽  
Crossmodal Correspondences ◽  
Screen Questionnaire ◽  
Video Clips ◽  
Digital World ◽  
Sensory Modalities ◽  
Tracking Device

AbstractA great deal of research effort has been put in exploring crossmodal correspondences in the field of cognitive science which refer to the systematic associations frequently made between different sensory modalities (e.g. high pitch is matched with angular shapes). However, the possibilities cross-modality opens in the digital world have been relatively unexplored. Therefore, we consider that studying the plasticity and the effects of crossmodal correspondences in a mulsemedia setup can bring novel insights about improving the human-computer dialogue and experience. Mulsemedia refers to the combination of three or more senses to create immersive experiences. In our experiments, users were shown six video clips associated with certain visual features based on color, brightness, and shape. We examined if the pairing with crossmodal matching sound and the corresponding auto-generated haptic effect, and smell would lead to an enhanced user QoE. For this, we used an eye-tracking device as well as a heart rate monitor wristband to capture users’ eye gaze and heart rate whilst they were experiencing mulsemedia. After each video clip, we asked the users to complete an on-screen questionnaire with a set of questions related to smell, sound and haptic effects targeting their enjoyment and perception of the experiment. Accordingly, the eye gaze and heart rate results showed significant influence of the cross-modally mapped multisensorial effects on the users’ QoE. Our results highlight that when the olfactory content is crossmodally congruent with the visual content, the visual attention of the users seems shifted towards the correspondent visual feature. Crosmodally matched media is also shown to result in an enhanced QoE compared to a video only condition.

Evaluative similarity hypothesis of crossmodal correspondences: A developmental view

2012 ◽  
Vol 25 (0) ◽  
pp. 127 ◽  
Author(s):  
Dragan Jankovic
Keyword(s):  
Multisensory Integration ◽  
Age Groups ◽  
Auditory Stimuli ◽  
Crossmodal Correspondences ◽  
Affective Valence ◽  
Similarity Hypothesis ◽  
Visual Patterns ◽  
Sensory Modalities ◽  
Valence And Arousal ◽  
Crossmodal Associations

Crossmodal correspondences have been widely demonstrated, although mechanisms that stand behind the phenomenon have not been fully established yet. According to the Evaluative similarity hypothesis crossmodal correspondences are influenced by evaluative (affective) similarity of stimuli from different sensory modalities (Jankovic, 2010, Journal of Vision 10(7), 859). From this view, detection of similar evaluative information in stimulation from different sensory modalities facilitates crossmodal correspondences and multisensory integration. The aim of this study was to explore the evaluative similarity hypothesis of crossmodal correspondences in children. In Experiment 1 two groups of participants (nine- and thirteen-year-olds) were asked to make explicit matches between presented auditory stimuli (1 s long sound clips) and abstract visual patterns. In Experiment 2 the same participants judged abstract visual patterns and auditory stimuli on the set of evaluative attributes measuring affective valence and arousal. The results showed that crossmodal correspondences are mostly influenced by evaluative similarity of visual and auditory stimuli in both age groups. The most frequently matched were visual and auditory stimuli congruent in both valence and arousal, followed by stimuli congruent in valence, and finally stimuli congruent in arousal. Evaluatively incongruent stimuli demonstrated low crossmodal associations especially in older group.

Sign in / Sign up

Export Citation Format

Share Document