Multimodal Integration and Expertise

Multi-modal integration often results in one modality dominating sensory perception. Such dominance is influenced by task demands, processing efficiency, and training. I assessed modality dominance between auditory and visual processing in a paradigm controlling for the first two factors while manipulating the third. In a uni-modal task auditory and visual processing was equated per individual participant. Pre and post training, participants completed a bimodal selective attention task where the relationship between relevant and irrelevant information, and the task-relevant modality changed across trials. Training in one modality was provided between pre and post-training tasks. Training resulted in non-specific speeding post-training. Pre-training, visual information impacted auditory responding more than vice versa and this pattern reversed following training, implying visual dominance pre, and auditory dominance post-training. Results suggest modality dominance is flexible and influenced by experimental design and participant abilities. Research should continue to uncover factors leading to sensory dominance by one modality.

Multimodal Integration and Expertise

Multi-modal integration often results in one modality dominating sensory perception. Such dominance is influenced by task demands, processing efficiency, and training. I assessed modality dominance between auditory and visual processing in a paradigm controlling for the first two factors while manipulating the third. In a uni-modal task auditory and visual processing was equated per individual participant. Pre and post training, participants completed a bimodal selective attention task where the relationship between relevant and irrelevant information, and the task-relevant modality changed across trials. Training in one modality was provided between pre and post-training tasks. Training resulted in non-specific speeding post-training. Pre-training, visual information impacted auditory responding more than vice versa and this pattern reversed following training, implying visual dominance pre, and auditory dominance post-training. Results suggest modality dominance is flexible and influenced by experimental design and participant abilities. Research should continue to uncover factors leading to sensory dominance by one modality.

Tones disrupt visual fixations and responding on a visual-spatial task

The current study used an eye tracker to examine how auditory input affects the latency of visual saccades, fixations, and response times while using variations of a Serial Response Time (SRT) task. In Experiment 1, participants viewed a repeating sequence of visual stimuli that appeared in different locations on a computer monitor and they had to quickly determine if each visual stimulus was red or blue. The visual sequence was either presented in silence or paired with tones. Compared to the silent condition, the tones slowed down red/blue discriminations and delayed the latency of first fixations to the visual stimuli. To ensure the interference was not occurring during the decision/response phase and to better understand the nature of auditory interference, we removed the red/blue discrimination task in Experiment 2, manipulated cognitive load, and developed a gaze-contingent procedure where the timing of each visual stimulus was dependent on a saccade crossing a gaze-contingent boundary surrounding the target. Participants were slower at initiating their saccades/fixations and made more fixations under high load and auditory interference was found with participants being more likely to fixate on the visual images and were faster at fixating on the visual stimuli when the visual sequences were presented in silence. These findings suggest that auditory interference effects occur early in the course of processing and provide insights into potential mechanisms underlying modality dominance effects.

Listen to your heart: Examining modality dominance using cross-modal oddball tasks

The current study used cross-modal oddball tasks to examine cardiac and behavioral responses to changing auditory and visual information. When instructed to press the same button for auditory and visual oddballs, auditory dominance was found with cross-modal presentation slowing down visual response times more than auditory response times (Experiment 1). When instructed to make separate responses to auditory and visual oddballs, visual dominance was found with cross-modal presentation decreasing auditory discrimination. Participants also made more visual-based than auditory-based errors on cross-modal trials (Experiment 2). Experiment 3 increased task demands while requiring a single button press and found evidence of auditory dominance, suggesting that it is unlikely that increased task demands can account for the reversal in Experiment 2. Examination of cardiac responses that were time-locked with stimulus onset showed cross-modal facilitation effects, with auditory and visual discrimination occurring earlier in the course of processing in the cross-modal condition than in the unimodal conditions. The current findings showing that response demand manipulations reversed modality dominance and that time-locked cardiac responses show cross-modal facilitation, not interference, suggest that auditory and visual dominance effects may both be occurring later in the course of processing, not from disrupted encoding.

Individual differences between the practising behaviours of six pianists: A challenge to Perceptual Learning Style theory

Perceptual Learning Style theory (PLS) claims that the presentation of information in either a visual, auditory or kinaesthetic format will improve the learning of selected individuals due to the dominance of one or more modality in their information gathering. The modality dominance of six Western classical pianists in higher music education was explored through observing their practising in four different situations totalling 80 minutes, and conducting interviews with them in a stimulated-recall protocol after each observation. The observation and interview data were analysed to 1) identify patterns in individuals’ approaches to learning new musical material from scores, to 2) identify differences between the individuals, and to 3) relate these differences to the predictions of PLS. Thirteen behavioural categories were identified that differentiated the pianists, but these could not be convincingly linked to PLS. The findings of the study question the applicability of the theory to musical practising.