Capture of Intermodal Visual/Tactile Apparent Motion by Moving and Static Sound

2011 ◽  
Vol 24 (4) ◽  
pp. 369-389 ◽  
Author(s):  
Lihan Chen ◽  
Xiaolin Zhou
Keyword(s):  
Apparent Motion ◽  
Visual Stimulus ◽  
Tactile Stimulus ◽  
Temporal Information ◽  
Auditory Information ◽  
Motion Direction ◽  
Temporal Ventriloquism ◽  
Temporal Interval ◽  
Visual Apparent Motion ◽  
Task Irrelevant

AbstractApparent motion can occur within a particular modality or between modalities, in which a visual or tactile stimulus at one location is perceived as moving towards the location of the subsequent tactile or visual stimulus. Intramodal apparent motion has been shown to be affected or 'captured' by information from another, task-irrelevant modality, as in spatial or temporal ventriloquism. Here we investigate whether and how intermodal apparent motion is affected by motion direction cues or temporal interval information from a third modality. We demonstrated that both moving and asynchronous static sounds can capture intermodal (visual–tactile and tactile–visual) apparent motion; moreover, while the auditory direction cues have less impact upon the perception of intramodal visual apparent motion than upon the perception of intramodal tactile or intermodal visual/tactile apparent motion, the auditory temporal information has equivalent impacts upon both intramodal and intermodal apparent motion. These findings suggest intermodal apparent motion is susceptible to the influence of dynamic or static auditory information in similar ways as intramodal visual or tactile apparent motion.

The Effect of Brief Auditory Stimuli on Visual Apparent Motion

Perception ◽  
10.1068/p5741 ◽  
2007 ◽  
Vol 36 (7) ◽  
pp. 1089-1103 ◽  
Author(s):  
Stephan Getzmann
Keyword(s):  
Apparent Motion ◽  
Visual Stimulus ◽  
Temporal Structure ◽  
Visual Stimuli ◽  
Auditory Stimuli ◽  
Temporal Dimension ◽  
Rapid Succession ◽  
Temporal Ventriloquism ◽  
Visual Apparent Motion

When two discrete stimuli are presented in rapid succession, observers typically report a movement of the lead stimulus toward the lag stimulus. The object of this study was to investigate crossmodal effects of irrelevant sounds on this illusion of visual apparent motion. Observers were presented with two visual stimuli that were temporally separated by interstimulus onset intervals from 0 to 350 ms. After each trial, observers classified their impression of the stimuli using a categorisation system. The presentation of short sounds intervening between the visual stimuli facilitated the impression of apparent motion relative to baseline (visual stimuli without sounds), whereas sounds presented before the first and after the second visual stimulus as well as simultaneously presented sounds reduced the motion impression. The results demonstrate an effect of the temporal structure of irrelevant sounds on visual apparent motion that is discussed in light of a related multisensory phenomenon, ‘temporal ventriloquism’, on the assumption that sounds can attract lights in the temporal dimension.

Mutual Influences of Intermodal Visual/Tactile Apparent Motion and Auditory Motion with Uncrossed and Crossed Arms

2013 ◽  
Vol 26 (1-2) ◽  
pp. 19-51 ◽  
Author(s):  
Yushi Jiang ◽  
Lihan Chen
Keyword(s):  
Apparent Motion ◽  
Congruency Effect ◽  
Reference Frames ◽  
Auditory Stimuli ◽  
Directional Information ◽  
Spatial Reference Frames ◽  
Visual Apparent Motion ◽  
Tactile Stimuli ◽  
Modality Combinations ◽  
Task Irrelevant

Intra-modal apparent motion has been shown to be affected or ‘captured’ by information from another, task-irrelevant modality, as shown in cross-modal dynamic capture effect. Here we created inter-modal apparent motion between visual and tactile stimuli and investigated whether there are mutual influences between auditory apparent motion and inter-modal visual/tactile apparent motion. Moreover, we examined whether and how the spatial remapping between somatotopic and external reference frames of tactile events affect the cross-modal capture between auditory apparent motion and inter-modal visual/tactile apparent motion, by introducing two arm postures: arms-uncrossed and arms-crossed. In Experiment 1, we used auditory stimuli (auditory apparent motion) as distractors and inter-modal visual/tactile stimuli (inter-modal apparent motion) as targets while in Experiment 2 we reversed the distractors and targets. In Experiment 1, we found a general detrimental influence of arms-crossed posture in the task of discrimination of direction in visual/tactile stream, but in Experiment 2, the influence of arms-uncrossed posture played a significant role in modulating the inter-modal visual/tactile stimuli capturing over auditory apparent motion. In both Experiments, the synchronously presented motion streams led to noticeable directional congruency effect in judging the target motion. Among the different modality combinations, tactile to tactile apparent motion (TT) and visual to visual apparent motion (VV) are two signatures revealing the asymmetric congruency effects. When the auditory stimuli were targets, the congruency effect was largest with VV distractors, lowest with TT distractors; the pattern was reversed when the auditory stimuli were distractors. In addition, across both experiments the congruency effect in visual to tactile (VT) and tactile to visual (TV) apparent motion was intermediate between the effect-sizes in VV and TT. We replicated the above findings with a block-wise design (Experiment 3). In Experiment 4, we introduced static distractor events (visual or tactile stimulus), and found the modulation of spatial remapping of distractors upon AA motion is reduced. These findings suggest that there are mutual but a robust asymmetric influence between intra-modal auditory apparent motion and intermodal visual/tactile apparent motion. We proposed that relative reliabilities in directional information between distractor and target streams, summed over a remapping process between two spatial reference frames, determined this asymmetric influence.

scholarly journals Adaptation to temporal interval modulates the perception of visual apparent motion

2012 ◽  
Vol 12 (9) ◽  
pp. 1029-1029
Author(s):  
L. Chen ◽  
H. Zhang ◽  
X. Zhou
Keyword(s):  
Apparent Motion ◽  
Temporal Interval ◽  
Visual Apparent Motion

scholarly journals Peripersonal space in the front, rear, left and right directions for audio-tactile multisensory integration

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Yusuke Matsuda ◽  
Maki Sugimoto ◽  
Masahiko Inami ◽  
Michiteru Kitazaki
Keyword(s):  
Multisensory Integration ◽  
Tactile Stimulus ◽  
Peripersonal Space ◽  
Auditory Information ◽  
Specific Direction ◽  
Left And Right ◽  
Audio Stimulus ◽  
Task Irrelevant

AbstractPeripersonal space (PPS) is important for humans to perform body–environment interactions. However, many previous studies only focused on the specific direction of the PPS, such as the front space, despite suggesting that there were PPSs in all directions. We aimed to measure and compare the peri-trunk PPS in four directions (front, rear, left, and right). To measure the PPS, we used a tactile and an audio stimulus because auditory information is available at any time in all directions. We used the approaching and receding task-irrelevant sounds in the experiment. Observers were asked to respond as quickly as possible when a tactile stimulus was applied to a vibrator on their chest. We found that peri-trunk PPS representations exist with an approaching sound, irrespective of the direction.

Previously reward-associated sounds interfere with goal-directed auditory processing

2021 ◽  
pp. 174702182199003
Author(s):  
Andy J Kim ◽  
David S Lee ◽  
Brian A Anderson
Keyword(s):  
Visual Processing ◽  
Auditory Processing ◽  
Dichotic Listening ◽  
Auditory Information ◽  
Correct Identification ◽  
Subsequent Test ◽  
Dichotic Listening Task ◽  
Listening Task ◽  
Task Irrelevant ◽  
Visual Domain

Previously reward-associated stimuli have consistently been shown to involuntarily capture attention in the visual domain. Although previously reward-associated but currently task-irrelevant sounds have also been shown to interfere with visual processing, it remains unclear whether such stimuli can interfere with the processing of task-relevant auditory information. To address this question, we modified a dichotic listening task to measure interference from task-irrelevant but previously reward-associated sounds. In a training phase, participants were simultaneously presented with a spoken letter and number in different auditory streams and learned to associate the correct identification of each of three letters with high, low, and no monetary reward, respectively. In a subsequent test phase, participants were again presented with the same auditory stimuli but were instead instructed to report the number while ignoring spoken letters. In both the training and test phases, response time measures demonstrated that attention was biased in favour of the auditory stimulus associated with high value. Our findings demonstrate that attention can be biased towards learned reward cues in the auditory domain, interfering with goal-directed auditory processing.

Cognitive Styles Differentiate Crossmodal Correspondences Between Pitch Glide and Visual Apparent Motion

2017 ◽  
Vol 30 (3-5) ◽  
pp. 363-385 ◽  
Author(s):  
Lu Guo ◽  
Ming Bao ◽  
Luyang Guan ◽  
Lihan Chen
Keyword(s):  
Apparent Motion ◽  
Moving Objects ◽  
High Efficiency ◽  
Cognitive Styles ◽  
Crossmodal Correspondences ◽  
Crossmodal Interaction ◽  
Visual Apparent Motion ◽  
Field Independent ◽  
Crossmodal Correspondence ◽  
Figure Test

Crossmodal correspondences are the automatic associations that most people have between different basic sensory stimulus attributes, dimensions, or features. For instance, people often show a systematic tendency to associate moving objects with changing pitches. Cognitive styles are defined as an individual’s consistent approach to think, perceive, and remember information, and they reflect qualitative rather than quantitative differences between individuals in their thinking processes. Here we asked whether cognitive styles played a role in modulating the crossmodal interaction. We used the visual Ternus display in our study, since it elicits two distinct apparent motion percepts: element motion (with a shorter interval between the two Ternus frames) and group motion (with a longer interval between the two frames). We examined the audiovisual correspondences between the visual Ternus movement directions (upward or downward) and the changes of pitches of concurrent glides (ascending frequency or descending frequency). Moreover, we measured the cognitive styles (with the Embedded Figure Test) for each participant. The results showed that congruent correspondence between pitch-ascending (decreasing) glides and moving upward (downward) visual directions led to a more dominant percept of ‘element motion’, and such an effect was typically observed in the field-independent group. Importantly, field-independent participants demonstrated a high efficiency for identifying the properties of audiovisual events and applying the crossmodal correspondence in crossmodal interaction. The results suggest cognitive styles could differentiate crossmodal correspondences in crossmodal interaction.

Auditory Motion Induces Directionally Dependent Receptive Field Shifts in Inferior Colliculus Neurons

1998 ◽  
Vol 79 (4) ◽  
pp. 2040-2062 ◽  
Author(s):  
Willard W. Wilson ◽  
William E. O'Neill
Keyword(s):  
Receptive Field ◽  
Inferior Colliculus ◽  
Apparent Motion ◽  
Target Location ◽  
Receptive Fields ◽  
Silent Interval ◽  
Auditory Motion ◽  
Motion Direction ◽  
Motion Sequence ◽  
And Shifts

Wilson, Willard W. and William E. O'Neill. Auditory motion induces directionally dependent receptive field shifts in inferior colliculus neurons. J. Neurophysiol. 79: 2040–2062, 1998. This research focused on the response of neurons in the inferior colliculus of the unanesthetized mustached bat, Pteronotus parnelli, to apparent auditory motion. We produced the apparent motion stimulus by broadcasting pure-tone bursts sequentially from an array of loudspeakers along horizontal, vertical, or oblique trajectories in the frontal hemifield. Motion direction had an effect on the response of 65% of the units sampled. In these cells, motion in opposite directions produced shifts in receptive field locations, differences in response magnitude, or a combination of the two effects. Receptive fields typically were shifted opposite the direction of motion (i.e., units showed a greater response to moving sounds entering the receptive field than exiting) and shifts were obtained to horizontal, vertical, and oblique motion orientations. Response latency also shifted as a function of motion direction, and stimulus locations eliciting greater spike counts also exhibited the shortest neural latency. Motion crossing the receptive field boundaries appeared to be both necessary and sufficient to produce receptive field shifts. Decreasing the silent interval between successive stimuli in the apparent motion sequence increased both the probability of obtaining a directional effect and the magnitude of receptive field shifts. We suggest that the observed directional effects might be explained by “spatial masking,” where the response of auditory neurons after stimulation from particularly effective locations in space would be diminished. The shift in auditory receptive fields would be expected to shift the perceived location of a moving sound and may explain shifts in localization of moving sources observed in psychophysical studies. Shifts in perceived target location caused by auditory motion might be exploited by auditory predators such as Pteronotus in a predictive tracking strategy to capture moving insect prey.

Sign in / Sign up

Export Citation Format

Share Document