scholarly journals Rapid, generalized adaptation to asynchronous audiovisual speech

2015 ◽  
Vol 282 (1804) ◽  
pp. 20143083 ◽  
Author(s):  
Erik Van der Burg ◽  
Patrick T. Goodbourn
Keyword(s):  
Preceding Trial ◽  
Audiovisual Speech ◽  
Temporal Recalibration ◽  
Visual Events ◽  
Point Of Subjective Simultaneity ◽  
Adaptation Procedure ◽  
Subjective Simultaneity ◽  
Speed Of Propagation ◽  
The Voice ◽  
The Brain

The brain is adaptive. The speed of propagation through air, and of low-level sensory processing, differs markedly between auditory and visual stimuli; yet the brain can adapt to compensate for the resulting cross-modal delays. Studies investigating temporal recalibration to audiovisual speech have used prolonged adaptation procedures, suggesting that adaptation is sluggish. Here, we show that adaptation to asynchronous audiovisual speech occurs rapidly. Participants viewed a brief clip of an actor pronouncing a single syllable. The voice was either advanced or delayed relative to the corresponding lip movements, and participants were asked to make a synchrony judgement. Although we did not use an explicit adaptation procedure, we demonstrate rapid recalibration based on a single audiovisual event. We find that the point of subjective simultaneity on each trial is highly contingent upon the modality order of the preceding trial. We find compelling evidence that rapid recalibration generalizes across different stimuli, and different actors. Finally, we demonstrate that rapid recalibration occurs even when auditory and visual events clearly belong to different actors. These results suggest that rapid temporal recalibration to audiovisual speech is primarily mediated by basic temporal factors, rather than higher-order factors such as perceived simultaneity and source identity.

Visuotactile Temporal Recalibration Transfers Across Different Locations

2015 ◽  
Vol 28 (3-4) ◽  
pp. 351-370 ◽  
Author(s):  
Hao Tam Ho ◽  
Hao Tam Ho ◽  
Emily Orchard-Mills ◽  
Hao Tam Ho ◽  
Emily Orchard-Mills ◽  
...  
Keyword(s):  
Prolonged Exposure ◽  
Left Hand ◽  
Timing Mechanism ◽  
Temporal Recalibration ◽  
Simultaneity Judgment ◽  
Point Of Subjective Simultaneity ◽  
Subjective Simultaneity ◽  
The Right ◽  
Audiovisual Asynchrony ◽  
Right And Left Hand

Following prolonged exposure to audiovisual asynchrony, an observer’s point of subjective simultaneity (PSS) shifts in the direction of the leading modality. It has been debated whether other sensory pairings, such as vision and touch, lead to a similar temporal recalibration, and if so, whether the internal timing mechanism underlying lag visuotactile adaptation is centralised or distributed. To address these questions, we adapted observers to vision- and tactile-leading visuotactile asynchrony on either their left or right hand side in different blocks. In one test condition, participants performed a simultaneity judgment on the adapted side (unilateral) and in another they performed a simultaneity judgment on the non-adapted side (contralateral). In a third condition, participants adapted concurrently to equal and opposite asynchronies on each side and were tested randomly on either hand (bilateral opposed). Results from the first two conditions show that observers recalibrate to visuotactile asynchronies, and that the recalibration transfers to the non-adapted side. These findings suggest a centralised recalibration mechanism not linked to the adapted side and predict no recalibration for the bilateral opposed condition, assuming the adapted effects were equal on each side. This was confirmed in the group of participants that adapted to vision- and tactile-leading asynchrony on the right and left hand side, respectively. However, the other group (vision-leading on the left and tactile-leading on the right) did show a recalibration effect, suggesting a distributed mechanism. We discuss these findings in terms of a hybrid model that assumes the co-existence of a centralised and distributed timing mechanism.

Telling the time with audiovisual speech and non-speech: Does the brain use multiple clocks?

2012 ◽  
Vol 25 (0) ◽  
pp. 14-15
Author(s):  
Alberta Ipser ◽  
Diana Paunoiu ◽  
Elliot D. Freeman
Keyword(s):  
Individual Differences ◽  
Temporal Distribution ◽  
Optimal Timing ◽  
Point Of Subjective Simultaneity ◽  
Subjective Simultaneity ◽  
The Mean ◽  
Mcgurk Illusion ◽  
Neural Delay ◽  
The Brain ◽  
Audiovisual Asynchrony

It has often been claimed that there is mutual dependence between the perceived synchrony of auditory and visual sources, and the extent to which they perceptually integrate (‘unity assumption’: Vroomen and Keetels, 2010; Welsh and Warren, 1980). However subjective audiovisual synchrony can vary widely between subjects (Stone, 2001) and between paradigms (van Eijk, 2008). Do such individual differences in subjective synchrony correlate positively with individual differences in optimal timing for integration, as expected under the unity assumption? In separate experiments we measured the optimal audiovisual asynchrony for the McGurk illusion (McGurk and MacDonald, 1976), and the stream-bounce illusion (Sekuler et al., 1997). We concurrently elicited either temporal order judgements (TOJ) or simultaneity judgements (SJ), in counterbalanced sessions, from which we derived the point of subjective simultaneity (PSS). For both experiments, the asynchrony for maximum illusion showed a significant positive correlation with PSS derived from SJ, following the unity assumption. But surprisingly, the analogous correlation with PSS derived from TOJ was significantly negative. The temporal mechanisms for this pairing of tasks seem neither unitary nor fully independent, but apparently antagonistic. A tentative temporal renormalisation mechanism explains these paradoxical results as follows: (1) subjective timing in our different tasks can depend on independent mechanisms subject to their own neural delays; (2) inter-modal synchronization is achieved by first discounting the mean neural delay within each modality; and (3) apparent antagonism between estimates of subjective timing emerges as the mean is attracted towards deviants in the unimodal temporal distribution.

scholarly journals Unity assumption between face and voice modulates audiovisual temporal recalibration

2021 ◽  
Author(s):  
Kyuto Uno ◽  
Kazuhiko Yokosawa
Keyword(s):  
Prolonged Exposure ◽  
Female Voice ◽  
Visual Signals ◽  
Spatial Proximity ◽  
Male Voice ◽  
Cluttered Environment ◽  
Temporal Recalibration ◽  
Point Of Subjective Simultaneity ◽  
Subjective Simultaneity ◽  
Audiovisual Stimuli

Audiovisual temporal recalibration refers to a shift in the point of subjective simultaneity (PSS) between audio and visual signals triggered by prolonged exposure to asynchronies between these signals. Previous research indicated that the spatial proximity of audiovisual signals can be a determinant of which pairs of signals are temporally recalibrated when multiple events compete for recalibration. Here we show that temporal recalibration is modulated by an observer’s assumption that the audiovisual signals originate from the same unitary event (“unity assumption”). Participants were shown alternating face photos and voices of the male and female speakers. These stimuli were presented equally spaced in time, and the voices were presented monaurally through headphones, such that no spatiotemporal-based grouping was implied for these stimuli. There were two conditions for the stimulus sequence in the adaptation phase: one in which a face photo always preceded its corresponding voice within each pairing of audiovisual stimuli (i.e., multiple repetitions of the sequence: female voice – male face – male voice – female voice), and the other one in which the corresponding voice always preceded its face photo. We found a shift in the PSS between these audiovisual signals towards the temporal order for the same gender person. The results show that the unity assumption between face photos and voices affects temporal recalibration, indicating the possibility that the brain selectively recalibrates the asynchronies of audiovisual signals that are considered to originate from the same unitary event in a cluttered environment.

Temporal-Order Judgment and Reaction Time to Stimuli of Different Rise Times

Perception ◽  
1993 ◽  
Vol 22 (8) ◽  
pp. 963-970 ◽  
Author(s):  
Piotr Jaśkowski
Keyword(s):  
Reaction Time ◽  
Simple Reaction Time ◽  
Temporal Order ◽  
Temporal Order Judgment ◽  
Stimulus Presentation ◽  
Stimulus Onset ◽  
Simple Reaction ◽  
Point Of Subjective Simultaneity ◽  
Subjective Simultaneity ◽  
The Subject

Point of subjective simultaneity and simple reaction time were compared for stimuli with different rise times. It was found that these measures behave differently. To explain the result it is suggested that in the case of temporal-order judgment the subject takes into account not only the stimulus onset but also other events connected with stimulus presentation.

The Origins of Music and of Tonal Languages

2019 ◽  
pp. 220-244
Author(s):  
Robert C. Ehle
Keyword(s):  
Absolute Pitch ◽  
Perinatal Period ◽  
Final Part ◽  
Specific Information ◽  
Tonal Language ◽  
Tone Language ◽  
The Voice ◽  
Musical Emotion ◽  
The Brain ◽  
The Way

This chapter offers the author's theory of the origins of music in ancient primates a million years ago, and what music would have sounded like. Origins of nasal and tone languages and the anatomy of larynx is discussed, and then a hypothesis is presented that these creatures would fashioned a tone language. They had absolute pitch that allowed them to recognize other voices, to read each other's emotions from the sounds they made with their voices, and to convey over long distances specific information about strategies, meeting places, etc. Having an acute sense of pitch, they would have sung, essentially using tonal language for aesthetic and subjective purposes. Thus, they would have invented music. Then the physicality of the human (or hominid) voice is discussed and the way an absolute pitch can be acquired, as the musicality still lies in the vocalisms it expresses. The reason for this is that music is actually contained in the way the brain works, and the ear and the voice are parts of this system. The final part discusses the origins of musical emotion as the case for imprinting in the perinatal period.

scholarly journals Visual delay affects force scaling and weight perception during object lifting in virtual reality

2019 ◽  
Vol 121 (4) ◽  
pp. 1398-1409 ◽  
Author(s):  
Vonne van Polanen ◽  
Robert Tibold ◽  
Atsuo Nuruki ◽  
Marco Davare
Keyword(s):  
Virtual Reality ◽  
Multisensory Integration ◽  
Force Feedback ◽  
Haptic Feedback ◽  
Visual Signals ◽  
Visual Events ◽  
Object Lifting ◽  
Relative Weighting ◽  
The Brain ◽  
Visual Delay

Lifting an object requires precise scaling of fingertip forces based on a prediction of object weight. At object contact, a series of tactile and visual events arise that need to be rapidly processed online to fine-tune the planned motor commands for lifting the object. The brain mechanisms underlying multisensory integration serially at transient sensorimotor events, a general feature of actions requiring hand-object interactions, are not yet understood. In this study we tested the relative weighting between haptic and visual signals when they are integrated online into the motor command. We used a new virtual reality setup to desynchronize visual feedback from haptics, which allowed us to probe the relative contribution of haptics and vision in driving participants’ movements when they grasped virtual objects simulated by two force-feedback robots. We found that visual delay changed the profile of fingertip force generation and led participants to perceive objects as heavier than when lifts were performed without visual delay. We further modeled the effect of vision on motor output by manipulating the extent to which delayed visual events could bias the force profile, which allowed us to determine the specific weighting the brain assigns to haptics and vision. Our results show for the first time how visuo-haptic integration is processed at discrete sensorimotor events for controlling object-lifting dynamics and further highlight the organization of multisensory signals online for controlling action and perception. NEW & NOTEWORTHY Dexterous hand movements require rapid integration of information from different senses, in particular touch and vision, at different key time points as movement unfolds. The relative weighting between vision and haptics for object manipulation is unknown. We used object lifting in virtual reality to desynchronize visual and haptic feedback and find out their relative weightings. Our findings shed light on how rapid multisensory integration is processed over a series of discrete sensorimotor control points.

scholarly journals Rapid Temporal Recalibration to Audiovisual Asynchrony Occurs Across the Difference in Neural Processing Speed Based on Spatial Frequency

i-Perception ◽  
2020 ◽  
Vol 11 (5) ◽  
pp. 204166952096661
Author(s):  
Yasuhiro Takeshima
Keyword(s):  
Spatial Frequency ◽  
Processing Speed ◽  
Audiovisual Integration ◽  
Auditory Stimuli ◽  
Neural Processing ◽  
Temporal Recalibration ◽  
Synchrony Perception ◽  
The Difference ◽  
The One ◽  
The Brain

Audiovisual integration relies on temporal synchrony between visual and auditory stimuli. The brain rapidly adapts to audiovisual asynchronous events by shifting the timing of subjective synchrony in the direction of the leading modality of the most recent event, a process called rapid temporal recalibration. This phenomenon is the flexible function of audiovisual synchrony perception. Previous studies found that neural processing speed based on spatial frequency (SF) affects the timing of subjective synchrony. This study examined the effects of SF on the rapid temporal recalibration process by discriminating whether the presentation of the visual and auditory stimuli was simultaneous. I compared the magnitudes of the recalibration effect between low and high SF visual stimuli using two techniques. First, I randomly presented each SF accompanied by a tone during one session, then in a second experiment, only a single SF was paired with the tone throughout the one session. The results indicated that rapid recalibration occurred regardless of difference in presented SF between preceding and test trials. The recalibration magnitude did not significantly differ between the SF conditions. These findings confirm that intersensory temporal process is important to produce rapid recalibration and suggest that rapid recalibration can be induced by the simultaneity judgment criterion changes attributed to the low-level temporal information of audiovisual events.

Perceived Onset Simultaneity of Stimuli with Unequal Durations

Perception ◽  
1991 ◽  
Vol 20 (6) ◽  
pp. 715-726 ◽  
Author(s):  
Piotr Jaśkowski
Keyword(s):  
Temporal Order ◽  
Temporal Order Judgment ◽  
Visual Stimuli ◽  
Constant Stimulus ◽  
Point Of Subjective Simultaneity ◽  
Subjective Simultaneity ◽  
The Subject ◽  
The Difference ◽  
Stimulus Offset ◽  
Method Of Adjustment

Temporal-order judgment was investigated for a pair of visual stimuli with different durations in order to check whether offset asynchrony can disturb the perception of the order/simultaneity of onset. In experiment 1 the point of subjective simultaneity was estimated by the method of adjustment. The difference in duration of the two stimuli in the pair was either 0 or 50 ms. It was found that the subject shifts the onset of the shorter stimulus towards the offset of the longer one to obtain a satisfying impression of simultaneity even though the subject was asked to ignore the events concerning the stimulus offset. In experiments 2 and 3 the method of constant stimulus was applied. Both experiments indicate that subjects, in spite of instruction, take into account the offset asynchrony in their judgment.

scholarly journals Genetic Analysis with Feature Reduction to Predict the Onset of Parkinson’s disease

2020 ◽  
Vol 9 (10) ◽  
pp. 12-16
Keyword(s):  
Parkinson’S Disease ◽  
Parkinson's Disease ◽  
Feature Reduction ◽  
Reduction Techniques ◽  
Parkinson’S Diseases ◽  
Efficient Prediction ◽  
Primary Focus ◽  
The Voice ◽  
The Brain ◽  
Optimal Feature

Parkinson is a disease which directly affects the brain cells and certain movement, voice and other disabilities. Hence curable medication is not available in market. The best solution is the early diagnosis to relieve the symptoms of Parkinson’s disease affected people. One major concern effecting public is Parkinson's disease (PD). This paper studies the bias of various traditional algorithms on the voice-based data that has various parameters recorded from Parkinson patients and healthy patients. A brief survey of techniques are mentioned for the prediction of Parkinson's diseases is presented. To accomplish this task, identifying the best feature reduction approach was the primary focus. This paper further applies feature reduction techniques using a genetic algorithm for efficient prediction of Parkinson's disease along with machine learning-based approaches. The proposed method also presents higher accuracy in prediction by using this optimal feature reduction technique.

Neurobiology of Auditory Hallucinations

2019 ◽  
Author(s):  
Judith M. Ford ◽  
Holly K. Hamilton ◽  
Alison Boos
Keyword(s):  
Help Seeking ◽  
Treatment Approach ◽  
Neural Mechanisms ◽  
Hearing Voices ◽  
Auditory Verbal Hallucinations ◽  
Self And Other ◽  
Neurological Conditions ◽  
Language Areas ◽  
The Voice ◽  
The Brain

Auditory verbal hallucinations (AVH), also referred to as “hearing voices,” are vivid perceptions of speech that occur in the absence of any corresponding external stimulus but seem very real to the voice hearer. They are experienced by the majority of people with schizophrenia, less frequently in other psychiatric and neurological conditions, and are relatively rare in the general population. Because antipsychotic medications are not always successful in reducing the severity or frequency of AVH, a better understanding is needed of their neurobiological basis, which may ultimately lead to more precise treatment targets. What voices say and how the voices sound, or their phenomenology, varies widely within and across groups of people who hear them. In help-seeking populations, such as people with schizophrenia, the voices tend to be threatening and menacing, typically spoken in a non-self-voice, often commenting and sometimes commanding the voice hearers to do things they would not otherwise do. In psychotic populations, voices differ from normal inner speech by being unbidden and unintended, co-opting the voice hearer’s attention. In healthy voice-hearing populations, voices are not typically distressing nor disabling, and are sometimes comforting and reassuring. Regardless of content and valence, voices tend to activate some speech and language areas of the brain. Efforts to silence these brain areas with neurostimulation have had mixed success in reducing the frequency and salience of voices. Progress with this treatment approach would likely benefit from more precise anatomical targets and more precisely dosed neurostimulation. Neural mechanisms that may underpin the experience of voices are being actively investigated and include mechanisms enabling context-based predictions and distinctions between experiences coming from self and other. Both these mechanisms can be studied in non-human animal “models” and both can provide new anatomical targets for neurostimulation.

Sign in / Sign up

Export Citation Format

Share Document