Human Vision Reconstructs Time to Satisfy Causal Constraints

The goal of perception is to infer the most plausible source of sensory stimulation. Unisensory perception of temporal order, however, appears to require no inference, because the order of events can be uniquely determined from the order in which sensory signals arrive. Here, we demonstrate a novel perceptual illusion that casts doubt on this intuition: In three experiments ( N = 607), the experienced event timings were determined by causality in real time. Adult participants viewed a simple three-item sequence, ACB, which is typically remembered as ABC in line with principles of causality. When asked to indicate the time at which events B and C occurred, participants’ points of subjective simultaneity shifted so that the assumed cause B appeared earlier and the assumed effect C later, despite participants’ full attention and repeated viewings. This first demonstration of causality reversing perceived temporal order cannot be explained by postperceptual distortion, lapsed attention, or saccades.

Perceived Timing of Cutaneous Vibration and Intracortical Microstimulation of Human Somatosensory Cortex

BackgroundElectrically stimulating the somatosensory cortex can partially restore the sense of touch. Though this technique bypasses much of the neuroaxis, prior studies with non-human primates have found that conscious detection of touch elicited by intracortical microstimulation (ICMS) lags behind the detection of vibration applied to the skin. These findings may have been influenced by a mismatch in stimulus intensity; typically, vibration is perceived as more intense than ICMS, which can significantly impact temporal perception.ObjectiveThe goal of this study was to evaluate the relative latency at which intensity-matched vibration and ICMS are perceived in a human subject.MethodsA human participant implanted with microelectrode arrays in somatosensory cortex performed a reaction time task and a temporal order judgment (TOJ) task. In the reaction time task, the participant was presented with ICMS or vibration and verbal response times were obtained. In the TOJ task, the participant was sequentially presented with a pair of stimuli – ICMS followed by vibration or vice versa – and reported which stimulus occurred first.ResultsWhen ICMS and vibration were matched in perceived intensity, the reaction time to vibration was ∼50 ms faster than ICMS. However, in the TOJ task, ICMS and vibratory sensations arose at comparable latencies, with points of subjective simultaneity that were not significantly different from zero.ConclusionsBecause the perception of ICMS is slower than that of intensity-matched vibration, it may be necessary to stimulate at stronger ICMS intensities (thus decreasing reaction time) when incorporating ICMS sensory feedback into neural prostheses.

Visual field differences in temporal synchrony processing for audio-visual stimuli

Audio-visual integration relies on temporal synchrony between visual and auditory inputs. However, differences in traveling and transmitting speeds between visual and auditory stimuli exist; therefore, audio-visual synchrony perception exhibits flexible functions. The processing speed of visual stimuli affects the perception of audio-visual synchrony. The present study examined the effects of visual fields, in which visual stimuli are presented, for the processing of audio-visual temporal synchrony. The point of subjective simultaneity, the temporal binding window, and the rapid recalibration effect were measured using temporal order judgment, simultaneity judgment, and stream/bounce perception, because different mechanisms of temporal processing have been suggested among these three paradigms. The results indicate that auditory stimuli should be presented earlier for visual stimuli in the central visual field than in the peripheral visual field condition in order to perceive subjective simultaneity in the temporal order judgment task conducted in this study. Meanwhile, the subjective simultaneity bandwidth was broader in the central visual field than in the peripheral visual field during the simultaneity judgment task. In the stream/bounce perception task, neither the point of subjective simultaneity nor the temporal binding window differed between the two types of visual fields. Moreover, rapid recalibration occurred in both visual fields during the simultaneity judgment tasks. However, during the temporal order judgment task and stream/bounce perception, rapid recalibration occurred only in the central visual field. These results suggest that differences in visual processing speed based on the visual field modulate the temporal processing of audio-visual stimuli. Furthermore, these three tasks, temporal order judgment, simultaneity judgment, and stream/bounce perception, each have distinct functional characteristics for audio-visual synchrony perception. Future studies are necessary to confirm the effects of compensation regarding differences in the temporal resolution of the visual field in later cortical visual pathways on visual field differences in audio-visual temporal synchrony.

Early Capture of Attention by Self-Face: Investigation Using a Temporal Order Judgment Task

Earlier work on self-face processing has reported a bias in the processing of self-face result in faster response to self-face in comparison to other familiar and unfamiliar faces (termed as self-face advantage or SFA). Even though most studies agree that the SFA occurs due to an attentional bias, there is little agreement regarding the stage at which it occurs. While a large number of studies show self-face influencing processing later at disengagement stage, early event-related potential components show differential activity for the self-face suggesting that SFA occurs early. We address this contradiction using a cueless temporal order judgment task that allows us to investigate early perceptual processing, while bias due to top-down expectation is controlled. A greater shift in point of subjective simultaneity for self-face would indicate a greater processing advantage at early perceptual stage. With help of two experiments, we show an early perceptual advantage for self-face, compared to both a friend’s face and an unfamiliar face (Experiment 1). This advantage is present even when the effect of criterion shift is minimized (Experiment 2). Interestingly, the magnitude of advantage is similar for self-friend and self-unfamiliar pair. The evidence from the two experiments suggests early capture of attention as a likely reason for the SFA, which is present for the self-face but not for other familiar faces.

Unity assumption between face and voice modulates audiovisual temporal recalibration

Audiovisual temporal recalibration refers to a shift in the point of subjective simultaneity (PSS) between audio and visual signals triggered by prolonged exposure to asynchronies between these signals. Previous research indicated that the spatial proximity of audiovisual signals can be a determinant of which pairs of signals are temporally recalibrated when multiple events compete for recalibration. Here we show that temporal recalibration is modulated by an observer’s assumption that the audiovisual signals originate from the same unitary event (“unity assumption”). Participants were shown alternating face photos and voices of the male and female speakers. These stimuli were presented equally spaced in time, and the voices were presented monaurally through headphones, such that no spatiotemporal-based grouping was implied for these stimuli. There were two conditions for the stimulus sequence in the adaptation phase: one in which a face photo always preceded its corresponding voice within each pairing of audiovisual stimuli (i.e., multiple repetitions of the sequence: female voice – male face – male voice – female voice), and the other one in which the corresponding voice always preceded its face photo. We found a shift in the PSS between these audiovisual signals towards the temporal order for the same gender person. The results show that the unity assumption between face photos and voices affects temporal recalibration, indicating the possibility that the brain selectively recalibrates the asynchronies of audiovisual signals that are considered to originate from the same unitary event in a cluttered environment.

Multisensory binding is driven by the strength of stimulus correlation

Our perceptual system is adept at exploiting sensory regularities to better extract information about our environment. One clear example of this is how the sensory and multisensory systems can utilize consistency to group sensory features into a perceptual object and segregate objects from each other and background noise. Leveraging tenets of object-based attention and multisensory binding, we sought whether this ability scaled with the strength of that consistency. We presented participants with amplitude modulated (AM) auditory and visual streams and asked them to detect imbedded orthogonal, near-threshold frequency modulation (FM) events. We modulated the correlation of the streams by varying the phase of the visual AM. In line with a previous report, we first observed peak performance that was shifted from 0°. After accounting for this, we found that across participants discriminability of the FM event linearly improved with correlation. Additionally, we sought to answer a question left dangling from our previous report as to the possible explanation for the phase shift. We found that phase shift correlated with auditory and visual response time differences, but not point of subjective simultaneity, suggesting differences in sensory processing times may account for the observed phase shift. These results suggest that our perceptual system can bind multisensory features across a spectrum of temporal correlations, a process necessary for multisensory binding in complex environments where unrelated signals may have small errant correlations.

Temporal Metrics of Multisensory Processing Change in the Elderly

Older adults exhibit greater multisensory response time (RT) facilitation by violating the race model more than young adults; this is commonly interpreted as an enhancement in perception. Older adults typically exhibit wider temporal binding windows (TBWs) and points of subjective simultaneity (PSS) that typically lie farther from true simultaneity as compared to young adults when simultaneity judgment (SJ) and temporal-order judgment (TOJ) tasks are utilized; this is commonly interpreted as an impairment in perception. Here we explore the relation between the three tasks in order to better assess audiovisual multisensory temporal processing in both young and older adults. Our results confirm previous reports showing that audiovisual RT, TBWs and PSSs change with age; however, we show for the first time a significant positive relation between the magnitude of race model violation in young adults as a function of the PSS obtained from the audiovisual TOJ task (r: 0.49, p: 0.007), that is absent in older adults (r: 0.13, p: 0.58). Furthermore, we find no evidence for the relation between race model violation as a function of the PSS obtained from the audiovisual SJ task in both young (r: −0.01, p: 0.94) and older adults (r: 0.1, p: 0.66). Our results confirm previous reports that (i) audiovisual temporal processing changes with age; (ii) distinct processes are likely involved in simultaneity and temporal-order perception; and (iii) common processing between race model violation and temporal-order judgment is impaired in the elderly.

Temporal Metrics of Multisensory Processing Change in the Elderly

Older adults exhibit greater multisensory response time (RT) facilitation by violating the race model more than younger adults; this is commonly interpreted as an enhancement in perception. Older adults typically exhibit wider temporal binding windows (TBWs) and points of subjective simultaneity (PSS) that are farther from true simultaneity as compared to younger adults when simultaneity judgment (SJ) and temporal order judgment (TOJ) tasks are utilized; this is commonly interpreted as an impairment in perception. Here we explore the relation between the three tasks in younger and older adults in order to better understand the underlying mechanisms that subserve audiovisual multisensory temporal processing. Our results confirm previous reports showing that audiovisual RT, TBWs and PSSs change with age, and we show for the first time a significant positive relation between the magnitude of race model violation in younger adults as a function of the PSS obtained from the audiovisual TOJ task with (r: 0.49, p: 0.007), that is absent among the elderly (r: 0.13, p: 0.58). Furthermore, we find no evidence for the relation between race model violation as a function of the PSS obtained from the audiovisual SJ task in both younger (r: −0.01, p: 0.94) and older adults (r: 0.1, p: 0.66). Our results confirm previous reports that i) audiovisual temporal processing changes with age; ii) there is evidence for distinct neural networks involved in simultaneity and temporal order perception; and iii) common processing between race model violation and temporal order judgment is impaired in the elderly.

I before U: Temporal order judgements reveal bias for self-owned objects

A multitude of studies demonstrate that self-relevant stimuli influence attention. Self-owned objects are a special class of self-relevant stimuli. If a self-owned object can indeed be characterised as a self-relevant stimulus then, consistent with theoretical predictions, a behavioural effect of ownership on attention should be present. To test this prediction, a task was selected that is known to be particularly sensitive measure of the prioritisation of visual information: the temporal order judgement. Participants completed temporal order judgements with pictures of “own” and “experimenter” owned objects (mugs) presented on either side of a central fixation cross. There was a variable onset delay between each picture, ranging between 0 ms and 105 ms, and participants were asked to indicate which mug appeared first. The results indicated a reliable change in the point of subjective simultaneity (PSS) in favour of their own mug. Such a change in the PSS was not observed for two groups of participants who were exposed to a mug but did not keep the mug. A further experiment indicated that the source of the bias in PSS was more consistent with a criterion shift or top-down attentional prioritisation rather than a perceptual bias. These findings suggest that ownership, beyond mere-touch, mere-choice, or familiarity, leads to prioritised processing and responses, but the mechanism underlying the effect is not likely to be perceptual in nature.