Causal inference regulates audiovisual spatial recalibration via its influence on audiovisual perception

To obtain a coherent perception of the world, our senses need to be in alignment. When we encounter misaligned cues from two sensory modalities, the brain must infer which cue is faulty and recalibrate the corresponding sense. We examined whether and how the brain uses cue reliability to identify the miscalibrated sense by measuring the audiovisual ventriloquism aftereffect for stimuli of varying visual reliability. To adjust for modality-specific biases, visual stimulus locations were chosen based on perceived alignment with auditory stimulus locations for each participant. During an audiovisual recalibration phase, participants were presented with bimodal stimuli with a fixed perceptual spatial discrepancy; they localized one modality, cued after stimulus presentation. Unimodal auditory and visual localization was measured before and after the audiovisual recalibration phase. We compared participants’ behavior to the predictions of three models of recalibration: (a) Reliability-based: each modality is recalibrated based on its relative reliability—less reliable cues are recalibrated more; (b) Fixed-ratio: the degree of recalibration for each modality is fixed; (c) Causal-inference: recalibration is directly determined by the discrepancy between a cue and its estimate, which in turn depends on the reliability of both cues, and inference about how likely the two cues derive from a common source. Vision was hardly recalibrated by audition. Auditory recalibration by vision changed idiosyncratically as visual reliability decreased: the extent of auditory recalibration either decreased monotonically, peaked at medium visual reliability, or increased monotonically. The latter two patterns cannot be explained by either the reliability-based or fixed-ratio models. Only the causal-inference model of recalibration captures the idiosyncratic influences of cue reliability on recalibration. We conclude that cue reliability, causal inference, and modality-specific biases guide cross-modal recalibration indirectly by determining the perception of audiovisual stimuli.

Ambiguity avoidance as a factor in the rise of the English dative alternation

This paper discusses the role of cognitive factors in language change; specifically, it investigates the potential impact of argument ambiguity avoidance on the emergence of one of the most well-studied syntactic alternations in English, viz. the dative alternation (We gave them cake vs We gave cake to them). Linking this development to other major changes in the history of English like the loss of case marking, I propose that morphological as well as semantic-pragmatic ambiguity between prototypical agents (subjects) and prototypical recipients (indirect objects) in ditransitive clauses plausibly gave a processing advantage to patterns with higher cue reliability such as prepositional marking, but also fixed clause-level (SVO) order. The main hypotheses are tested through a quantitative analysis of ditransitives in a corpus of Middle English, which (i) confirms that the spread of the PP-construction is impacted by argument ambiguity and (ii) demonstrates that this change reflects a complex restructuring of disambiguation strategies.

The Effects of Cue Reliability on Crossmodal Recalibration in Adults and Children

Reliability-based cue combination is a hallmark of multisensory integration, while the role of cue reliability for crossmodal recalibration is less understood. The present study investigated whether visual cue reliability affects audiovisual recalibration in adults and children. Participants had to localize sounds, which were presented either alone or in combination with a spatially discrepant high- or low-reliability visual stimulus. In a previous study we had shown that the ventriloquist effect (indicating multisensory integration) was overall larger in the children groups and that the shift in sound localization toward the spatially discrepant visual stimulus decreased with visual cue reliability in all groups. The present study replicated the onset of the immediate ventriloquist aftereffect (a shift in unimodal sound localization following a single exposure of a spatially discrepant audiovisual stimulus) at the age of 6–7 years. In adults the immediate ventriloquist aftereffect depended on visual cue reliability, whereas the cumulative ventriloquist aftereffect (reflecting the audiovisual spatial discrepancies over the complete experiment) did not. In 6–7-year-olds the immediate ventriloquist aftereffect was independent of visual cue reliability. The present results are compatible with the idea of immediate and cumulative crossmodal recalibrations being dissociable processes and that the immediate ventriloquist aftereffect is more closely related to genuine multisensory integration.

Modeling Input Factors in Second Language Acquisition of the English Article Construction

Based on the Competition Model, the current study investigated how cue availability and cue reliability as two important input factors influenced second language (L2) learners' cue learning of the English article construction. Written corpus data of university-level Chinese-L1 learners of English were sampled for a comparison of English majors and non-English majors who demonstrated two levels of L2 competence in English article usage. The path model analysis in structural equation modeling was utilized to investigate the relationship between the input factors and L2 usage (frequency and accuracy of article cue production). The findings contribute novel and scarce empirical evidence that confirms a central claim of the Competition Model, i.e., the changing importance of cue availability and cue reliability in the frequency and accuracy of production. Cue availability was found to determine L2 production frequency regardless of level of L2 competence. Cue reliability was the input factor that differentiated competence levels. When learners stayed at a relatively lower L2 proficiency, cue reliability played an important role in influencing L2 frequency of usage rather than accuracy of usage. When learners developed increased exposure to and stronger competence in the target language, cue reliability played a significant role in determining learners' success of cue learning. The study is methodologically innovative and expands the empirical applicability of the Competition Model to the domain of second language production and construction learning.

Communicative efficiency and differential case marking: a reverse-engineering approach

The use of differential case marking of A and P has been explained in terms of efficiency (economy) and markedness. The present study tests predictions based on these accounts, using conditional probabilities of a particular feature given the syntactic role (cue availability), and conditional probabilities of a particular syntactic role given the feature in question (cue reliability). Cue availability serves as a measure of markedness, whereas cue reliability is central for the efficiency account. Similar to reverse engineering, we determine which of the probabilistic measures could have been responsible for the recurrent cross-linguistic patterns described in the literature. The probabilities are estimated from spontaneous informal dialogues in English and Russian (Indo-European), Lao (Tai-Kadai), N||ng (Tuu) and Ruuli (Bantu). The analyses, which involve a series of mixed-effects Poisson models, clearly demonstrate that cue reliability matches the observed cross-linguistic patterns better than cue availability. Thus, the results support the efficiency account of differential marking.

Causal-inference regulates audiovisual spatial recalibration via its influence on audiovisual perception

To obtain a coherent perception of the world, our senses need to be in alignment. When we encounter misaligned cues from two sensory modalities, the brain must infer which cue is faulty and recalibrate the corresponding sense. We examined whether and how the brain uses cue reliability to identify the miscalibrated sense by measuring the audiovisual ventriloquism aftereffect for stimuli of varying reliability. Visual spatial reliability was smaller, comparable to and greater than that of auditory stimuli. To adjust for modality-specific biases, visual stimulus locations were chosen based on perceived alignment with auditory stimulus locations for each participant. During audiovisual recalibration, participants were presented with bimodal stimuli with a fixed perceptual spatial discrepancy; they localized one modality, cued after stimulus presentation. Unimodal auditory and visual localization was measured before and after the recalibration phase. We compared participants’ behavior to the predictions of three models of recalibration: (a) Reliability-based: each modality is recalibrated based on its relative reliability—less reliable cues are recalibrated more; (b) Fixed-ratio: the degree of recalibration for each modality is fixed; (c) Causal-inference: recalibration is directly determined by the discrepancy between a cue and its final estimate, which in turn depends on the reliability of both cues, and inference about how likely the two cues derive from a common source. Vision was hardly recalibrated by audition. Auditory recalibration by vision changed idiosyncratically as visual reliability decreased: the extent of auditory recalibration either decreased monotonically, first increased and then decreased, or increased monotonically. The latter two patterns cannot be explained by either the reliability-based or fixed-ratio models. Only the causal-inference model of recalibration captures the idiosyncratic influences of cue reliability on recalibration. We conclude that cue reliability, causal inference, and modality-specific biases guide cross-modal recalibration indirectly by determining the perception of audiovisual stimuli.Author summaryAudiovisual recalibration of spatial perception occurs when we receive audiovisual stimuli with a systematic spatial discrepancy. The brain must determine to which extent both modalities should be recalibrated. In this study, we scrutinized the mechanisms the brain employs to do so. To this aim, we conducted a classical recalibration task in which participants were adapted to spatially discrepant audiovisual stimuli. The visual component of the bimodal stimulus was either less, equally, or more reliable than the auditory component. We measured the amount of recalibration by computing the difference between participants’ unimodal localization responses before and after the recalibration task. Across participants, the influence of visual reliability on auditory recalibration varied fundamentally. We compared three models of recalibration. Only a causal-inference model of recalibration captured the diverse influences of cue reliability on recalibration found in our study, and this model is able to replicate contradictory results found in previous studies. In this model, recalibration depends on the discrepancy between a cue and its final estimate. Cue reliability, perceptual biases, and the degree to which participants infer that the two cues come from a common source govern audiovisual perception and therefore audiovisual recalibration.

Cue reliability, salience and early comprehension of agreement: Evidence from Greek

Previous studies across languages (English, Spanish, French) have argued that perceptual salience and cue reliability can explain cross-linguistic differences in early comprehension of verbal agreement. Here we tested this hypothesis further by investigating early comprehension in Greek, where markers have high salience and reliability (compared to Spanish and English) predicting early comprehension, as in French. We investigated two and three-year-old Greek-speaking children's ability to distinguish third person singular and plural agreement in a picture-selection task. We also examined the frequency of these morphemes in child-directed speech to address input effects. Results showed that three-year-olds are sensitive to both singular and plural agreement, earlier than children acquiring English and Spanish, but later than French, and despite singular agreement being more frequent than plural agreement in the child corpus. These findings provide further support for the role of salience and reliability during early acquisition, while highlighting a potential effect of morpheme position.

Neural dynamics of cue reliability in perceptual decisions

To extract meaningful information from scenes, the visual system must combine local cues that can vary greatly in their degree of reliability. Here, we asked whether cue reliability mostly affects visual or decision-related processes, using visual evoked potentials (VEPs) and a model-based approach to identify when and where stimulus-evoked brain activity reflects cue reliability. Participants performed a shape discrimination task on Gaborized ellipses while we parametrically, and independently, varied the reliability of contour or surface cues. We modelled the expected behavioral performance as a linear function of cue reliability and established at what latencies and electrodes VEP activity reflected behavioral sensitivity to cue reliability. We found that VEPs were linearly related to the individual behavioral predictors at around 400 ms post-stimulus, at electrodes over parietal and lateral temporal cortex. The observed cue reliability effects were similar for variations in contour and surface cues. Notably, effects of cue reliability were absent at earlier latencies where visual shape information is typically reported, and also in data time-locked to the behavioral response, suggesting the effects are not decision-related. These results indicate that reliability of visual cues is reflected in late distributed perceptual processes.