Newly learned novel cues to location are combined with familiar cues but not always with each other

Mature perceptual systems can learn new arbitrary sensory signals (novel cues) to properties of the environment, but little is known about the extent to which novel cues are integrated into normal perception. In normal perception, multiple uncertain familiar cues are combined, often near-optimally (reliability-weighted averaging), to increase perceptual precision. We trained observers to use abstract novel cues to estimate horizontal locations of hidden objects on a monitor. In Experiment 1, four groups of observers each learned to use a different novel cue. All groups benefitted from a suboptimal but significant gain in precision using novel and familiar cues together after short-term training (3 x ~1.5 hour sessions), extending previous reports of novel-familiar cue combination. In Experiment 2, we tested whether two novel cues may also be combined with each other. One pair of novel cues could be combined to improve precision but the other could not, at least not after three sessions of repeated training. Overall, our results provide extensive evidence that novel cues can be learned and combined with familiar cues to enhance perception, but mixed evidence for whether perceptual and decision-making systems can extend this ability to the combination of multiple novel cues with only short-term training.

Central tendency biases must be accounted for to consistently capture Bayesian cue combination in continuous response data

Observers in perceptual tasks are often reported to combine multiple sensory cues in a weighted average that improves precision—in some studies, approaching statistically optimal (Bayesian) weighting, but in others departing from optimality, or not benefitting from combined cues at all. To correctly conclude which combination rules observers use, it is crucial to have accurate measures of their sensory precision and cue weighting. Here, we present a new approach for accurately recovering these parameters in perceptual tasks with continuous responses. Continuous responses have many advantages, but are susceptible to a central tendency bias, where responses are biased towards the central stimulus value. We show that such biases lead to inaccuracies in estimating both precision gains and cue weightings, two key measures used to assess sensory cue combination. We introduce a method that estimates sensory precision by regressing continuous responses on targets and dividing the variance of the residuals by the squared slope of the regression line, “correcting-out” the error introduced by the central bias and increasing statistical power. We also suggest a complementary analysis that recovers the sensory cue weights. Using both simulations and empirical data, we show that the proposed methods can accurately estimate sensory precision and cue weightings in the presence of central tendency biases. We conclude that central tendency biases should be (and can easily be) accounted for to consistently capture Bayesian cue combination in continuous response data.

The Effects of Cue Reliability on Crossmodal Recalibration in Adults and Children

Reliability-based cue combination is a hallmark of multisensory integration, while the role of cue reliability for crossmodal recalibration is less understood. The present study investigated whether visual cue reliability affects audiovisual recalibration in adults and children. Participants had to localize sounds, which were presented either alone or in combination with a spatially discrepant high- or low-reliability visual stimulus. In a previous study we had shown that the ventriloquist effect (indicating multisensory integration) was overall larger in the children groups and that the shift in sound localization toward the spatially discrepant visual stimulus decreased with visual cue reliability in all groups. The present study replicated the onset of the immediate ventriloquist aftereffect (a shift in unimodal sound localization following a single exposure of a spatially discrepant audiovisual stimulus) at the age of 6–7 years. In adults the immediate ventriloquist aftereffect depended on visual cue reliability, whereas the cumulative ventriloquist aftereffect (reflecting the audiovisual spatial discrepancies over the complete experiment) did not. In 6–7-year-olds the immediate ventriloquist aftereffect was independent of visual cue reliability. The present results are compatible with the idea of immediate and cumulative crossmodal recalibrations being dissociable processes and that the immediate ventriloquist aftereffect is more closely related to genuine multisensory integration.

EXPRESS: Cue Combination in Goal-Oriented Navigation

This study examined cue combination of self-motion and landmark cues in goal-localization. In an immersive virtual environment, before walking a two-leg path, participants learned the locations of three goal objects (one at the path origin, i.e., home) and landmarks. After walking the path without seeing landmarks or goals, participants indicated the locations of the home and non-home goals in four conditions: 1) path integration only, 2) landmarks only, 3) both path integration and the landmarks, and 4) path integration and rotated landmarks. The ratio of the length between the testing position (P) and the turning point (T) over the length between the T and the three goals (G) (i.e., PT/TG) was manipulated. The results showed the cue combination consistently for participants’ heading estimates but not for goal-localization. In Experiments 1-2 (using distal landmarks), the cue combination for goal estimates appeared in a small length ratio (PT/TG=0.5) but disappeared in a large length ratio (PT/TG=2). In Experiments 3-4 (using proximal landmarks), while the cue combination disappeared for the home with a medium length ratio (PT/TG=1), it appeared for the non-home goal with a large length ratio (PT/TG=2) and only disappeared with a very large length ratio (PT/TG=3). These findings are explained by a model stipulating that cue combination occurs in self-localization (e.g., heading estimates), which leads to one estimate of the goal location; proximal landmarks produce another goal location estimate; these two goal estimates are then combined, which may only occur for non-home goals.

Central tendency biases must be accounted for to consistently capture Bayesian cue combination in continuous response data

Observers in perceptual tasks are often reported to combine multiple sensory cues in a weighted average that improves precision – in some studies, approaching statistically-optimal (Bayesian) weighting, but in others departing from optimality, or not benefitting from combined cues at all. To correctly conclude which combination rules observers use, it is crucial to have accurate measures of their sensory precision and cue weighting. Here, we present a new approach for accurately recovering these parameters in perceptual tasks with continuous responses. Continuous responses have many advantages, but are susceptible to a central tendency bias, where responses are biased towards the central stimulus value. We show such biases lead to inaccuracies in estimating both precision gains and cue weightings, two key measures used to assess sensory cue combination. We introduce a method that estimates sensory precision by regressing continuous responses on targets and dividing the variance of the residuals by the squared slope of the regression line, “correcting-out” the error introduced by the central bias and increasing statistical power. We also suggest a complementary analysis that recovers the sensory cue weights. Using both simulations and empirical data, we show that the proposed methods can accurately estimate sensory precision and cue weightings in the presence of central tendency biases. We conclude that central tendency biases should be (and can easily be) accounted for to consistently capture Bayesian cue combination in continuous response data.

Adaptive weighting of taste and odor cues during flavor choice

The authors demonstrate that rats make choices about which flavor solution (i.e., taste-odor mixture) to consume by weighting the individual taste and odor components according to the reliability of the information they provide about which solution is the preferred one. A similar weighting operation underlies multisensory cue combination in other domains and offers novel insight into the computations underlying multisensory flavor perception and food choice behavior.