Integration of allocentric and egocentric visual information in a convolutional / multilayer perceptron network model of goal-directed gaze shifts

Allocentric (landmark-centered) and egocentric (eye-centered) visual codes are fundamental for spatial cognition, navigation, and goal-directed movement. Neuroimaging and neurophysiology suggest these codes are segregated initially, but then reintegrated in frontal cortex for movement control. We created and validated a theoretical framework for this process using physiologically constrained inputs and outputs. To implement a general framework, we integrated a Convolutional Neural Network (CNN) of the visual system with a Multilayer Perceptron (MLP) model of the sensorimotor transformation. The network was trained on a task where a landmark shifted relative to the saccade target. These visual parameters were input to the CNN, the CNN output and initial gaze position to the MLP, and a decoder transformed MLP output into saccade vectors. Decoded saccade output replicated idealized training sets with various allocentric weightings, and actual monkey data where the landmark shift had a partial influence (R2 = 0.8). Furthermore, MLP output units accurately simulated prefrontal response field shifts recorded from monkeys during the same paradigm. In summary, our model replicated both the general properties of the visuomotor transformations for gaze and specific experimental results obtained during allocentric-egocentric integration, suggesting it can provide a general framework for understanding these and other complex visuomotor behaviors.

Precise visuomotor transformations underlying collective behavior in larval zebrafish

Complex schooling behaviors result from local interactions among individuals. Yet, how sensory signals from neighbors are analyzed in the visuomotor stream of animals is poorly understood. Here, we studied aggregation behavior in larval zebrafish and found that over development larvae transition from overdispersed groups to tight shoals. Using a virtual reality assay, we characterized the algorithms fish use to transform visual inputs from neighbors into movement decisions. We found that young larvae turn away from virtual neighbors by integrating and averaging retina-wide visual occupancy within each eye, and by using a winner-take-all strategy for binocular integration. As fish mature, their responses expand to include attraction to virtual neighbors, which is based on similar algorithms of visual integration. Using model simulations, we show that the observed algorithms accurately predict group structure over development. These findings allow us to make testable predictions regarding the neuronal circuits underlying collective behavior in zebrafish.

Trial-by-trial modulation of express visuomotor responses induced by symbolic or barely detectable cues

Human cerebral cortex can produce visuomotor responses that are modulated by contextual and task-specific constraints. However, the distributed cortical network for visuomotor transformations limits the minimal response time of that pathway. Notably, humans can generate express visuomotor responses in arm muscles that are inflexibly tuned to the target location and occur 80-120ms from stimulus presentation (stimulus-locked responses, SLRs). This suggests a subcortical pathway for visuomotor transformations that might involve the superior colliculus and its downstream reticulo-spinal projections. Here we investigated whether cognitive expectations can modulate the SLR. In one experiment, we recorded surface EMG from shoulder muscles as participants reached toward a visual target whose location was unpredictable in control conditions, and partially predictable in cue conditions by interpreting a symbolic cue (75% validity). Valid symbolic cues led to earlier and larger SLRs than control conditions; invalid symbolic cues produced later and smaller SLRs than control conditions. This is consistent with a cortical top-down modulation of the putative subcortical SLR-network. In a second experiment, we presented high-contrast targets in isolation (control) or ~24ms after low-contrast stimuli, which could appear at the same (valid cue) or opposite (invalid cue) location as the target, and with equal probability (50% cue validity). We observed earlier SLRs than control with the valid low-contrast cues, whereas the invalid cues led to the opposite results. These findings may reflect bottom-up attentional mechanisms, potentially evolving subcortically via the superior colliculus. Overall, our results support both top-down and bottom-up modulations of the putative subcortical SLR network in humans.

Precise visuomotor transformations underlying collective behavior in larval zebrafish

Complex schooling behaviors result from local interactions among individuals. Yet, how sensory signals from neighbors are analyzed in the visuomotor stream of animals is poorly understood. Here, we studied aggregation behavior in larval zebrafish and found that over development larvae transition from overdispersed groups to tight shoals. Using a virtual reality assay, we characterized the algorithms fish use to transform visual inputs from neighbors into movement decisions. We found that young larvae turn away from retinal "clutter" by integrating and averaging retina-wide visual inputs within each eye, and by using a winner-take-all strategy for binocular integration. As fish mature, their responses expand to include attraction to low retinal clutter, that is based on similar algorithms of visual integration. Using model simulations, we show that the observed algorithms accurately predict group structure over development. These findings allow us to make testable predictions regarding the neuronal circuits underlying collective behavior in zebrafish.

Trial-by-trial modulation of express visuomotor responses induced by symbolic or barely detectable cues

ABSTRACTHuman cerebral cortex can produce visuomotor responses that are modulated by contextual and task-specific constraints. However, the distributed cortical network for visuomotor transformations limits the minimal response time of that pathway. Notably, humans can generate express visuomotor responses that are inflexibly tuned to the target location and occur 80-120ms from stimulus presentation (stimulus-locked responses, SLRs). This suggests a subcortical pathway for visuomotor transformations involving the superior colliculus and its downstream reticulo-spinal projections. Here we investigated whether cognitive expectations can modulate the SLR. In one experiment, we recorded surface EMG from shoulder muscles as participants reached toward a visual target whose location was unpredictable in control conditions, and partially predictable in cue conditions by extrapolating a symbolic cue (75% validity). Valid symbolic cues led to faster and larger SLRs than control conditions; invalid symbolic cues produced slower and smaller SLRs than control conditions. This is consistent with a cortical top-down modulation of the putative subcortical SLR-network. In a second experiment, we presented high-contrast targets in isolation (control) or ~24ms after low-contrast stimuli, which could appear at the same (valid cue) or opposite (invalid cue) location as the target, and with equal probability (50% cue validity). We observed faster SLRs than control with the valid low-contrast cues, whereas the invalid cues led to the opposite results. These findings may reflect exogenous priming mechanisms of the SLR network, potentially evolving subcortically via the superior colliculus. Overall, our results support both top-down and bottom-up modulations of the putative subcortical SLR network in humans.NEW & NOTEWORTHYExpress visuomotor responses in humans appear to reflect subcortical sensorimotor transformation of visual inputs, potentially conveyed via the tecto-reticulo-spinal pathway. Here we show that the express responses are influenced both by symbolic and barely detectable spatial cues about stimulus location. The symbolic cue-induced effects suggest cortical top-down modulation of the putative subcortical visuomotor network. The effects of barely detectable cues may reflect exogenous priming mechanisms of the tecto-reticulo-spinal pathway.

The influence of temporal predictability on express visuomotor responses

Humans are able to generate target-directed visuomotor responses in less than 100ms after stimulus onset. These "express" responses have been termed stimulus-locked responses (SLRs) and are proposed to be modulated by visuomotor transformations performed subcortically via the superior colliculus. Unfortunately, these responses have proven difficult to detect consistently across individuals. The recent report of an effective paradigm for generating SLRs in 100% of participants appears to change this. The task required the interception of a target moving at a constant velocity that emerged from behind a barrier. Here we aimed to reproduce the efficacy of this paradigm for eliciting SLRs and to test the hypothesis that its effectiveness derives from the predictability of target onset time as opposed to target motion per se. In one experiment, we recorded surface EMG from shoulder muscles as participants made reaches to intercept temporally predictable or unpredictable targets. Consistent with our hypothesis, predictably timed targets produced more frequent and stronger SLRs than unpredictably timed targets. In a second experiment, we compared different temporally predictable stimuli and observed that transiently presented targets produced larger and earlier SLRs than sustained moving targets. Our results suggest that target motion is not critical for facilitating the SLR expression and that timing predictability does not rely on extrapolation of a physically plausible motion trajectory. These findings provide support for a mechanism whereby an internal timer, probably located in cerebral cortex, primes the processing of both visual input and motor output within the superior colliculus to produce SLRs.

The influence of temporal predictability on express visuomotor responses

ABSTRACTVolitional visuomotor responses in humans are generally thought to manifest 100ms or more after stimulus onset. Under appropriate conditions, however, much faster target-directed responses can be produced at upper limb and neck muscles. These “express” responses have been termed stimulus-locked responses (SLRs) and are proposed to be modulated by visuomotor transformations performed subcortically via the superior colliculus. Unfortunately, for those interested in studying SLRs, these responses have proven difficult to detect consistently across individuals. The recent report of an effective paradigm for generating SLRs in 100% of participants appears to change this. The task required the interception of a moving target that emerged from behind a barrier at a time consistent with the target velocity. Here we aimed to reproduce the efficacy of this paradigm for eliciting SLRs and to test the hypothesis that its effectiveness derives from the predictability of target onset time as opposed to target motion per se. In one experiment, we recorded surface EMG from shoulder muscles as participants made reaches to intercept temporally predictable or unpredictable targets. Consistent with our hypothesis, predictably timed targets produced more frequent and stronger SLRs than unpredictably timed targets. In a second experiment, we compared different temporally predictable stimuli and observed that transiently presented targets produced larger and earlier SLRs than sustained moving targets. Our results suggest that target motion is not critical for facilitating the expression of an SLR and that timing predictability does not rely on extrapolation of a physically plausible motion trajectory. These findings provide support for a mechanism whereby an internal timer, probably located in cerebral cortex, primes the processing of both visual input and motor output within the superior colliculus to produce SLRs.

An emerging target paradigm evokes fast visuomotor responses on human upper limb muscles

ABSTRACTTo reach towards a seen object, visual information has to be transformed into motor commands. Visual information such as the object’s colour, shape, and size is processed and integrated within numerous brain areas, then ultimately relayed to the motor periphery. In some instances we must react as fast as possible. These fast visuomotor transformations, and their underlying neurological substrates, are poorly understood in humans as they have lacked a reliable biomarker. Stimulus-locked responses (SLRs) are short latency (<100 ms) bursts of electromyographic (EMG) activity representing the first wave of muscle recruitment influenced by visual stimulus presentation. SLRs provide a quantifiable output of rapid visuomotor transformations, but SLRs have not been consistently observed in all subjects in past studies. Here we describe a new, behavioural paradigm featuring the sudden emergence of a moving target below an obstacle that consistently evokes robust SLRs. Human participants generated visually-guided reaches toward or away from the emerging target using a robotic manipulandum while surface electrodes recorded EMG activity from the pectoralis major muscle. In comparison to previous studies that investigated SLRs using static stimuli, the SLRs evoked with this emerging target paradigm were larger, evolved earlier, and were present in all participants. Reach reaction times (RTs) were also expedited in the emerging target paradigm. This paradigm affords numerous opportunities for modification that could permit systematic study of the impact of various sensory, cognitive, and motor manipulations on fast visuomotor responses. Overall, our results demonstrate that an emerging target paradigm is capable of consistently and robustly evoking activity within a fast visuomotor system.SUMMARYWe present a new behavioual paradigm that elicits robust fast visuomotor responses on human upper limb muscles during visually guided reaches.