Focusing on the face or getting distracted by social signals? The effect of distracting gestures on attentional focus in natural interaction

Attentional orienting towards others’ gaze direction or pointing has been wellinvestigated in laboratory conditions. However, less is known about the operation ofattentional mechanisms in online naturalistic social interaction scenarios. It is equally plausible that following social directional cues (gaze, pointing) occurs reflexively, and/orthat it is influenced by top-down cognitive factors. In a mobile eye-tracking experiment,we show that under natural interaction conditions overt attentional orienting is notnecessarily reflexively triggered by pointing gestures or a combination of gaze shifts andpointing gestures. We found that participants conversing with an experimenter, who,during the interaction, would play out pointing gestures as well as directional gaze movements, continued to mostly focus their gaze on the face of the experimenter, demonstrating the significance of attending to the face of the interaction partner – in linewith effective top-down control over reflexive orienting of attention in the direction of social cues.

Focusing on the face or getting distracted by social signals? The effect of distracting gestures on attentional focus in natural interaction

Attentional orienting towards others’ gaze direction or pointing has been well investigated in laboratory conditions. However, less is known about the operation of attentional mechanisms in online naturalistic social interaction scenarios. It is equally plausible that following social directional cues (gaze, pointing) occurs reflexively, and/or that it is influenced by top-down cognitive factors. In a mobile eye-tracking experiment, we show that under natural interaction conditions overt attentional orienting is not necessarily reflexively triggered by pointing gestures or a combination of gaze shifts and pointing gestures. We found that participants conversing with an experimenter, who, during the interaction, would play out pointing gestures as well as directional gaze movements, continued to mostly focus their gaze on the face of the experimenter, demonstrating the significance of attending to the face of the interaction partner – in line with effective top-down control over reflexive orienting of attention in the direction of social cues.

Vestigial auriculomotor activity indicates the direction of auditory attention in humans

Unlike dogs and cats, people do not point their ears as they focus attention on novel, salient, or task-relevant stimuli. Our species may nevertheless have retained a vestigial pinna-orienting system that has persisted as a 'neural fossil’ within in the brain for about 25 million years. Consistent with this hypothesis, we demonstrate that the direction of auditory attention is reflected in sustained electrical activity of muscles within the vestigial auriculomotor system. Surface electromyograms (EMGs) were taken from muscles that either move the pinna or alter its shape. To assess reflexive, stimulus-driven attention we presented novel sounds from speakers at four different lateral locations while the participants silently read a boring text in front of them. To test voluntary, goal-directed attention we instructed participants to listen to a short story coming from one of these speakers, while ignoring a competing story from the corresponding speaker on the opposite side. In both experiments, EMG recordings showed larger activity at the ear on the side of the attended stimulus, but with slightly different patterns. Upward movement (perking) differed according to the lateral focus of attention only during voluntary orienting; rearward folding of the pinna’s upper-lateral edge exhibited such differences only during reflexive orienting. The existence of a pinna-orienting system in humans, one that is experimentally accessible, offers opportunities for basic as well as applied science.

Vestigial Auriculomotor Activity Indicates the Direction of Auditory Attention in Humans

It is commonly assumed that, unlike dogs and cats, we humans do not make ear movements when focusing our attention reflexively toward novel sounds or voluntarily toward those that are goal–relevant. In fact, it has been suggested that we do have a pinna–orienting system. Although this system became vestigial about 25 million years ago, it still exists as a “neural fossil” within the brain. Consistent with this hypothesis, we demonstrate for the first time that the direction of auditory attention is reflected in the sustained electrical activity of muscles within the vestigial auriculomotor system.Surface electromyograms (EMGs) were taken from muscles that either move the pinna or alter its shape. To assess reflexive, stimulus-driven attention we presented novel sounds from speakers at four different lateral locations while the participants silently read a boring text in front of them. To test voluntary, goal-directed attention we instructed participants to listen to a short story coming from one of these speakers, while ignoring a competing story from the corresponding speaker on the opposite side.In both experiments, EMG recordings showed larger activity at the ear on the side of the attended stimulus, but with slightly different patterns. Upward movement (perking) differed according to the lateral focus of attention only during voluntary orienting; rearward folding of the pinna’s upper-lateral edge exhibited such differences only during reflexive orienting. The existence of a pinna-orienting system in humans, one that is experimentally accessible, offers opportunities for basic as well as applied science. It could lead to a better understanding of the evolution of auditory attention and support the near real–time decoding of auditory attention in technical applications, for example, for attentionally controlled hearing aids that preferentially amplify sounds the user is attempting to listen to.

On the role of eye contact in gaze cueing

Most experimental protocols examining joint attention with the gaze cueing paradigm are “observational” and “offline”, thereby not involving social interaction. We examined whether within a naturalistic online interaction, real-time eye contact influences the gaze cueing effect (GCE). We embedded gaze cueing in an interactive protocol with the iCub humanoid robot. This has the advantage of ecological validity combined with excellent experimental control. Critically, before averting the gaze, iCub either established eye contact or not, a manipulation enabled by an algorithm detecting position of the human eyes. For non-predictive gaze cueing procedure (Experiment 1), only the eye contact condition elicited GCE, while for counter-predictive procedure (Experiment 2), only the condition with no eye contact induced GCE. These results reveal an interactive effect of strategic (gaze validity) and social (eye contact) top-down components on the reflexive orienting of attention induced by gaze cues. More generally, we propose that naturalistic protocols with an embodied presence of an agent can cast a new light on mechanisms of social cognition.

Endogenous orienting in the archer fish

The literature has long emphasized the neocortex’s role in volitional processes. In this work, we examined endogenous orienting in an evolutionarily older species, the archer fish, which lacks neocortex-like cells. We used Posner’s classic endogenous cuing task, in which a centrally presented, spatially informative cue is followed by a target. The fish responded to the target by shooting a stream of water at it. Interestingly, the fish demonstrated a human-like “volitional” facilitation effect: their reaction times to targets that appeared on the side indicated by the precue were faster than their reaction times to targets on the opposite side. The fish also exhibited inhibition of return, an aftermath of orienting that commonly emerges only in reflexive orienting tasks in human participants. We believe that this pattern demonstrates the acquisition of an arbitrary connection between spatial orienting and a nonspatial feature of a centrally presented stimulus in nonprimate species. In the literature on human attention, orienting in response to such contingencies has been strongly associated with volitional control. We discuss the implications of these results for the evolution of orienting, and for the study of volitional processes in all species, including humans.