Eyes on Emotion: Dynamic Gaze Allocation During Emotion Perception From Speech-Like Stimuli

Abstract The majority of emotional expressions used in daily communication are multimodal and dynamic in nature. Consequently, one would expect that human observers utilize specific perceptual strategies to process emotions and to handle the multimodal and dynamic nature of emotions. However, our present knowledge on these strategies is scarce, primarily because most studies on emotion perception have not fully covered this variation, and instead used static and/or unimodal stimuli with few emotion categories. To resolve this knowledge gap, the present study examined how dynamic emotional auditory and visual information is integrated into a unified percept. Since there is a broad spectrum of possible forms of integration, both eye movements and accuracy of emotion identification were evaluated while observers performed an emotion identification task in one of three conditions: audio-only, visual-only video, or audiovisual video. In terms of adaptations of perceptual strategies, eye movement results showed a shift in fixations toward the eyes and away from the nose and mouth when audio is added. Notably, in terms of task performance, audio-only performance was mostly significantly worse than video-only and audiovisual performances, but performance in the latter two conditions was often not different. These results suggest that individuals flexibly and momentarily adapt their perceptual strategies to changes in the available information for emotion recognition, and these changes can be comprehensively quantified with eye tracking.

Download Full-text

Baseball Players’ Eye Movements and Higher Coincident-Timing Task Performance

Perceptual and Motor Skills ◽

10.1177/0031512520905435 ◽

2020 ◽

Vol 127 (3) ◽

pp. 571-586

Author(s):

Ikumi Tochikura ◽

Daisuke Sato ◽

Daiki Imoto ◽

Atsuo Nuruki ◽

Koya Yamashiro ◽

...

Keyword(s):

Decision Making ◽

Information Processing ◽

Eye Movements ◽

Task Performance ◽

Visual Information ◽

Absolute Error ◽

Visual Information Processing ◽

Baseball Players ◽

Mobile Targets ◽

Timing Task

Previous studies have reported that baseball players have higher than average visual information processing abilities and outstanding motor control. The speed and position of the baseball and the batter are constantly changing, leading skilled players to acquire highly accurate visual information processing and decision-making. This study sought to clarify how movement of the eyes is associated with baseball players’ higher coincident-timing task performance. We recruited 15 right-handed baseball players and 15 age-matched track and field athletes. On a computer-based coincident-timing task, we instructed participants to stop a computer image of a moving target by pressing a button at a designated point. We presented bidirectional moving targets with various velocities, presented in a random order. The targets’ moving angular velocity varied between 100, 83, 71, 63, 56, 50, and 46 deg/s. We conducted 168 repetitions (42 reps × 4 sets) of this coincident-timing task and measured participants’ eye movements during the task using Pupil Centre Corneal Reflection. Mixed-design analysis of variance results revealed participant group effects in favor of baseball players for timing absolute error and low absolute error, as predicted from prior visual processing and decision-making research with baseball players. However, in contrast to prior research, we found significantly shorter smooth-pursuit onset latency in elite baseball players, and there were no significant group differences for saccade onset and offset latencies. This may be explained by the difference in our research paradigm with mobile targets randomly presented at various velocities from the left and right. Our data showed baseball players’ higher than normal simultaneous timing execution for making decisions and movements based on visual information, even under laboratory conditions with randomly moving mobile targets.

Download Full-text

Directing voluntary temporal attention increases fixational stability

10.1101/449264 ◽

2018 ◽

Author(s):

Rachel N. Denison ◽

Shlomit Yuval-Greenberg ◽

Marisa Carrasco

Keyword(s):

Eye Movements ◽

Visual Perception ◽

Task Performance ◽

Visual Information ◽

Perceptual Sensitivity ◽

Temporal Attention ◽

Relevant Stimulus ◽

Gaze Stabilization ◽

Cortical Areas ◽

Fixational Eye Movements

AbstractOur visual input is constantly changing, but not all moments are equally relevant. Temporal attention, the prioritization of visual information at specific points in time, increases perceptual sensitivity at behaviorally relevant times. The dynamic processes underlying this increase are unclear. During fixation, humans make small eye movements called microsaccades, and inhibiting microsaccades improves perception of brief stimuli. Here we asked whether temporal attention changes the pattern of microsaccades in anticipation of brief stimuli. Human observers (female and male) judged brief stimuli presented within a short sequence. They were given either an informative precue to attend to one of the stimuli, which was likely to be probed, or an uninformative (neutral) precue. We found strong microsaccadic inhibition before the stimulus sequence, likely due to its predictable onset. Critically, this anticipatory inhibition was stronger when the first target in the sequence (T1) was precued (task-relevant) than when the precue was uninformative. Moreover, the timing of the last microsaccade before T1 and the first microsaccade after T1 shifted, such that both occurred earlier when T1 was precued than when the precue was uninformative. Finally, the timing of the nearest pre- and post-T1 microsaccades affected task performance. Directing voluntary temporal attention therefore impacts microsaccades, helping to stabilize fixation at the most relevant moments, over and above the effect of predictability. Just as saccading to a relevant stimulus can be an overt correlate of the allocation of spatial attention, precisely timed gaze stabilization can be an overt correlate of the allocation of temporal attention.Significance statementWe pay attention at moments in time when a relevant event is likely to occur. Such temporal attention improves our visual perception, but how it does so is not well understood. Here we discovered a new behavioral correlate of voluntary, or goal-directed, temporal attention. We found that the pattern of small fixational eye movements called microsaccades changes around behaviorally relevant moments in a way that stabilizes the position of the eyes. Microsaccades during a brief visual stimulus can impair perception of that stimulus. Therefore, such fixation stabilization may contribute to the improvement of visual perception at attended times. This link suggests that in addition to cortical areas, subcortical areas mediating eye movements may be recruited with temporal attention.

Download Full-text

A Sensory Memory to Preserve Visual Representations Across Eye Movements

10.1101/2020.02.28.970277 ◽

2020 ◽

Author(s):

Amir Akbarian ◽

Kaiser Niknam ◽

Kelsey Clark ◽

Behrad Noudoost ◽

Neda Nategh

Keyword(s):

Eye Movements ◽

Visual Information ◽

Continuous Flow ◽

Neuronal Response ◽

Visual Representations ◽

Extrastriate Cortex ◽

Visual World ◽

Temporal Precision ◽

Abrupt Changes ◽

Available Information

SUMMARYDuring eye movements, the continuous flow of visual information is frequently disrupted due to abrupt changes of the retinal image, yet our perception of the visual world is uninterrupted. In order to identify the neuronal response components necessary for the integration of perception across eye movements, we developed a computational model to trace the changes in the visuospatial sensitivity of neurons in the extrastriate cortex of macaque monkeys with high temporal precision. Employing the model, we examined the perceptual implications of these changes and found that by maintaining a memory of the visual scene, extrastriate neurons produce an uninterrupted representation of the visual world. These results reveal how our brain exploits available information to maintain the sense of vision in the absence of visual information.

Download Full-text

Situational Factors Competing for Attention

Journal of Media Psychology Theories Methods and Applications ◽

10.1027/1864-1105/a000002 ◽

2010 ◽

Vol 22 (1) ◽

pp. 2-13 ◽

Cited By ~ 39

Author(s):

Weiyu Zhang ◽

Se-Hoon Jeong ◽

Martin Fishbein†

Keyword(s):

Task Performance ◽

Visual Information ◽

Situational Factors ◽

Sexual Content ◽

Sexually Explicit ◽

Explicit Content ◽

Audio Information

This study investigates how multitasking interacts with levels of sexually explicit content to influence an individual’s ability to recognize TV content. A 2 (multitasking vs. nonmultitasking) by 3 (low, medium, and high sexual content) between-subjects experiment was conducted. The analyses revealed that multitasking not only impaired task performance, but also decreased TV recognition. An inverted-U relationship between degree of sexually explicit content and recognition of TV content was found, but only when subjects were multitasking. In addition, multitasking interfered with subjects’ ability to recognize audio information more than their ability to recognize visual information.

Download Full-text

Eye movements in sports research and practice: Immersive technologies as optimal environments for the study of gaze behaviour

10.31234/osf.io/fbwvk ◽

2020 ◽

Author(s):

David Harris ◽

Mark Wilson ◽

Tim Holmes ◽

Toby de Burgh ◽

Samuel James Vine

Keyword(s):

Virtual Reality ◽

Eye Movements ◽

Eye Tracking ◽

Visual Information ◽

Successful Performance ◽

Experimental Control ◽

Research And Practice ◽

Gaze Behaviour ◽

Immersive Technologies

Head-mounted eye tracking has been fundamental for developing an understanding of sporting expertise, as the way in which performers sample visual information from the environment is a major determinant of successful performance. There is, however, a long running tension between the desire to study realistic, in-situ gaze behaviour and the difficulties of acquiring accurate ocular measurements in dynamic and fast-moving sporting tasks. Here, we describe how immersive technologies, such as virtual reality, offer an increasingly compelling approach for conducting eye movement research in sport. The possibility of studying gaze behaviour in representative and realistic environments, but with high levels of experimental control, could enable significant strides forward for eye tracking in sport and improve understanding of how eye movements underpin sporting skills. By providing a rationale for virtual reality as an optimal environment for eye tracking research, as well as outlining practical considerations related to hardware, software and data analysis, we hope to guide researchers and practitioners in the use of this approach.

Download Full-text