Meaning and Expected Surfaces Combine to Guide Attention During Visual Search in Scenes

How do spatial constraints and meaningful scene regions interact to control overt attention during visual search for objects in real-world scenes? To answer this question, we combined novel surface maps of the likely locations of target objects with maps of the spatial distribution of scene semantic content. The surface maps captured likely target surfaces as continuous probabilities. Meaning was represented by meaning maps highlighting the distribution of semantic content in local scene regions and objects. Attention was indexed by eye movements during search for target objects that varied in the likelihood they would appear on specific surfaces. The interaction between surface maps and meaning maps was analyzed to test whether fixations were directed to meaningful scene regions on target-related surfaces. Overall, meaningful scene regions were more likely to be fixated if they appeared on target-related surfaces than if they appeared on target-unrelated surfaces. These findings suggest that the visual system prioritizes meaningful scene regions on target-related surfaces during visual search in scenes.

Download Full-text

The effects of target template specificity on visual search in real-world scenes: Evidence from eye movements

Journal of Vision ◽

10.1167/9.11.8 ◽

2009 ◽

Vol 9 (11) ◽

pp. 8-8 ◽

Cited By ~ 71

Author(s):

G. L. Malcolm ◽

J. M. Henderson

Keyword(s):

Visual Search ◽

Eye Movements ◽

Real World ◽

Target Template ◽

Template Specificity

Download Full-text

The Effect of Scene Context on Visual Search in Real‐World Scenes

Inquiry@Queen's Undergraduate Research Conference Proceedings ◽

10.24908/iqurcp.7518 ◽

2017 ◽

Author(s):

Samia Hussein

Keyword(s):

Visual Search ◽

Eye Movements ◽

Visual Attention ◽

Real World ◽

Target Object ◽

Scene Context ◽

Efficient Search ◽

Object Attention ◽

Study Participants

The present study examined the effect of scene context on guidance of attention during visual search in real‐world scenes. Prior research has demonstrated that when searching for an object, attention is usually guided to the region of a scene that most likely contains that target object. This study examined two possible mechanisms of attention that underlie efficient search: enhancement of attention (facilitation) and a deficiency of attention (inhibition). In this study, participants (N=20) were shown an object name and then required to search through scenes for the target while their eye movements were tracked. Scenes were divided into target‐relevant contextual regions (upper, middle, lower) and participants searched repeatedly in the same scene for different targets either in the same region or in different regions. Comparing repeated searches within the same scene across different regions, we expect to find that visual search is faster and more efficient (facilitation of attention) in regions of a scene where attention was previously deployed. At the same time, when searching across different regions, we expect searches to be slower and less efficient (inhibition of attention) because those regions were previously ignored. Results from this study help to better understand how mechanisms of visual attention operate within scene contexts during visual search.

Download Full-text

The 35th Sir Frederick Bartlett Lecture: Eye movements and attention in reading, scene perception, and visual search

Quarterly Journal of Experimental Psychology ◽

10.1080/17470210902816461 ◽

2009 ◽

Vol 62 (8) ◽

pp. 1457-1506 ◽

Cited By ~ 1103

Author(s):

Keith Rayner

Keyword(s):

Visual Search ◽

Eye Movements ◽

Cognitive Processes ◽

Real World ◽

Scene Perception ◽

Movement Control ◽

Preview Benefit ◽

Visual World Paradigm ◽

Perceptual Span ◽

Eye Movements During Reading

Eye movements are now widely used to investigate cognitive processes during reading, scene perception, and visual search. In this article, research on the following topics is reviewed with respect to reading: (a) the perceptual span (or span of effective vision), (b) preview benefit, (c) eye movement control, and (d) models of eye movements. Related issues with respect to eye movements during scene perception and visual search are also reviewed. It is argued that research on eye movements during reading has been somewhat advanced over research on eye movements in scene perception and visual search and that some of the paradigms developed to study reading should be more widely adopted in the study of scene perception and visual search. Research dealing with “real-world” tasks and research utilizing the visual-world paradigm are also briefly discussed.

Download Full-text

Meaning Guides Attention in Real-World Scene Images: Evidence from Eye Movements and Meaning Maps

10.1101/207076 ◽

2017 ◽

Author(s):

John M. Henderson ◽

Taylor R. Hayes

Keyword(s):

Spatial Distribution ◽

Eye Movements ◽

Real World ◽

Semantic Information ◽

Relevance Theory ◽

Image Features ◽

Attentional Guidance ◽

Unique Variance

AbstractWe compared the influences of meaning and salience on attentional guidance in scene images. Meaning was captured by “meaning maps” representing the spatial distribution of semantic information in scenes. Meaning maps were coded in a format that could be directly compared to maps of image salience generated from image features. We investigated the degree to which meaning versus image salience predicted human viewers’ spatial distribution of attention over scenes, with attention operationalized as duration-weighted fixation density. The results showed that both meaning and salience predicted the distribution of attention, but that when the correlation between meaning and salience was statistically controlled, meaning accounted for unique variance in attention but salience did not. This pattern was observed for early as well as late fixations, for fixations following short as well as long saccades, and for fixations including or excluding the centers of the scenes. The results strongly suggest that meaning guides attention in real world scenes. We discuss the results from the perspective of a cognitive relevance theory of attentional guidance in scenes.

Download Full-text

Caudal Prefrontal Cortex

10.1093/oso/9780198844570.003.0005 ◽

2021 ◽

pp. 153-190

Author(s):

Richard E. Passingham

Keyword(s):

Visual Search ◽

Eye Movements ◽

Covert Attention ◽

Frontal Eye Field ◽

Top Down ◽

Cluttered Environment ◽

Eye Field ◽

Sensory Responses ◽

Search For Objects ◽

Oculomotor Nuclei

The caudal prefrontal (PF) cortex supports the visual search for objects such as foods both through eye movements and covert attention, and its connections explain how it can do this. The caudal PF cortex, which includes the frontal eye field, has connections with both the dorsal and ventral visual streams. The direction of eye movements depends on its connections with the superior colliculus and oculomotor nuclei. Covert attention depends on enhanced sensory responses that are mediated through top-down interactions with posterior sensory areas. Along with the granular parts of the orbital PF cortex, the caudal PF cortex evolved in early primates. Together, these two new areas led to improvements in searching for and evaluating objects that are hidden in a cluttered environment.

Download Full-text

Human Visual Search Follows Suboptimal Bayesian Strategy Revealed by a Spatiotemporal Computational Model

10.1101/779652 ◽

2019 ◽

Author(s):

Yunhui Zhou ◽

Yuguo Yu

Keyword(s):

Visual Search ◽

Eye Movements ◽

Visual System ◽

Eye Movement ◽

Search Strategy ◽

Memory Capacity ◽

Search Performance ◽

Human Eye ◽

Visibility Map ◽

Temporal Course

AbstractHumans perform sequences of eye movements to search for a target in complex environment, but the efficiency of human search strategy is still controversial. Previous studies showed that humans can optimally integrate information across fixations and determine the next fixation location. However, their models ignored the temporal control of eye movement, ignored the limited human memory capacity, and the model prediction did not agree with details of human eye movement metrics well. Here, we measured the temporal course of human visibility map and recorded the eye movements of human subjects performing a visual search task. We further built a continuous-time eye movement model which considered saccadic inaccuracy, saccadic bias, and memory constraints in the visual system. This model agreed with many spatial and temporal properties of human eye movements, and showed several similar statistical dependencies between successive eye movements. In addition, our model also predicted that the human saccade decision is shaped by a memory capacity of around 8 recent fixations. These results suggest that human visual search strategy is not strictly optimal in the sense of fully utilizing the visibility map, but instead tries to balance between search performance and the costs to perform the task.Author SummaryDuring visual search, how do humans determine when and where to make eye movement is an important unsolved issue. Previous studies suggested that human can optimally use the visibility map to determine fixation locations, but we found that such model didn’t agree with details of human eye movement metrics because it ignored several realistic biological limitations of human brain functions, and couldn’t explain the temporal control of eye movements. Instead, we showed that considering the temporal course of visual processing and several constrains of the visual system could greatly improve the prediction on the spatiotemporal properties of human eye movement while only slightly affected the search performance in terms of median fixation numbers. Therefore, humans may not use the visibility map in a strictly optimal sense, but tried to balance between search performance and the costs to perform the task.

Download Full-text

Individual differences in visual search performance extend from artificial arrays to naturalistic environments.

10.1101/2021.10.15.464609 ◽

2021 ◽

Author(s):

Thomas L. Botch ◽

Brenda D. Garcia ◽

Yeo Bi Choi ◽

Caroline E. Robertson

Keyword(s):

Individual Differences ◽

Visual Search ◽

Eye Movements ◽

Real World ◽

Search Task ◽

Visual Search Task ◽

Search Behavior ◽

Search Performance ◽

Set Size ◽

Visual Clutter

Visual search is a universal human activity in naturalistic environments. Traditionally, visual search is investigated under tightly controlled conditions, where head-restricted participants locate a minimalistic target in a cluttered array presented on a computer screen. Do classic findings of visual search extend to naturalistic settings, where participants actively explore complex, real-world scenes? Here, we leverage advances in virtual reality (VR) technology to relate individual differences in classic visual search paradigms to naturalistic search behavior. In a naturalistic visual search task, participants looked for an object within their environment via a combination of head-turns and eye-movements using a head-mounted display. Then, in a classic visual search task, participants searched for a target within a simple array of colored letters using only eye-movements. We tested how set size, a property known to limit visual search within computer displays, predicts the efficiency of search behavior inside immersive, real-world scenes that vary in levels of visual clutter. We found that participants' search performance was impacted by the level of visual clutter within real-world scenes. Critically, we also observed that individual differences in visual search efficiency in classic search predicted efficiency in real-world search, but only when the comparison was limited to the forward-facing field of view for real-world search. These results demonstrate that set size is a reliable predictor of individual performance across computer-based and active, real-world visual search behavior.

Download Full-text