Estimating Difficulty Score of Visual Search in Images for Semi-supervised Object Detection

AbstractWhat limits our ability to find an object we are looking for? There are two competing models: one explains attentional limitations during visual search in terms of a serial processing computation, the other attributes limitations to noisy parallel processing. Both models predict human visual search behavior when applied to the simplified stimuli often used in experiments, but it remains unclear how to extend them to account for search of complex natural scenes. Models exist of natural scene search, but they do not predict whether a given scene will limit search accuracy. Here we propose an alternate mechanism to explain limitations across stimuli types: visual search is limited by an “untangling” computation, proposed to underlie object recognition. To test this idea, we ask whether models of object recognition account for visual search behavior. The current best-in-class models are artificial neural networks (ANNs) that accurately predict both behavior and neural activity in the primate visual system during object recognition tasks. Unlike dominant visual search models, ANNs can provide predictions for any image. First we test ANN-based object recognition models with simplified stimuli typically used in studies of visual search. We find these models exhibit a hallmark effect of such studies: a drop in target detection accuracy as the number of distractors increases. Further experiments show this effect results from learned representations: networks that are not pre-trained for object recognition can achieve near perfect accuracy. Next we test these models with complex natural images, using a version of the Pascal VOC dataset where each image has a visual search difficulty score, derived from human reaction times. We find models exhibit a drop in accuracy as search difficulty score increases. We conclude that ANN-based object recognition models account for aspects of visual search behavior across stimuli types, and discuss how to extend these results.

Download Full-text

The Influences of Goal and Laterality of a Prepared Hand Response on a Co-Occurring Visual Search

Swiss Journal of Psychology ◽

10.1024/1421-0185/a000148 ◽

2015 ◽

Vol 74 (1) ◽

pp. 55-60 ◽

Cited By ~ 2

Author(s):

Alexandre Coutté ◽

Gérard Olivier ◽

Sylvane Faure

Keyword(s):

Visual Search ◽

Visual Target ◽

Motor Planning ◽

Search Performance ◽

Response Preparation ◽

Manual Response ◽

Response Hand ◽

Computer Screen ◽

Left Hand ◽

Manual Interaction

Computer use generally requires manual interaction with human-computer interfaces. In this experiment, we studied the influence of manual response preparation on co-occurring shifts of attention to information on a computer screen. The participants were to carry out a visual search task on a computer screen while simultaneously preparing to reach for either a proximal or distal switch on a horizontal device, with either their right or left hand. The response properties were not predictive of the target’s spatial position. The results mainly showed that the preparation of a manual response influenced visual search: (1) The visual target whose location was congruent with the goal of the prepared response was found faster; (2) the visual target whose location was congruent with the laterality of the response hand was found faster; (3) these effects have a cumulative influence on visual search performance; (4) the magnitude of the influence of the response goal on visual search is marginally negatively correlated with the rapidity of response execution. These results are discussed in the general framework of structural coupling between perception and motor planning.

Download Full-text

Scientific Problem Solving in a Virtual Laboratory: A Comparison Between Individuals and Pairs

Swiss Journal of Psychology ◽

10.1024/1421-0185.67.2.71 ◽

2008 ◽

Vol 67 (2) ◽

pp. 71-83 ◽

Cited By ~ 5

Author(s):

Yolanda A. Métrailler ◽

Ester Reijnen ◽

Cornelia Kneser ◽

Klaus Opwis

Keyword(s):

Visual Search ◽

Problem Solving ◽

Process Data ◽

Scientific Problem ◽

Domain Specific ◽

Scientific Problem Solving ◽

Before And After ◽

Verbal Data ◽

Psychological Knowledge ◽

Problem Solving Task

This study compared individuals with pairs in a scientific problem-solving task. Participants interacted with a virtual psychological laboratory called Virtue to reason about a visual search theory. To this end, they created hypotheses, designed experiments, and analyzed and interpreted the results of their experiments in order to discover which of five possible factors affected the visual search process. Before and after their interaction with Virtue, participants took a test measuring theoretical and methodological knowledge. In addition, process data reflecting participants’ experimental activities and verbal data were collected. The results showed a significant but equal increase in knowledge for both groups. We found differences between individuals and pairs in the evaluation of hypotheses in the process data, and in descriptive and explanatory statements in the verbal data. Interacting with Virtue helped all students improve their domain-specific and domain-general psychological knowledge.

Download Full-text

Visual Search Facilitation in Repeated Displays Depends on Visuospatial Working Memory

Experimental Psychology (formerly Zeitschrift für Experimentelle Psychologie) ◽

10.1027/1618-3169/a000125 ◽

2012 ◽

Vol 59 (1) ◽

pp. 47-54 ◽

Cited By ~ 16

Author(s):

Angela A. Manginelli ◽

Franziska Geringswald ◽

Stefan Pollmann

Keyword(s):

Working Memory ◽

Visual Search ◽

Control Experiment ◽

Memory Load ◽

Contextual Cueing ◽

Long Term Memory ◽

Visuospatial Working Memory ◽

Working Memory Load ◽

Memory Resources

When distractor configurations are repeated over time, visual search becomes more efficient, even if participants are unaware of the repetition. This contextual cueing is a form of incidental, implicit learning. One might therefore expect that contextual cueing does not (or only minimally) rely on working memory resources. This, however, is debated in the literature. We investigated contextual cueing under either a visuospatial or a nonspatial (color) visual working memory load. We found that contextual cueing was disrupted by the concurrent visuospatial, but not by the color working memory load. A control experiment ruled out that unspecific attentional factors of the dual-task situation disrupted contextual cueing. Visuospatial working memory may be needed to match current display items with long-term memory traces of previously learned displays.

Download Full-text