Saliency-Based Gaze Visualization for Eye Movement Analysis

Gaze movement and visual stimuli have been utilized to analyze human visual attention intuitively. Gaze behavior studies mainly show statistical analyses of eye movements and human visual attention. During these analyses, eye movement data and the saliency map are presented to the analysts as separate views or merged views. However, the analysts become frustrated when they need to memorize all of the separate views or when the eye movements obscure the saliency map in the merged views. Therefore, it is not easy to analyze how visual stimuli affect gaze movements since existing techniques focus excessively on the eye movement data. In this paper, we propose a novel visualization technique for analyzing gaze behavior using saliency features as visual clues to express the visual attention of an observer. The visual clues that represent visual attention are analyzed to reveal which saliency features are prominent for the visual stimulus analysis. We visualize the gaze data with the saliency features to interpret the visual attention. We analyze the gaze behavior with the proposed visualization to evaluate that our approach to embedding saliency features within the visualization supports us to understand the visual attention of an observer.

MemX

This work presents MemX: a biologically-inspired attention-aware eyewear system developed with the goal of pursuing the long-awaited vision of a personalized visual Memex. MemX captures human visual attention on the fly, analyzes the salient visual content, and records moments of personal interest in the form of compact video snippets. Accurate attentive scene detection and analysis on resource-constrained platforms is challenging because these tasks are computation and energy intensive. We propose a new temporal visual attention network that unifies human visual attention tracking and salient visual content analysis. Attention tracking focuses computation-intensive video analysis on salient regions, while video analysis makes human attention detection and tracking more accurate. Using the YouTube-VIS dataset and 30 participants, we experimentally show that MemX significantly improves the attention tracking accuracy over the eye-tracking-alone method, while maintaining high system energy efficiency. We have also conducted 11 in-field pilot studies across a range of daily usage scenarios, which demonstrate the feasibility and potential benefits of MemX.

High-definition human visual attention mapping using wavelets

In this work, three novel approaches to detecting visual attention in images are presented. The idea behind detecting areas within images or video that naturally attract a viewer’s attention is based on the concept of generating pre-attentive saliency maps. Saliency, in and of itself, relates to some measure of “conspicuity” in the visual field and is believed to be an important precursor for many tasks in computer vision. One of the proposed methods in this thesis detects salient regions, while the other two detect salient edges. The classical approach to saliency detection proposed by Itti is extended by introducing wavelets as a lossless resizing tool while maintaining the aspect of biological inspiration. In addition to this, the spectral residual method and the frequency tuned method are modified using wavelets to allow for salient edge detection. Tests show that the proposed methods yield results that are not only comparable to leading,cutting-edge methods, but also exceed them in terms of correct and complete object detection as well as noise reduction.

High-definition human visual attention mapping using wavelets

In this work, three novel approaches to detecting visual attention in images are presented. The idea behind detecting areas within images or video that naturally attract a viewer’s attention is based on the concept of generating pre-attentive saliency maps. Saliency, in and of itself, relates to some measure of “conspicuity” in the visual field and is believed to be an important precursor for many tasks in computer vision. One of the proposed methods in this thesis detects salient regions, while the other two detect salient edges. The classical approach to saliency detection proposed by Itti is extended by introducing wavelets as a lossless resizing tool while maintaining the aspect of biological inspiration. In addition to this, the spectral residual method and the frequency tuned method are modified using wavelets to allow for salient edge detection. Tests show that the proposed methods yield results that are not only comparable to leading,cutting-edge methods, but also exceed them in terms of correct and complete object detection as well as noise reduction.

Few-Shot Personalized Saliency Prediction Based on Adaptive Image Selection Considering Object and Visual Attention

A few-shot personalized saliency prediction based on adaptive image selection considering object and visual attention is presented in this paper. Since general methods predicting personalized saliency maps (PSMs) need a large number of training images, the establishment of a theory using a small number of training images is needed. To tackle this problem, although finding persons who have visual attention similar to that of a target person is effective, all persons have to commonly gaze at many images. Thus, it becomes difficult and unrealistic when considering their burden. On the other hand, this paper introduces a novel adaptive image selection (AIS) scheme that focuses on the relationship between human visual attention and objects in images. AIS focuses on both a diversity of objects in images and a variance of PSMs for the objects. Specifically, AIS selects images so that selected images have various kinds of objects to maintain their diversity. Moreover, AIS guarantees the high variance of PSMs for persons since it represents the regions that many persons commonly gaze at or do not gaze at. The proposed method enables selecting similar users from a small number of images by selecting images that have high diversities and variances. This is the technical contribution of this paper. Experimental results show the effectiveness of our personalized saliency prediction including the new image selection scheme.

Bottom-up visual attention model for still image: a preliminary study

The philosophy of human visual attention is scientifically explained in the field of cognitive psychology and neuroscience then computationally modeled in the field of computer science and engineering. Visual attention models have been applied in computer vision systems such as object detection, object recognition, image segmentation, image and video compression, action recognition, visual tracking, and so on. This work studies bottom-up visual attention, namely human fixation prediction and salient object detection models. The preliminary study briefly covers from the biological perspective of visual attention, including visual pathway, the theory of visual attention, to the computational model of bottom-up visual attention that generates saliency map. The study compares some models at each stage and observes whether the stage is inspired by biological architecture, concept, or behavior of human visual attention. From the study, the use of low-level features, center-surround mechanism, sparse representation, and higher-level guidance with intrinsic cues dominate the bottom-up visual attention approaches. The study also highlights the correlation between bottom-up visual attention and curiosity.

A Seamless Image-Stitching Method Based on Human Visual Discrimination and Attention

Stitching gaps and misalignments in mosaic images can severely degrade the human visual perception of mosaic effects. Image stitching plays a key role in eliminating these unpleasant defects. In this paper, an image-stitching method for mosaic images with invisible seams is proposed, according to the research on the human visual system (HVS). By quantifying the human visual attention of images and visual discrimination about luminance difference and fine dislocations, each pixel in the stitching region is given a priority value for tracing a stitching line. Coupled with the processing of an optimal stitching line locating method and the multi-band blending algorithm, the pixels of discontinuous items in mosaic images decrease significantly and the stitching line is almost invisible. This study provides a new insight into the image-stitching field, and the experiments show that the results of the proposed method are more consistent with the human visual system in creating high-quality image mosaics.