Salient Object Detection Techniques in Computer Vision—A Survey

Detection and localization of regions of images that attract immediate human visual attention is currently an intensive area of research in computer vision. The capability of automatic identification and segmentation of such salient image regions has immediate consequences for applications in the field of computer vision, computer graphics, and multimedia. A large number of salient object detection (SOD) methods have been devised to effectively mimic the capability of the human visual system to detect the salient regions in images. These methods can be broadly categorized into two categories based on their feature engineering mechanism: conventional or deep learning-based. In this survey, most of the influential advances in image-based SOD from both conventional as well as deep learning-based categories have been reviewed in detail. Relevant saliency modeling trends with key issues, core techniques, and the scope for future research work have been discussed in the context of difficulties often faced in salient object detection. Results are presented for various challenging cases for some large-scale public datasets. Different metrics considered for assessment of the performance of state-of-the-art salient object detection models are also covered. Some future directions for SOD are presented towards end.

Download Full-text

Saliency detection in deep learning era: trends of development

Information and Control Systems ◽

10.31799/1684-8853-2019-3-10-36 ◽

2019 ◽

pp. 10-36 ◽

Cited By ~ 2

Author(s):

M. N. Favorskaya ◽

L. C. Jain

Keyword(s):

Deep Learning ◽

Object Detection ◽

Event Detection ◽

Visual Analysis ◽

Saliency Detection ◽

Salient Object Detection ◽

Public Image ◽

Detection Methods ◽

Salient Object ◽

Salient Event

Introduction:Saliency detection is a fundamental task of computer vision. Its ultimate aim is to localize the objects of interest that grab human visual attention with respect to the rest of the image. A great variety of saliency models based on different approaches was developed since 1990s. In recent years, the saliency detection has become one of actively studied topic in the theory of Convolutional Neural Network (CNN). Many original decisions using CNNs were proposed for salient object detection and, even, event detection.Purpose:A detailed survey of saliency detection methods in deep learning era allows to understand the current possibilities of CNN approach for visual analysis conducted by the human eyes’ tracking and digital image processing.Results:A survey reflects the recent advances in saliency detection using CNNs. Different models available in literature, such as static and dynamic 2D CNNs for salient object detection and 3D CNNs for salient event detection are discussed in the chronological order. It is worth noting that automatic salient event detection in durable videos became possible using the recently appeared 3D CNN combining with 2D CNN for salient audio detection. Also in this article, we have presented a short description of public image and video datasets with annotated salient objects or events, as well as the often used metrics for the results’ evaluation.Practical relevance:This survey is considered as a contribution in the study of rapidly developed deep learning methods with respect to the saliency detection in the images and videos.

Download Full-text

Overview of deep-learning based methods for salient object detection in videos

Pattern Recognition ◽

10.1016/j.patcog.2020.107340 ◽

2020 ◽

Vol 104 ◽

pp. 107340

Author(s):

Qiong Wang ◽

Lu Zhang ◽

Yan Li ◽

Kidiyo Kpalma

Keyword(s):

Deep Learning ◽

Object Detection ◽

Salient Object Detection ◽

Salient Object

Download Full-text

Salient Object Detection on Large-Scale Video Data

2007 IEEE Conference on Computer Vision and Pattern Recognition ◽

10.1109/cvpr.2007.383495 ◽

2007 ◽

Cited By ~ 5

Author(s):

Shile Zhang ◽

Jianping Fan ◽

Hong Lu ◽

Xiangyang Xue

Keyword(s):

Object Detection ◽

Large Scale ◽

Salient Object Detection ◽

Video Data ◽

Salient Object

Download Full-text

SuperVAE: Superpixelwise Variational Autoencoder for Salient Object Detection

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v33i01.33018569 ◽

2019 ◽

Vol 33 ◽

pp. 8569-8576 ◽

Cited By ~ 2

Author(s):

Bo Li ◽

Zhengxing Sun ◽

Yuqi Guo

Keyword(s):

Deep Learning ◽

Object Detection ◽

Saliency Detection ◽

Salient Object Detection ◽

Salient Object ◽

Image Saliency ◽

Spatial Consistency ◽

Variational Autoencoder ◽

Benchmark Datasets ◽

Supervised Methods

Image saliency detection has recently witnessed rapid progress due to deep neural networks. However, there still exist many important problems in the existing deep learning based methods. Pixel-wise convolutional neural network (CNN) methods suffer from blurry boundaries due to the convolutional and pooling operations. While region-based deep learning methods lack spatial consistency since they deal with each region independently. In this paper, we propose a novel salient object detection framework using a superpixelwise variational autoencoder (SuperVAE) network. We first use VAE to model the image background and then separate salient objects from the background through the reconstruction residuals. To better capture semantic and spatial contexts information, we also propose a perceptual loss to take advantage from deep pre-trained CNNs to train our SuperVAE network. Without the supervision of mask-level annotated data, our method generates high quality saliency results which can better preserve object boundaries and maintain the spatial consistency. Extensive experiments on five wildly-used benchmark datasets show that the proposed method achieves superior or competitive performance compared to other algorithms including the very recent state-of-the-art supervised methods.

Download Full-text

RGBD Salient Object Detection using Spatially Coherent Deep Learning Framework

2018 IEEE 23rd International Conference on Digital Signal Processing (DSP) ◽

10.1109/icdsp.2018.8631584 ◽

2018 ◽

Cited By ~ 4

Author(s):

Posheng Huang ◽

Chin-Han Shen ◽

Hsu-Feng Hsiao

Keyword(s):

Deep Learning ◽

Object Detection ◽

Salient Object Detection ◽

Salient Object ◽

Learning Framework

Download Full-text

SaliencyGAN: Deep Learning Semisupervised Salient Object Detection in the Fog of IoT

IEEE Transactions on Industrial Informatics ◽

10.1109/tii.2019.2945362 ◽

2020 ◽

Vol 16 (4) ◽

pp. 2667-2676 ◽

Cited By ~ 23

Author(s):

Chengjia Wang ◽

Shizhou Dong ◽

Xiaofeng Zhao ◽

Giorgos Papanastasiou ◽

Heye Zhang ◽

...

Keyword(s):

Deep Learning ◽

Object Detection ◽

Salient Object Detection ◽

Salient Object

Download Full-text

Single-Pixel Salient Object Detection via Discrete Cosine Spectrum Acquisition and Deep Learning

IEEE Photonics Technology Letters ◽

10.1109/lpt.2020.3026472 ◽

2020 ◽

Vol 32 (21) ◽

pp. 1381-1384

Author(s):

Yonghao Li ◽

Jianhong Shi ◽

Lei Sun ◽

Xiaoyan Wu ◽

Guihua Zeng

Keyword(s):

Deep Learning ◽

Object Detection ◽

Salient Object Detection ◽

Salient Object ◽

Single Pixel

Download Full-text

RGB-D salient object detection: A survey

Computational Visual Media ◽

10.1007/s41095-020-0199-z ◽

2021 ◽

Author(s):

Tao Zhou ◽

Deng-Ping Fan ◽

Ming-Ming Cheng ◽

Jianbing Shen ◽

Ling Shao

Keyword(s):

Object Detection ◽

Spatial Information ◽

Salient Object Detection ◽

Future Research ◽

Salient Object ◽

Human Visual Perception ◽

Depth Sensors ◽

Depth Maps ◽

Benchmark Datasets ◽

Comprehensive Survey

AbstractSalient object detection, which simulates human visual perception in locating the most significant object(s) in a scene, has been widely applied to various computer vision tasks. Now, the advent of depth sensors means that depth maps can easily be captured; this additional spatial information can boost the performance of salient object detection. Although various RGB-D based salient object detection models with promising performance have been proposed over the past several years, an in-depth understanding of these models and the challenges in this field remains lacking. In this paper, we provide a comprehensive survey of RGB-D based salient object detection models from various perspectives, and review related benchmark datasets in detail. Further, as light fields can also provide depth maps, we review salient object detection models and popular benchmark datasets from this domain too. Moreover, to investigate the ability of existing models to detect salient objects, we have carried out a comprehensive attribute-based evaluation of several representative RGB-D based salient object detection models. Finally, we discuss several challenges and open directions of RGB-D based salient object detection for future research. All collected models, benchmark datasets, datasets constructed for attribute-based evaluation, and related code are publicly available at https://github.com/taozh2017/RGBD-SODsurvey.

Download Full-text