Saliency detection in deep learning era: trends of development

Introduction:Saliency detection is a fundamental task of computer vision. Its ultimate aim is to localize the objects of interest that grab human visual attention with respect to the rest of the image. A great variety of saliency models based on different approaches was developed since 1990s. In recent years, the saliency detection has become one of actively studied topic in the theory of Convolutional Neural Network (CNN). Many original decisions using CNNs were proposed for salient object detection and, even, event detection.Purpose:A detailed survey of saliency detection methods in deep learning era allows to understand the current possibilities of CNN approach for visual analysis conducted by the human eyes’ tracking and digital image processing.Results:A survey reflects the recent advances in saliency detection using CNNs. Different models available in literature, such as static and dynamic 2D CNNs for salient object detection and 3D CNNs for salient event detection are discussed in the chronological order. It is worth noting that automatic salient event detection in durable videos became possible using the recently appeared 3D CNN combining with 2D CNN for salient audio detection. Also in this article, we have presented a short description of public image and video datasets with annotated salient objects or events, as well as the often used metrics for the results’ evaluation.Practical relevance:This survey is considered as a contribution in the study of rapidly developed deep learning methods with respect to the saliency detection in the images and videos.

Download Full-text

SuperVAE: Superpixelwise Variational Autoencoder for Salient Object Detection

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v33i01.33018569 ◽

2019 ◽

Vol 33 ◽

pp. 8569-8576 ◽

Cited By ~ 2

Author(s):

Bo Li ◽

Zhengxing Sun ◽

Yuqi Guo

Keyword(s):

Deep Learning ◽

Object Detection ◽

Saliency Detection ◽

Salient Object Detection ◽

Salient Object ◽

Image Saliency ◽

Spatial Consistency ◽

Variational Autoencoder ◽

Benchmark Datasets ◽

Supervised Methods

Image saliency detection has recently witnessed rapid progress due to deep neural networks. However, there still exist many important problems in the existing deep learning based methods. Pixel-wise convolutional neural network (CNN) methods suffer from blurry boundaries due to the convolutional and pooling operations. While region-based deep learning methods lack spatial consistency since they deal with each region independently. In this paper, we propose a novel salient object detection framework using a superpixelwise variational autoencoder (SuperVAE) network. We first use VAE to model the image background and then separate salient objects from the background through the reconstruction residuals. To better capture semantic and spatial contexts information, we also propose a perceptual loss to take advantage from deep pre-trained CNNs to train our SuperVAE network. Without the supervision of mask-level annotated data, our method generates high quality saliency results which can better preserve object boundaries and maintain the spatial consistency. Extensive experiments on five wildly-used benchmark datasets show that the proposed method achieves superior or competitive performance compared to other algorithms including the very recent state-of-the-art supervised methods.

Download Full-text

Detecting Salient Image Objects Using Color Histogram Clustering for Region Granularity

Journal of Imaging ◽

10.3390/jimaging7090187 ◽

2021 ◽

Vol 7 (9) ◽

pp. 187

Author(s):

Seena Joseph ◽

Oludayo O. Olugbara

Keyword(s):

Object Detection ◽

Saliency Detection ◽

Contrast Ratio ◽

Salient Object Detection ◽

Color Images ◽

Color Histogram ◽

Detection Methods ◽

Color Contrast ◽

Salient Object ◽

Selection Of

Salient object detection represents a novel preprocessing stage of many practical image applications in the discipline of computer vision. Saliency detection is generally a complex process to copycat the human vision system in the processing of color images. It is a convoluted process because of the existence of countless properties inherent in color images that can hamper performance. Due to diversified color image properties, a method that is appropriate for one category of images may not necessarily be suitable for others. The selection of image abstraction is a decisive preprocessing step in saliency computation and region-based image abstraction has become popular because of its computational efficiency and robustness. However, the performances of the existing region-based salient object detection methods are extremely hooked on the selection of an optimal region granularity. The incorrect selection of region granularity is potentially prone to under- or over-segmentation of color images, which can lead to a non-uniform highlighting of salient objects. In this study, the method of color histogram clustering was utilized to automatically determine suitable homogenous regions in an image. Region saliency score was computed as a function of color contrast, contrast ratio, spatial feature, and center prior. Morphological operations were ultimately performed to eliminate the undesirable artifacts that may be present at the saliency detection stage. Thus, we have introduced a novel, simple, robust, and computationally efficient color histogram clustering method that agglutinates color contrast, contrast ratio, spatial feature, and center prior for detecting salient objects in color images. Experimental validation with different categories of images selected from eight benchmarked corpora has indicated that the proposed method outperforms 30 bottom-up non-deep learning and seven top-down deep learning salient object detection methods based on the standard performance metrics.

Download Full-text

Deep Embedding Features for Salient Object Detection

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v33i01.33019340 ◽

2019 ◽

Vol 33 ◽

pp. 9340-9347 ◽

Cited By ~ 5

Author(s):

Yunzhi Zhuge ◽

Yu Zeng ◽

Huchuan Lu

Keyword(s):

Object Detection ◽

Rapid Development ◽

Saliency Detection ◽

Salient Object Detection ◽

Detection Methods ◽

Salient Object ◽

Novel Approach ◽

Deep Embedding ◽

Salient Regions ◽

Multi Level

Benefiting from the rapid development of Convolutional Neural Networks (CNNs), some salient object detection methods have achieved remarkable results by utilizing multi-level convolutional features. However, the saliency training datasets is of limited scale due to the high cost of pixel-level labeling, which leads to a limited generalization of the trained model on new scenarios during testing. Besides, some FCN-based methods directly integrate multi-level features, ignoring the fact that the noise in some features are harmful to saliency detection. In this paper, we propose a novel approach that transforms prior information into an embedding space to select attentive features and filter out outliers for salient object detection. Our network firstly generates a coarse prediction map through an encorder-decorder structure. Then a Feature Embedding Network (FEN) is trained to embed each pixel of the coarse map into a metric space, which incorporates much attentive features that highlight salient regions and suppress the response of non-salient regions. Further, the embedded features are refined through a deep-to-shallow Recursive Feature Integration Network (RFIN) to improve the details of prediction maps. Moreover, to alleviate the blurred boundaries, we propose a Guided Filter Refinement Network (GFRN) to jointly optimize the predicted results and the learnable guidance maps. Extensive experiments on five benchmark datasets demonstrate that our method outperforms state-of-the-art results. Our proposed method is end-to-end and achieves a realtime speed of 38 FPS.

Download Full-text

Overview of deep-learning based methods for salient object detection in videos

Pattern Recognition ◽

10.1016/j.patcog.2020.107340 ◽

2020 ◽

Vol 104 ◽

pp. 107340

Author(s):

Qiong Wang ◽

Lu Zhang ◽

Yan Li ◽

Kidiyo Kpalma

Keyword(s):

Deep Learning ◽

Object Detection ◽

Salient Object Detection ◽

Salient Object

Download Full-text

RGBD Salient Object Detection using Spatially Coherent Deep Learning Framework

2018 IEEE 23rd International Conference on Digital Signal Processing (DSP) ◽

10.1109/icdsp.2018.8631584 ◽

2018 ◽

Cited By ~ 4

Author(s):

Posheng Huang ◽

Chin-Han Shen ◽

Hsu-Feng Hsiao

Keyword(s):

Deep Learning ◽

Object Detection ◽

Salient Object Detection ◽

Salient Object ◽

Learning Framework

Download Full-text

Salient Object Detection Techniques in Computer Vision—A Survey

Entropy ◽

10.3390/e22101174 ◽

2020 ◽

Vol 22 (10) ◽

pp. 1174

Author(s):

Ashish Kumar Gupta ◽

Ayan Seal ◽

Mukesh Prasad ◽

Pritee Khanna

Keyword(s):

Computer Vision ◽

Deep Learning ◽

Object Detection ◽

Large Scale ◽

Research Work ◽

Salient Object Detection ◽

Future Research ◽

Automatic Identification ◽

Salient Object ◽

Detection Techniques

Detection and localization of regions of images that attract immediate human visual attention is currently an intensive area of research in computer vision. The capability of automatic identification and segmentation of such salient image regions has immediate consequences for applications in the field of computer vision, computer graphics, and multimedia. A large number of salient object detection (SOD) methods have been devised to effectively mimic the capability of the human visual system to detect the salient regions in images. These methods can be broadly categorized into two categories based on their feature engineering mechanism: conventional or deep learning-based. In this survey, most of the influential advances in image-based SOD from both conventional as well as deep learning-based categories have been reviewed in detail. Relevant saliency modeling trends with key issues, core techniques, and the scope for future research work have been discussed in the context of difficulties often faced in salient object detection. Results are presented for various challenging cases for some large-scale public datasets. Different metrics considered for assessment of the performance of state-of-the-art salient object detection models are also covered. Some future directions for SOD are presented towards end.

Download Full-text

Double Feature Combination: Region Contrast for Visual Salient Object Detection

Applied Mechanics and Materials ◽

10.4028/www.scientific.net/amm.239-240.811 ◽

2012 ◽

Vol 239-240 ◽

pp. 811-815

Author(s):

Zhi Hai Sun ◽

Teng Song ◽

Wen Hui Zhou ◽

Hua Zhang

Keyword(s):

Object Detection ◽

Saliency Detection ◽

Visual Saliency ◽

Recall Rate ◽

Detection Algorithm ◽

Salient Object Detection ◽

Color Contrast ◽

Salient Object ◽

Histograms Of Oriented Gradients ◽

Visual Saliency Detection

Visual saliency detection has become an important step between computer vision and digital image processing. Recent methods almost form a computational model based on color, which are difficult to overcome the shortcoming with cluttered and textured background. This paper proposes a novel salient object detection algorithm integrating with region color contrast and histograms of oriented gradients (HoG). Extensively experiments show that our algorithm outperforms other state-of-art saliency methods, yielding higher precision and better recall rate, even lower mean absolution error.

Download Full-text