A saliency model-oriented convolution neural network for cloud detection in remote sensing images

Remote sensing is an indispensable technical way for monitoring earth resources and environmental changes. However, optical remote sensing images often contain a large number of cloud, especially in tropical rain forest areas, make it difficult to obtain completely cloud-free remote sensing images. Therefore, accurate cloud detection is of great research value for optical remote sensing applications. In this paper, we propose a saliency model-oriented convolution neural network for cloud detection in remote sensing images. Firstly, we adopt Kernel Principal Component Analysis (KCPA) to unsupervised pre-training the network. Secondly, small labeled samples are used to fine-tune the network structure. And, remote sensing images are performed with super-pixel approach before cloud detection to eliminate the irrelevant backgrounds and non-clouds object. Thirdly, the image blocks are input into the trained convolutional neural network (CNN) for cloud detection. Meanwhile, the segmented image will be recovered. Fourth, we fuse the detected result with the saliency map of raw image to further improve the accuracy of detection result. Experiments show that the proposed method can accurately detect cloud. Compared to other state-of-the-art cloud detection method, the new method has better robustness.

A dual-branch balance saliency model based on discriminative feature for fabric defect detection

PurposeThe purpose of this paper is to focus on the design of a dual-branch balance saliency model based on fully convolutional network (FCN) for automatic fabric defect detection, and improve quality control in textile manufacturing.Design/methodology/approachThis paper proposed a dual-branch balance saliency model based on discriminative feature for fabric defect detection. A saliency branch is firstly designed to address the problems of scale variation and contextual information integration, which is realized through the cooperation of a multi-scale discriminative feature extraction module (MDFEM) and a bidirectional stage-wise integration module (BSIM). These modules are respectively adopted to extract multi-scale discriminative context information and enrich the contextual information of features at each stage. In addition, another branch is proposed to balance the network, in which a bootstrap refinement module (BRM) is trained to guide the restoration of feature details.FindingsTo evaluate the performance of the proposed network, we conduct extensive experiments, and the experimental results demonstrate that the proposed method outperforms state-of-the-art (SOTA) approaches on seven evaluation metrics. We also conduct adequate ablation analyses that provide a full understanding of the design principles of the proposed method.Originality/valueThe dual-branch balance saliency model was proposed and applied into the fabric defect detection. The qualitative and quantitative experimental results show the effectiveness of the detection method. Therefore, the proposed method can be used for accurate fabric defect detection and even surface defect detection of other industrial products.

Multiplicative Watermarking Method with the Visual Saliency Model Using Contourlet Transform

We have proposed an image adaptive watermarking method by using contourlet transform. Firstly, we have selected high-energy image blocks as the watermark embedding space through segmenting the original image into nonoverlapping blocks and designed a watermark embedded strength factor by taking advantage of the human visual saliency model. To achieve dynamic adjustability of the multiplicative watermark embedding parameter, the relationship between watermark embedded strength factor and watermarked image quality is developed through experiments with the peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM), respectively. Secondly, to detect the watermark information, the generalized Gaussian distribution (GGD) has been utilized to model the contourlet coefficients. Furthermore, positions of the blocks selected, watermark embedding factor, and watermark size have been used as side information for watermark decoding. Finally, several experiments have been conducted on eight images, and the results prove the effectiveness of the proposed watermarking approach. Concretely, our watermarking method has good imperceptibility and strong robustness when against Gaussian noise, JPEG compression, scaling, rotation, median filtering, and Gaussian filtering attack.

Deep saliency models learn low-, mid-, and high-level features to predict scene attention

Deep saliency models represent the current state-of-the-art for predicting where humans look in real-world scenes. However, for deep saliency models to inform cognitive theories of attention, we need to know how deep saliency models prioritize different scene features to predict where people look. Here we open the black box of three prominent deep saliency models (MSI-Net, DeepGaze II, and SAM-ResNet) using an approach that models the association between attention, deep saliency model output, and low-, mid-, and high-level scene features. Specifically, we measured the association between each deep saliency model and low-level image saliency, mid-level contour symmetry and junctions, and high-level meaning by applying a mixed effects modeling approach to a large eye movement dataset. We found that all three deep saliency models were most strongly associated with high-level and low-level features, but exhibited qualitatively different feature weightings and interaction patterns. These findings suggest that prominent deep saliency models are primarily learning image features associated with high-level scene meaning and low-level image saliency and highlight the importance of moving beyond simply benchmarking performance.

Construction of a Visual Saliency Model for Neighborhood Building Landmarks Based on K-Means Clustering

In this paper, firstly, based on the quantitative relationship between K-means clustering and visual saliency of neighborhood building landmarks, the weights occupied by each index of composite visual factors are obtained by using multiple statistical regression methods, and, finally, we try to construct a saliency model of multiple visual index composites and analyze and test the model. As regards decomposition and quantification of visual saliency influencing factors, to describe and quantify these visual significance factors of the landmarks, the significant factors are decomposed into several quantifiable secondary indicators. Considering that the visual saliency of the landmarks in the neighborhood is reflected by the variance of the influencing factors and that the scope of the landmarks is localized, the local outlier detection algorithm is used to solve the variance of the secondary indicators. Since the visual significance of neighborhood building landmarks is influenced by a combination of influencing factors, the overall difference degree of secondary indicators is calculated by K-means clustering. To facilitate the factor calculation, a factor-controlled virtual environment was built to carry out the experimental study of landmark perception and calculate the different degrees of each index of the building. The data of visual indicators of the neighborhood buildings for this experiment were also collected, and the significance values of the neighborhood buildings were calculated. The influence weights of the indicators were obtained by using multiple linear regression analysis, the visual significance model of the landmarks of the neighborhood buildings in the factor-controlled environment was constructed, and the model was analyzed and tested.

What do deep saliency models learn about where we look in scenes?

Deep saliency models represent the current state-of-the-art for predicting where humans look in real-world scenes. However, for deep saliency models to inform cognitive theories of attention, we need to know how deep saliency models predict where people look. Here we open the black box of deep saliency models using an approach that models the association between the output of 3 prominent deep saliency models (MSI-Net, DeepGaze II, and SAM-ResNet) and low-, mid-, and high-level scene features. Specifically, we measured the association between each deep saliency model and low-level image saliency, mid-level contour symmetry and junctions, and high-level meaning by applying a mixed effects modeling approach to a large eye movement dataset. We found that despite different architectures, training regimens, and loss functions, all three deep saliency models were most strongly associated with high-level meaning. These findings suggest that deep saliency models are primarily learning image features associated with scene meaning.

AFI-Net: Attention-Guided Feature Integration Network for RGBD Saliency Detection

This article proposes an innovative RGBD saliency model, that is, attention-guided feature integration network, which can extract and fuse features and perform saliency inference. Specifically, the model first extracts multimodal and level deep features. Then, a series of attention modules are deployed to the multilevel RGB and depth features, yielding enhanced deep features. Next, the enhanced multimodal deep features are hierarchically fused. Lastly, the RGB and depth boundary features, that is, low-level spatial details, are added to the integrated feature to perform saliency inference. The key points of the AFI-Net are the attention-guided feature enhancement and the boundary-aware saliency inference, where the attention module indicates salient objects coarsely, and the boundary information is used to equip the deep feature with more spatial details. Therefore, salient objects are well characterized, that is, well highlighted. The comprehensive experiments on five challenging public RGBD datasets clearly exhibit the superiority and effectiveness of the proposed AFI-Net.