Multiscale Deep Network with Centerness-Aware Loss for Salient Object Detection

Deep encoder-decoder networks have been adopted for saliency detection and achieved state-of-the-art performance. However, most existing saliency models usually fail to detect very small salient objects. In this paper, we propose a multitask architecture, M2Net, and a novel centerness-aware loss for salient object detection. The proposed M2Net aims to solve saliency prediction and centerness prediction simultaneously. Specifically, the network architecture is composed of a bottom-up encoder module, top-down decoder module, and centerness prediction module. In addition, different from binary cross entropy, the proposed centerness-aware loss can guide the proposed M2Net to uniformly highlight the entire salient regions with well-defined object boundaries. Experimental results on five benchmark saliency datasets demonstrate that M2Net outperforms state-of-the-art methods on different evaluation metrics.

Application of region-based video surveillance in smart cities using deep learning

Smart video surveillance helps to build more robust smart city environment. The varied angle cameras act as smart sensors and collect visual data from smart city environment and transmit it for further visual analysis. The transmitted visual data is required to be in high quality for efficient analysis which is a challenging task while transmitting videos on low capacity bandwidth communication channels. In latest smart surveillance cameras, high quality of video transmission is maintained through various video encoding techniques such as high efficiency video coding. However, these video coding techniques still provide limited capabilities and the demand of high-quality based encoding for salient regions such as pedestrians, vehicles, cyclist/motorcyclist and road in video surveillance systems is still not met. This work is a contribution towards building an efficient salient region-based surveillance framework for smart cities. The proposed framework integrates a deep learning-based video surveillance technique that extracts salient regions from a video frame without information loss, and then encodes it in reduced size. We have applied this approach in diverse case studies environments of smart city to test the applicability of the framework. The successful result in terms of bitrate 56.92%, peak signal to noise ratio 5.35 bd and SR based segmentation accuracy of 92% and 96% for two different benchmark datasets is the outcome of proposed work. Consequently, the generation of less computational region-based video data makes it adaptable to improve surveillance solution in Smart Cities.

Active Shift Attention Based Object Tracking System

<p>The human vision system (HVS) collects a huge amount of information and performs a variety of biological mechanisms to select relevant information. Computational models based on these biological mechanisms are used in machine vision to select interesting or salient regions in the images for application in scene analysis, object detection and object tracking. Different object tracking techniques have been proposed often using complex processing methods. On the other hand, attention-based computational models have shown significant performance advantages in various applications. We hypothesise the integration of a visual attention model with object tracking can be effective in increasing the performance by reducing the detection complexity in challenging environments such as illumination change, occlusion, and camera moving. The overall objective of this thesis is to develop a visual saliency based object tracker that alternates between targets using a measure of current uncertainty derived from a Kalman filter. This thesis presents the results by showing the effectiveness of the tracker using the mean square error when compared to a tracker without the uncertainty mechanism. Specific colour spaces can contribute to the identification of salient regions. The investigation is done between the non-uniform red, green and blue (RGB) derived opponencies with the hue, saturation and value (HSV) colour space using video information. The main motivation for this particular comparison is to improve the quality of saliency detection in challenging situations such as lighting changes. Precision-Recall curves are used to compare the colour spaces using pyramidal and non-pyramidal saliency models.</p>

Active Shift Attention Based Object Tracking System

<p>The human vision system (HVS) collects a huge amount of information and performs a variety of biological mechanisms to select relevant information. Computational models based on these biological mechanisms are used in machine vision to select interesting or salient regions in the images for application in scene analysis, object detection and object tracking. Different object tracking techniques have been proposed often using complex processing methods. On the other hand, attention-based computational models have shown significant performance advantages in various applications. We hypothesise the integration of a visual attention model with object tracking can be effective in increasing the performance by reducing the detection complexity in challenging environments such as illumination change, occlusion, and camera moving. The overall objective of this thesis is to develop a visual saliency based object tracker that alternates between targets using a measure of current uncertainty derived from a Kalman filter. This thesis presents the results by showing the effectiveness of the tracker using the mean square error when compared to a tracker without the uncertainty mechanism. Specific colour spaces can contribute to the identification of salient regions. The investigation is done between the non-uniform red, green and blue (RGB) derived opponencies with the hue, saturation and value (HSV) colour space using video information. The main motivation for this particular comparison is to improve the quality of saliency detection in challenging situations such as lighting changes. Precision-Recall curves are used to compare the colour spaces using pyramidal and non-pyramidal saliency models.</p>

A Generative Adversarial Network with Dual Discriminators for Infrared and Visible Image Fusion Based on Saliency Detection

Infrared and visible image fusion needs to preserve both the salient target of the infrared image and the texture details of the visible image. Therefore, an infrared and visible image fusion method based on saliency detection is proposed. Firstly, the saliency map of the infrared image is obtained by saliency detection. Then, the specific loss function and network architecture are designed based on the saliency map to improve the performance of the fusion algorithm. Specifically, the saliency map is normalized to [0, 1], used as a weight map to constrain the loss function. At the same time, the saliency map is binarized to extract salient regions and nonsalient regions. And, a generative adversarial network with dual discriminators is obtained. The two discriminators are used to distinguish the salient regions and the nonsalient regions, respectively, to promote the generator to generate better fusion results. The experimental results show that the fusion results of our method are better than those of the existing methods in both subjective and objective aspects.

Attention U-Net ensemble for interpretable polyp and instrument segmentation

The difficulty associated with screening and treating colorectal polyps alongside other gastrointestinal pathology presents an opportunity to incorporate computer-aided systems. This paper develops a deep learning pipeline that accurately segments colorectal polyps and various instruments used during endoscopic procedures. To improve transparency, we leverage the Attention U-Net architecture, enabling visualisation of the attention coefficients to identify salient regions. Moreover, we improve performance by incorporating transfer learning using a pre-trained encoder, together with test-time augmentation, softmax averaging, softmax thresholding and connected component labeling to further refine predictions.

A Comparison of Bottom-Up Models for Spatial Saliency Predictions in Autonomous Driving

Bottom-up saliency models identify the salient regions of an image based on features such as color, intensity and orientation. These models are typically used as predictors of human visual behavior and for computer vision tasks. In this paper, we conduct a systematic evaluation of the saliency maps computed with four selected bottom-up models on images of urban and highway traffic scenes. Saliency both over whole images and on object level is investigated and elaborated in terms of the energy and the entropy of the saliency maps. We identify significant differences with respect to the amount, size and shape-complexity of the salient areas computed by different models. Based on these findings, we analyze the likelihood that object instances fall within the salient areas of an image and investigate the agreement between the segments of traffic participants and the saliency maps of the different models. The overall and object-level analysis provides insights on the distinctive features of salient areas identified by different models, which can be used as selection criteria for prospective applications in autonomous driving such as object detection and tracking.

A 3D Keypoints Voting Network for 6DoF Pose Estimation in Indoor Scene

This paper addresses the problem of instance-level 6DoF pose estimation from a single RGBD image in an indoor scene. Many recent works have shown that a two-stage network, which first detects the keypoints and then regresses the keypoints for 6d pose estimation, achieves remarkable performance. However, the previous methods concern little about channel-wise attention and the keypoints are not selected by comprehensive use of RGBD information, which limits the performance of the network. To enhance RGB feature representation ability, a modular Split-Attention block that enables attention across feature-map groups is proposed. In addition, by combining the Oriented FAST and Rotated BRIEF (ORB) keypoints and the Farthest Point Sample (FPS) algorithm, a simple but effective keypoint selection method named ORB-FPS is presented to avoid the keypoints appear on the non-salient regions. The proposed algorithm is tested on the Linemod and the YCB-Video dataset, the experimental results demonstrate that our method outperforms the current approaches, achieves ADD(S) accuracy of 94.5% on the Linemod dataset and 91.4% on the YCB-Video dataset.

A Non-Global Disturbance Targeted Adversarial Example Algorithm Combined with C&W and Grad-Cam

Adversarial examples are artificially crafted to mislead deep learning systems into making wrong decisions. In the research of attack algorithms against multi-class image classifiers, an improved strategy of applying category explanation to the generation control of targeted adversarial example is proposed to reduce the perturbation noise and improve the adversarial robustness. On the basis of C&W adversarial attack algorithm, the method uses Grad-Cam, a category visualization explanation algorithm of CNN, to dynamically obtain the salient regions according to the signal features of source and target categories during the iterative generation process. The adversarial example of non-global perturbation is finally achieved by gradually shielding the non salient regions and fine-tuning the perturbation signals. Compared with other similar algorithms under the same conditions, the method enhances the effects of the original image category signal on the perturbation position. Experimental results show that, the improved adversarial examples have higher PSNR. In addition, in a variety of different defense processing tests, the examples can keep high adversarial performance and show strong attacking robustness.

DeepVASP-E: A Flexible Analysis of Electrostatic Isopotentials for Finding and Explaining Mechanisms that Control Binding Specificity

Amino acids that play a role in binding specificity can be identified with many methods, but few techniques identify the biochemical mechanisms by which they act. To address a part of this problem, we present DeepVASP-E, an algorithm that can suggest electrostatic mechanisms that influence specificity. DeepVASP-E uses convolutional neural networks to classify an electrostatic representation of ligand binding sites into specificity categories. It also uses class activation mapping to identify regions of electrostatic potential that are salient for classification. We hypothesize that electrostatic regions that are salient for classification are also likely to play a biochemical role in achieving specificity. Our findings, on two families of proteins with electrostatic influences on specificity, demonstrate that large salient regions can identify amino acids that have an electrostatic role in binding, and that DeepVASP-E is an effective classifier of ligand binding sites.