Multi-level attention model for tracking and segmentation of objects under complex occlusion

Learning-based methods suffer from a deficiency of clean annotations, especially in biomedical segmentation. Although many semi-supervised methods have been proposed to provide extra training data, automatically generated labels are usually too noisy to retrain models effectively. In this paper, we propose a Two-Stream Mutual Attention Network (TSMAN) that weakens the influence of back-propagated gradients caused by incorrect labels, thereby rendering the network robust to unclean data. The proposed TSMAN consists of two sub-networks that are connected by three types of attention models in different layers. The target of each attention model is to indicate potentially incorrect gradients in a certain layer for both sub-networks by analyzing their inferred features using the same input. In order to achieve this purpose, the attention models are designed based on the propagation analysis of noisy gradients at different layers. This allows the attention models to effectively discover incorrect labels and weaken their influence during parameter updating process. By exchanging multi-level features within two-stream architecture, the effects of noisy labels in each sub-network are reduced by decreasing the noisy gradients. Furthermore, a hierarchical distillation is developed to provide reliable pseudo labels for unlabelded data, which further boosts the performance of TSMAN. The experiments using both HVSMR 2016 and BRATS 2015 benchmarks demonstrate that our semi-supervised learning framework surpasses the state-of-the-art fully-supervised results.

Download Full-text

Multi-Level Contextual RNNs With Attention Model for Scene Labeling

IEEE Transactions on Intelligent Transportation Systems ◽

10.1109/tits.2017.2775628 ◽

2018 ◽

Vol 19 (11) ◽

pp. 3475-3485 ◽

Cited By ~ 11

Author(s):

Heng Fan ◽

Xue Mei ◽

Danil Prokhorov ◽

Haibin Ling

Keyword(s):

Attention Model ◽

Scene Labeling ◽

Multi Level

Download Full-text

Multi-level attention model for person re-identification

Pattern Recognition Letters ◽

10.1016/j.patrec.2018.08.024 ◽

2019 ◽

Vol 127 ◽

pp. 156-164 ◽

Cited By ~ 6

Author(s):

Yichao Yan ◽

Bingbing Ni ◽

Jinxian Liu ◽

Xiaokang Yang

Keyword(s):

Attention Model ◽

Multi Level

Download Full-text

Multi-level Stereo Attention Model for Center Channel Extraction

2019 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB) ◽

10.1109/bmsb47279.2019.8971885 ◽

2019 ◽

Author(s):

Wootaek Lim ◽

Seungkwon Beack ◽

Taejin Lee

Keyword(s):

Attention Model ◽

Center Channel ◽

Multi Level

Download Full-text

Multi-level dictionary learning for fine-grained images categorization with attention model

Neurocomputing ◽

10.1016/j.neucom.2020.07.147 ◽

2021 ◽

Author(s):

Jinsheng Ji ◽

Yiyou Guo ◽

Zhen Yang ◽

Tao Zhang ◽

Xiankai Lu

Keyword(s):

Dictionary Learning ◽

Attention Model ◽

Fine Grained ◽

Multi Level

Download Full-text

A Multi-Level Attention Model for Remote Sensing Image Captions

Remote Sensing ◽

10.3390/rs12060939 ◽

2020 ◽

Vol 12 (6) ◽

pp. 939 ◽

Cited By ~ 1

Author(s):

Yangyang Li ◽

Shuangkang Fang ◽

Licheng Jiao ◽

Ruijiao Liu ◽

Ronghua Shang

Keyword(s):

Remote Sensing ◽

State Of The Art ◽

Remote Sensing Image ◽

Attention Mechanism ◽

Complex Task ◽

Human Beings ◽

Image Captioning ◽

Attention Model ◽

Multi Level ◽

Image Caption

The task of image captioning involves the generation of a sentence that can describe an image appropriately, which is the intersection of computer vision and natural language. Although the research on remote sensing image captions has just started, it has great significance. The attention mechanism is inspired by the way humans think, which is widely used in remote sensing image caption tasks. However, the attention mechanism currently used in this task is mainly aimed at images, which is too simple to express such a complex task well. Therefore, in this paper, we propose a multi-level attention model, which is a closer imitation of attention mechanisms of human beings. This model contains three attention structures, which represent the attention to different areas of the image, the attention to different words, and the attention to vision and semantics. Experiments show that our model has achieved better results than before, which is currently state-of-the-art. In addition, the existing datasets for remote sensing image captioning contain a large number of errors. Therefore, in this paper, a lot of work has been done to modify the existing datasets in order to promote the research of remote sensing image captioning.

Download Full-text

Multi-Level Two-Stream Fusion-Based Spatio-Temporal Attention Model for Violence Detection and Localization

International Journal of Pattern Recognition and Artificial Intelligence ◽

10.1142/s0218001422550023 ◽

2021 ◽

Author(s):

Mujtaba Asad ◽

He Jiang ◽

Jie Yang ◽

Enmei Tu ◽

Aftab A. Malik

Keyword(s):

Human Behavior ◽

Superior Performance ◽

Temporal Attention ◽

Attention Model ◽

Violence Detection ◽

Temporal Features ◽

Proposed Model ◽

Multi Level ◽

Spatio Temporal ◽

Detection And Localization

Detection of violent human behavior is necessary for public safety and monitoring. However, it demands constant human observation and attention in human-based surveillance systems, which is a challenging task. Autonomous detection of violent human behavior is therefore essential for continuous uninterrupted video surveillance. In this paper, we propose a novel method for violence detection and localization in videos using the fusion of spatio-temporal features and attention model. The model consists of Fusion Convolutional Neural Network (Fusion-CNN), spatio-temporal attention modules and Bi-directional Convolutional LSTMs (BiConvLSTM). The Fusion-CNN learns both spatial and temporal features by combining multi-level inter-layer features from both RGB and Optical flow input frames. The spatial attention module is used to generate an importance mask to focus on the most important areas of the image frame. The temporal attention part, which is based on BiConvLSTM, identifies the most significant video frames which are related to violent activity. The proposed model can also localize and discriminate prominent regions in both spatial and temporal domains, given the weakly supervised training with only video-level classification labels. Experimental results evaluated on different publicly available benchmarking datasets show the superior performance of the proposed model in comparison with the existing methods. Our model achieves the improved accuracies (ACC) of 89.1%, 99.1% and 98.15% for RWF-2000, HockeyFight and Crowd-Violence datasets, respectively. For CCTV-FIGHTS dataset, we choose the mean average precision (mAp) performance metric and our model obtained 80.7% mAp.

Download Full-text