scholarly journals Dual Visual Attention Network for Visual Dialog

2019 ◽  
Author(s):  
Dan Guo ◽  
Hui Wang ◽  
Meng Wang
Keyword(s):  
Visual Attention ◽  
Experimental Results ◽  
Visual Features ◽  
Attention Network ◽  
Attention Model ◽  
Semantic Correlation ◽  
Object Based ◽  
Global Vision ◽  
Modal Semantic ◽  
Vision And Language

Visual dialog is a challenging task, which involves multi-round semantic transformations between vision and language. This paper aims to address cross-modal semantic correlation for visual dialog. Motivated by that Vg (global vision), Vl (local vision), Q (question) and H (history) have inseparable relevances, the paper proposes a novel Dual Visual Attention Network (DVAN) to realize (Vg, Vl, Q, H)--> A. DVAN is a three-stage query-adaptive attention model. In order to acquire accurate A (answer), it first explores the textual attention, which imposes the question on history to pick out related context H'. Then, based on Q and H', it implements respective visual attentions to discover related global image visual hints Vg' and local object-based visual hints Vl'. Next, a dual crossing visual attention is proposed. Vg' and Vl' are mutually embedded to learn the complementary of visual semantics. Finally, the attended textual and visual features are combined to infer the answer. Experimental results on the VisDial v0.9 and v1.0 datasets validate the effectiveness of the proposed approach.

scholarly journals Scene Recognition Based on Recurrent Memorized Attention Network

Electronics ◽  
2020 ◽  
Vol 9 (12) ◽  
pp. 2038
Author(s):  
Xi Shao ◽  
Xuan Zhang ◽  
Guijin Tang ◽  
Bingkun Bao
Keyword(s):  
Recognition Accuracy ◽  
State Of The Art ◽  
Classification Performance ◽  
Scene Recognition ◽  
Experimental Results ◽  
Scene Classification ◽  
Attention Network ◽  
Attention Model ◽  
Scene Image ◽  
Object Based

We propose a new end-to-end scene recognition framework, called a Recurrent Memorized Attention Network (RMAN) model, which performs object-based scene classification by recurrently locating and memorizing objects in the image. Based on the proposed framework, we introduce a multi-task mechanism that contiguously attends on the different essential objects in a scene image and recurrently performs memory fusion of the features of object focused by an attention model to improve the scene recognition accuracy. The experimental results show that the RMAN model has achieved better classification performance on the constructed dataset and two public scene datasets, surpassing state-of-the-art image scene recognition approaches.

scholarly journals The emergence of object‐based visual attention in infancy: A role for family socioeconomic status and competing visual features

Infancy ◽  
2019 ◽  
Vol 24 (5) ◽  
pp. 752-767 ◽  
Author(s):  
Denise M. Werchan ◽  
Andrew Lynn ◽  
Natasha Z. Kirkham ◽  
Dima Amso
Keyword(s):  
Socioeconomic Status ◽  
Visual Attention ◽  
Visual Features ◽  
Family Socioeconomic Status ◽  
Object Based

scholarly journals A NOVEL SHIP DETECTION METHOD FOR LARGE-SCALE OPTICAL SATELLITE IMAGES BASED ON VISUAL LBP FEATURE AND VISUAL ATTENTION MODEL

2016 ◽  
Vol XLI-B3 ◽  
pp. 917-921
Author(s):  
Sui Haigang ◽  
Song Zhina
Keyword(s):  
Visual Attention ◽  
Large Scale ◽  
Satellite Images ◽  
Visual Features ◽  
Support Vector ◽  
Biologically Inspired ◽  
Visual Attention Model ◽  
Attention Model ◽  
Ship Detection ◽  
Optical Satellite Images

Reliably ship detection in optical satellite images has a wide application in both military and civil fields. However, this problem is very difficult in complex backgrounds, such as waves, clouds, and small islands. Aiming at these issues, this paper explores an automatic and robust model for ship detection in large-scale optical satellite images, which relies on detecting statistical signatures of ship targets, in terms of biologically-inspired visual features. This model first selects salient candidate regions across large-scale images by using a mechanism based on biologically-inspired visual features, combined with visual attention model with local binary pattern (CVLBP). Different from traditional studies, the proposed algorithm is high-speed and helpful to focus on the suspected ship areas avoiding the separation step of land and sea. Largearea images are cut into small image chips and analyzed in two complementary ways: Sparse saliency using visual attention model and detail signatures using LBP features, thus accordant with sparseness of ship distribution on images. Then these features are employed to classify each chip as containing ship targets or not, using a support vector machine (SVM). After getting the suspicious areas, there are still some false alarms such as microwaves and small ribbon clouds, thus simple shape and texture analysis are adopted to distinguish between ships and nonships in suspicious areas. Experimental results show the proposed method is insensitive to waves, clouds, illumination and ship size.

scholarly journals A NOVEL SHIP DETECTION METHOD FOR LARGE-SCALE OPTICAL SATELLITE IMAGES BASED ON VISUAL LBP FEATURE AND VISUAL ATTENTION MODEL

2016 ◽  
Vol XLI-B3 ◽  
pp. 917-921 ◽  
Author(s):  
Sui Haigang ◽  
Song Zhina
Keyword(s):  
Visual Attention ◽  
Large Scale ◽  
Satellite Images ◽  
Visual Features ◽  
Support Vector ◽  
Biologically Inspired ◽  
Visual Attention Model ◽  
Attention Model ◽  
Ship Detection ◽  
Optical Satellite Images

Reliably ship detection in optical satellite images has a wide application in both military and civil fields. However, this problem is very difficult in complex backgrounds, such as waves, clouds, and small islands. Aiming at these issues, this paper explores an automatic and robust model for ship detection in large-scale optical satellite images, which relies on detecting statistical signatures of ship targets, in terms of biologically-inspired visual features. This model first selects salient candidate regions across large-scale images by using a mechanism based on biologically-inspired visual features, combined with visual attention model with local binary pattern (CVLBP). Different from traditional studies, the proposed algorithm is high-speed and helpful to focus on the suspected ship areas avoiding the separation step of land and sea. Largearea images are cut into small image chips and analyzed in two complementary ways: Sparse saliency using visual attention model and detail signatures using LBP features, thus accordant with sparseness of ship distribution on images. Then these features are employed to classify each chip as containing ship targets or not, using a support vector machine (SVM). After getting the suspicious areas, there are still some false alarms such as microwaves and small ribbon clouds, thus simple shape and texture analysis are adopted to distinguish between ships and nonships in suspicious areas. Experimental results show the proposed method is insensitive to waves, clouds, illumination and ship size.

scholarly journals Remote Sensing Image Retrieval with Combined Features Of Salient Region

2014 ◽  
Vol XL-6 ◽  
pp. 83-88 ◽  
Author(s):  
Z. F. Shao ◽  
W. X. Zhou ◽  
Q. M. Cheng
Keyword(s):  
Remote Sensing ◽  
Visual Attention ◽  
Image Retrieval ◽  
Texture Features ◽  
Remote Sensing Image ◽  
Experimental Results ◽  
Semantic Gap ◽  
Visual Attention Model ◽  
Attention Model ◽  
Combined Features

Low-level features tend to achieve unsatisfactory retrieval results in remote sensing image retrieval community because of the existence of semantic gap. In order to improve retrieval precision, visual attention model is used to extract salient objects from image according to their saliency. Then color and texture features are extracted from salient objects and regarded as feature vectors for image retrieval. Experimental results demonstrate that our method improves retrieval results and obtains higher precision.

Robot-to-human feedback and automatic object grasping using an RGB-D camera–projector system

Robotica ◽  
2017 ◽  
Vol 36 (2) ◽  
pp. 241-260 ◽  
Author(s):  
Jinglin Shen ◽  
Nicholas Gans
Keyword(s):  
Visual Attention ◽  
Visual Servoing ◽  
Robot Manipulator ◽  
Human Robot Interaction ◽  
Experimental Results ◽  
Robotic Arm ◽  
Depth Information ◽  
Robot Interaction ◽  
Attention Model ◽  
Robot Grasping

SUMMARYThis paper presents a novel system for human–robot interaction in object-grasping applications. Consisting of an RGB-D camera, a projector and a robot manipulator, the proposed system provides intuitive information to the human by analyzing the scene, detecting graspable objects and directly projecting numbers or symbols in front of objects. Objects are detected using a visual attention model that incorporates color, shape and depth information. The positions and orientations of the projected numbers are based on the shapes, positions and orientations of the corresponding objects. Users select a grasping target by indicating the corresponding number. Projected arrows are then created on the fly to guide a robotic arm to grasp the selected object using visual servoing and deliver the object to the human user. Experimental results are presented to demonstrate how the system is used in robot grasping tasks.

scholarly journals Adversarial Hard Attention Adaptation

Information ◽  
2020 ◽  
Vol 11 (4) ◽  
pp. 224
Author(s):  
Hui Tao ◽  
Jun He ◽  
Quanjie Cao ◽  
Lei Zhang
Keyword(s):  
Visual Attention ◽  
Domain Knowledge ◽  
Domain Adaptation ◽  
Experimental Results ◽  
Target Domain ◽  
Source Domain ◽  
Visual Attention Model ◽  
Attention Model ◽  
Cross Domain ◽  
Adversarial Training

Domain adaptation is critical to transfer the invaluable source domain knowledge to the target domain. In this paper, for a particular visual attention model, saying hard attention, we consider to adapt the learned hard attention to the unlabeled target domain. To tackle this kind of hard attention adaptation, a novel adversarial reward strategy is proposed to train the policy of the target domain agent. In this adversarial training framework, the target domain agent competes with the discriminator which takes the attention features generated from the both domain agents as input and tries its best to distinguish them, and thus the target domain policy is learned to align the local attention feature to its source domain counterpart. We evaluated our model on the benchmarks of the cross-domain tasks, such as the centered digits datasets and the enlarged non-centered digits datasets. The experimental results show that our model outperforms the ADDA and other existing methods.

Sign in / Sign up

Export Citation Format

Share Document