Development of Hierarchical Attention Network Based Architecture for Cloze-Style Question Answering

To date, visual question answering (VQA) (i.e., image QA and video QA) is still a holy grail in vision and language understanding, especially for video QA. Compared with image QA that focuses primarily on understanding the associations between image region-level details and corresponding questions, video QA requires a model to jointly reason across both spatial and long-range temporal structures of a video as well as text to provide an accurate answer. In this paper, we specifically tackle the problem of video QA by proposing a Structured Two-stream Attention network, namely STA, to answer a free-form or open-ended natural language question about the content of a given video. First, we infer rich longrange temporal structures in videos using our structured segment component and encode text features. Then, our structured two-stream attention component simultaneously localizes important visual instance, reduces the influence of background video and focuses on the relevant text. Finally, the structured two-stream fusion component incorporates different segments of query and video aware context representation and infers the answers. Experiments on the large-scale video QA dataset TGIF-QA show that our proposed method significantly surpasses the best counterpart (i.e., with one representation for the video input) by 13.0%, 13.5%, 11.0% and 0.3 for Action, Trans., TrameQA and Count tasks. It also outperforms the best competitor (i.e., with two representations) on the Action, Trans., TrameQA tasks by 4.1%, 4.7%, and 5.1%.

Download Full-text

Multi-tier attention network using term-weighted question features for Visual Question Answering

Image and Vision Computing ◽

10.1016/j.imavis.2021.104291 ◽

2021 ◽

pp. 104291

Author(s):

Sruthy Manmadhan ◽

Binsu C. Kovoor

Keyword(s):

Question Answering ◽

Attention Network ◽

Visual Question Answering

Download Full-text

Aspect Sentiment Classification Towards Question-Answering with Reinforced Bidirectional Attention Network

10.18653/v1/p19-1345 ◽

2019 ◽

Cited By ~ 4

Author(s):

Jingjing Wang ◽

Changlong Sun ◽

Shoushan Li ◽

Xiaozhong Liu ◽

Luo Si ◽

...

Keyword(s):

Question Answering ◽

Sentiment Classification ◽

Attention Network

Download Full-text

KGAnet: a knowledge graph attention network for enhancing natural language inference

Neural Computing and Applications ◽

10.1007/s00521-020-04851-5 ◽

2020 ◽

Vol 32 (18) ◽

pp. 14963-14973

Author(s):

Meina Song ◽

Wen Zhao ◽

E. HaiHong

Keyword(s):

Natural Language ◽

Question Answering ◽

Knowledge Graph ◽

Attention Network ◽

External Knowledge ◽

Basic Task ◽

Joint Training

Abstract Natural language inference (NLI) is the basic task of many applications such as question answering and paraphrase recognition. Existing methods have solved the key issue of how the NLI model can benefit from external knowledge. Inspired by this, we attempt to further explore the following two problems: (1) how to make better use of external knowledge when the total amount of such knowledge is constant and (2) how to bring external knowledge to the NLI model more conveniently in the application scenario. In this paper, we propose a novel joint training framework that consists of a modified graph attention network, called the knowledge graph attention network, and an NLI model. We demonstrate that the proposed method outperforms the existing method which introduces external knowledge, and we improve the performance of multiple NLI models without additional external knowledge.

Download Full-text

Multi-Modality Global Fusion Attention Network for Visual Question Answering

Electronics ◽

10.3390/electronics9111882 ◽

2020 ◽

Vol 9 (11) ◽

pp. 1882

Author(s):

Cheng Yang ◽

Weijia Wu ◽

Yuxing Wang ◽

Hong Zhou

Keyword(s):

Correct Answer ◽

Question Answering ◽

Human Beings ◽

Single Model ◽

Attention Network ◽

Attention Model ◽

Global Perspectives ◽

Visual Question Answering ◽

Previous State ◽

High Level

Visual question answering (VQA) requires a high-level understanding of both questions and images, along with visual reasoning to predict the correct answer. Therefore, it is important to design an effective attention model to associate key regions in an image with key words in a question. Up to now, most attention-based approaches only model the relationships between individual regions in an image and words in a question. It is not enough to predict the correct answer for VQA, as human beings always think in terms of global information, not only local information. In this paper, we propose a novel multi-modality global fusion attention network (MGFAN) consisting of stacked global fusion attention (GFA) blocks, which can capture information from global perspectives. Our proposed method computes co-attention and self-attention at the same time, rather than computing them individually. We validate our proposed method on the two most commonly used benchmarks, the VQA-v2 datasets. Experimental results show that the proposed method outperforms the previous state-of-the-art. Our best single model achieves 70.67% accuracy on the test-dev set of VQA-v2.

Download Full-text