scholarly journals Multi-Level Visual-Semantic Alignments with Relation-Wise Dual Attention Network for Image and Text Matching

2019 ◽  
Author(s):  
Zhibin Hu ◽  
Yongsheng Luo ◽  
Jiong Lin ◽  
Yan Yan ◽  
Jian Chen
Keyword(s):  
Semantic Network ◽  
Attention Network ◽  
Main Challenge ◽  
Concrete Objects ◽  
Benchmark Datasets ◽  
Dual Pathway ◽  
Local Correlations ◽  
Complete Set ◽  
Multi Level ◽  
Text Matching

Image-text matching is central to visual-semantic cross-modal retrieval and has been attracting extensive attention recently. Previous studies have been devoted to finding the latent correspondence between image regions and words, e.g., connecting key words to specific regions of salient objects. However, existing methods are usually committed to handle concrete objects, rather than abstract ones, e.g., a description of some action, which in fact are also ubiquitous in description texts of real-world. The main challenge in dealing with abstract objects is that there is no explicit connections between them, unlike their concrete counterparts. One therefore has to alternatively find the implicit and intrinsic connections between them. In this paper, we propose a relation-wise dual attention network (RDAN) for image-text matching. Specifically, we maintain an over-complete set that contains pairs of regions and words. Then built upon this set, we encode the local correlations and the global dependencies between regions and words by training a visual-semantic network. Then a dual pathway attention network is presented to infer the visual-semantic alignments and image-text similarity. Extensive experiments validate the efficacy of our method, by achieving the state-of-the-art performance on several public benchmark datasets.

Attend, Correct And Focus: A Bidirectional Correct Attention Network For Image-Text Matching

2021 ◽  
Author(s):  
Yang Liu ◽  
Huaqiu Wang ◽  
Fanyang Meng ◽  
Mengyuan Liu ◽  
Hong Liu
Keyword(s):  
Attention Network ◽  
Text Matching

scholarly journals k-Nearest Neighbor Learning with Graph Neural Networks

Mathematics ◽  
2021 ◽  
Vol 9 (8) ◽  
pp. 830
Author(s):  
Seokho Kang
Keyword(s):  
Neural Network ◽  
Nearest Neighbor ◽  
Learning Algorithm ◽  
Weighting Function ◽  
High Sensitivity ◽  
Training Data ◽  
K Nearest Neighbor ◽  
Main Challenge ◽  
Benchmark Datasets ◽  
Graph Neural Networks

k-nearest neighbor (kNN) is a widely used learning algorithm for supervised learning tasks. In practice, the main challenge when using kNN is its high sensitivity to its hyperparameter setting, including the number of nearest neighbors k, the distance function, and the weighting function. To improve the robustness to hyperparameters, this study presents a novel kNN learning method based on a graph neural network, named kNNGNN. Given training data, the method learns a task-specific kNN rule in an end-to-end fashion by means of a graph neural network that takes the kNN graph of an instance to predict the label of the instance. The distance and weighting functions are implicitly embedded within the graph neural network. For a query instance, the prediction is obtained by performing a kNN search from the training data to create a kNN graph and passing it through the graph neural network. The effectiveness of the proposed method is demonstrated using various benchmark datasets for classification and regression tasks.

scholarly journals Multi-granularity Deep Local Representations for Irregular Scene Text Recognition

2021 ◽  
Vol 2 (2) ◽  
pp. 1-18
Author(s):  
Hongchao Gao ◽  
Yujia Li ◽  
Jiao Dai ◽  
Xi Wang ◽  
Jizhong Han ◽  
...  
Keyword(s):  
State Of The Art ◽  
Visual Representation ◽  
Text Recognition ◽  
Natural Scene ◽  
Attention Network ◽  
Training Time ◽  
Scene Text ◽  
Benchmark Datasets ◽  
Local Representations ◽  
Scene Text Recognition

Recognizing irregular text from natural scene images is challenging due to the unconstrained appearance of text, such as curvature, orientation, and distortion. Recent recognition networks regard this task as a text sequence labeling problem and most networks capture the sequence only from a single-granularity visual representation, which to some extent limits the performance of recognition. In this article, we propose a hierarchical attention network to capture multi-granularity deep local representations for recognizing irregular scene text. It consists of several hierarchical attention blocks, and each block contains a Local Visual Representation Module (LVRM) and a Decoder Module (DM). Based on the hierarchical attention network, we propose a scene text recognition network. The extensive experiments show that our proposed network achieves the state-of-the-art performance on several benchmark datasets including IIIT-5K, SVT, CUTE, SVT-Perspective, and ICDAR datasets under shorter training time.

Unified Adaptive Relevance Distinguishable Attention Network for Image-Text Matching

2022 ◽  
pp. 1-1
Author(s):  
Kun Zhang ◽  
Zhendong Mao ◽  
Anan Liu ◽  
Yongdong Zhang
Keyword(s):  
Attention Network ◽  
Text Matching

SANTM: Efficient Self-attention-driven Network for Text Matching

2022 ◽  
Vol 22 (3) ◽  
pp. 1-21
Author(s):  
Prayag Tiwari ◽  
Amit Kumar Jaiswal ◽  
Sahil Garg ◽  
Ilsun You
Keyword(s):  
Natural Language ◽  
State Of The Art ◽  
The State ◽  
Attention Mechanism ◽  
Matching Problems ◽  
Attention Model ◽  
Extra Information ◽  
Textual Entailment ◽  
Benchmark Datasets ◽  
Text Matching

Self-attention mechanisms have recently been embraced for a broad range of text-matching applications. Self-attention model takes only one sentence as an input with no extra information, i.e., one can utilize the final hidden state or pooling. However, text-matching problems can be interpreted either in symmetrical or asymmetrical scopes. For instance, paraphrase detection is an asymmetrical task, while textual entailment classification and question-answer matching are considered asymmetrical tasks. In this article, we leverage attractive properties of self-attention mechanism and proposes an attention-based network that incorporates three key components for inter-sequence attention: global pointwise features, preceding attentive features, and contextual features while updating the rest of the components. Our model follows evaluation on two benchmark datasets cover tasks of textual entailment and question-answer matching. The proposed efficient Self-attention-driven Network for Text Matching outperforms the state of the art on the Stanford Natural Language Inference and WikiQA datasets with much fewer parameters.

scholarly journals Interactive Dual Attention Network for Text Sentiment Classification

2020 ◽  
Vol 2020 ◽  
pp. 1-11
Author(s):  
Yinglin Zhu ◽  
Wenbin Zheng ◽  
Hong Tang
Keyword(s):  
Language Processing ◽  
Classification Performance ◽  
Research Field ◽  
Sentiment Classification ◽  
Attention Network ◽  
Linguistic Resources ◽  
Benchmark Datasets ◽  
Interactive Relationship ◽  
Conventional Machine ◽  
Result Analysis

Text sentiment classification is an essential research field of natural language processing. Recently, numerous deep learning-based methods for sentiment classification have been proposed and achieved better performances compared with conventional machine learning methods. However, most of the proposed methods ignore the interactive relationship between contextual semantics and sentimental tendency while modeling their text representation. In this paper, we propose a novel Interactive Dual Attention Network (IDAN) model that aims to interactively learn the representation between contextual semantics and sentimental tendency information. Firstly, we design an algorithm that utilizes linguistic resources to obtain sentimental tendency information from text and then extract word embeddings from the BERT (Bidirectional Encoder Representations from Transformers) pretraining model as the embedding layer of IDAN. Next, we use two Bidirectional LSTM (BiLSTM) networks to learn the long-range dependencies of contextual semantics and sentimental tendency information, respectively. Finally, two types of attention mechanisms are implemented in IDAN. One is multihead attention, which is the next layer of BiLSTM and is used to learn the interactive relationship between contextual semantics and sentimental tendency information. The other is global attention that aims to make the model focus on the important parts of the sequence and generate the final representation for classification. These two attention mechanisms enable IDAN to interactively learn the relationship between semantics and sentimental tendency information and improve the classification performance. A large number of experiments on four benchmark datasets show that our IDAN model is superior to competitive methods. Moreover, both the result analysis and the attention weight visualization further demonstrate the effectiveness of our proposed method.

Sign in / Sign up

Export Citation Format

Share Document