Subset feature learning for fine-grained category classification

Fine-grained object retrieval has attracted extensive research focus recently. Its state-of-the-art schemesare typically based upon convolutional neural network (CNN) features. Despite the extensive progress, two issues remain open. On one hand, the deep features are coarsely extracted at image level rather than precisely at object level, which are interrupted by background clutters. On the other hand, training CNN features with a standard triplet loss is time consuming and incapable to learn discriminative features. In this paper, we present a novel fine-grained object retrieval scheme that conquers these issues in a unified framework. Firstly, we introduce a novel centralized ranking loss (CRL), which achieves a very efficient (1,000times training speedup comparing to the triplet loss) and discriminative feature learning by a ?centralized? global pooling. Secondly, a weakly supervised attractive feature extraction is proposed, which segments object contours with top-down saliency. Consequently, the contours are integrated into the CNN response map to precisely extract features ?within? the target object. Interestingly, we have discovered that the combination of CRL and weakly supervised learning can reinforce each other. We evaluate the performance ofthe proposed scheme on widely-used benchmarks including CUB200-2011 and CARS196. We havereported significant gains over the state-of-the-art schemes, e.g., 5.4% over SCDA [Wei et al., 2017]on CARS196, and 3.7% on CUB200-2011.

Download Full-text

Fine-grained image analysis via progressive feature learning

Neurocomputing ◽

10.1016/j.neucom.2018.07.100 ◽

2020 ◽

Vol 396 ◽

pp. 254-265

Author(s):

Yichao Yan ◽

Bingbing Ni ◽

Huawei Wei ◽

Xiaokang Yang

Keyword(s):

Image Analysis ◽

Feature Learning ◽

Fine Grained

Download Full-text

PointSite: a point cloud segmentation tool for identification of protein ligand binding atoms

10.1101/831131 ◽

2019 ◽

Author(s):

Zhen Li ◽

Xu Yan ◽

Qing Wei ◽

Xin Gao ◽

Sheng Wang ◽

...

Keyword(s):

Protein Structure ◽

Ligand Binding ◽

Protein Function ◽

Feature Learning ◽

Point Clouds ◽

Accurate Identification ◽

3D Protein Structure ◽

Fine Grained ◽

Atom Level ◽

Cascaded Filter

AbstractAccurate identifications of ligand binding sites (LBS) on protein structure is critical for understanding protein function and designing structure-based drug. As the previous pocket-centric methods are usually based on the investigation of pseudo surface points (PSPs) outside the protein structure, thus inherently cannot incorporate the local connectivity and global 3D geometrical information of the protein structure. In this paper, we propose a novel point clouds segmentation method, PointSite, for accurate identification of protein ligand binding atoms, which performs protein LBS identification at the atom-level in a protein-centric manner. Specifically, we first transfer the original 3D protein structure to point clouds and then conduct segmentation through Submanifold Sparse Convolution (SSC) based U-Net. With the fine-grained atom-level binding atoms representation and enhanced feature learning, PointSite can outperform previous methods in atom-IoU by a large margin. Furthermore, our segmented binding atoms can work as a filter on predictions achieved by previous pocket-centric approaches, which significantly decreases the false-positive of LBS candidates. Through cascaded filter and re-ranking aided by the segmented atoms, state-of-the-art performance can be achieved over various canonical benchmarks and CAMEO hard targets in terms of the commonly used DCA criteria. Our code is publicly available through https://github.com/PointSite.

Download Full-text

Multi-attention Meta Learning for Few-shot Fine-grained Image Recognition

Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2020/152 ◽

2020 ◽

Author(s):

Yaohui Zhu ◽

Chenlong Liu ◽

Shuqiang Jiang

Keyword(s):

Image Recognition ◽

Feature Learning ◽

Learning Approaches ◽

Fine Grained ◽

Meta Learning ◽

Benchmark Datasets ◽

Gradient Based ◽

General Object ◽

Base Learner ◽

Discriminative Parts

The goal of few-shot image recognition is to distinguish different categories with only one or a few training samples. Previous works of few-shot learning mainly work on general object images. And current solutions usually learn a global image representation from training tasks to adapt novel tasks. However, fine-gained categories are distinguished by subtle and local parts, which could not be captured by global representations effectively. This may hinder existing few-shot learning approaches from dealing with fine-gained categories well. In this work, we propose a multi-attention meta-learning (MattML) method for few-shot fine-grained image recognition (FSFGIR). Instead of using only base learner for general feature learning, the proposed meta-learning method uses attention mechanisms of the base learner and task learner to capture discriminative parts of images. The base learner is equipped with two convolutional block attention modules (CBAM) and a classifier. The two CBAM can learn diverse and informative parts. And the initial weights of classifier are attended by the task learner, which gives the classifier a task-related sensitive initialization. For adaptation, the gradient-based meta-learning approach is employed by updating the parameters of two CBAM and the attended classifier, which facilitates the updated base learner to adaptively focus on discriminative parts. We experimentally analyze the different components of our method, and experimental results on four benchmark datasets demonstrate the effectiveness and superiority of our method.

Download Full-text

Syntactically Meaningful and Transferable Recursive Neural Networks for Aspect and Opinion Extraction

Computational Linguistics ◽

10.1162/coli_a_00362 ◽

2020 ◽

Vol 45 (4) ◽

pp. 705-736

Author(s):

Wenya Wang ◽

Sinno Jialin Pan

Keyword(s):

Neural Network ◽

Neural Networks ◽

Syntactic Structure ◽

Feature Learning ◽

Single Domain ◽

Fine Grained ◽

Adversarial Network ◽

Dependency Tree ◽

Recursive Neural Networks ◽

Syntactic Relations

In fine-grained opinion mining, extracting aspect terms (a.k.a. opinion targets) and opinion terms (a.k.a. opinion expressions) from user-generated texts is the most fundamental task in order to generate structured opinion summarization. Existing studies have shown that the syntactic relations between aspect and opinion words play an important role for aspect and opinion terms extraction. However, most of the works either relied on predefined rules or separated relation mining with feature learning. Moreover, these works only focused on single-domain extraction, which failed to adapt well to other domains of interest where only unlabeled data are available. In real-world scenarios, annotated resources are extremely scarce for many domains, motivating knowledge transfer strategies from labeled source domain(s) to any unlabeled target domain. We observe that syntactic relations among target words to be extracted are not only crucial for single-domain extraction, but also serve as invariant “pivot” information to bridge the gap between different domains. In this article, we explore the constructions of recursive neural networks based on the dependency tree of each sentence for associating syntactic structure with feature learning. Furthermore, we construct transferable recursive neural networks to automatically learn the domain-invariant fine-grained interactions among aspect words and opinion words. The transferability is built on an auxiliary task and a conditional domain adversarial network to reduce domain distribution difference in the hidden spaces effectively in word level through syntactic relations. Specifically, the auxiliary task builds structural correspondences across domains by predicting the dependency relation for each path of the dependency tree in the recursive neural network. The conditional domain adversarial network helps to learn domain-invariant hidden representation for each word conditioned on the syntactic structure. In the end, we integrate the recursive neural network with a sequence labeling classifier on top that models contextual influence in the final predictions. Extensive experiments and analysis are conducted to demonstrate the effectiveness of the proposed model and each component on three benchmark data sets.

Download Full-text

Fine-Grained 3D-Attention Prototypes for Few-Shot Learning

Neural Computation ◽

10.1162/neco_a_01302 ◽

2020 ◽

Vol 32 (9) ◽

pp. 1664-1684

Author(s):

Xin Hu ◽

Jun Liu ◽

Jie Ma ◽

Yudai Pan ◽

Lingling Zhang

Keyword(s):

Image Reconstruction ◽

Feature Learning ◽

Local Features ◽

Image Features ◽

Superior Performance ◽

Fine Grained ◽

Learning Module ◽

Class Distribution ◽

Label Distribution ◽

3D Attention

In the real world, a limited number of labeled finely grained images per class can hardly represent the class distribution effectively. Due to the more subtle visual differences in fine-grained images than simple images with obvious objects, that is, there exist smaller interclass and larger intraclass variations. To solve these issues, we propose an end-to-end attention-based model for fine-grained few-shot image classification (AFG) with the recent episode training strategy. It is composed mainly of a feature learning module, an image reconstruction module, and a label distribution module. The feature learning module mainly devises a 3D-Attention mechanism, which considers both the spatial positions and different channel attentions of the image features, in order to learn more discriminative local features to better represent the class distribution. The image reconstruction module calculates the mappings between local features and the original images. It is constrained by a designed loss function as auxiliary supervised information, so that the learning of each local feature does not need extra annotations. The label distribution module is used to predict the label distribution of a given unlabeled sample, and we use the local features to represent the image features for classification. By conducting comprehensive experiments on Mini-ImageNet and three fine-grained data sets, we demonstrate that the proposed model achieves superior performance over the competitors.

Download Full-text

Multi-layer and multi-order fine-grained feature learning for artwork attribute recognition

Computer Communications ◽

10.1016/j.comcom.2021.03.006 ◽

2021 ◽

Author(s):

Yang Gao ◽

Neng Chang ◽

Kai Shang

Keyword(s):

Feature Learning ◽

Fine Grained ◽

Attribute Recognition

Download Full-text

Filtration and Distillation: Enhancing Region Attention for Fine-Grained Visual Categorization

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v34i07.6822 ◽

2020 ◽

Vol 34 (07) ◽

pp. 11555-11562 ◽

Cited By ~ 1

Author(s):

Chuanbin Liu ◽

Hongtao Xie ◽

Zheng-Jun Zha ◽

Lingfeng Ma ◽

Lingyun Yu ◽

...

Keyword(s):

State Of The Art ◽

Critical Role ◽

Feature Learning ◽

Direct Optimization ◽

Visual Categorization ◽

Fine Grained ◽

Performance Metric ◽

Object Based ◽

Feature Extractor ◽

Entire Object

Delicate attention of the discriminative regions plays a critical role in Fine-Grained Visual Categorization (FGVC). Unfortunately, most of the existing attention models perform poorly in FGVC, due to the pivotal limitations in discriminative regions proposing and region-based feature learning. 1) The discriminative regions are predominantly located based on the filter responses over the images, which can not be directly optimized with a performance metric. 2) Existing methods train the region-based feature extractor as a one-hot classification task individually, while neglecting the knowledge from the entire object. To address the above issues, in this paper, we propose a novel “Filtration and Distillation Learning” (FDL) model to enhance the region attention of discriminate parts for FGVC. Firstly, a Filtration Learning (FL) method is put forward for discriminative part regions proposing based on the matchability between proposing and predicting. Specifically, we utilize the proposing-predicting matchability as the performance metric of Region Proposal Network (RPN), thus enable a direct optimization of RPN to filtrate most discriminative regions. Go in detail, the object-based feature learning and region-based feature learning are formulated as “teacher” and “student”, which can furnish better supervision for region-based feature learning. Accordingly, our FDL can enhance the region attention effectively, and the overall framework can be trained end-to-end without neither object nor parts annotations. Extensive experiments verify that FDL yields state-of-the-art performance under the same backbone with the most competitive approaches on several FGVC tasks.

Download Full-text