scholarly journals Centralized Ranking Loss with Weakly Supervised Localization for Fine-Grained Object Retrieval

2018 ◽  
Author(s):  
Xiawu Zheng ◽  
Rongrong Ji ◽  
Xiaoshuai Sun ◽  
Yongjian Wu ◽  
Feiyue Huang ◽  
...  
Keyword(s):  
State Of The Art ◽  
Feature Learning ◽  
Target Object ◽  
Object Retrieval ◽  
Unified Framework ◽  
Fine Grained ◽  
Discriminative Feature ◽  
Triplet Loss ◽  
Weakly Supervised ◽  
Ranking Loss

Fine-grained object retrieval has attracted extensive research focus recently. Its state-of-the-art schemesare typically based upon convolutional neural network (CNN) features. Despite the extensive progress, two issues remain open. On one hand, the deep features are coarsely extracted at image level rather than precisely at object level, which are interrupted by background clutters. On the other hand, training CNN features with a standard triplet loss is time consuming and incapable to learn discriminative features. In this paper, we present a novel fine-grained object retrieval scheme that conquers these issues in a unified framework. Firstly, we introduce a novel centralized ranking loss (CRL), which achieves a very efficient (1,000times training speedup comparing to the triplet loss) and discriminative feature learning by a ?centralized? global pooling. Secondly, a weakly supervised attractive feature extraction is proposed, which segments object contours with top-down saliency. Consequently, the contours are integrated into the CNN response map to precisely extract features ?within? the target object. Interestingly, we have discovered that the combination of CRL and weakly supervised learning can reinforce each other. We evaluate the performance ofthe proposed scheme on widely-used benchmarks including CUB200-2011 and CARS196. We havereported significant gains over the state-of-the-art schemes, e.g., 5.4% over SCDA [Wei et al., 2017]on CARS196, and 3.7% on CUB200-2011.  

scholarly journals Bilinear CNN Model for Fine-Grained Classification Based on Subcategory-Similarity Measurement

2019 ◽  
Vol 9 (2) ◽  
pp. 301 ◽  
Author(s):  
Xinghua Dai ◽  
Shengrong Gong ◽  
Shan Zhong ◽  
Zongming Bao
Keyword(s):  
Background Noise ◽  
Supervised Classification ◽  
State Of The Art ◽  
Equal Treatment ◽  
Training Set ◽  
Significant Similarity ◽  
Fine Grained ◽  
Triplet Loss ◽  
Weakly Supervised ◽  
Weakly Supervised Classification

One of the challenges in fine-grained classification is that subcategories with significant similarity are hard to be distinguished due to the equal treatment of all subcategories in existing algorithms. In order to solve this problem, a fine-grained image classification method by combining a bilinear convolutional neural network (B-CNN) and the measurement of subcategory similarities is proposed. Firstly, an improved weakly supervised localization method is designed to obtain the bounding box of the main object, which allows the model to eliminate the influence of background noise and obtain more accurate features. Then, sample features in the training set are computed by B-CNN so that the fuzzing similarity matrix for measuring interclass similarities can be obtained. To further improve classification accuracy, the loss function is designed by weighting triplet loss and softmax loss. Extensive experiments implemented on two benchmarks datasets, Stanford Cars-196 and Caltech-UCSD Birds-200-2011 (CUB-200-2011), show that the newly proposed method outperforms in accuracy several state-of-the-art weakly supervised classification models.

scholarly journals Multiproxies Adaptive Distribution Loss with Weakly Supervised Feature Aggregation for Fine-Grained Retrieval

2021 ◽  
Vol 2021 ◽  
pp. 1-16
Author(s):  
Hongwei Zhao ◽  
Danyang Zhang ◽  
Jiaxin Wu ◽  
Pingping Liu
Keyword(s):  
Image Retrieval ◽  
Image Feature ◽  
Cluster Center ◽  
Unified Framework ◽  
Retrieval Method ◽  
Fine Grained ◽  
Feature Aggregation ◽  
Benchmark Datasets ◽  
Weakly Supervised ◽  
Representative Image

Fine-grained retrieval is one of the complex problems in computer vision. Compared with general content-based image retrieval, fine-grained image retrieval faces more difficult challenges. In fine-grained image retrieval tasks, all classes belong to a subclass of a meta-class, so there will be small interclass variance and large intraclass variance. In order to solve this problem, in this paper, we propose a fine-grained retrieval method to improve loss and feature aggregation, which can achieve better retrieval results under a unified framework. Firstly, we propose a novel multiproxies adaptive distribution loss which can better characterize the intraclass variations and the degree of dispersion of each cluster center. Secondly, we propose a weakly supervised feature aggregation method based on channel weighting, which distinguishes the importance of different feature channels to obtain more representative image feature descriptors. We verify the performance of our proposed method on the universal benchmark datasets such as CUB200-2011 and Stanford Dog. Higher Recall@K demonstrates the advantage of our proposed method over the state of the art.

scholarly journals Graph-Propagation Based Correlation Learning for Weakly Supervised Fine-Grained Image Classification

2020 ◽  
Vol 34 (07) ◽  
pp. 12289-12296 ◽  
Author(s):  
Zhuhui Wang ◽  
Shijie Wang ◽  
Haojie Li ◽  
Zhi Dou ◽  
Jianjun Li
Keyword(s):  
Image Classification ◽  
Feature Representation ◽  
Spatial Context ◽  
Discriminative Power ◽  
Fine Grained ◽  
Semantic Correlation ◽  
Discriminative Feature ◽  
Correlation Learning ◽  
Weakly Supervised ◽  
Graph Propagation

The key of Weakly Supervised Fine-grained Image Classification (WFGIC) is how to pick out the discriminative regions and learn the discriminative features from them. However, most recent WFGIC methods pick out the discriminative regions independently and utilize their features directly, while neglecting the facts that regions' features are mutually semantic correlated and region groups can be more discriminative. To address these issues, we propose an end-to-end Graph-propagation based Correlation Learning (GCL) model to fully mine and exploit the discriminative potentials of region correlations for WFGIC. Specifically, in discriminative region localization phase, a Criss-cross Graph Propagation (CGP) sub-network is proposed to learn region correlations, which establishes correlation between regions and then enhances each region by weighted aggregating other regions in a criss-cross way. By this means each region's representation encodes the global image-level context and local spatial context simultaneously, thus the network is guided to implicitly discover the more powerful discriminative region groups for WFGIC. In discriminative feature representation phase, the Correlation Feature Strengthening (CFS) sub-network is proposed to explore the internal semantic correlation among discriminative patches' feature vectors, to improve their discriminative power by iteratively enhancing informative elements while suppressing the useless ones. Extensive experiments demonstrate the effectiveness of proposed CGP and CFS sub-networks, and show that the GCL model achieves better performance both in accuracy and efficiency.

scholarly journals Hierarchical Online Instance Matching for Person Search

2020 ◽  
Vol 34 (07) ◽  
pp. 10518-10525
Author(s):  
Di Chen ◽  
Shanshan Zhang ◽  
Wanli Ouyang ◽  
Jian Yang ◽  
Bernt Schiele
Keyword(s):  
State Of The Art ◽  
Pedestrian Detection ◽  
Feature Learning ◽  
Optimal Solution ◽  
Unified Framework ◽  
Discrimination Power ◽  
Hierarchical Relationship ◽  
Instance Matching ◽  
Image Dataset ◽  
Person Search

Person Search is a challenging task which requires to retrieve a person's image and the corresponding position from an image dataset. It consists of two sub-tasks: pedestrian detection and person re-identification (re-ID). One of the key challenges is to properly combine the two sub-tasks into a unified framework. Existing works usually adopt a straightforward strategy by concatenating a detector and a re-ID model directly, either into an integrated model or into separated models. We argue that simply concatenating detection and re-ID is a sub-optimal solution, and we propose a Hierarchical Online Instance Matching (HOIM) loss which exploits the hierarchical relationship between detection and re-ID to guide the learning of our network. Our novel HOIM loss function harmonizes the objectives of the two sub-tasks and encourages better feature learning. In addition, we improve the loss update policy by introducing Selective Memory Refreshment (SMR) for unlabeled persons, which takes advantage of the potential discrimination power of unlabeled data. From the experiments on two standard person search benchmarks, i.e. CUHK-SYSU and PRW, we achieve state-of-the-art performance, which justifies the effectiveness of our proposed HOIM loss on learning robust features.

scholarly journals Filtration and Distillation: Enhancing Region Attention for Fine-Grained Visual Categorization

2020 ◽  
Vol 34 (07) ◽  
pp. 11555-11562 ◽  
Author(s):  
Chuanbin Liu ◽  
Hongtao Xie ◽  
Zheng-Jun Zha ◽  
Lingfeng Ma ◽  
Lingyun Yu ◽  
...  
Keyword(s):  
State Of The Art ◽  
Critical Role ◽  
Feature Learning ◽  
Direct Optimization ◽  
Visual Categorization ◽  
Fine Grained ◽  
Performance Metric ◽  
Object Based ◽  
Feature Extractor ◽  
Entire Object

Delicate attention of the discriminative regions plays a critical role in Fine-Grained Visual Categorization (FGVC). Unfortunately, most of the existing attention models perform poorly in FGVC, due to the pivotal limitations in discriminative regions proposing and region-based feature learning. 1) The discriminative regions are predominantly located based on the filter responses over the images, which can not be directly optimized with a performance metric. 2) Existing methods train the region-based feature extractor as a one-hot classification task individually, while neglecting the knowledge from the entire object. To address the above issues, in this paper, we propose a novel “Filtration and Distillation Learning” (FDL) model to enhance the region attention of discriminate parts for FGVC. Firstly, a Filtration Learning (FL) method is put forward for discriminative part regions proposing based on the matchability between proposing and predicting. Specifically, we utilize the proposing-predicting matchability as the performance metric of Region Proposal Network (RPN), thus enable a direct optimization of RPN to filtrate most discriminative regions. Go in detail, the object-based feature learning and region-based feature learning are formulated as “teacher” and “student”, which can furnish better supervision for region-based feature learning. Accordingly, our FDL can enhance the region attention effectively, and the overall framework can be trained end-to-end without neither object nor parts annotations. Extensive experiments verify that FDL yields state-of-the-art performance under the same backbone with the most competitive approaches on several FGVC tasks.

scholarly journals Mis-Classified Vector Guided Softmax Loss for Face Recognition

2020 ◽  
Vol 34 (07) ◽  
pp. 12241-12248 ◽  
Author(s):  
Xiaobo Wang ◽  
Shifeng Zhang ◽  
Shuo Wang ◽  
Tianyu Fu ◽  
Hailin Shi ◽  
...  
Keyword(s):  
Face Recognition ◽  
Loss Function ◽  
State Of The Art ◽  
Feature Learning ◽  
Ground Truth ◽  
Significant Progress ◽  
Deep Convolutional Neural Networks ◽  
Face Features ◽  
Discriminative Feature ◽  
Feature Mining

Face recognition has witnessed significant progress due to the advances of deep convolutional neural networks (CNNs), the central task of which is how to improve the feature discrimination. To this end, several margin-based (e.g., angular, additive and additive angular margins) softmax loss functions have been proposed to increase the feature margin between different classes. However, despite great achievements have been made, they mainly suffer from three issues: 1) Obviously, they ignore the importance of informative features mining for discriminative learning; 2) They encourage the feature margin only from the ground truth class, without realizing the discriminability from other non-ground truth classes; 3) The feature margin between different classes is set to be same and fixed, which may not adapt the situations very well. To cope with these issues, this paper develops a novel loss function, which adaptively emphasizes the mis-classified feature vectors to guide the discriminative feature learning. Thus we can address all the above issues and achieve more discriminative face features. To the best of our knowledge, this is the first attempt to inherit the advantages of feature margin and feature mining into a unified loss function. Experimental results on several benchmarks have demonstrated the effectiveness of our method over state-of-the-art alternatives. Our code is available at http://www.cbsr.ia.ac.cn/users/xiaobowang/.

scholarly journals Deep Robust Unsupervised Multi-Modal Network

2019 ◽  
Vol 33 ◽  
pp. 5652-5659
Author(s):  
Yang Yang ◽  
Yi-Feng Wu ◽  
De-Chuan Zhan ◽  
Zhi-Bin Liu ◽  
Yuan Jiang
Keyword(s):  
Real World ◽  
Feature Learning ◽  
Learning Performance ◽  
Real Problem ◽  
Learning Approaches ◽  
Unified Framework ◽  
Heterogeneous Information ◽  
Feature Representations ◽  
Discriminative Feature ◽  
Different Sources

In real-world applications, data are often with multiple modalities, and many multi-modal learning approaches are proposed for integrating the information from different sources. Most of the previous multi-modal methods utilize the modal consistency to reduce the complexity of the learning problem, therefore the modal completeness needs to be guaranteed. However, due to the data collection failures, self-deficiencies, and other various reasons, multi-modal instances are often incomplete in real applications, and have the inconsistent anomalies even in the complete instances, which jointly result in the inconsistent problem. These degenerate the multi-modal feature learning performance, and will finally affect the generalization abilities in different tasks. In this paper, we propose a novel Deep Robust Unsupervised Multi-modal Network structure (DRUMN) for solving this real problem within a unified framework. The proposed DRUMN can utilize the extrinsic heterogeneous information from unlabeled data against the insufficiency caused by the incompleteness. On the other hand, the inconsistent anomaly issue is solved with an adaptive weighted estimation, rather than adjusting the complex thresholds. As DRUMN can extract the discriminative feature representations for each modality, experiments on real-world multimodal datasets successfully validate the effectiveness of our proposed method.

scholarly journals Learning Visual Words for Weakly-Supervised Semantic Segmentation

2021 ◽  
Author(s):  
Lixiang Ru ◽  
Bo Du ◽  
Chen Wu
Keyword(s):  
State Of The Art ◽  
Local Maximum ◽  
Semantic Segmentation ◽  
Input Image ◽  
Feature Maps ◽  
Visual Words ◽  
Fine Grained ◽  
Spatial Pyramid Pooling ◽  
Weakly Supervised ◽  
Spatial Pyramid

Current weakly-supervised semantic segmentation (WSSS) methods with image-level labels mainly adopt class activation maps (CAM) to generate the initial pseudo labels. However, CAM usually only identifies the most discriminative object extents, which is attributed to the fact that the network doesn't need to discover the integral object to recognize image-level labels. In this work, to tackle this problem, we proposed to simultaneously learn the image-level labels and local visual word labels. Specifically, in each forward propagation, the feature maps of the input image will be encoded to visual words with a learnable codebook. By enforcing the network to classify the encoded fine-grained visual words, the generated CAM could cover more semantic regions. Besides, we also proposed a hybrid spatial pyramid pooling module that could preserve local maximum and global average values of feature maps, so that more object details and less background were considered. Based on the proposed methods, we conducted experiments on the PASCAL VOC 2012 dataset. Our proposed method achieved 67.2% mIoU on the val set and 67.3% mIoU on the test set, which outperformed recent state-of-the-art methods.

Sign in / Sign up

Export Citation Format

Share Document