HIERARCHICAL DISTANCE METRIC LEARNING FOR LARGE MARGIN NEAREST NEIGHBOR CLASSIFICATION

2011 ◽  
Vol 25 (07) ◽  
pp. 1073-1087 ◽  
Author(s):  
SHILIANG SUN ◽  
QIAONA CHEN
Keyword(s):  
Nearest Neighbor ◽  
Metric Learning ◽  
Data Sets ◽  
Distance Metric Learning ◽  
Distance Metric ◽  
Real World Data ◽  
K Nearest Neighbors ◽  
Large Margin ◽  
Nearest Neighbor Classification ◽  
Neighbor Classification

Distance metric learning is a powerful tool to improve performance in classification, clustering and regression tasks. Many techniques have been proposed for distance metric learning based on convex programming, kernel learning, dimension reduction and large margin. The recently proposed large margin nearest neighbor classification (LMNN) improves the performance of k-nearest neighbors classification (k-nn) by a learned global distance metric. However, it does not consider the locality of data distributions. We demonstrate a novel local distance metric learning method called hierarchical distance metric learning (HDM) which first builds a hierarchical structure by grouping data points according to the overlapping ratios defined by us and then learns distance metrics sequentially. In this paper, we combine HDM with LMNN and further propose a new method named hierarchical distance metric learning for large margin nearest neighbor classification (HLMNN). Experiments are performed on many artificial and real-world data sets. Comparisons with the traditional k-nn and the state-of-the-art LMNN show the effectiveness of the proposed HLMNN.

scholarly journals Acceleration of Large Margin Metric Learning for Nearest Neighbor Classification Using Triplet Mining and Stratified Sampling

2021 ◽  
Vol 6 (1) ◽  
pp. 1-5
Author(s):  
Parisa Abdolrahim Poorheravi ◽  
Benyamin Ghojogh ◽  
Vincent Gaudet ◽  
Fakhri Karray ◽  
Mark Crowley
Keyword(s):  
Nearest Neighbor ◽  
Metric Learning ◽  
Stratified Sampling ◽  
Hierarchical Approach ◽  
Large Margin ◽  
Nearest Neighbor Classification ◽  
Triplet Loss ◽  
Mining Methods ◽  
Siamese Networks ◽  
Neighbor Classification

Metric learning is a technique in manifold learning to find a projection subspace for increasing and decreasing the inter- and intra-class variances, respectively. Some metric learning methods are based on triplet learning with anchor-positive-negative triplets. Large margin metric learning for nearest neighbor classification is one of the fundamental methods to do this. Recently, Siamese networks have been introduced with the triplet loss. Many triplet mining methods have been developed for Siamese nets; however, these techniques have not been applied on the triplets of large margin metric learning. In this work, inspired by the mining methods for Siamese nets, we propose several triplet mining techniques for large margin metric learning. Moreover, a hierarchical approach is proposed, for acceleration and scalability of optimization, where triplets are selected by stratified sampling in hierarchical hyper-spheres. We analyze the proposed methods on three publicly available datasets.

scholarly journals Chi-Squared Distance Metric Learning for Histogram Data

2015 ◽  
Vol 2015 ◽  
pp. 1-12 ◽  
Author(s):  
Wei Yang ◽  
Luhui Xu ◽  
Xiaopan Chen ◽  
Fengbin Zheng ◽  
Yang Liu
Keyword(s):  
Nearest Neighbor ◽  
State Of The Art ◽  
Metric Learning ◽  
Nearest Neighbors ◽  
Distance Metric Learning ◽  
Distance Metric ◽  
Projected Gradient Method ◽  
Proper Distance ◽  
Chi Squared ◽  
Real World Datasets

Learning a proper distance metric for histogram data plays a crucial role in many computer vision tasks. The chi-squared distance is a nonlinear metric and is widely used to compare histograms. In this paper, we show how to learn a general form of chi-squared distance based on the nearest neighbor model. In our method, the margin of sample is first defined with respect to the nearest hits (nearest neighbors from the same class) and the nearest misses (nearest neighbors from the different classes), and then the simplex-preserving linear transformation is trained by maximizing the margin while minimizing the distance between each sample and its nearest hits. With the iterative projected gradient method for optimization, we naturally introduce thel2,1norm regularization into the proposed method for sparse metric learning. Comparative studies with the state-of-the-art approaches on five real-world datasets verify the effectiveness of the proposed method.

scholarly journals Pattern and Feature Selection by Genetic Algorithms in Nearest Neighbor Classification

2000 ◽  
Vol 4 (2) ◽  
pp. 138-145 ◽  
Author(s):  
Hisao Ishibuchi ◽  
◽  
Tomoharu Nakashima
Keyword(s):  
Genetic Algorithm ◽  
Genetic Algorithms ◽  
Feature Selection ◽  
Computer Simulations ◽  
Nearest Neighbor ◽  
Classification Performance ◽  
Data Sets ◽  
Nearest Neighbor Classification ◽  
Reference Set ◽  
Neighbor Classification

This paper proposes a genetic-algorithm-based approach for finding a compact reference set in nearest neighbor classification. The reference set is designed by selecting a small number of reference patterns from a large number of training patterns using a genetic algorithm. The genetic algorithm also removes unnecessary features. The reference set in our nearest neighbor classification consists of selected patterns with selected features. A binary string is used for representing the inclusion (or exclusion) of each pattern and feature in the reference set. Our goal is to minimize the number of selected patterns, to minimize the number of selected features, and to maximize the classification performance of the reference set. Computer simulations on commonly used data sets examine the effectiveness of our approach.

A nearest-neighbor search model for distance metric learning

2021 ◽  
Vol 552 ◽  
pp. 261-277
Author(s):  
Yibang Ruan ◽  
Yanshan Xiao ◽  
Zhifeng Hao ◽  
Bo Liu
Keyword(s):  
Nearest Neighbor ◽  
Metric Learning ◽  
Nearest Neighbor Search ◽  
Search Model ◽  
Distance Metric Learning ◽  
Distance Metric ◽  
Neighbor Search
Sign in / Sign up

Export Citation Format

Share Document