scholarly journals Learning Robust Distance Metric with Side Information via Ratio Minimization of Orthogonally Constrained L21-Norm Distances

2019 ◽  
Author(s):  
Kai Liu ◽  
Lodewijk Brand ◽  
Hua Wang ◽  
Feiping Nie
Keyword(s):  
State Of The Art ◽  
Side Information ◽  
Metric Learning ◽  
Iterative Solution ◽  
Distance Metric ◽  
Data Set ◽  
The Past ◽  
Robust Model ◽  
L2 Norm ◽  
Data Objects

Metric Learning, which aims at learning a distance metric for a given data set, plays an important role in measuring the distance or similarity between data objects. Due to its broad usefulness, it has attracted a lot of interest in machine learning and related areas in the past few decades. This paper proposes to learn the distance metric from the side information in the forms of must-links and cannot-links. Given the pairwise constraints, our goal is to learn a Mahalanobis distance that minimizes the ratio of the distances of the data pairs in the must-links to those in the cannot-links. Different from many existing papers that use the traditional squared L2-norm distance, we develop a robust model that is less sensitive to data noise or outliers by using the not-squared L2-norm distance. In our objective, the orthonormal constraint is enforced to avoid degenerate solutions. To solve our objective, we have derived an efficient iterative solution algorithm. We have conducted extensive experiments, which demonstrated the superiority of our method over state-of-the-art.

scholarly journals Chi-Squared Distance Metric Learning for Histogram Data

2015 ◽  
Vol 2015 ◽  
pp. 1-12 ◽  
Author(s):  
Wei Yang ◽  
Luhui Xu ◽  
Xiaopan Chen ◽  
Fengbin Zheng ◽  
Yang Liu
Keyword(s):  
Nearest Neighbor ◽  
State Of The Art ◽  
Metric Learning ◽  
Nearest Neighbors ◽  
Distance Metric Learning ◽  
Distance Metric ◽  
Projected Gradient Method ◽  
Proper Distance ◽  
Chi Squared ◽  
Real World Datasets

Learning a proper distance metric for histogram data plays a crucial role in many computer vision tasks. The chi-squared distance is a nonlinear metric and is widely used to compare histograms. In this paper, we show how to learn a general form of chi-squared distance based on the nearest neighbor model. In our method, the margin of sample is first defined with respect to the nearest hits (nearest neighbors from the same class) and the nearest misses (nearest neighbors from the different classes), and then the simplex-preserving linear transformation is trained by maximizing the margin while minimizing the distance between each sample and its nearest hits. With the iterative projected gradient method for optimization, we naturally introduce thel2,1norm regularization into the proposed method for sparse metric learning. Comparative studies with the state-of-the-art approaches on five real-world datasets verify the effectiveness of the proposed method.

scholarly journals Analyze COVID-19 CT images based on evolutionary algorithm with dynamic searching space

2021 ◽  
Author(s):  
Yunhong Gong ◽  
Yanan Sun ◽  
Dezhong Peng ◽  
Peng Chen ◽  
Zhongtai Yan ◽  
...  
Keyword(s):  
Common Sense ◽  
Expert Knowledge ◽  
State Of The Art ◽  
Data Set ◽  
The Past ◽  
Computed Tomography Images ◽  
Intensive Use ◽  
Hospital Resources ◽  
The Common ◽  
Ct Data

AbstractThe COVID-19 pandemic has caused a global alarm. With the advances in artificial intelligence, the COVID-19 testing capabilities have been greatly expanded, and hospital resources are significantly alleviated. Over the past years, computer vision researches have focused on convolutional neural networks (CNNs), which can significantly improve image analysis ability. However, CNN architectures are usually manually designed with rich expertise that is scarce in practice. Evolutionary algorithms (EAs) can automatically search for the proper CNN architectures and voluntarily optimize the related hyperparameters. The networks searched by EAs can be used to effectively process COVID-19 computed tomography images without expert knowledge and manual setup. In this paper, we propose a novel EA-based algorithm with a dynamic searching space to design the optimal CNN architectures for diagnosing COVID-19 before the pathogenic test. The experiments are performed on the COVID-CT data set against a series of state-of-the-art CNN models. The experiments demonstrate that the architecture searched by the proposed EA-based algorithm achieves the best performance yet without any preprocessing operations. Furthermore, we found through experimentation that the intensive use of batch normalization may deteriorate the performance. This contrasts with the common sense approach of manually designing CNN architectures and will help the related experts in handcrafting CNN models to achieve the best performance without any preprocessing operations

scholarly journals Learning Mahalanobis Distance Metric: Considering Instance Disturbance Helps

2017 ◽  
Author(s):  
Han-Jia Ye ◽  
De-Chuan Zhan ◽  
Xue-Min Si ◽  
Yuan Jiang
Keyword(s):  
Mahalanobis Distance ◽  
Side Information ◽  
Metric Learning ◽  
Learning Approach ◽  
Distance Metric Learning ◽  
Distance Metric ◽  
Information Utilization ◽  
Learning Methods ◽  
Feature Weights ◽  
Distance Computation

Mahalanobis distance metric takes feature weights and correlation into account in the distance computation, which can improve the performance of many similarity/dissimilarity based methods, such as kNN. Most existing distance metric learning methods obtain metric based on the raw features and side information but neglect the reliability of them. Noises or disturbances on instances will make changes on their relationships, so as to affect the learned metric.In this paper, we claim that considering disturbance of instances may help the distance metric learning approach get a robust metric, and propose the Distance metRIc learning Facilitated by disTurbances (DRIFT) approach. In DRIFT, the noise or the disturbance of each instance is learned. Therefore, the distance between each pair of (noisy) instances can be better estimated, which facilitates side information utilization and metric learning.Experiments on prediction and visualization clearly indicate the effectiveness of the proposed approach.

scholarly journals Distance metric learning from uncertain side information for automated photo tagging

2011 ◽  
Vol 2 (2) ◽  
pp. 1-28 ◽  
Author(s):  
Lei Wu ◽  
Steven C.H. Hoi ◽  
Rong Jin ◽  
Jianke Zhu ◽  
Nenghai Yu
Keyword(s):  
Side Information ◽  
Metric Learning ◽  
Distance Metric Learning ◽  
Distance Metric

SEMI-SUPERVISED FUZZY CLUSTERING WITH LEARNABLE CLUSTER DEPENDENT KERNELS

2013 ◽  
Vol 22 (03) ◽  
pp. 1350013 ◽  
Author(s):  
OUIEM BCHIR ◽  
HICHEM FRIGUI ◽  
MOHAMED MAHER BEN ISMAIL
Keyword(s):  
Fuzzy Clustering ◽  
Side Information ◽  
Metric Learning ◽  
Real Data ◽  
Distance Functions ◽  
Gaussian Kernel ◽  
Cost Functions ◽  
Data Sets ◽  
Learning Approaches ◽  
Data Set

Many machine learning applications rely on learning distance functions with side information. Most of these distance metric learning approaches learns a Mahalanobis distance. While these approaches may work well when data is in low dimensionality, they become computationally expensive or even infeasible for high dimensional data. In this paper, we propose a novel method of learning nonlinear distance functions with side information while clustering the data. The new semi-supervised clustering approach is called Semi-Supervised Fuzzy clustering with Learnable Cluster dependent Kernels (SS-FLeCK). The proposed algorithm learns the underlying cluster-dependent dissimilarity measure while finding compact clusters in the given data set. The learned dissimilarity is based on a Gaussian kernel function with cluster dependent parameters. This objective function integrates penalty and reward cost functions. These cost functions are weighted by fuzzy membership degrees. Moreover, they use side-information in the form of a small set of constraints on which instances should or should not reside in the same cluster. The proposed algorithm uses only the pairwise relation between the feature vectors. This makes it applicable when similar objects cannot be represented by a single prototype. Using synthetic and real data sets, we show that SS-FLeCK outperforms several other algorithms.

Distance Metric Learning Based on Side Information Autogeneration for Time Series

2014 ◽  
Vol 24 (11) ◽  
pp. 2642-2655
Author(s):  
Peng-Cheng ZOU ◽  
Jian-Dong WANG ◽  
Guo-Qing YANG ◽  
Xia ZHANG ◽  
Li-Na WANG
Keyword(s):  
Time Series ◽  
Side Information ◽  
Metric Learning ◽  
Distance Metric Learning ◽  
Distance Metric

scholarly journals Adversarial Metric Learning

2018 ◽  
Author(s):  
Shuo Chen ◽  
Chen Gong ◽  
Jian Yang ◽  
Xiang Li ◽  
Yang Wei ◽  
...  
Keyword(s):  
State Of The Art ◽  
Metric Learning ◽  
Sampling Bias ◽  
Training Data ◽  
Loss Functions ◽  
Original Training ◽  
Training Set ◽  
Learning Problem ◽  
Optimization Framework ◽  
The Past

In the past decades, intensive efforts have been put to design various loss functions and metric forms for metric learning problem. These improvements have shown promising results when the test data is similar to the training data. However, the trained models often fail to produce reliable distances on the ambiguous test pairs due to the different samplings between training set and test set. To address this problem, the Adversarial Metric Learning (AML) is proposed in this paper, which automatically generates adversarial pairs to remedy the sampling bias and facilitate robust metric learning. Specifically, AML consists of two adversarial stages, i.e. confusion and distinguishment. In confusion stage, the ambiguous but critical adversarial data pairs are adaptively generated to mislead the learned metric. In distinguishment stage, a metric is exhaustively learned to try its best to distinguish both adversarial pairs and original training pairs. Thanks to the challenges posed by the confusion stage in such competing process, the AML model is able to grasp plentiful difficult knowledge that has not been contained by the original training pairs, so the discriminability of AML can be significantly improved. The entire model is formulated into optimization framework, of which the global convergence is theoretically proved. The experimental results on toy data and practical datasets clearly demonstrate the superiority of AML to representative state-of-the-art metric learning models.

Zero-shot Metric Learning

2019 ◽  
Author(s):  
Xinyi Xu ◽  
Huanhuan Cao ◽  
Yanhua Yang ◽  
Erkun Yang ◽  
Cheng Deng
Keyword(s):  
State Of The Art ◽  
Metric Learning ◽  
Visual Similarity ◽  
Distance Metric ◽  
Learning Problem ◽  
Combine Data ◽  
Benchmark Datasets ◽  
Novel Method ◽  
Multiple Relation ◽  
Continuous Relation

In this work, we tackle the zero-shot metric learning problem and propose a novel method abbreviated as ZSML, with the purpose to learn a distance metric that measures the similarity of unseen categories (even unseen datasets). ZSML achieves strong transferability by capturing multi-nonlinear yet continuous relation among data. It is motivated by two facts: 1) relations can be essentially described from various perspectives; and 2) traditional binary supervision is insufficient to represent continuous visual similarity. Specifically, we first reformulate a collection of specific-shaped convolutional kernels to combine data pairs and generate multiple relation vectors. Furthermore, we design a new cross-update regression loss to discover continuous similarity. Extensive experiments including intra-dataset transfer and inter-dataset transfer on four benchmark datasets demonstrate that ZSML can achieve state-of-the-art performance.

scholarly journals Multi-Label Causal Feature Selection

2020 ◽  
Vol 34 (04) ◽  
pp. 6430-6437 ◽  
Author(s):  
Xingyu Wu ◽  
Bingbing Jiang ◽  
Kui Yu ◽  
Huanhuan Chen ◽  
Chunyan Miao
Keyword(s):  
Feature Selection ◽  
State Of The Art ◽  
Causal Mechanism ◽  
Data Sets ◽  
Causal Relationships ◽  
Real World Data ◽  
Data Set ◽  
Complete Representation ◽  
The Past ◽  
Different Types

Multi-label feature selection has received considerable attentions during the past decade. However, existing algorithms do not attempt to uncover the underlying causal mechanism, and individually solve different types of variable relationships, ignoring the mutual effects between them. Furthermore, these algorithms lack of interpretability, which can only select features for all labels, but cannot explain the correlation between a selected feature and a certain label. To address these problems, in this paper, we theoretically study the causal relationships in multi-label data, and propose a novel Markov blanket based multi-label causal feature selection (MB-MCF) algorithm. MB-MCF mines the causal mechanism of labels and features first, to obtain a complete representation of information about labels. Based on the causal relationships, MB-MCF then selects predictive features and simultaneously distinguishes common features shared by multiple labels and label-specific features owned by single labels. Experiments on real-world data sets validate that MB-MCF could automatically determine the number of selected features and simultaneously achieve the best performance compared with state-of-the-art methods. An experiment in Emotions data set further demonstrates the interpretability of MB-MCF.

Sign in / Sign up

Export Citation Format

Share Document