Cross-Domain Metric and Multiple Kernel Learning Based on Information Theory

2018 ◽  
Vol 30 (3) ◽  
pp. 820-855 ◽  
Author(s):  
Wei Wang ◽  
Hao Wang ◽  
Chen Zhang ◽  
Yang Gao
Keyword(s):  
Domain Adaptation ◽  
Convex Combination ◽  
Learning Algorithms ◽  
Metric Learning ◽  
Multiple Kernel Learning ◽  
Kernel Learning ◽  
Target Domain ◽  
Source Domain ◽  
Cross Domain ◽  
Multiple Kernel

Learning an appropriate distance metric plays a substantial role in the success of many learning machines. Conventional metric learning algorithms have limited utility when the training and test samples are drawn from related but different domains (i.e., source domain and target domain). In this letter, we propose two novel metric learning algorithms for domain adaptation in an information-theoretic setting, allowing for discriminating power transfer and standard learning machine propagation across two domains. In the first one, a cross-domain Mahalanobis distance is learned by combining three goals: reducing the distribution difference between different domains, preserving the geometry of target domain data, and aligning the geometry of source domain data with label information. Furthermore, we devote our efforts to solving complex domain adaptation problems and go beyond linear cross-domain metric learning by extending the first method to a multiple kernel learning framework. A convex combination of multiple kernels and a linear transformation are adaptively learned in a single optimization, which greatly benefits the exploration of prior knowledge and the description of data characteristics. Comprehensive experiments in three real-world applications (face recognition, text classification, and object categorization) verify that the proposed methods outperform state-of-the-art metric learning and domain adaptation methods.

scholarly journals Optimal Transport with Dimensionality Reduction for Domain Adaptation

Symmetry ◽  
2020 ◽  
Vol 12 (12) ◽  
pp. 1994
Author(s):  
Ping Li ◽  
Zhiwei Ni ◽  
Xuhui Zhu ◽  
Juan Song ◽  
Wenying Wu
Keyword(s):  
Dimensionality Reduction ◽  
Optimal Transport ◽  
Domain Adaptation ◽  
Wasserstein Distance ◽  
Local Information ◽  
Target Domain ◽  
Source Domain ◽  
Second Stage ◽  
Cross Domain ◽  
Feature Based

Domain adaptation manages to learn a robust classifier for target domain, using the source domain, but they often follow different distributions. To bridge distribution shift between the two domains, most of previous works aim to align their feature distributions through feature transformation, of which optimal transport for domain adaptation has attract researchers’ interest, as it can exploit the local information of the two domains in the process of mapping the source instances to the target ones by minimizing Wasserstein distance between their feature distributions. However, it may weaken the feature discriminability of source domain, thus degrade domain adaptation performance. To address this problem, this paper proposes a two-stage feature-based adaptation approach, referred to as optimal transport with dimensionality reduction (OTDR). In the first stage, we apply the dimensionality reduction with intradomain variant maximization but source intraclass compactness minimization, to separate data samples as much as possible and enhance the feature discriminability of the source domain. In the second stage, we leverage optimal transport-based technique to preserve the local information of the two domains. Notably, the desirable properties in the first stage can mitigate the degradation of feature discriminability of the source domain in the second stage. Extensive experiments on several cross-domain image datasets validate that OTDR is superior to its competitors in classification accuracy.

scholarly journals Distributional Correspondence Indexing for Cross-Lingual and Cross-Domain Sentiment Classification (Extended Abstract)

2018 ◽  
Author(s):  
Alejandro Moreo Fernández ◽  
Andrea Esuli ◽  
Fabrizio Sebastiani
Keyword(s):  
Domain Adaptation ◽  
State Of The Art ◽  
Sentiment Classification ◽  
Training Data ◽  
Target Domain ◽  
Source Domain ◽  
Machine Learning Methods ◽  
Cross Domain ◽  
Current State ◽  
Cross Lingual

Domain Adaptation (DA) techniques aim at enabling machine learning methods learn effective classifiers for a “target” domain when the only available training data belongs to a different “source” domain. In this extended abstract, we briefly describe our new DA method called Distributional Correspondence Indexing (DCI) for sentiment classification. DCI derives term representations in a vector space common to both domains where each dimension reflects its distributional correspondence to a pivot, i.e., to a highly predictive term that behaves similarly across domains. The experiments we have conducted show that DCI obtains better performance than current state-of-the-art techniques for cross-lingual and cross-domain sentiment classification.

scholarly journals Few shot domain adaptation for in situ macromolecule structural classification in cryoelectron tomograms

2020 ◽  
Author(s):  
Liangyong Yu ◽  
Ran Li ◽  
Xiangrui Zeng ◽  
Hongyi Wang ◽  
Jie Jin ◽  
...  
Keyword(s):  
Deep Learning ◽  
Large Scale ◽  
Spatial Organization ◽  
Domain Adaptation ◽  
Single Cells ◽  
Supplementary Information ◽  
Target Domain ◽  
Source Domain ◽  
Cellular Processes ◽  
Cross Domain

Abstract Motivation Cryoelectron tomography (cryo-ET) visualizes structure and spatial organization of macromolecules and their interactions with other subcellular components inside single cells in the close-to-native state at submolecular resolution. Such information is critical for the accurate understanding of cellular processes. However, subtomogram classification remains one of the major challenges for the systematic recognition and recovery of the macromolecule structures in cryo-ET because of imaging limits and data quantity. Recently, deep learning has significantly improved the throughput and accuracy of large-scale subtomogram classification. However, often it is difficult to get enough high-quality annotated subtomogram data for supervised training due to the enormous expense of labeling. To tackle this problem, it is beneficial to utilize another already annotated dataset to assist the training process. However, due to the discrepancy of image intensity distribution between source domain and target domain, the model trained on subtomograms in source domain may perform poorly in predicting subtomogram classes in the target domain. Results In this article, we adapt a few shot domain adaptation method for deep learning-based cross-domain subtomogram classification. The essential idea of our method consists of two parts: (i) take full advantage of the distribution of plentiful unlabeled target domain data, and (ii) exploit the correlation between the whole source domain dataset and few labeled target domain data. Experiments conducted on simulated and real datasets show that our method achieves significant improvement on cross domain subtomogram classification compared with baseline methods. Availability and implementation Software is available online https://github.com/xulabs/aitom. Supplementary information Supplementary data are available at Bioinformatics online.

scholarly journals Semi Supervised Multiple Kernel Learning using Distance Metric Learning Techniques

2017 ◽  
Vol 14 (1) ◽  
pp. 53-70
Author(s):  
Tahereh Zare Bidoki ◽  
Mohammad Taghi Sadeghi ◽  
Hamid Reza Abutalebi ◽  
◽  
◽  
...  
Keyword(s):  
Metric Learning ◽  
Multiple Kernel Learning ◽  
Kernel Learning ◽  
Distance Metric Learning ◽  
Distance Metric ◽  
Multiple Kernel ◽  
Learning Techniques

scholarly journals Absent Multiple Kernel Learning Algorithms

2020 ◽  
Vol 42 (6) ◽  
pp. 1303-1316 ◽  
Author(s):  
Xinwang Liu ◽  
Lei Wang ◽  
Xinzhong Zhu ◽  
Miaomiao Li ◽  
En Zhu ◽  
...  
Keyword(s):  
Learning Algorithms ◽  
Multiple Kernel Learning ◽  
Kernel Learning ◽  
Multiple Kernel

Multiple Kernel Learning with Gaussianity Measures

2012 ◽  
Vol 24 (7) ◽  
pp. 1853-1881 ◽  
Author(s):  
Hideitsu Hino ◽  
Nima Reyhani ◽  
Noboru Murata
Keyword(s):  
Kernel Methods ◽  
Convex Combination ◽  
Multiple Kernel Learning ◽  
Covariance Structure ◽  
Feature Space ◽  
Kernel Functions ◽  
Kernel Learning ◽  
Empirical Characteristic Function ◽  
Fisher Discriminant Analysis ◽  
Multiple Kernel

Kernel methods are known to be effective for nonlinear multivariate analysis. One of the main issues in the practical use of kernel methods is the selection of kernel. There have been a lot of studies on kernel selection and kernel learning. Multiple kernel learning (MKL) is one of the promising kernel optimization approaches. Kernel methods are applied to various classifiers including Fisher discriminant analysis (FDA). FDA gives the Bayes optimal classification axis if the data distribution of each class in the feature space is a gaussian with a shared covariance structure. Based on this fact, an MKL framework based on the notion of gaussianity is proposed. As a concrete implementation, an empirical characteristic function is adopted to measure gaussianity in the feature space associated with a convex combination of kernel functions, and two MKL algorithms are derived. From experimental results on some data sets, we show that the proposed kernel learning followed by FDA offers strong classification power.

Sign in / Sign up

Export Citation Format

Share Document