A Graph Embedding Framework for Maximum Mean Discrepancy-Based Domain Adaptation Algorithms

Convolutional neural networks raised the bar for machine learning and artificial intelligence applications, mainly due to the abundance of data and computations. However, there is not always enough data for training, especially when it comes to historical collections of cultural heritage where the original artworks have been destroyed or damaged over time. Transfer Learning and domain adaptation techniques are possible solutions to tackle the issue of data scarcity. This article presents a new method for domain adaptation based on Knowledge graph embeddings. Knowledge Graph embedding forms a projection of a knowledge graph into a lower-dimensional where entities and relations are represented into continuous vector spaces. Our method incorporates these semantic vector spaces as a key ingredient to guide the domain adaptation process. We combined knowledge graph embeddings with visual embeddings from the images and trained a neural network with the combined embeddings as anchors using an extension of Fisher’s linear discriminant. We evaluated our approach on two cultural heritage datasets of images containing medieval and renaissance musical instruments. The experimental results showed a significant increase in the baselines and state-of-the-art performance compared with other domain adaptation methods.

Download Full-text

Domain Adaptation for Intelligent Fault Diagnosis under Different Working Conditions

MATEC Web of Conferences ◽

10.1051/matecconf/202031903001 ◽

2020 ◽

Vol 319 ◽

pp. 03001

Author(s):

Weigui Li ◽

Zhuqing Yuan ◽

Wenyu Sun ◽

Yongpan Liu

Keyword(s):

Fault Diagnosis ◽

Working Conditions ◽

Domain Adaptation ◽

Transfer Problem ◽

Intelligent Manufacturing ◽

Maximum Mean Discrepancy ◽

Bearing Fault Diagnosis ◽

Cross Domain ◽

Conditional Maximum ◽

Adaptation Method

Recently, deep learning algorithms have been widely into fault diagnosis in the intelligent manufacturing field. To tackle the transfer problem due to various working conditions and insufficient labeled samples, a conditional maximum mean discrepancy (CMMD) based domain adaptation method is proposed. Existing transfer approaches mainly focus on aligning the single representation distributions, which only contains partial feature information. Inspired by the Inception module, multi-representation domain adaptation is introduced to improve classification accuracy and generalization ability for cross-domain bearing fault diagnosis. And CMMD-based method is adopted to minimize the discrepancy between the source and the target. Finally, the unsupervised learning method with unlabeled target data can promote the practical application of the proposed algorithm. According to the experimental results on the standard dataset, the proposed method can effectively alleviate the domain shift problem.

Download Full-text

Balanced joint maximum mean discrepancy for deep transfer learning

Analysis and Applications ◽

10.1142/s0219530520400035 ◽

2020 ◽

pp. 1-18

Author(s):

Chuangji Meng ◽

Cunlu Xu ◽

Qin Lei ◽

Wei Su ◽

Jinzhao Wu

Keyword(s):

Network Architecture ◽

Reproducing Kernel ◽

Domain Adaptation ◽

Reproducing Kernel Hilbert Space ◽

Stochastic Gradient Descent ◽

Backpropagation Algorithm ◽

Maximum Mean Discrepancy ◽

Feature Representations ◽

Joint Distributions ◽

Balanced Distribution

Recent studies have revealed that deep networks can learn transferable features that generalize well to novel tasks with little or unavailable labeled data for domain adaptation. However, justifying which components of the feature representations can reason about original joint distributions using JMMD within the regime of deep architecture remains unclear. We present a new backpropagation algorithm for JMMD called the Balanced Joint Maximum Mean Discrepancy (B-JMMD) to further reduce the domain discrepancy. B-JMMD achieves the effect of balanced distribution adaptation for deep network architecture, and can be treated as an improved version of JMMD’s backpropagation algorithm. The proposed method leverages the importance of marginal and conditional distributions behind multiple domain-specific layers across domains adaptively to get a good match for the joint distributions in a second-order reproducing kernel Hilbert space. The learning of the proposed method can be performed technically by a special form of stochastic gradient descent, in which the gradient is computed by backpropagation with a strategy of balanced distribution adaptation. Theoretical analysis shows that the proposed B-JMMD is superior to JMMD method. Experiments confirm that our method yields state-of-the-art results with standard datasets.

Download Full-text

Knowledge Preserving and Distribution Alignment for Heterogeneous Domain Adaptation

ACM Transactions on Information Systems ◽

10.1145/3469856 ◽

2022 ◽

Vol 40 (1) ◽

pp. 1-29

Author(s):

Hanrui Wu ◽

Qingyao Wu ◽

Michael K. Ng

Keyword(s):

Domain Adaptation ◽

Iteration Scheme ◽

Target Space ◽

Target Domain ◽

Maximum Mean Discrepancy ◽

Learning Tasks ◽

Feature Spaces ◽

Latent Space ◽

Laplacian Graph ◽

Novel Model

Domain adaptation aims at improving the performance of learning tasks in a target domain by leveraging the knowledge extracted from a source domain. To this end, one can perform knowledge transfer between these two domains. However, this problem becomes extremely challenging when the data of these two domains are characterized by different types of features, i.e., the feature spaces of the source and target domains are different, which is referred to as heterogeneous domain adaptation (HDA). To solve this problem, we propose a novel model called Knowledge Preserving and Distribution Alignment (KPDA), which learns an augmented target space by jointly minimizing information loss and maximizing domain distribution alignment. Specifically, we seek to discover a latent space, where the knowledge is preserved by exploiting the Laplacian graph terms and reconstruction regularizations. Moreover, we adopt the Maximum Mean Discrepancy to align the distributions of the source and target domains in the latent space. Mathematically, KPDA is formulated as a minimization problem with orthogonal constraints, which involves two projection variables. Then, we develop an algorithm based on the Gauss–Seidel iteration scheme and split the problem into two subproblems, which are solved by searching algorithms based on the Barzilai–Borwein (BB) stepsize. Promising results demonstrate the effectiveness of the proposed method.

Download Full-text