scholarly journals Visual Domain Adaptation by Consensus-Based Transfer to Intermediate Domain

2020 ◽  
Vol 34 (07) ◽  
pp. 10655-10662
Author(s):  
Jongwon Choi ◽  
Youngjoon Choi ◽  
Jihoon Kim ◽  
Jinyeop Chang ◽  
Ilhwan Kwon ◽  
...  
Keyword(s):  
Real World ◽  
Domain Adaptation ◽  
Ensemble Classifiers ◽  
Training Algorithm ◽  
Target Domain ◽  
Intermediate Domain ◽  
Previous State ◽  
Real World Datasets ◽  
Visual Domain ◽  
Adaptation Scenarios

We describe an unsupervised domain adaptation framework for images by a transform to an abstract intermediate domain and ensemble classifiers seeking a consensus. The intermediate domain can be thought as a latent domain where both the source and target domains can be transferred easily. The proposed framework aligns both domains to the intermediate domain, which greatly improves the adaptation performance when the source and target domains are notably dissimilar. In addition, we propose an ensemble model trained by confusing multiple classifiers and letting them make a consensus alternately to enhance the adaptation performance for ambiguous samples. To estimate the hidden intermediate domain and the unknown labels of the target domain simultaneously, we develop a training algorithm using a double-structured architecture. We validate the proposed framework in hard adaptation scenarios with real-world datasets from simple synthetic domains to complex real-world domains. The proposed algorithm outperforms the previous state-of-the-art algorithms on various environments.

scholarly journals Unsupervised multi-source domain adaptation with no observable source data

PLoS ONE ◽  
2021 ◽  
Vol 16 (7) ◽  
pp. e0253415
Author(s):  
Hyunsik Jeon ◽  
Seongmin Lee ◽  
U Kang
Keyword(s):  
Real World ◽  
Domain Adaptation ◽  
State Of The Art ◽  
Multiple Source ◽  
Multiple Sources ◽  
Target Domain ◽  
Source Domain ◽  
Source Data ◽  
Real World Datasets ◽  
Target Data

Given trained models from multiple source domains, how can we predict the labels of unlabeled data in a target domain? Unsupervised multi-source domain adaptation (UMDA) aims for predicting the labels of unlabeled target data by transferring the knowledge of multiple source domains. UMDA is a crucial problem in many real-world scenarios where no labeled target data are available. Previous approaches in UMDA assume that data are observable over all domains. However, source data are not easily accessible due to privacy or confidentiality issues in a lot of practical scenarios, although classifiers learned in source domains are readily available. In this work, we target data-free UMDA where source data are not observable at all, a novel problem that has not been studied before despite being very realistic and crucial. To solve data-free UMDA, we propose DEMS (Data-free Exploitation of Multiple Sources), a novel architecture that adapts target data to source domains without exploiting any source data, and estimates the target labels by exploiting pre-trained source classifiers. Extensive experiments for data-free UMDA on real-world datasets show that DEMS provides the state-of-the-art accuracy which is up to 27.5% point higher than that of the best baseline.

scholarly journals A Subspace Based Transfer Joint Matching with Laplacian Regularization for Visual Domain Adaptation

Sensors ◽  
2020 ◽  
Vol 20 (16) ◽  
pp. 4367
Author(s):  
Rakesh Kumar Sanodiya ◽  
Leehter Yao
Keyword(s):  
Real World ◽  
Nearest Neighbor ◽  
Domain Adaptation ◽  
Clustering Algorithms ◽  
K Nearest Neighbor ◽  
Target Domain ◽  
Illumination Variation ◽  
Machine Learning Classification ◽  
Visual Domain ◽  
Laplacian Regularization

In a real-world application, the images taken by different cameras with different conditions often incur illumination variation, low-resolution, different poses, blur, etc., which leads to a large distribution difference or gap between training (source) and test (target) images. This distribution gap is challenging for many primitive machine learning classification and clustering algorithms such as k-Nearest Neighbor (k-NN) and k-means. In order to minimize this distribution gap, we propose a novel Subspace based Transfer Joint Matching with Laplacian Regularization (STJML) method for visual domain adaptation by jointly matching the features and re-weighting the instances across different domains. Specifically, the proposed STJML-based method includes four key components: (1) considering subspaces of both domains; (2) instance re-weighting; (3) it simultaneously reduces the domain shift in both marginal distribution and conditional distribution between the source domain and the target domain; (4) preserving the original similarity of data points by using Laplacian regularization. Experiments on three popular real-world domain adaptation problem datasets demonstrate a significant performance improvement of our proposed method over published state-of-the-art primitive and domain adaptation methods.

scholarly journals Fuzzy Overclustering: Semi-Supervised Classification of Fuzzy Labels with Overclustering and Inverse Cross-Entropy

Sensors ◽  
2021 ◽  
Vol 21 (19) ◽  
pp. 6661
Author(s):  
Lars Schmarje ◽  
Johannes Brünger ◽  
Monty Santarossa ◽  
Simon-Martin Schröder ◽  
Rainer Kiko ◽  
...  
Keyword(s):  
Deep Learning ◽  
Real World ◽  
Supervised Classification ◽  
Limited Information ◽  
Classification Problems ◽  
Previous State ◽  
Supervised Methods ◽  
Real World Datasets ◽  
Fuzzy Labels

Deep learning has been successfully applied to many classification problems including underwater challenges. However, a long-standing issue with deep learning is the need for large and consistently labeled datasets. Although current approaches in semi-supervised learning can decrease the required amount of annotated data by a factor of 10 or even more, this line of research still uses distinct classes. For underwater classification, and uncurated real-world datasets in general, clean class boundaries can often not be given due to a limited information content in the images and transitional stages of the depicted objects. This leads to different experts having different opinions and thus producing fuzzy labels which could also be considered ambiguous or divergent. We propose a novel framework for handling semi-supervised classifications of such fuzzy labels. It is based on the idea of overclustering to detect substructures in these fuzzy labels. We propose a novel loss to improve the overclustering capability of our framework and show the benefit of overclustering for fuzzy labels. We show that our framework is superior to previous state-of-the-art semi-supervised methods when applied to real-world plankton data with fuzzy labels. Moreover, we acquire 5 to 10% more consistent predictions of substructures.

scholarly journals Fuzzy Graph Learning Regularized Sparse Filtering for Visual Domain Adaptation

2021 ◽  
Vol 11 (10) ◽  
pp. 4503
Author(s):  
Lingtong Min ◽  
Deyun Zhou ◽  
Xiaoyang Li ◽  
Qinyi Lv ◽  
Yuanjie Zhi
Keyword(s):  
Domain Adaptation ◽  
Weather Conditions ◽  
Optimization Method ◽  
Superior Performance ◽  
Target Domain ◽  
Graph Learning ◽  
Learning Framework ◽  
Cross Domain ◽  
Sparse Filtering ◽  
Visual Domain

Distribution mismatch can be easily found in multi-sensor systems, which may be caused by different shoot angles, weather conditions and so on. Domain adaptation aims to build robust classifiers using the knowledge from a well-labeled source domain, while applied on a related but different target domain. Pseudo labeling is a prevalent technique for class-wise distribution alignment. Therefore, numerous efforts have been spent on alleviating the issue of mislabeling. In this paper, unlike existing selective hard labeling works, we propose a fuzzy labeling based graph learning framework for matching conditional distribution. Specifically, we construct the cross-domain affinity graph by considering the fuzzy label matrix of target samples. In order to solve the problem of representation shrinkage, the paradigm of sparse filtering is introduced. Finally, a unified optimization method based on gradient descent is proposed. Extensive experiments show that our method achieves comparable or superior performance when compared to state-of-the-art works.

scholarly journals Differentially Private Optimal Transport: Application to Domain Adaptation

2019 ◽  
Author(s):  
Nam LeTien ◽  
Amaury Habrard ◽  
Marc Sebban
Keyword(s):  
Optimal Transport ◽  
Transport Model ◽  
Domain Adaptation ◽  
Wasserstein Distance ◽  
Source Distribution ◽  
Target Domain ◽  
The Past ◽  
Series Of Experiments ◽  
Adaptation Scenarios ◽  
Transportation Plan

Optimal transport has received much attention during the past few years to deal with domain adaptation tasks. The goal is to transfer knowledge from a source domain to a target domain by finding a transportation of minimal cost moving the source distribution to the target one. In this paper, we address the challenging task of privacy preserving domain adaptation by optimal transport. Using the Johnson-Lindenstrauss transform together with some noise, we present the first differentially private optimal transport model and show how it can be directly applied on both unsupervised and semi-supervised domain adaptation scenarios. Our theoretically grounded method allows the optimization of the transportation plan and the Wasserstein distance between the two distributions while protecting the data of both domains.We perform an extensive series of experiments on various benchmarks (VisDA, Office-Home and Office-Caltech datasets) that demonstrates the efficiency of our method compared to non-private strategies.

scholarly journals MotionTransformer: Transferring Neural Inertial Tracking between Domains

2019 ◽  
Vol 33 ◽  
pp. 8009-8016 ◽  
Author(s):  
Changhao Chen ◽  
Yishu Miao ◽  
Chris Xiaoxuan Lu ◽  
Linhai Xie ◽  
Phil Blunsom ◽  
...  
Keyword(s):  
Information Processing ◽  
Mobile Phone ◽  
Real World ◽  
Mobile Agents ◽  
Domain Adaptation ◽  
Target Domain ◽  
Paired Data ◽  
Source Domain ◽  
Invariant Features ◽  
Inertial Measurements

Inertial information processing plays a pivotal role in egomotion awareness for mobile agents, as inertial measurements are entirely egocentric and not environment dependent. However, they are affected greatly by changes in sensor placement/orientation or motion dynamics, and it is infeasible to collect labelled data from every domain. To overcome the challenges of domain adaptation on long sensory sequences, we propose MotionTransformer - a novel framework that extracts domain-invariant features of raw sequences from arbitrary domains, and transforms to new domains without any paired data. Through the experiments, we demonstrate that it is able to efficiently and effectively convert the raw sequence from a new unlabelled target domain into an accurate inertial trajectory, benefiting from the motion knowledge transferred from the labelled source domain. We also conduct real-world experiments to show our framework can reconstruct physically meaningful trajectories from raw IMU measurements obtained with a standard mobile phone in various attachments.

scholarly journals Semi-Supervised Domain Adaptation for Holistic Counting under Label Gap

2021 ◽  
Vol 7 (10) ◽  
pp. 198
Author(s):  
Mattia Litrico ◽  
Sebastiano Battiato ◽  
Sotirios A. Tsaftaris ◽  
Mario Valerio Giuffrida
Keyword(s):  
Mean Squared Error ◽  
Domain Adaptation ◽  
Input Image ◽  
Cell Counting ◽  
Global Optimality ◽  
Target Domain ◽  
Generative Adversarial Network ◽  
Specific Domain ◽  
Counting Problem ◽  
Real World Datasets

This paper proposes a novel approach for semi-supervised domain adaptation for holistic regression tasks, where a DNN predicts a continuous value y∈R given an input image x. The current literature generally lacks specific domain adaptation approaches for this task, as most of them mostly focus on classification. In the context of holistic regression, most of the real-world datasets not only exhibit a covariate (or domain) shift, but also a label gap—the target dataset may contain labels not included in the source dataset (and vice versa). We propose an approach tackling both covariate and label gap in a unified training framework. Specifically, a Generative Adversarial Network (GAN) is used to reduce covariate shift, and label gap is mitigated via label normalisation. To avoid overfitting, we propose a stopping criterion that simultaneously takes advantage of the Maximum Mean Discrepancy and the GAN Global Optimality condition. To restore the original label range—that was previously normalised—a handful of annotated images from the target domain are used. Our experimental results, run on 3 different datasets, demonstrate that our approach drastically outperforms the state-of-the-art across the board. Specifically, for the cell counting problem, the mean squared error (MSE) is reduced from 759 to 5.62; in the case of the pedestrian dataset, our approach lowered the MSE from 131 to 1.47. For the last experimental setup, we borrowed a task from plant biology, i.e., counting the number of leaves in a plant, and we ran two series of experiments, showing the MSE is reduced from 2.36 to 0.88 (intra-species), and from 1.48 to 0.6 (inter-species).

scholarly journals Adversarial Spectral Kernel Matching for Unsupervised Time Series Domain Adaptation

2021 ◽  
Author(s):  
Qiao Liu ◽  
Hui Xue
Keyword(s):  
Time Series ◽  
Real World ◽  
Domain Adaptation ◽  
Kernel Learning ◽  
Target Domain ◽  
Maximum Mean Discrepancy ◽  
Local Statistics ◽  
Unsupervised Domain Adaptation ◽  
Non Local ◽  
Spectral Kernel

Unsupervised domain adaptation (UDA) has been received increasing attention since it does not require labels in target domain. Most existing UDA methods learn domain-invariant features by minimizing discrepancy distance computed by a certain metric between domains. However, these discrepancy-based methods cannot be robustly applied to unsupervised time series domain adaptation (UTSDA). That is because discrepancy metrics in these methods contain only low-order and local statistics, which have limited expression for time series distributions and therefore result in failure of domain matching. Actually, the real-world time series are always non-local distributions, i.e., with non-stationary and non-monotonic statistics. In this paper, we propose an Adversarial Spectral Kernel Matching (AdvSKM) method, where a hybrid spectral kernel network is specifically designed as inner kernel to reform the Maximum Mean Discrepancy (MMD) metric for UTSDA. The hybrid spectral kernel network can precisely characterize non-stationary and non-monotonic statistics in time series distributions. Embedding hybrid spectral kernel network to MMD not only guarantees precise discrepancy metric but also benefits domain matching. Besides, the differentiable architecture of the spectral kernel network enables adversarial kernel learning, which brings more discriminatory expression for discrepancy matching. The results of extensive experiments on several real-world UTSDA tasks verify the effectiveness of our proposed method.

scholarly journals Unsupervised Domain Adaptation by Matching Distributions Based on the Maximum Mean Discrepancy via Unilateral Transformations

2019 ◽  
Vol 33 ◽  
pp. 4106-4113 ◽  
Author(s):  
Atsutoshi Kumagai ◽  
Tomoharu Iwata
Keyword(s):  
Domain Adaptation ◽  
Feature Space ◽  
Classification Performance ◽  
Target Domain ◽  
Target Feature ◽  
Maximum Mean Discrepancy ◽  
Source Domain ◽  
Unsupervised Domain Adaptation ◽  
Real World Datasets ◽  
Target Data

We propose a simple yet effective method for unsupervised domain adaptation. When training and test distributions are different, standard supervised learning methods perform poorly. Semi-supervised domain adaptation methods have been developed for the case where labeled data in the target domain are available. However, the target data are often unlabeled in practice. Therefore, unsupervised domain adaptation, which does not require labels for target data, is receiving a lot of attention. The proposed method minimizes the discrepancy between the source and target distributions of input features by transforming the feature space of the source domain. Since such unilateral transformations transfer knowledge in the source domain to the target one without reducing dimensionality, the proposed method can effectively perform domain adaptation without losing information to be transfered. With the proposed method, it is assumed that the transformed features and the original features differ by a small residual to preserve the relationship between features and labels. This transformation is learned by aligning the higher-order moments of the source and target feature distributions based on the maximum mean discrepancy, which enables to compare two distributions without density estimation. Once the transformation is found, we learn supervised models by using the transformed source data and their labels. We use two real-world datasets to demonstrate experimentally that the proposed method achieves better classification performance than existing methods for unsupervised domain adaptation.

Sign in / Sign up

Export Citation Format

Share Document