scholarly journals Domain Adaptation for Syntactic and Semantic Dependency Parsing Using Deep Belief Networks

2015 ◽  
Vol 3 ◽  
pp. 271-282 ◽  
Author(s):  
Haitong Yang ◽  
Tao Zhuang ◽  
Chengqing Zong
Keyword(s):  
Test Data ◽  
Domain Adaptation ◽  
Feature Space ◽  
Feature Representation ◽  
Dependency Parsing ◽  
Shared Task ◽  
Target Domain ◽  
Current Systems ◽  
Semantic Dependency ◽  
Very High

In current systems for syntactic and semantic dependency parsing, people usually define a very high-dimensional feature space to achieve good performance. But these systems often suffer severe performance drops on out-of-domain test data due to the diversity of features of different domains. This paper focuses on how to relieve this domain adaptation problem with the help of unlabeled target domain data. We propose a deep learning method to adapt both syntactic and semantic parsers. With additional unlabeled target domain data, our method can learn a latent feature representation (LFR) that is beneficial to both domains. Experiments on English data in the CoNLL 2009 shared task show that our method largely reduced the performance drop on out-of-domain test data. Moreover, we get a Macro F1 score that is 2.32 points higher than the best system in the CoNLL 2009 shared task in out-of-domain tests.

scholarly journals Joint Domain Alignment and Discriminative Feature Learning for Unsupervised Deep Domain Adaptation

2019 ◽  
Vol 33 ◽  
pp. 3296-3303 ◽  
Author(s):  
Chao Chen ◽  
Zhihong Chen ◽  
Boyuan Jiang ◽  
Xinyu Jin
Keyword(s):  
Domain Adaptation ◽  
Feature Learning ◽  
Feature Space ◽  
Feature Representation ◽  
Learning Method ◽  
Target Domain ◽  
Shared Feature ◽  
Class Separability ◽  
Discriminative Feature ◽  
Domain Alignment

Recently, considerable effort has been devoted to deep domain adaptation in computer vision and machine learning communities. However, most of existing work only concentrates on learning shared feature representation by minimizing the distribution discrepancy across different domains. Due to the fact that all the domain alignment approaches can only reduce, but not remove the domain shift, target domain samples distributed near the edge of the clusters, or far from their corresponding class centers are easily to be misclassified by the hyperplane learned from the source domain. To alleviate this issue, we propose to joint domain alignment and discriminative feature learning, which could benefit both domain alignment and final classification. Specifically, an instance-based discriminative feature learning method and a center-based discriminative feature learning method are proposed, both of which guarantee the domain invariant features with better intra-class compactness and inter-class separability. Extensive experiments show that learning the discriminative features in the shared feature space can significantly boost the performance of deep domain adaptation methods.

scholarly journals Joint Partial Optimal Transport for Open Set Domain Adaptation

2020 ◽  
Author(s):  
Renjun Xu ◽  
Pelen Liu ◽  
Yin Zhang ◽  
Fang Cai ◽  
Jindong Wang ◽  
...  
Keyword(s):  
Optimal Transport ◽  
Transport Model ◽  
Domain Adaptation ◽  
General Setting ◽  
Feature Space ◽  
Set Domain ◽  
Target Domain ◽  
Source Domain ◽  
Open Set ◽  
Proposed Model

Domain adaptation (DA) has achieved a resounding success to learn a good classifier by leveraging labeled data from a source domain to adapt to an unlabeled target domain. However, in a general setting when the target domain contains classes that are never observed in the source domain, namely in Open Set Domain Adaptation (OSDA), existing DA methods failed to work because of the interference of the extra unknown classes. This is a much more challenging problem, since it can easily result in negative transfer due to the mismatch between the unknown and known classes. Existing researches are susceptible to misclassification when target domain unknown samples in the feature space distributed near the decision boundary learned from the labeled source domain. To overcome this, we propose Joint Partial Optimal Transport (JPOT), fully utilizing information of not only the labeled source domain but also the discriminative representation of unknown class in the target domain. The proposed joint discriminative prototypical compactness loss can not only achieve intra-class compactness and inter-class separability, but also estimate the mean and variance of the unknown class through backpropagation, which remains intractable for previous methods due to the blindness about the structure of the unknown classes. To our best knowledge, this is the first optimal transport model for OSDA. Extensive experiments demonstrate that our proposed model can significantly boost the performance of open set domain adaptation on standard DA datasets.

scholarly journals Multi-Source Distilling Domain Adaptation

2020 ◽  
Vol 34 (07) ◽  
pp. 12975-12983
Author(s):  
Sicheng Zhao ◽  
Guangzhi Wang ◽  
Shanghang Zhang ◽  
Yang Gu ◽  
Yaxian Li ◽  
...  
Keyword(s):  
Domain Adaptation ◽  
Feature Space ◽  
Wasserstein Distance ◽  
Training Data ◽  
Source Distribution ◽  
Single Source ◽  
Multiple Sources ◽  
Target Domain ◽  
Target Feature ◽  
Training Samples

Deep neural networks suffer from performance decay when there is domain shift between the labeled source domain and unlabeled target domain, which motivates the research on domain adaptation (DA). Conventional DA methods usually assume that the labeled data is sampled from a single source distribution. However, in practice, labeled data may be collected from multiple sources, while naive application of the single-source DA algorithms may lead to suboptimal solutions. In this paper, we propose a novel multi-source distilling domain adaptation (MDDA) network, which not only considers the different distances among multiple sources and the target, but also investigates the different similarities of the source samples to the target ones. Specifically, the proposed MDDA includes four stages: (1) pre-train the source classifiers separately using the training data from each source; (2) adversarially map the target into the feature space of each source respectively by minimizing the empirical Wasserstein distance between source and target; (3) select the source training samples that are closer to the target to fine-tune the source classifiers; and (4) classify each encoded target feature by corresponding source classifier, and aggregate different predictions using respective domain weight, which corresponds to the discrepancy between each source and target. Extensive experiments are conducted on public DA benchmarks, and the results demonstrate that the proposed MDDA significantly outperforms the state-of-the-art approaches. Our source code is released at: https://github.com/daoyuan98/MDDA.

scholarly journals ADVERSARIAL DOMAIN ADAPTATION FOR THE CLASSIFICATION OF AERIAL IMAGES AND HEIGHT DATA USING CONVOLUTIONAL NEURAL NETWORKS

2019 ◽  
Vol IV-2/W7 ◽  
pp. 197-204 ◽  
Author(s):  
D. Wittich ◽  
F. Rottensteiner
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Domain Adaptation ◽  
Feature Representation ◽  
Training Data ◽  
Aerial Images ◽  
Remotely Sensed Data ◽  
Target Domain ◽  
Deep Convolutional Neural Networks

<p><strong>Abstract.</strong> Domain adaptation (DA) can drastically decrease the amount of training data needed to obtain good classification models by leveraging available data from a source domain for the classification of a new (target) domains. In this paper, we address deep DA, i.e. DA with deep convolutional neural networks (CNN), a problem that has not been addressed frequently in remote sensing. We present a new method for semi-supervised DA for the task of pixel-based classification by a CNN. After proposing an encoder-decoder-based fully convolutional neural network (FCN), we adapt a method for adversarial discriminative DA to be applicable to the pixel-based classification of remotely sensed data based on this network. It tries to learn a feature representation that is domain invariant; domain-invariance is measured by a classifier’s incapability of predicting from which domain a sample was generated. We evaluate our FCN on the ISPRS labelling challenge, showing that it is close to the best-performing models. DA is evaluated on the basis of three domains. We compare different network configurations and perform the representation transfer at different layers of the network. We show that when using a proper layer for adaptation, our method achieves a positive transfer and thus an improved classification accuracy in the target domain for all evaluated combinations of source and target domains.</p>

scholarly journals Unsupervised Domain Adaptation via Structured Prediction Based Selective Pseudo-Labeling

2020 ◽  
Vol 34 (04) ◽  
pp. 6243-6250 ◽  
Author(s):  
Qian Wang ◽  
Toby Breckon
Keyword(s):  
Domain Adaptation ◽  
Feature Space ◽  
Structured Prediction ◽  
Target Domain ◽  
Source Domain ◽  
Domain Specific ◽  
Unsupervised Domain Adaptation ◽  
Deep Feature ◽  
Significant Performance ◽  
Error Accumulation

Unsupervised domain adaptation aims to address the problem of classifying unlabeled samples from the target domain whilst labeled samples are only available from the source domain and the data distributions are different in these two domains. As a result, classifiers trained from labeled samples in the source domain suffer from significant performance drop when directly applied to the samples from the target domain. To address this issue, different approaches have been proposed to learn domain-invariant features or domain-specific classifiers. In either case, the lack of labeled samples in the target domain can be an issue which is usually overcome by pseudo-labeling. Inaccurate pseudo-labeling, however, could result in catastrophic error accumulation during learning. In this paper, we propose a novel selective pseudo-labeling strategy based on structured prediction. The idea of structured prediction is inspired by the fact that samples in the target domain are well clustered within the deep feature space so that unsupervised clustering analysis can be used to facilitate accurate pseudo-labeling. Experimental results on four datasets (i.e. Office-Caltech, Office31, ImageCLEF-DA and Office-Home) validate our approach outperforms contemporary state-of-the-art methods.

FUZZY BRIDGED REFINEMENT DOMAIN ADAPTATION: LONG-TERM BANK FAILURE PREDICTION

2013 ◽  
Vol 12 (01) ◽  
pp. 1350003 ◽  
Author(s):  
VAHID BEHBOOD ◽  
JIE LU ◽  
GUANGQUAN ZHANG
Keyword(s):  
Test Data ◽  
Prediction Models ◽  
Predictive Accuracy ◽  
Domain Adaptation ◽  
Failure Prediction ◽  
Bank Failure ◽  
Training Data ◽  
Support Vector ◽  
Target Domain

Machine learning methods, such as neural network (NN) and support vector machine, assume that the training data and the test data are drawn from the same distribution. This assumption may not be satisfied in many real world applications, like long-term financial failure prediction, because the training and test data may each come from different time periods or domains. This paper proposes a novel algorithm known as fuzzy bridged refinement-based domain adaptation to solve the problem of long-term prediction. The algorithm utilizes the fuzzy system and similarity concepts to modify the target instances' labels which were initially predicted by a shift-unaware prediction model. The experiments are performed using three shift-unaware prediction models based on nine different settings including two main situations: (1) there is no labeled instance in the target domain; (2) there are a few labeled instances in the target domain. In these experiments bank failure financial data is used to validate the algorithm. The results demonstrate a significant improvement in the predictive accuracy, particularly in the second situation identified above.

scholarly journals Cross-Domain Visual Representations via Unsupervised Graph Alignment

2019 ◽  
Vol 33 ◽  
pp. 5613-5620 ◽  
Author(s):  
Baoyao Yang ◽  
Pong C. Yuen
Keyword(s):  
Domain Adaptation ◽  
Visual Representations ◽  
Feature Space ◽  
Source Model ◽  
Target Domain ◽  
Graph Alignment ◽  
Adversarial Network ◽  
Cross Domain ◽  
Data Representations ◽  
The Difference

In unsupervised domain adaptation, distributions of visual representations are mismatched across domains, which leads to the performance drop of a source model in the target domain. Therefore, distribution alignment methods have been proposed to explore cross-domain visual representations. However, most alignment methods have not considered the difference in distribution structures across domains, and the adaptation would subject to the insufficient aligned cross-domain representations. To avoid the misclassification/misidentification due to the difference in distribution structures, this paper proposes a novel unsupervised graph alignment method that aligns both data representations and distribution structures across the source and target domains. An adversarial network is developed for unsupervised graph alignment, which maps both source and target data to a feature space where data are distributed with unified structure criteria. Experimental results show that the graph-aligned visual representations achieve good performance on both crossdataset recognition and cross-modal re-identification.

scholarly journals Wasserstein Distance Learns Domain Invariant Feature Representations for Drift Compensation of E-Nose

Sensors ◽  
2019 ◽  
Vol 19 (17) ◽  
pp. 3703 ◽  
Author(s):  
Yang Tao ◽  
Chunyan Li ◽  
Zhifang Liang ◽  
Haocheng Yang ◽  
Juan Xu
Keyword(s):  
Feature Space ◽  
Representation Learning ◽  
Wasserstein Distance ◽  
Recognition Algorithm ◽  
Feature Representation ◽  
Target Domain ◽  
Feature Representations ◽  
Invariant Feature ◽  
Sensor Drift ◽  
Drift Compensation

Electronic nose (E-nose), a kind of instrument which combines with the gas sensor and the corresponding pattern recognition algorithm, is used to detect the type and concentration of gases. However, the sensor drift will occur in realistic application scenario of E-nose, which makes a variation of data distribution in feature space and causes a decrease in prediction accuracy. Therefore, studies on the drift compensation algorithms are receiving increasing attention in the field of the E-nose. In this paper, a novel method, namely Wasserstein Distance Learned Feature Representations (WDLFR), is put forward for drift compensation, which is based on the domain invariant feature representation learning. It regards a neural network as a domain discriminator to measure the empirical Wasserstein distance between the source domain (data without drift) and target domain (drift data). The WDLFR minimizes Wasserstein distance by optimizing the feature extractor in an adversarial manner. The Wasserstein distance for domain adaption has good gradient and generalization bound. Finally, the experiments are conducted on a real dataset of E-nose from the University of California, San Diego (UCSD). The experimental results demonstrate that the effectiveness of the proposed method outperforms all compared drift compensation methods, and the WDLFR succeeds in significantly reducing the sensor drift.

scholarly journals Category-Sensitive Domain Adaptation for Land Cover Mapping in Aerial Scenes

2019 ◽  
Vol 11 (22) ◽  
pp. 2631 ◽  
Author(s):  
Bo Fang ◽  
Rong Kou ◽  
Li Pan ◽  
Pengfei Chen
Keyword(s):  
Land Cover ◽  
Domain Adaptation ◽  
Feature Space ◽  
Aerial Images ◽  
Land Cover Mapping ◽  
Target Domain ◽  
Generative Adversarial Network ◽  
Source Domain ◽  
Adversarial Network ◽  
Semantic Labeling

Since manually labeling aerial images for pixel-level classification is expensive and time-consuming, developing strategies for land cover mapping without reference labels is essential and meaningful. As an efficient solution for this issue, domain adaptation has been widely utilized in numerous semantic labeling-based applications. However, current approaches generally pursue the marginal distribution alignment between the source and target features and ignore the category-level alignment. Therefore, directly applying them to land cover mapping leads to unsatisfactory performance in the target domain. In our research, to address this problem, we embed a geometry-consistent generative adversarial network (GcGAN) into a co-training adversarial learning network (CtALN), and then develop a category-sensitive domain adaptation (CsDA) method for land cover mapping using very-high-resolution (VHR) optical aerial images. The GcGAN aims to eliminate the domain discrepancies between labeled and unlabeled images while retaining their intrinsic land cover information by translating the features of the labeled images from the source domain to the target domain. Meanwhile, the CtALN aims to learn a semantic labeling model in the target domain with the translated features and corresponding reference labels. By training this hybrid framework, our method learns to distill knowledge from the source domain and transfers it to the target domain, while preserving not only global domain consistency, but also category-level consistency between labeled and unlabeled images in the feature space. The experimental results between two airborne benchmark datasets and the comparison with other state-of-the-art methods verify the robustness and superiority of our proposed CsDA.

scholarly journals Domain Adaptation for Structural Fault Detection under Model Uncertainty

2021 ◽  
Vol 12 (2) ◽  
Author(s):  
Ali Ozdagli ◽  
Xenofon Koutsoukos
Keyword(s):  
Neural Network ◽  
Experimental Data ◽  
Damage Detection ◽  
Domain Adaptation ◽  
Feature Space ◽  
Health Condition ◽  
Target Domain ◽  
Detection And Localization ◽  
Adaptation Method ◽  
Target Data

In the last decade, the interest in machine learning (ML) has grown significantly within the structural health monitoring (SHM) community. Traditional supervised ML approaches for detecting faults assume that the training and test data come from similar distributions. However, real-world applications, where an ML model is trained, for example, on numerical simulation data and tested on experimental data, are deemed to fail in detecting the damage. The deterioration in the prediction performance is mainly related to the fact that the numerical and experimental data are collected under different conditions and they do not share the same underlying features. This paper proposes a domain adaptation approach for ML-based damage detection and localization problems where the classifier has access to the labeled training (source) and unlabeled test (target) data, but the source and target domains are statistically different. The proposed domain adaptation method seeks to form a feature space that is capable of representing both source and target domains by implementing a domain-adversarial neural network. This neural network uses H-divergence criteria to minimize the discrepancy between the source and target domain in a latent feature space. To evaluate the performance, we present two case studies where we design a neural network model for classifying the health condition of a variety of systems. The effectiveness of the domain adaptation is shown by computing the classification accuracy of the unlabeled target data with and without domain adaptation. Furthermore, the performance gain of the domain adaptation over a well-known transfer knowledge approach called Transfer Component Analysis is also demonstrated. Overall, the results demonstrate that the domain adaption is a valid approach for damage detection applications where access to labeled experimental data is limited.

Sign in / Sign up

Export Citation Format

Share Document