scholarly journals Understanding How Feature Structure Transfers in Transfer Learning

2017 ◽  
Author(s):  
Tongliang Liu ◽  
Qiang Yang ◽  
Dacheng Tao
Keyword(s):  
Transfer Learning ◽  
Domain Adaptation ◽  
Common Knowledge ◽  
Learning Performance ◽  
Learning To Learn ◽  
Target Domain ◽  
Feature Structure ◽  
Analysis Scheme ◽  
Feature Structures ◽  
Regularization Matrix

Transfer learning transfers knowledge across domains to improve the learning performance. Since feature structures generally represent the common knowledge across different domains, they can be transferred successfully even though the labeling functions across domains differ arbitrarily. However, theoretical justification for this success has remained elusive. In this paper, motivated by self-taught learning, we regard a set of bases as a feature structure of a domain if the bases can (approximately) reconstruct any observation in this domain. We propose a general analysis scheme to theoretically justify that if the source and target domains share similar feature structures, the source domain feature structure is transferable to the target domain, regardless of the change of the labeling functions across domains. The transferred structure is interpreted to function as a regularization matrix which benefits the learning process of the target domain task. We prove that such transfer enables the corresponding learning algorithms to be uniformly stable. Specifically, we illustrate the existence of feature structure transfer in two well-known transfer learning settings: domain adaptation and learning to learn.

scholarly journals Spectral Normalization for Domain Adaptation

Information ◽  
2020 ◽  
Vol 11 (2) ◽  
pp. 68
Author(s):  
Liquan Zhao ◽  
Yan Liu
Keyword(s):  
Transfer Learning ◽  
Predictive Accuracy ◽  
Domain Adaptation ◽  
Learning Performance ◽  
Constraint Condition ◽  
Adversarial Network ◽  
Network Methods ◽  
The Stability ◽  
Spectral Normalization ◽  
Adaptation Method

The transfer learning method is used to extend our existing model to more difficult scenarios, thereby accelerating the training process and improving learning performance. The conditional adversarial domain adaptation method proposed in 2018 is a particular type of transfer learning. It uses the domain discriminator to identify which images the extracted features belong to. The features are obtained from the feature extraction network. The stability of the domain discriminator directly affects the classification accuracy. Here, we propose a new algorithm to improve the predictive accuracy. First, we introduce the Lipschitz constraint condition into domain adaptation. If the constraint condition can be satisfied, the method will be stable. Second, we analyze how to make the gradient satisfy the condition, thereby deducing the modified gradient via the spectrum regularization method. The modified gradient is then used to update the parameter matrix. The proposed method is compared to the ResNet-50, deep adaptation network, domain adversarial neural network, joint adaptation network, and conditional domain adversarial network methods using the datasets that are found in Office-31, ImageCLEF-DA, and Office-Home. The simulations demonstrate that the proposed method has a better performance than other methods with respect to accuracy.

scholarly journals Domain Adversarial Neural Networks for Large-Scale Land Cover Classification

2019 ◽  
Vol 11 (10) ◽  
pp. 1153 ◽  
Author(s):  
Mesay Belete Bejiga ◽  
Farid Melgani ◽  
Pietro Beraldini
Keyword(s):  
Neural Networks ◽  
Land Cover ◽  
Transfer Learning ◽  
Large Scale ◽  
Domain Adaptation ◽  
Land Cover Classification ◽  
Target Domain ◽  
Training Samples ◽  
Latent Space ◽  
Large Scale Land

Learning classification models require sufficiently labeled training samples, however, collecting labeled samples for every new problem is time-consuming and costly. An alternative approach is to transfer knowledge from one problem to another, which is called transfer learning. Domain adaptation (DA) is a type of transfer learning that aims to find a new latent space where the domain discrepancy between the source and the target domain is negligible. In this work, we propose an unsupervised DA technique called domain adversarial neural networks (DANNs), composed of a feature extractor, a class predictor, and domain classifier blocks, for large-scale land cover classification. Contrary to the traditional methods that perform representation and classifier learning in separate stages, DANNs combine them into a single stage, thereby learning a new representation of the input data that is both domain-invariant and discriminative. Once trained, the classifier of a DANN can be used to predict both source and target domain labels. Additionally, we also modify the domain classifier of a DANN to evaluate its suitability for multi-target domain adaptation problems. Experimental results obtained for both single and multiple target DA problems show that the proposed method provides a performance gain of up to 40%.

A domain adaptation model for early gear pitting fault diagnosis based on deep transfer learning network

2019 ◽  
Vol 234 (1) ◽  
pp. 168-182 ◽  
Author(s):  
Jialin Li ◽  
Xueyi Li ◽  
David He ◽  
Yongzhi Qu
Keyword(s):  
Neural Network ◽  
Fault Diagnosis ◽  
Transfer Learning ◽  
Working Conditions ◽  
Deep Neural Network ◽  
Domain Adaptation ◽  
Diagnostic Model ◽  
Target Domain ◽  
Vibration Signals ◽  
Learning Network

In recent years, research on gear pitting fault diagnosis has been conducted. Most of the research has focused on feature extraction and feature selection process, and diagnostic models are only suitable for one working condition. To diagnose early gear pitting faults under multiple working conditions, this article proposes to develop a domain adaptation diagnostic model–based improved deep neural network and transfer learning with raw vibration signals. A particle swarm optimization algorithm and L2 regularization are used to optimize the improved deep neural network to improve the stability and accuracy of the diagnosis. When using the domain adaptation diagnostic model for fault diagnosis, it is necessary to discriminate whether the target domain (test data) is the same as the source domain (training data). If the target domain and the source domain are consistent, the trained improved deep neural network can be used directly for diagnosis. Otherwise, the transfer learning is combined with improved deep neural network to develop a deep transfer learning network to improve the domain adaptability of the diagnostic model. Vibration signals for seven gear types with early pitting faults under 25 working conditions collected from a gear test rig are used to validate the proposed method. It is confirmed by the validation results that the developed domain adaptation diagnostic model has a significant improvement in the adaptability of multiple working conditions.

scholarly journals Disjoint Label Space Transfer Learning with Common Factorised Space

2019 ◽  
Vol 33 ◽  
pp. 3288-3295 ◽  
Author(s):  
Xiaobin Chang ◽  
Yongxin Yang ◽  
Tao Xiang ◽  
Timothy M. Hospedales
Keyword(s):  
Transfer Learning ◽  
Domain Adaptation ◽  
Unified Approach ◽  
Target Domain ◽  
Single Model ◽  
Unsupervised Domain Adaptation ◽  
Common Representation ◽  
Wide Range

In this paper, a unified approach is presented to transfer learning that addresses several source and target domain labelspace and annotation assumptions with a single model. It is particularly effective in handling a challenging case, where source and target label-spaces are disjoint, and outperforms alternatives in both unsupervised and semi-supervised settings. The key ingredient is a common representation termed Common Factorised Space. It is shared between source and target domains, and trained with an unsupervised factorisation loss and a graph-based loss. With a wide range of experiments, we demonstrate the flexibility, relevance and efficacy of our method, both in the challenging cases with disjoint label spaces, and in the more conventional cases such as unsupervised domain adaptation, where the source and target domains share the same label-sets.

Feature-Based Transfer Learning for Bearing Fault Recognition Without Available Fault Data

2020 ◽  
Author(s):  
Clayton Cooper ◽  
Dongdong Liu ◽  
Jianjing Zhang ◽  
Robert X. Gao
Keyword(s):  
Transfer Learning ◽  
Domain Adaptation ◽  
Operating Conditions ◽  
Data Availability ◽  
Detection Methods ◽  
Target Domain ◽  
Bearing Fault ◽  
Fault Recognition ◽  
Training Samples ◽  
Feature Based

Abstract Machine learning has demonstrated its effectiveness in fault recognition for mechanical systems. However, sufficient data for establishing accurate and reliable fault detection methods is not always available in real-world applications. Transfer learning leverages the knowledge learned from a source domain in order to bypass limitations in data availability and facilitate effective analysis in a target domain. For mechanical fault recognition, existing transfer learning methods mainly focus on transferring knowledge between different operating conditions which require training samples corresponding to all desired fault conditions from the target domain in order to realize domain adaptation. However faulted data in real applications is usually unavailable and impractical to collect. In this paper, a transfer learning-based cross-machine bearing fault recognition method is investigated. This new method sees domain adaptation take place without faulted data being available in the target domain, and thus alleviates data availability limitations. The effectiveness of the method is demonstrated in a case study in which the bearing diagnostic method is transferred from an electric motor to a wind turbine.

Extreme Learning Machine-Based Traffic Incidents Detection with Domain Adaptation Transfer Learning

2017 ◽  
Vol 26 (4) ◽  
pp. 601-612
Author(s):  
Chaimae Elhatri ◽  
Mohammed Tahifa ◽  
Jaouad Boumhidi
Keyword(s):  
Extreme Learning Machine ◽  
Transfer Learning ◽  
Domain Adaptation ◽  
Back Propagation ◽  
Back Propagation Neural Network ◽  
Incident Detection ◽  
Target Domain ◽  
Traffic Incidents ◽  
Learning Machine ◽  
Past Experiences

AbstractTraffic incidents in big cities are increasing alongside economic growth, causing traffic delays and deteriorating road safety conditions. Thus, developing a universal freeway automatic incident detection (AID) algorithm is a task that took the interest of researchers. This paper presents a novel automatic traffic incident detection method based on the extreme learning machine (ELM) algorithm. Furthermore, transfer learning has recently gained popularity as it can successfully generalise information across multiple tasks. This paper aimed to develop a new approach for the traffic domain-based domain adaptation. The ELM was used as a classifier for detection, and target domain adaptation transfer ELM (TELM-TDA) was used as a tool to transfer knowledge between environments to benefit from past experiences. The detection performance was evaluated by common criteria including detection rate, false alarm rate, and others. To prove the efficiency of the proposed method, a comparison was first made between back-propagation neural network and ELM; then, another comparison was made between ELM and TELM-TDA.

scholarly journals Generative Transfer Learning for Intelligent Fault Diagnosis of the Wind Turbine Gearbox

Sensors ◽  
2020 ◽  
Vol 20 (5) ◽  
pp. 1361 ◽  
Author(s):  
Jianwen Guo ◽  
Jiapeng Wu ◽  
Shaohui Zhang ◽  
Jianyu Long ◽  
Weidong Chen ◽  
...  
Keyword(s):  
Fault Diagnosis ◽  
Wind Turbine ◽  
Transfer Learning ◽  
Domain Adaptation ◽  
Industrial Applications ◽  
Classification Model ◽  
Intelligent Fault Diagnosis ◽  
Target Domain ◽  
Wind Turbine Gearbox ◽  
Time Frequency

Intelligent fault diagnosis algorithms based on machine learning and deep learning techniques have been widely used in industrial applications and have obtained much attention as well as achievements. In real industrial applications, working loads of machines are always changing. Hence, directly applying the traditional algorithms will cause significant degradation of performance with changing conditions. In this paper, a novel domain adaptation method, named generative transfer learning (GTL), is proposed to tackle this problem. First, raw datasets were transformed to time–frequency domain based on short-time Fourier transformation. A domain discriminator was then built to distinguish whether the data came from the source or the target domain. A target domain classification model was finally acquired by the feature extractor and the classifier. Experiments were carried out for the fault diagnosis of a wind turbine gearbox. The t-distributed stochastic neighbor embedding technique was used to visualize the output features for checking the effectiveness of the proposed algorithm in feature extraction. The results showed that the proposed GTL could improve classification rates under various working loads. Compared with other domain adaptation algorithms, the proposed method exhibited not only higher accuracy but faster convergence speed as well.

scholarly journals Inductive and Transductive Transfer Learning for Zero-day Attack Detection

2019 ◽  
Vol 8 (11) ◽  
pp. 1765-1768
Keyword(s):  
Transfer Learning ◽  
Domain Adaptation ◽  
Attack Detection ◽  
Supervised Machine Learning ◽  
Intrusion Detection Systems ◽  
Machine Learning Techniques ◽  
Target Domain ◽  
Detection Systems ◽  
Learning Techniques ◽  
Manifold Alignment

Upon application of supervised machine learning techniques Intrusion Detection Systems (IDSs) are successful in detecting known attacks as they use predefined attack signatures. However, detecting zero-day attacks is challenged because of the scarcity of the labeled instances for zero-day attacks. Advanced research on IDS applies the concept of Transfer Learning (TL) to compensate the scarcity of labeled instances of zero-day attacks by making use of abundant labeled instances present in related domain(s). This paper explores the potential of Inductive and Transductive transfer learning for detecting zero-day attacks experimentally, where inductive TL deals with the presence of minimal labeled instances in the target domain and transductive TL deals with the complete absence of labeled instances in the target domain. The concept of domain adaptation with manifold alignment (DAMA) is applied in inductive TL where the variant of DAMA is proposed to handle transductive TL due to non-availability of labeled instances. NSL_KDD dataset is used for experimentation

scholarly journals Comparison of domain adaptation techniques for white matter hyperintensity segmentation in brain MR images

2021 ◽  
Author(s):  
Vaanathi Sundaresan ◽  
Giovanna Zamboni ◽  
Nicola K. Dinsdale ◽  
Peter M. Rothwell ◽  
Ludovica Griffanti ◽  
...  
Keyword(s):  
White Matter ◽  
Transfer Learning ◽  
Domain Adaptation ◽  
White Matter Hyperintensity ◽  
Mr Images ◽  
Target Domain ◽  
Baseline Model ◽  
Adversarial Learning ◽  
Adversarial Training ◽  
Brain Mr Images

AbstractRobust automated segmentation of white matter hyperintensities (WMHs) in different datasets (domains) is highly challenging due to differences in acquisition (scanner, sequence), population (WMH amount and location) and limited availability of manual segmentations to train supervised algorithms. In this work we explore various domain adaptation techniques such as transfer learning and domain adversarial learning methods, including domain adversarial neural networks and domain unlearning, to improve the generalisability of our recently proposed triplanar ensemble network, which is our baseline model. We evaluated the domain adaptation techniques on source and target domains consisting of 5 different datasets with variations in intensity profile, lesion characteristics and acquired using different scanners. For transfer learning, we also studied various training options such as minimal number of unfrozen layers and subjects required for finetuning in the target domain. On comparing the performance of different techniques on the target dataset, unsupervised domain adversarial training of neural network gave the best performance, making the technique promising for robust WMH segmentation.

Sign in / Sign up

Export Citation Format

Share Document