scholarly journals Domain Adversarial Transfer Learning for Generalized Tool Wear Prediction

2020 ◽  
Vol 12 (1) ◽  
pp. 8
Author(s):  
Peng (Edward) Wang ◽  
Matthew Russell
Keyword(s):  
Tool Wear ◽  
Transfer Learning ◽  
Network Performance ◽  
Generative Adversarial Networks ◽  
Smart Manufacturing ◽  
Successful Implementation ◽  
Target Domain ◽  
Source Domain ◽  
Network Training ◽  
And Task

Given its demonstrated ability in analyzing and revealing patterns underlying data, Deep Learning (DL) has been increasingly investigated to complement physics-based models in various aspects of smart manufacturing, such as machine condition monitoring and fault diagnosis, complex manufacturing process modeling, and quality inspection. However, successful implementation of DL techniques relies greatly on the amount, variety, and veracity of data for robust network training. Also, the distributions of data used for network training and application should be identical to avoid the internal covariance shift problem that reduces the network performance applicability. As a promising solution to address these challenges, Transfer Learning (TL) enables DL networks trained on a source domain and task to be applied to a separate target domain and task. This paper presents a domain adversarial TL approach, based upon the concepts of generative adversarial networks. In this method, the optimizer seeks to minimize the loss (i.e., regression or classification accuracy) across the labeled training examples from the source domain while maximizing the loss of the domain classifier across the source and target data sets (i.e., maximizing the similarity of source and target features). The developed domain adversarial TL method has been implemented on a 1-D CNN backbone network and evaluated for prediction of tool wear propagation, using NASA's milling dataset. Performance has been compared to other TL techniques, and the results indicate that domain adversarial TL can successfully allow DL models trained on certain scenarios to be applied to new target tasks.

scholarly journals Deep Transfer Learning Method Based on 1D-CNN for Bearing Fault Diagnosis

2021 ◽  
Vol 2021 ◽  
pp. 1-16
Author(s):  
Jun He ◽  
Xiang Li ◽  
Yong Chen ◽  
Danfeng Chen ◽  
Jing Guo ◽  
...  
Keyword(s):  
Fault Diagnosis ◽  
Transfer Learning ◽  
Vibration Signal ◽  
Rolling Bearing ◽  
Learning Method ◽  
Target Domain ◽  
Source Domain ◽  
Second Order Statistics ◽  
Bearing Fault ◽  
Bearing Fault Diagnosis

In mechanical fault diagnosis, it is impossible to collect massive labeled samples with the same distribution in real industry. Transfer learning, a promising method, is usually used to address the critical problem. However, as the number of samples increases, the interdomain distribution discrepancy measurement of the existing method has a higher computational complexity, which may make the generalization ability of the method worse. To solve the problem, we propose a deep transfer learning method based on 1D-CNN for rolling bearing fault diagnosis. First, 1-dimension convolutional neural network (1D-CNN), as the basic framework, is used to extract features from vibration signal. The CORrelation ALignment (CORAL) is employed to minimize marginal distribution discrepancy between the source domain and target domain. Then, the cross-entropy loss function and Adam optimizer are used to minimize the classification errors and the second-order statistics of feature distance between the source domain and target domain, respectively. Finally, based on the bearing datasets of Case Western Reserve University and Jiangnan University, seven transfer fault diagnosis comparison experiments are carried out. The results show that our method has better performance.

scholarly journals Transfer learning for Twitter sentiment analysis: Choosing an effective source dataset

2020 ◽  
Author(s):  
Eliseu Guimarães ◽  
Jonnathan Carvalho ◽  
Aline Paes ◽  
Alexandre Plastino
Keyword(s):  
Sentiment Analysis ◽  
Transfer Learning ◽  
Distance Metrics ◽  
Learning Approaches ◽  
Target Domain ◽  
Social Media Data ◽  
Inverse Document Frequency ◽  
Source Domain ◽  
Document Frequency ◽  
Media Data

Sentiment analysis on social media data can be a challenging task, among other reasons, because labeled data for training is not always available. Transfer learning approaches address this problem by leveraging a labeled source domain to obtain a model for a target domain that is different but related to the source domain. However, the question that arises is how to choose proper source data for training the target classifier, which can be made considering the similarity between source and target data using distance metrics. This article investigates the relation between these distance metrics and the classifiers’ performance. For this purpose, we propose to evaluate four metrics combined with distinct dataset representations. Computational experiments, conducted in the Twitter sentiment analysis scenario, showed that the cosine similarity metric combined with bag-of-words normalized with term frequency-inverse document frequency presented the best results in terms of predictive power, outperforming even the classifiers trained with the target dataset in many cases.

scholarly journals TLGP: a flexible transfer learning algorithm for gene prioritization based on heterogeneous source domain

2021 ◽  
Vol 22 (S9) ◽  
Author(s):  
Yan Wang ◽  
Zuheng Xia ◽  
Jingjing Deng ◽  
Xianghua Xie ◽  
Maoguo Gong ◽  
...  
Keyword(s):  
Transfer Learning ◽  
Learning Algorithm ◽  
Genomic Data ◽  
Gene Prioritization ◽  
Affinity Matrix ◽  
Target Domain ◽  
Gene Ranking ◽  
Underlying Assumption ◽  
Source Domain ◽  
Target Cancer

Abstract Background Gene prioritization (gene ranking) aims to obtain the centrality of genes, which is critical for cancer diagnosis and therapy since keys genes correspond to the biomarkers or targets of drugs. Great efforts have been devoted to the gene ranking problem by exploring the similarity between candidate and known disease-causing genes. However, when the number of disease-causing genes is limited, they are not applicable largely due to the low accuracy. Actually, the number of disease-causing genes for cancers, particularly for these rare cancers, are really limited. Therefore, there is a critical needed to design effective and efficient algorithms for gene ranking with limited prior disease-causing genes. Results In this study, we propose a transfer learning based algorithm for gene prioritization (called TLGP) in the cancer (target domain) without disease-causing genes by transferring knowledge from other cancers (source domain). The underlying assumption is that knowledge shared by similar cancers improves the accuracy of gene prioritization. Specifically, TLGP first quantifies the similarity between the target and source domain by calculating the affinity matrix for genes. Then, TLGP automatically learns a fusion network for the target cancer by fusing affinity matrix, pathogenic genes and genomic data of source cancers. Finally, genes in the target cancer are prioritized. The experimental results indicate that the learnt fusion network is more reliable than gene co-expression network, implying that transferring knowledge from other cancers improves the accuracy of network construction. Moreover, TLGP outperforms state-of-the-art approaches in terms of accuracy, improving at least 5%. Conclusion The proposed model and method provide an effective and efficient strategy for gene ranking by integrating genomic data from various cancers.

scholarly journals Temperature Prediction of Heating Furnace Based on Deep Transfer Learning

Sensors ◽  
2020 ◽  
Vol 20 (17) ◽  
pp. 4676
Author(s):  
Naiju Zhai ◽  
Xiaofeng Zhou
Keyword(s):  
Transfer Learning ◽  
Heating Temperature ◽  
Direct Method ◽  
Fine Tuning ◽  
Generative Adversarial Networks ◽  
Heating Process ◽  
Heating Zone ◽  
Billet Temperature ◽  
Source Domain ◽  
Target Model

Heating temperature is very important in the process of billet production, and it directly affects the quality of billet. However, there is no direct method to measure billet temperature, so we need to accurately predict the temperature of each heating zone in the furnace in order to approximate the billet temperature. Due to the complexity of the heating process, it is difficult to accurately predict the temperature of each heating zone and each heating zone sensor datum to establish a model, which will increase the cost of calculation. To solve these two problems, a two-layer transfer learning framework based on a temporal convolution network (TL-TCN) is proposed for the first time, which transfers the knowledge learned from the source heating zone to the target heating zone. In the first layer, the TCN model is built for the source domain data, and the self-transfer learning method is used to optimize the TCN model to obtain the basic model, which improves the prediction accuracy of the source domain. In the second layer, we propose two frameworks: one is to generate the target model directly by using fine-tuning, and the other is to generate the target model by using generative adversarial networks (GAN) for domain adaption. Case studies demonstrated that the proposed TL-TCN framework achieves state-of-the-art prediction results on each dataset, and the prediction errors are significantly reduced. Consistent results applied to each dataset indicate that this framework is the most advanced method to solve the above problem under the condition of limited samples.

scholarly journals Multi-Source Deep Transfer Neural Network Algorithm

Sensors ◽  
2019 ◽  
Vol 19 (18) ◽  
pp. 3992 ◽  
Author(s):  
Jingmei Li ◽  
Weifei Wu ◽  
Di Xue ◽  
Peng Gao
Keyword(s):  
Neural Network ◽  
Probability Distribution ◽  
Convolutional Neural Network ◽  
Transfer Learning ◽  
Classification Performance ◽  
Classification Error ◽  
Target Domain ◽  
Source Domain ◽  
Network Algorithm ◽  
Neural Network Algorithm

Transfer learning can enhance classification performance of a target domain with insufficient training data by utilizing knowledge relating to the target domain from source domain. Nowadays, it is common to see two or more source domains available for knowledge transfer, which can improve performance of learning tasks in the target domain. However, the classification performance of the target domain decreases due to mismatching of probability distribution. Recent studies have shown that deep learning can build deep structures by extracting more effective features to resist the mismatching. In this paper, we propose a new multi-source deep transfer neural network algorithm, MultiDTNN, based on convolutional neural network and multi-source transfer learning. In MultiDTNN, joint probability distribution adaptation (JPDA) is used for reducing the mismatching between source and target domains to enhance features transferability of the source domain in deep neural networks. Then, the convolutional neural network is trained by utilizing the datasets of each source and target domain to obtain a set of classifiers. Finally, the designed selection strategy selects classifier with the smallest classification error on the target domain from the set to assemble the MultiDTNN framework. The effectiveness of the proposed MultiDTNN is verified by comparing it with other state-of-the-art deep transfer learning on three datasets.

scholarly journals Autoencoder-based transfer learning in brain–computer interface for rehabilitation robot

2019 ◽  
Vol 16 (2) ◽  
pp. 172988141984086 ◽  
Author(s):  
Chuanqi Tan ◽  
Fuchun Sun ◽  
Bin Fang ◽  
Tao Kong ◽  
Wenchang Zhang
Keyword(s):  
Transfer Learning ◽  
Negative Transfer ◽  
Brain Computer Interface ◽  
Training Data ◽  
Computer Interface ◽  
Rehabilitation Robot ◽  
Target Domain ◽  
Source Domain ◽  
Adversarial Network ◽  
The Brain

The brain–computer interface-based rehabilitation robot has quickly become a very important research area due to its natural interaction. One of the most important problems in brain–computer interface is that large-scale annotated electroencephalography data sets required by advanced classifiers are almost impossible to acquire because biological data acquisition is challenging and quality annotation is costly. Transfer learning relaxes the hypothesis that the training data must be independent and identically distributed with the test data. It can be considered a powerful tool for solving the problem of insufficient training data. There are two basic issues with transfer learning, under transfer and negative transfer. We proposed a novel brain–computer interface framework by using autoencoder-based transfer learning, which includes three main components: an autoencoder framework, a joint adversarial network, and a regularized manifold constraint. The autoencoder framework automatically encodes and reconstructs data from source and target domains and forces the neural network to learn to represent these domains reliably. The joint adversarial network aims to force the network to learn to encode more appropriately for the source domain and target domain simultaneously, thereby overcoming the problem of under transfer. The regularized manifold constraint aims to avoid the problem of negative transfer by avoiding geometric manifold structure in the target domain being destroyed by the source domain. Experiments show that the brain–computer interface framework proposed by us can achieve better results than state-of-the-art approaches in electroencephalography signal classification tasks. This is helpful in aiding our rehabilitation robot to understand the intention of patients and can help patients to carry out rehabilitation exercises effectively.

scholarly journals Transfer Learning by Mapping and Revising Boosted Relational Dependency Networks

2020 ◽  
Author(s):  
Rodrigo Azevedo Santos ◽  
Aline Paes ◽  
Gerson Zaverucha
Keyword(s):  
Transfer Learning ◽  
Uncertain Data ◽  
Machine Learning Algorithms ◽  
Theory Revision ◽  
Statistical Machine Learning ◽  
Target Domain ◽  
Real World Data ◽  
Source Domain ◽  
Dependency Networks ◽  
Starting Point

Statistical machine learning algorithms usually assume that there is considerably-size data to train the models. However, they would fail in addressing domains where data is difficult or expensive to obtain. Transfer learning has emerged to address this problem of learning from scarce data by relying on a model learned in a source domain where data is easy to obtain to be a starting point for the target domain. On the other hand, real-world data contains objects and their relations, usually gathered from noisy environment. Finding patterns through such uncertain relational data has been the focus of the Statistical Relational Learning (SRL) area. Thus, to address domains with scarce, relational, and uncertain data, in this paper, we propose TreeBoostler, an algorithm that transfers the SRL state-of-the-art Boosted Relational Dependency Networks learned in a source domain to the target domain. TreeBoostler first finds a mapping between pairs of predicates to accommodate the additive trees into the target vocabulary. After, it employs two theory revision operators devised to handle incorrect relational regression trees aiming at improving the performance of the mapped trees. In the experiments presented in this paper, TreeBoostler has successfully transferred knowledge among several distinct domains. Moreover, it performs comparably or better than learning from scratch methods in terms of accuracy and outperforms a transfer learning approach in terms of accuracy and runtime.

scholarly journals Sentiment Classification for Financial Texts Based on Deep Learning

2021 ◽  
Vol 2021 ◽  
pp. 1-9
Author(s):  
Shanshan Dong ◽  
Chang Liu
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Transfer Learning ◽  
Loss Function ◽  
Classification Accuracy ◽  
Sentiment Classification ◽  
Target Domain ◽  
Text Data ◽  
Source Domain ◽  
Cross Domain

Sentiment classification for financial texts is of great importance for predicting stock markets and financial crises. At present, with the popularity of applications in the field of natural language processing (NLP) adopting deep learning, the application of automatic text classification and text-based sentiment classification has become more and more extensive. However, in the field of financial text-based sentiment classification, due to a lack of labeled samples, such applications are limited. A domain-adaptation-based financial text sentiment classification method is proposed in this paper, which can adopt source domain (SD) text data with sentiment labels and a large amount of unlabeled target domain (TD) financial text data as training samples for the proposed neural network. The proposed method is a cross-domain transfer-learning-based method. The domain classification subnetwork is added to the original neural network, and the domain classification loss function is also added to the original training loss function. Therefore, the network can simultaneously adapt to the target domain and then accomplish the classification task. The experiment of the proposed sentiment classification transfer learning method is carried out through an open-source dataset. The proposed method in this paper uses the reviews of Amazon Books, DVDs, electronics, and kitchen appliances as the source domain for cross-domain learning, and the classification accuracy rates can reach 65.0%, 61.2%, 61.6%, and 66.3%, respectively. Compared with nontransfer learning, the classification accuracy rate has improved by 11.0%, 7.6%, 11.4%, and 13.4%, respectively.

scholarly journals One Network for Multi-Domains: Domain Adaptive Hashing with Intersectant Generative Adversarial Networks

2019 ◽  
Author(s):  
Tao He ◽  
Yuan-Fang Li ◽  
Lianli Gao ◽  
Dongxiang Zhang ◽  
Jingkuan Song
Keyword(s):  
Adaptive Learning ◽  
Digital Data ◽  
Generative Adversarial Networks ◽  
Target Domain ◽  
Source Domain ◽  
Learning Framework ◽  
Common Space ◽  
Adversarial Networks ◽  
Shared Space ◽  
Benchmark Datasets

With the recent explosive increase of digital data, image recognition and retrieval become a critical practical application. Hashing is an effective solution to this problem, due to its low storage requirement and high query speed. However, most of past works focus on hashing in a single (source) domain. Thus, the learned hash function may not adapt well in a new (target) domain that has a large distributional difference with the source domain. In this paper, we explore an end-to-end domain adaptive learning framework that simultaneously and precisely generates discriminative hash codes and classifies target domain images. Our method encodes two domains images into a semantic common space, followed by two independent generative adversarial networks arming at crosswise reconstructing two domains’ images, reducing domain disparity and improving alignment in the shared space. We evaluate our framework on four public benchmark datasets, all of which show that our method is superior to the other state-of-the-art methods on the tasks of object recognition and image retrieval.

scholarly journals Deep Transfer Learning for Biology Cross-Domain Image Classification

2021 ◽  
Vol 2021 ◽  
pp. 1-19
Author(s):  
Chunfeng Guo ◽  
Bin Wei ◽  
Kun Yu
Keyword(s):  
Image Classification ◽  
Transfer Learning ◽  
Poor Performance ◽  
Target Domain ◽  
Deep Convolutional Neural Networks ◽  
Specific Domain ◽  
Source Domain ◽  
Cross Domain ◽  
Domain Transfer ◽  
Domain Transfer Learning

Automatic biology image classification is essential for biodiversity conservation and ecological study. Recently, due to the record-shattering performance, deep convolutional neural networks (DCNNs) have been used more often in biology image classification. However, training DCNNs requires a large amount of labeled data, which may be difficult to collect for some organisms. This study was carried out to exploit cross-domain transfer learning for DCNNs with limited data. According to the literature, previous studies mainly focus on transferring from ImageNet to a specific domain or transferring between two closely related domains. While this study explores deep transfer learning between species from different domains and analyzes the situation when there is a huge difference between the source domain and the target domain. Inspired by the analysis of previous studies, the effect of biology cross-domain image classification in transfer learning is proposed. In this work, the multiple transfer learning scheme is designed to exploit deep transfer learning on several biology image datasets from different domains. There may be a huge difference between the source domain and the target domain, causing poor performance on transfer learning. To address this problem, multistage transfer learning is proposed by introducing an intermediate domain. The experimental results show the effectiveness of cross-domain transfer learning and the importance of data amount and validate the potential of multistage transfer learning.

Sign in / Sign up

Export Citation Format

Share Document