scholarly journals Learning Domain-Independent Deep Representations by Mutual Information Minimization

2019 ◽  
Vol 2019 ◽  
pp. 1-14 ◽  
Author(s):  
Ke Wang ◽  
Jiayong Liu ◽  
Jing-Yan Wang
Keyword(s):  
Mutual Information ◽  
Transfer Learning ◽  
Representation Learning ◽  
Target Domain ◽  
Source Domain ◽  
Cross Domain ◽  
Data Points ◽  
Data Point ◽  
Domain Transfer ◽  
Domain Transfer Learning

Domain transfer learning aims to learn common data representations from a source domain and a target domain so that the source domain data can help the classification of the target domain. Conventional transfer representation learning imposes the distributions of source and target domain representations to be similar, which heavily relies on the characterization of the distributions of domains and the distribution matching criteria. In this paper, we proposed a novel framework for domain transfer representation learning. Our motive is to make the learned representations of data points independent from the domains which they belong to. In other words, from an optimal cross-domain representation of a data point, it is difficult to tell which domain it is from. In this way, the learned representations can be generalized to different domains. To measure the dependency between the representations and the corresponding domain which the data points belong to, we propose to use the mutual information between the representations and the domain-belonging indicators. By minimizing such mutual information, we learn the representations which are independent from domains. We build a classwise deep convolutional network model as a representation model and maximize the margin of each data point of the corresponding class, which is defined over the intraclass and interclass neighborhood. To learn the parameters of the model, we construct a unified minimization problem where the margins are maximized while the representation-domain mutual information is minimized. In this way, we learn representations which are not only discriminate but also independent from domains. An iterative algorithm based on the Adam optimization method is proposed to solve the minimization to learn the classwise deep model parameters and the cross-domain representations simultaneously. Extensive experiments over benchmark datasets show its effectiveness and advantage over existing domain transfer learning methods.

scholarly journals Deep Transfer Learning for Biology Cross-Domain Image Classification

2021 ◽  
Vol 2021 ◽  
pp. 1-19
Author(s):  
Chunfeng Guo ◽  
Bin Wei ◽  
Kun Yu
Keyword(s):  
Image Classification ◽  
Transfer Learning ◽  
Poor Performance ◽  
Target Domain ◽  
Deep Convolutional Neural Networks ◽  
Specific Domain ◽  
Source Domain ◽  
Cross Domain ◽  
Domain Transfer ◽  
Domain Transfer Learning

Automatic biology image classification is essential for biodiversity conservation and ecological study. Recently, due to the record-shattering performance, deep convolutional neural networks (DCNNs) have been used more often in biology image classification. However, training DCNNs requires a large amount of labeled data, which may be difficult to collect for some organisms. This study was carried out to exploit cross-domain transfer learning for DCNNs with limited data. According to the literature, previous studies mainly focus on transferring from ImageNet to a specific domain or transferring between two closely related domains. While this study explores deep transfer learning between species from different domains and analyzes the situation when there is a huge difference between the source domain and the target domain. Inspired by the analysis of previous studies, the effect of biology cross-domain image classification in transfer learning is proposed. In this work, the multiple transfer learning scheme is designed to exploit deep transfer learning on several biology image datasets from different domains. There may be a huge difference between the source domain and the target domain, causing poor performance on transfer learning. To address this problem, multistage transfer learning is proposed by introducing an intermediate domain. The experimental results show the effectiveness of cross-domain transfer learning and the importance of data amount and validate the potential of multistage transfer learning.

Tri-training for Dependency Parsing Domain Adaptation

2022 ◽  
Vol 21 (3) ◽  
pp. 1-17
Author(s):  
Shu Jiang ◽  
Zuchao Li ◽  
Hai Zhao ◽  
Bao-Liang Lu ◽  
Rui Wang
Keyword(s):  
Transfer Learning ◽  
High Performance ◽  
Domain Adaptation ◽  
Language Model ◽  
Training Methods ◽  
Dependency Parsing ◽  
Cross Domain ◽  
Cross Lingual ◽  
Domain Transfer ◽  
Domain Transfer Learning

In recent years, the research on dependency parsing focuses on improving the accuracy of the domain-specific (in-domain) test datasets and has made remarkable progress. However, there are innumerable scenarios in the real world that are not covered by the dataset, namely, the out-of-domain dataset. As a result, parsers that perform well on the in-domain data usually suffer from significant performance degradation on the out-of-domain data. Therefore, to adapt the existing in-domain parsers with high performance to a new domain scenario, cross-domain transfer learning methods are essential to solve the domain problem in parsing. This paper examines two scenarios for cross-domain transfer learning: semi-supervised and unsupervised cross-domain transfer learning. Specifically, we adopt a pre-trained language model BERT for training on the source domain (in-domain) data at the subword level and introduce self-training methods varied from tri-training for these two scenarios. The evaluation results on the NLPCC-2019 shared task and universal dependency parsing task indicate the effectiveness of the adopted approaches on cross-domain transfer learning and show the potential of self-learning to cross-lingual transfer learning.

scholarly journals Automatic Microsurgical Skill Assessment Based on Cross-Domain Transfer Learning

2020 ◽  
Vol 5 (3) ◽  
pp. 4148-4155
Author(s):  
Dandan Zhang ◽  
Zicong Wu ◽  
Junhong Chen ◽  
Anzhu Gao ◽  
Xu Chen ◽  
...  
Keyword(s):  
Transfer Learning ◽  
Skill Assessment ◽  
Cross Domain ◽  
Domain Transfer ◽  
Domain Transfer Learning

scholarly journals Damage detection using in-domain and cross-domain transfer learning

2021 ◽  
Author(s):  
Zaharah A. Bukhsh ◽  
Nils Jansen ◽  
Aaqib Saeed
Keyword(s):  
Damage Detection ◽  
Learning Strategies ◽  
Transfer Learning ◽  
Large Scale ◽  
Visual Inspection ◽  
Superior Performance ◽  
Cross Domain ◽  
Domain Transfer ◽  
Domain Transfer Learning ◽  
The Impact

AbstractWe investigate the capabilities of transfer learning in the area of structural health monitoring. In particular, we are interested in damage detection for concrete structures. Typical image datasets for such problems are relatively small, calling for the transfer of learned representation from a related large-scale dataset. Past efforts of damage detection using images have mainly considered cross-domain transfer learning approaches using pre-trained ImageNet models that are subsequently fine-tuned for the target task. However, there are rising concerns about the generalizability of ImageNet representations for specific target domains, such as for visual inspection and medical imaging. We, therefore, evaluate a combination of in-domain and cross-domain transfer learning strategies for damage detection in bridges. We perform comprehensive comparisons to study the impact of cross-domain and in-domain transfer, with various initialization strategies, using six publicly available visual inspection datasets. The pre-trained models are also evaluated for their ability to cope with the extremely low-data regime. We show that the combination of cross-domain and in-domain transfer persistently shows superior performance specially with tiny datasets. Likewise, we also provide visual explanations of predictive models to enable algorithmic transparency and provide insights to experts about the intrinsic decision logic of typically black-box deep models.

scholarly journals Sentiment Classification for Financial Texts Based on Deep Learning

2021 ◽  
Vol 2021 ◽  
pp. 1-9
Author(s):  
Shanshan Dong ◽  
Chang Liu
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Transfer Learning ◽  
Loss Function ◽  
Classification Accuracy ◽  
Sentiment Classification ◽  
Target Domain ◽  
Text Data ◽  
Source Domain ◽  
Cross Domain

Sentiment classification for financial texts is of great importance for predicting stock markets and financial crises. At present, with the popularity of applications in the field of natural language processing (NLP) adopting deep learning, the application of automatic text classification and text-based sentiment classification has become more and more extensive. However, in the field of financial text-based sentiment classification, due to a lack of labeled samples, such applications are limited. A domain-adaptation-based financial text sentiment classification method is proposed in this paper, which can adopt source domain (SD) text data with sentiment labels and a large amount of unlabeled target domain (TD) financial text data as training samples for the proposed neural network. The proposed method is a cross-domain transfer-learning-based method. The domain classification subnetwork is added to the original neural network, and the domain classification loss function is also added to the original training loss function. Therefore, the network can simultaneously adapt to the target domain and then accomplish the classification task. The experiment of the proposed sentiment classification transfer learning method is carried out through an open-source dataset. The proposed method in this paper uses the reviews of Amazon Books, DVDs, electronics, and kitchen appliances as the source domain for cross-domain learning, and the classification accuracy rates can reach 65.0%, 61.2%, 61.6%, and 66.3%, respectively. Compared with nontransfer learning, the classification accuracy rate has improved by 11.0%, 7.6%, 11.4%, and 13.4%, respectively.

Sign in / Sign up

Export Citation Format

Share Document