Transfer Learning for Collaborative Filtering Using a Psychometrics Model

In a real e-commerce website, usually only a small number of users will give ratings to the items they purchased, and this can lead to the very sparse user-item rating data. The data sparsity issue will greatly limit the recommendation performance of most recommendation algorithms. However, a user may register accounts in many e-commerce websites. If such users’ historical purchasing data on these websites can be integrated, the recommendation performance could be improved. But it is difficult to align the users and items between these websites, and thus how to effectively borrow the users’ rating data of one website (source domain) to help improve the recommendation performance of another website (target domain) is very challenging. To this end, this paper extended the traditional one-dimensional psychometrics model to multidimension. The extended model can effectively capture users’ multiple interests. Based on this multidimensional psychometrics model, we further propose a novel transfer learning algorithm. It can effectively transfer users’ rating preferences from the source domain to the target domain. Experimental results show that the proposed method can significantly improve the recommendation performance.

Download Full-text

TLGP: a flexible transfer learning algorithm for gene prioritization based on heterogeneous source domain

BMC Bioinformatics ◽

10.1186/s12859-021-04190-9 ◽

2021 ◽

Vol 22 (S9) ◽

Author(s):

Yan Wang ◽

Zuheng Xia ◽

Jingjing Deng ◽

Xianghua Xie ◽

Maoguo Gong ◽

...

Keyword(s):

Transfer Learning ◽

Learning Algorithm ◽

Genomic Data ◽

Gene Prioritization ◽

Affinity Matrix ◽

Target Domain ◽

Gene Ranking ◽

Underlying Assumption ◽

Source Domain ◽

Target Cancer

Abstract Background Gene prioritization (gene ranking) aims to obtain the centrality of genes, which is critical for cancer diagnosis and therapy since keys genes correspond to the biomarkers or targets of drugs. Great efforts have been devoted to the gene ranking problem by exploring the similarity between candidate and known disease-causing genes. However, when the number of disease-causing genes is limited, they are not applicable largely due to the low accuracy. Actually, the number of disease-causing genes for cancers, particularly for these rare cancers, are really limited. Therefore, there is a critical needed to design effective and efficient algorithms for gene ranking with limited prior disease-causing genes. Results In this study, we propose a transfer learning based algorithm for gene prioritization (called TLGP) in the cancer (target domain) without disease-causing genes by transferring knowledge from other cancers (source domain). The underlying assumption is that knowledge shared by similar cancers improves the accuracy of gene prioritization. Specifically, TLGP first quantifies the similarity between the target and source domain by calculating the affinity matrix for genes. Then, TLGP automatically learns a fusion network for the target cancer by fusing affinity matrix, pathogenic genes and genomic data of source cancers. Finally, genes in the target cancer are prioritized. The experimental results indicate that the learnt fusion network is more reliable than gene co-expression network, implying that transferring knowledge from other cancers improves the accuracy of network construction. Moreover, TLGP outperforms state-of-the-art approaches in terms of accuracy, improving at least 5%. Conclusion The proposed model and method provide an effective and efficient strategy for gene ranking by integrating genomic data from various cancers.

Download Full-text

Domain Adaptation for Pedestrian Detection Based on Prediction Consistency

The Scientific World JOURNAL ◽

10.1155/2014/280382 ◽

2014 ◽

Vol 2014 ◽

pp. 1-7

Author(s):

Yu Li-ping ◽

Tang Huan-ling ◽

An Zhi-yong

Keyword(s):

Computer Vision ◽

Detection Rate ◽

Domain Adaptation ◽

Learning Algorithm ◽

Pedestrian Detection ◽

Experimental Results ◽

Challenging Problem ◽

Target Domain ◽

Source Domain ◽

Adaptation Model

Pedestrian detection is an active area of research in computer vision. It remains a quite challenging problem in many applications where many factors cause a mismatch between source dataset used to train the pedestrian detector and samples in the target scene. In this paper, we propose a novel domain adaptation model for merging plentiful source domain samples with scared target domain samples to create a scene-specific pedestrian detector that performs as well as rich target domain simples are present. Our approach combines the boosting-based learning algorithm with an entropy-based transferability, which is derived from the prediction consistency with the source classifications, to selectively choose the samples showing positive transferability in source domains to the target domain. Experimental results show that our approach can improve the detection rate, especially with the insufficient labeled data in target scene.

Download Full-text

A Study of the Impact of Base Traditional Learners on Transfer Learning Algorithms

International Journal of Artificial Intelligence Tools ◽

10.1142/s0218213018500227 ◽

2018 ◽

Vol 27 (06) ◽

pp. 1850022

Author(s):

Karl R. Weiss ◽

Taghi M. Khoshkoftaar

Keyword(s):

Machine Learning ◽

Transfer Learning ◽

Domain Adaptation ◽

Learning Algorithm ◽

Learning Algorithms ◽

Target Domain ◽

Source Domain ◽

High Performing ◽

Machine Learning Methods ◽

The Impact

A transfer learning environment is characterized by not having sufficient labeled training data from the domain of interest (target domain) to build a high-performing machine learner. Transfer learning algorithms use labeled data from an alternate domain (source domain), that is similar to the target domain, to build high-performing learners. The design of a transfer learning algorithm is typically comprised of a domain adaptation step following by a learning step. The domain adaptation step attempts to align the distribution differences between the source domain and the target domain. Then, the aligned data from the domain adaptation step is used in the learning step, which is typically implemented with a traditional machine learning algorithm. Our research studies the impact of the learning step on the performance of various transfer learning algorithms. In our experiment, we use five unique domain adaptation methods coupled with seven different traditional machine learning methods to create 35 different transfer learning algorithms. We perform comparative performance analyses of the 35 transfer learning algorithms, along with the seven stand-alone traditional machine learning methods. This research will aid machine learning practitioners in the algorithm selection process for a transfer learning environment in the absence of reliable validation techniques.

Download Full-text

Deep Transfer Learning Method Based on 1D-CNN for Bearing Fault Diagnosis

Shock and Vibration ◽

10.1155/2021/6687331 ◽

2021 ◽

Vol 2021 ◽

pp. 1-16

Author(s):

Jun He ◽

Xiang Li ◽

Yong Chen ◽

Danfeng Chen ◽

Jing Guo ◽

...

Keyword(s):

Fault Diagnosis ◽

Transfer Learning ◽

Vibration Signal ◽

Rolling Bearing ◽

Learning Method ◽

Target Domain ◽

Source Domain ◽

Second Order Statistics ◽

Bearing Fault ◽

Bearing Fault Diagnosis

In mechanical fault diagnosis, it is impossible to collect massive labeled samples with the same distribution in real industry. Transfer learning, a promising method, is usually used to address the critical problem. However, as the number of samples increases, the interdomain distribution discrepancy measurement of the existing method has a higher computational complexity, which may make the generalization ability of the method worse. To solve the problem, we propose a deep transfer learning method based on 1D-CNN for rolling bearing fault diagnosis. First, 1-dimension convolutional neural network (1D-CNN), as the basic framework, is used to extract features from vibration signal. The CORrelation ALignment (CORAL) is employed to minimize marginal distribution discrepancy between the source domain and target domain. Then, the cross-entropy loss function and Adam optimizer are used to minimize the classification errors and the second-order statistics of feature distance between the source domain and target domain, respectively. Finally, based on the bearing datasets of Case Western Reserve University and Jiangnan University, seven transfer fault diagnosis comparison experiments are carried out. The results show that our method has better performance.

Download Full-text

Improving Human Happiness Analysis based on Transfer Learning：Algorithm Development and Validation (Preprint)

10.2196/preprints.28292 ◽

2021 ◽

Author(s):

Lele Yu ◽

Shaowu Zhang ◽

Yijia Zhang ◽

Hongfei Lin

Keyword(s):

Transfer Learning ◽

Human Life ◽

Language Model ◽

Experimental Results ◽

Target Domain ◽

Detection Model ◽

Human Happiness ◽

Comparison Results ◽

Semantically Enhanced ◽

Voting Strategy

BACKGROUND Happiness refers to the joyful and pleasant emotions that humans produce subjectively. It is the positive part of emotions, and it affects the quality of human life. Therefore, understanding human happiness is a meaningful task in sentiment analysis. We mainly discuss two facets (Agency/Sociality) of happiness in this study. Through analysis and research on happiness, we can expand on new concepts that define happiness and enrich our understanding of emotions. OBJECTIVE In this paper, we treated each happy moment as a sequence of short sentences, then proposed a short happiness detection model based on transfer learning to analyze the Agency and Sociality aspects of happiness. METHODS Happiness analysis is a novel and challenging research task. However, the current dataset in the field of happiness is small. To solve this problem，we utilized the unlabeled training set and transfer learning to train a semantically enhanced language model in the target domain. Then, the trained language model with domain characteristics was further combined with other deep learning models to obtain various models. Finally, we used the improved voting strategy to further improve the experimental results. RESULTS The proposed approach was evaluated on the public dataset. Experimental results showed that our approach significantly outperforms the baselines. When predicting the Agency aspect of happiness, our approach achieved an accuracy of 0.8574 and an F1 score of 0.90, repectively. When predicting Sociality, our approach achieved an accuracy of 0.928 and an F1 score of 0.9360, respectively. CONCLUSIONS Through the evaluation of the dataset, the comparison results demonstrated the effectiveness of our approach for happiness analysis. Experimental results confirmed that our method achieved state-of-the-art performance and transfer learning effectively improved happiness analysis.

Download Full-text

Transfer learning for Twitter sentiment analysis: Choosing an effective source dataset

10.5753/kdmile.2020.11972 ◽

2020 ◽

Author(s):

Eliseu Guimarães ◽

Jonnathan Carvalho ◽

Aline Paes ◽

Alexandre Plastino

Keyword(s):

Sentiment Analysis ◽

Transfer Learning ◽

Distance Metrics ◽

Learning Approaches ◽

Target Domain ◽

Social Media Data ◽

Inverse Document Frequency ◽

Source Domain ◽

Document Frequency ◽

Media Data

Sentiment analysis on social media data can be a challenging task, among other reasons, because labeled data for training is not always available. Transfer learning approaches address this problem by leveraging a labeled source domain to obtain a model for a target domain that is different but related to the source domain. However, the question that arises is how to choose proper source data for training the target classifier, which can be made considering the similarity between source and target data using distance metrics. This article investigates the relation between these distance metrics and the classifiers’ performance. For this purpose, we propose to evaluate four metrics combined with distinct dataset representations. Computational experiments, conducted in the Twitter sentiment analysis scenario, showed that the cosine similarity metric combined with bag-of-words normalized with term frequency-inverse document frequency presented the best results in terms of predictive power, outperforming even the classifiers trained with the target dataset in many cases.

Download Full-text

Multi-Source Domain Adaptation for Text Classification via DistanceNet-Bandits

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v34i05.6288 ◽

2020 ◽

Vol 34 (05) ◽

pp. 7830-7838 ◽

Cited By ~ 1

Author(s):

Han Guo ◽

Ramakanth Pasunuru ◽

Mohit Bansal

Keyword(s):

Loss Function ◽

Optimal Trajectory ◽

Domain Adaptation ◽

Learning Algorithm ◽

Data Distribution ◽

Distance Measures ◽

Target Domain ◽

Source Domain ◽

Unsupervised Domain Adaptation ◽

Additional Loss

Domain adaptation performance of a learning algorithm on a target domain is a function of its source domain error and a divergence measure between the data distribution of these two domains. We present a study of various distance-based measures in the context of NLP tasks, that characterize the dissimilarity between domains based on sample estimates. We first conduct analysis experiments to show which of these distance measures can best differentiate samples from same versus different domains, and are correlated with empirical results. Next, we develop a DistanceNet model which uses these distance measures, or a mixture of these distance measures, as an additional loss function to be minimized jointly with the task's loss function, so as to achieve better unsupervised domain adaptation. Finally, we extend this model to a novel DistanceNet-Bandit model, which employs a multi-armed bandit controller to dynamically switch between multiple source domains and allow the model to learn an optimal trajectory and mixture of domains for transfer to the low-resource target domain. We conduct experiments on popular sentiment analysis datasets with several diverse domains and show that our DistanceNet model, as well as its dynamic bandit variant, can outperform competitive baselines in the context of unsupervised domain adaptation.

Download Full-text

Multi-Source Deep Transfer Neural Network Algorithm

Sensors ◽

10.3390/s19183992 ◽

2019 ◽

Vol 19 (18) ◽

pp. 3992 ◽

Cited By ~ 2

Author(s):

Jingmei Li ◽

Weifei Wu ◽

Di Xue ◽

Peng Gao

Keyword(s):

Neural Network ◽

Probability Distribution ◽

Convolutional Neural Network ◽

Transfer Learning ◽

Classification Performance ◽

Classification Error ◽

Target Domain ◽

Source Domain ◽

Network Algorithm ◽

Neural Network Algorithm

Transfer learning can enhance classification performance of a target domain with insufficient training data by utilizing knowledge relating to the target domain from source domain. Nowadays, it is common to see two or more source domains available for knowledge transfer, which can improve performance of learning tasks in the target domain. However, the classification performance of the target domain decreases due to mismatching of probability distribution. Recent studies have shown that deep learning can build deep structures by extracting more effective features to resist the mismatching. In this paper, we propose a new multi-source deep transfer neural network algorithm, MultiDTNN, based on convolutional neural network and multi-source transfer learning. In MultiDTNN, joint probability distribution adaptation (JPDA) is used for reducing the mismatching between source and target domains to enhance features transferability of the source domain in deep neural networks. Then, the convolutional neural network is trained by utilizing the datasets of each source and target domain to obtain a set of classifiers. Finally, the designed selection strategy selects classifier with the smallest classification error on the target domain from the set to assemble the MultiDTNN framework. The effectiveness of the proposed MultiDTNN is verified by comparing it with other state-of-the-art deep transfer learning on three datasets.

Download Full-text

Autoencoder-based transfer learning in brain–computer interface for rehabilitation robot

International Journal of Advanced Robotic Systems ◽

10.1177/1729881419840860 ◽

2019 ◽

Vol 16 (2) ◽

pp. 172988141984086 ◽

Cited By ~ 4

Author(s):

Chuanqi Tan ◽

Fuchun Sun ◽

Bin Fang ◽

Tao Kong ◽

Wenchang Zhang

Keyword(s):

Transfer Learning ◽

Negative Transfer ◽

Brain Computer Interface ◽

Training Data ◽

Computer Interface ◽

Rehabilitation Robot ◽

Target Domain ◽

Source Domain ◽

Adversarial Network ◽

The Brain

The brain–computer interface-based rehabilitation robot has quickly become a very important research area due to its natural interaction. One of the most important problems in brain–computer interface is that large-scale annotated electroencephalography data sets required by advanced classifiers are almost impossible to acquire because biological data acquisition is challenging and quality annotation is costly. Transfer learning relaxes the hypothesis that the training data must be independent and identically distributed with the test data. It can be considered a powerful tool for solving the problem of insufficient training data. There are two basic issues with transfer learning, under transfer and negative transfer. We proposed a novel brain–computer interface framework by using autoencoder-based transfer learning, which includes three main components: an autoencoder framework, a joint adversarial network, and a regularized manifold constraint. The autoencoder framework automatically encodes and reconstructs data from source and target domains and forces the neural network to learn to represent these domains reliably. The joint adversarial network aims to force the network to learn to encode more appropriately for the source domain and target domain simultaneously, thereby overcoming the problem of under transfer. The regularized manifold constraint aims to avoid the problem of negative transfer by avoiding geometric manifold structure in the target domain being destroyed by the source domain. Experiments show that the brain–computer interface framework proposed by us can achieve better results than state-of-the-art approaches in electroencephalography signal classification tasks. This is helpful in aiding our rehabilitation robot to understand the intention of patients and can help patients to carry out rehabilitation exercises effectively.

Download Full-text

Domain Adversarial Transfer Learning for Generalized Tool Wear Prediction

Annual Conference of the PHM Society ◽

10.36001/phmconf.2020.v12i1.1137 ◽

2020 ◽

Vol 12 (1) ◽

pp. 8

Author(s):

Peng (Edward) Wang ◽

Matthew Russell

Keyword(s):

Tool Wear ◽

Transfer Learning ◽

Network Performance ◽

Generative Adversarial Networks ◽

Smart Manufacturing ◽

Successful Implementation ◽

Target Domain ◽

Source Domain ◽

Network Training ◽

And Task

Given its demonstrated ability in analyzing and revealing patterns underlying data, Deep Learning (DL) has been increasingly investigated to complement physics-based models in various aspects of smart manufacturing, such as machine condition monitoring and fault diagnosis, complex manufacturing process modeling, and quality inspection. However, successful implementation of DL techniques relies greatly on the amount, variety, and veracity of data for robust network training. Also, the distributions of data used for network training and application should be identical to avoid the internal covariance shift problem that reduces the network performance applicability. As a promising solution to address these challenges, Transfer Learning (TL) enables DL networks trained on a source domain and task to be applied to a separate target domain and task. This paper presents a domain adversarial TL approach, based upon the concepts of generative adversarial networks. In this method, the optimizer seeks to minimize the loss (i.e., regression or classification accuracy) across the labeled training examples from the source domain while maximizing the loss of the domain classifier across the source and target data sets (i.e., maximizing the similarity of source and target features). The developed domain adversarial TL method has been implemented on a 1-D CNN backbone network and evaluated for prediction of tool wear propagation, using NASA's milling dataset. Performance has been compared to other TL techniques, and the results indicate that domain adversarial TL can successfully allow DL models trained on certain scenarios to be applied to new target tasks.

Download Full-text