A transfer learning model with multi-source domains for biomedical event trigger extraction

Abstract Background Automatic extraction of biomedical events from literature, that allows for faster update of the latest discoveries automatically, is a heated research topic now. Trigger word recognition is a critical step in the process of event extraction. Its performance directly influences the results of the event extraction. In general, machine learning-based trigger recognition approaches such as neural networks must to be trained on a dataset with plentiful annotations to achieve high performances. However, the problem of the datasets in wide coverage event domains is that their annotations are insufficient and imbalance. One of the methods widely used to deal with this problem is transfer learning. In this work, we aim to extend the transfer learning to utilize multiple source domains. Multiple source domain datasets can be jointly trained to help achieve a higher recognition performance on a target domain with wide coverage events. Results Based on the study of previous work, we propose an improved multi-source domain neural network transfer learning architecture and a training approach for biomedical trigger detection task, which can share knowledge between the multi-source and target domains more comprehensively. We extend the ability of traditional adversarial networks to extract common features between source and target domains, when there is more than one dataset in the source domains. Multiple feature extraction channels to simultaneously capture global and local common features are designed. Moreover, under the constraint of an extra classifier, the multiple local common feature sub-channels can extract and transfer more diverse common features from the related multi-source domains effectively. In the experiments, MLEE corpus is used to train and test the proposed model to recognize the wide coverage triggers as a target dataset. Other four corpora with the varying degrees of relevance with MLEE from different domains are used as source datasets, respectively. Our proposed approach achieves recognition improvement compared with traditional adversarial networks. Moreover, its performance is competitive compared with the results of other leading systems on the same MLEE corpus. Conclusions The proposed Multi-Source Transfer Learning-based Trigger Recognizer (MSTLTR) can further improve the performance compared with the traditional method, when the source domains are more than one. The most essential improvement is that our approach represents common features in two aspects: the global common features and the local common features. Hence, these more sharable features improve the performance and generalization of the model on the target domain effectively.

Download Full-text

Deep Transfer Learning Method Based on 1D-CNN for Bearing Fault Diagnosis

Shock and Vibration ◽

10.1155/2021/6687331 ◽

2021 ◽

Vol 2021 ◽

pp. 1-16

Author(s):

Jun He ◽

Xiang Li ◽

Yong Chen ◽

Danfeng Chen ◽

Jing Guo ◽

...

Keyword(s):

Fault Diagnosis ◽

Transfer Learning ◽

Vibration Signal ◽

Rolling Bearing ◽

Learning Method ◽

Target Domain ◽

Source Domain ◽

Second Order Statistics ◽

Bearing Fault ◽

Bearing Fault Diagnosis

In mechanical fault diagnosis, it is impossible to collect massive labeled samples with the same distribution in real industry. Transfer learning, a promising method, is usually used to address the critical problem. However, as the number of samples increases, the interdomain distribution discrepancy measurement of the existing method has a higher computational complexity, which may make the generalization ability of the method worse. To solve the problem, we propose a deep transfer learning method based on 1D-CNN for rolling bearing fault diagnosis. First, 1-dimension convolutional neural network (1D-CNN), as the basic framework, is used to extract features from vibration signal. The CORrelation ALignment (CORAL) is employed to minimize marginal distribution discrepancy between the source domain and target domain. Then, the cross-entropy loss function and Adam optimizer are used to minimize the classification errors and the second-order statistics of feature distance between the source domain and target domain, respectively. Finally, based on the bearing datasets of Case Western Reserve University and Jiangnan University, seven transfer fault diagnosis comparison experiments are carried out. The results show that our method has better performance.

Download Full-text

Transfer learning for Twitter sentiment analysis: Choosing an effective source dataset

10.5753/kdmile.2020.11972 ◽

2020 ◽

Author(s):

Eliseu Guimarães ◽

Jonnathan Carvalho ◽

Aline Paes ◽

Alexandre Plastino

Keyword(s):

Sentiment Analysis ◽

Transfer Learning ◽

Distance Metrics ◽

Learning Approaches ◽

Target Domain ◽

Social Media Data ◽

Inverse Document Frequency ◽

Source Domain ◽

Document Frequency ◽

Media Data

Sentiment analysis on social media data can be a challenging task, among other reasons, because labeled data for training is not always available. Transfer learning approaches address this problem by leveraging a labeled source domain to obtain a model for a target domain that is different but related to the source domain. However, the question that arises is how to choose proper source data for training the target classifier, which can be made considering the similarity between source and target data using distance metrics. This article investigates the relation between these distance metrics and the classifiers’ performance. For this purpose, we propose to evaluate four metrics combined with distinct dataset representations. Computational experiments, conducted in the Twitter sentiment analysis scenario, showed that the cosine similarity metric combined with bag-of-words normalized with term frequency-inverse document frequency presented the best results in terms of predictive power, outperforming even the classifiers trained with the target dataset in many cases.

Download Full-text

TLGP: a flexible transfer learning algorithm for gene prioritization based on heterogeneous source domain

BMC Bioinformatics ◽

10.1186/s12859-021-04190-9 ◽

2021 ◽

Vol 22 (S9) ◽

Author(s):

Yan Wang ◽

Zuheng Xia ◽

Jingjing Deng ◽

Xianghua Xie ◽

Maoguo Gong ◽

...

Keyword(s):

Transfer Learning ◽

Learning Algorithm ◽

Genomic Data ◽

Gene Prioritization ◽

Affinity Matrix ◽

Target Domain ◽

Gene Ranking ◽

Underlying Assumption ◽

Source Domain ◽

Target Cancer

Abstract Background Gene prioritization (gene ranking) aims to obtain the centrality of genes, which is critical for cancer diagnosis and therapy since keys genes correspond to the biomarkers or targets of drugs. Great efforts have been devoted to the gene ranking problem by exploring the similarity between candidate and known disease-causing genes. However, when the number of disease-causing genes is limited, they are not applicable largely due to the low accuracy. Actually, the number of disease-causing genes for cancers, particularly for these rare cancers, are really limited. Therefore, there is a critical needed to design effective and efficient algorithms for gene ranking with limited prior disease-causing genes. Results In this study, we propose a transfer learning based algorithm for gene prioritization (called TLGP) in the cancer (target domain) without disease-causing genes by transferring knowledge from other cancers (source domain). The underlying assumption is that knowledge shared by similar cancers improves the accuracy of gene prioritization. Specifically, TLGP first quantifies the similarity between the target and source domain by calculating the affinity matrix for genes. Then, TLGP automatically learns a fusion network for the target cancer by fusing affinity matrix, pathogenic genes and genomic data of source cancers. Finally, genes in the target cancer are prioritized. The experimental results indicate that the learnt fusion network is more reliable than gene co-expression network, implying that transferring knowledge from other cancers improves the accuracy of network construction. Moreover, TLGP outperforms state-of-the-art approaches in terms of accuracy, improving at least 5%. Conclusion The proposed model and method provide an effective and efficient strategy for gene ranking by integrating genomic data from various cancers.

Download Full-text

Multi-Source Deep Transfer Neural Network Algorithm

Sensors ◽

10.3390/s19183992 ◽

2019 ◽

Vol 19 (18) ◽

pp. 3992 ◽

Cited By ~ 2

Author(s):

Jingmei Li ◽

Weifei Wu ◽

Di Xue ◽

Peng Gao

Keyword(s):

Neural Network ◽

Probability Distribution ◽

Convolutional Neural Network ◽

Transfer Learning ◽

Classification Performance ◽

Classification Error ◽

Target Domain ◽

Source Domain ◽

Network Algorithm ◽

Neural Network Algorithm

Transfer learning can enhance classification performance of a target domain with insufficient training data by utilizing knowledge relating to the target domain from source domain. Nowadays, it is common to see two or more source domains available for knowledge transfer, which can improve performance of learning tasks in the target domain. However, the classification performance of the target domain decreases due to mismatching of probability distribution. Recent studies have shown that deep learning can build deep structures by extracting more effective features to resist the mismatching. In this paper, we propose a new multi-source deep transfer neural network algorithm, MultiDTNN, based on convolutional neural network and multi-source transfer learning. In MultiDTNN, joint probability distribution adaptation (JPDA) is used for reducing the mismatching between source and target domains to enhance features transferability of the source domain in deep neural networks. Then, the convolutional neural network is trained by utilizing the datasets of each source and target domain to obtain a set of classifiers. Finally, the designed selection strategy selects classifier with the smallest classification error on the target domain from the set to assemble the MultiDTNN framework. The effectiveness of the proposed MultiDTNN is verified by comparing it with other state-of-the-art deep transfer learning on three datasets.

Download Full-text

Autoencoder-based transfer learning in brain–computer interface for rehabilitation robot

International Journal of Advanced Robotic Systems ◽

10.1177/1729881419840860 ◽

2019 ◽

Vol 16 (2) ◽

pp. 172988141984086 ◽

Cited By ~ 4

Author(s):

Chuanqi Tan ◽

Fuchun Sun ◽

Bin Fang ◽

Tao Kong ◽

Wenchang Zhang

Keyword(s):

Transfer Learning ◽

Negative Transfer ◽

Brain Computer Interface ◽

Training Data ◽

Computer Interface ◽

Rehabilitation Robot ◽

Target Domain ◽

Source Domain ◽

Adversarial Network ◽

The Brain

The brain–computer interface-based rehabilitation robot has quickly become a very important research area due to its natural interaction. One of the most important problems in brain–computer interface is that large-scale annotated electroencephalography data sets required by advanced classifiers are almost impossible to acquire because biological data acquisition is challenging and quality annotation is costly. Transfer learning relaxes the hypothesis that the training data must be independent and identically distributed with the test data. It can be considered a powerful tool for solving the problem of insufficient training data. There are two basic issues with transfer learning, under transfer and negative transfer. We proposed a novel brain–computer interface framework by using autoencoder-based transfer learning, which includes three main components: an autoencoder framework, a joint adversarial network, and a regularized manifold constraint. The autoencoder framework automatically encodes and reconstructs data from source and target domains and forces the neural network to learn to represent these domains reliably. The joint adversarial network aims to force the network to learn to encode more appropriately for the source domain and target domain simultaneously, thereby overcoming the problem of under transfer. The regularized manifold constraint aims to avoid the problem of negative transfer by avoiding geometric manifold structure in the target domain being destroyed by the source domain. Experiments show that the brain–computer interface framework proposed by us can achieve better results than state-of-the-art approaches in electroencephalography signal classification tasks. This is helpful in aiding our rehabilitation robot to understand the intention of patients and can help patients to carry out rehabilitation exercises effectively.

Download Full-text

Diagnosis of Sepsis by AI-Aided Proteomics Using 2D Electrophoresis Images of Patient Serum Incorporating Transfer Learning for Deep Neural Networks

Applied Sciences ◽

10.3390/app11041967 ◽

2021 ◽

Vol 11 (4) ◽

pp. 1967

Author(s):

Nobuhiro Hayashi ◽

Yoshihide Sawada ◽

Kei Ujimoto ◽

Syunta Yamaguchi ◽

Yoshikuni Sato ◽

...

Keyword(s):

Transfer Learning ◽

Recognition Performance ◽

Polyacrylamide Gel Electrophoresis ◽

Diagnostic Technique ◽

Small Scale ◽

2D Electrophoresis ◽

Two Dimensional ◽

Target Domain ◽

Serum Samples ◽

Practical Utilization

An accuracy of ≥98% was achieved in sepsis diagnosis using serum samples from 30 sepsis patients and 68 healthy individuals and a high-performance two-dimensional polyacrylamide gel electrophoresis (HP-2D-PAGE) method developed here with deep learning and transfer learning algorithms. In this method, small-scale target domain data, which are collected to achieve our objective, are inputted directly into a model constructed with source domain data which are collected from a different domain from the target; target vectors are estimated with the outputted target domain data and applied to refine the model. Recognition performance of small-scale data is improved by reusing all layers, including the output layers of the neural network. Proteomics is generally considered the ultimate bio-diagnostic technique and provides extremely high information density in its two-dimensional electrophoresis images, but extracting the data has posed a basic problem. The present study is expected to solve that problem and will be an important breakthrough for practical utilization and future perspectives of proteomics in clinics after evaluation in clinical settings.

Download Full-text

Domain Adversarial Transfer Learning for Generalized Tool Wear Prediction

Annual Conference of the PHM Society ◽

10.36001/phmconf.2020.v12i1.1137 ◽

2020 ◽

Vol 12 (1) ◽

pp. 8

Author(s):

Peng (Edward) Wang ◽

Matthew Russell

Keyword(s):

Tool Wear ◽

Transfer Learning ◽

Network Performance ◽

Generative Adversarial Networks ◽

Smart Manufacturing ◽

Successful Implementation ◽

Target Domain ◽

Source Domain ◽

Network Training ◽

And Task

Given its demonstrated ability in analyzing and revealing patterns underlying data, Deep Learning (DL) has been increasingly investigated to complement physics-based models in various aspects of smart manufacturing, such as machine condition monitoring and fault diagnosis, complex manufacturing process modeling, and quality inspection. However, successful implementation of DL techniques relies greatly on the amount, variety, and veracity of data for robust network training. Also, the distributions of data used for network training and application should be identical to avoid the internal covariance shift problem that reduces the network performance applicability. As a promising solution to address these challenges, Transfer Learning (TL) enables DL networks trained on a source domain and task to be applied to a separate target domain and task. This paper presents a domain adversarial TL approach, based upon the concepts of generative adversarial networks. In this method, the optimizer seeks to minimize the loss (i.e., regression or classification accuracy) across the labeled training examples from the source domain while maximizing the loss of the domain classifier across the source and target data sets (i.e., maximizing the similarity of source and target features). The developed domain adversarial TL method has been implemented on a 1-D CNN backbone network and evaluated for prediction of tool wear propagation, using NASA's milling dataset. Performance has been compared to other TL techniques, and the results indicate that domain adversarial TL can successfully allow DL models trained on certain scenarios to be applied to new target tasks.

Download Full-text

Transfer Learning by Mapping and Revising Boosted Relational Dependency Networks

10.5753/ctd.2020.11371 ◽

2020 ◽

Author(s):

Rodrigo Azevedo Santos ◽

Aline Paes ◽

Gerson Zaverucha

Keyword(s):

Transfer Learning ◽

Uncertain Data ◽

Machine Learning Algorithms ◽

Theory Revision ◽

Statistical Machine Learning ◽

Target Domain ◽

Real World Data ◽

Source Domain ◽

Dependency Networks ◽

Starting Point

Statistical machine learning algorithms usually assume that there is considerably-size data to train the models. However, they would fail in addressing domains where data is difficult or expensive to obtain. Transfer learning has emerged to address this problem of learning from scarce data by relying on a model learned in a source domain where data is easy to obtain to be a starting point for the target domain. On the other hand, real-world data contains objects and their relations, usually gathered from noisy environment. Finding patterns through such uncertain relational data has been the focus of the Statistical Relational Learning (SRL) area. Thus, to address domains with scarce, relational, and uncertain data, in this paper, we propose TreeBoostler, an algorithm that transfers the SRL state-of-the-art Boosted Relational Dependency Networks learned in a source domain to the target domain. TreeBoostler first finds a mapping between pairs of predicates to accommodate the additive trees into the target vocabulary. After, it employs two theory revision operators devised to handle incorrect relational regression trees aiming at improving the performance of the mapped trees. In the experiments presented in this paper, TreeBoostler has successfully transferred knowledge among several distinct domains. Moreover, it performs comparably or better than learning from scratch methods in terms of accuracy and outperforms a transfer learning approach in terms of accuracy and runtime.

Download Full-text

Sentiment Classification for Financial Texts Based on Deep Learning

Computational Intelligence and Neuroscience ◽

10.1155/2021/9524705 ◽

2021 ◽

Vol 2021 ◽

pp. 1-9

Author(s):

Shanshan Dong ◽

Chang Liu

Keyword(s):

Neural Network ◽

Deep Learning ◽

Transfer Learning ◽

Loss Function ◽

Classification Accuracy ◽

Sentiment Classification ◽

Target Domain ◽

Text Data ◽

Source Domain ◽

Cross Domain

Sentiment classification for financial texts is of great importance for predicting stock markets and financial crises. At present, with the popularity of applications in the field of natural language processing (NLP) adopting deep learning, the application of automatic text classification and text-based sentiment classification has become more and more extensive. However, in the field of financial text-based sentiment classification, due to a lack of labeled samples, such applications are limited. A domain-adaptation-based financial text sentiment classification method is proposed in this paper, which can adopt source domain (SD) text data with sentiment labels and a large amount of unlabeled target domain (TD) financial text data as training samples for the proposed neural network. The proposed method is a cross-domain transfer-learning-based method. The domain classification subnetwork is added to the original neural network, and the domain classification loss function is also added to the original training loss function. Therefore, the network can simultaneously adapt to the target domain and then accomplish the classification task. The experiment of the proposed sentiment classification transfer learning method is carried out through an open-source dataset. The proposed method in this paper uses the reviews of Amazon Books, DVDs, electronics, and kitchen appliances as the source domain for cross-domain learning, and the classification accuracy rates can reach 65.0%, 61.2%, 61.6%, and 66.3%, respectively. Compared with nontransfer learning, the classification accuracy rate has improved by 11.0%, 7.6%, 11.4%, and 13.4%, respectively.

Download Full-text

One Network for Multi-Domains: Domain Adaptive Hashing with Intersectant Generative Adversarial Networks

Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2019/344 ◽

2019 ◽

Cited By ~ 1

Author(s):

Tao He ◽

Yuan-Fang Li ◽

Lianli Gao ◽

Dongxiang Zhang ◽

Jingkuan Song

Keyword(s):

Adaptive Learning ◽

Digital Data ◽

Generative Adversarial Networks ◽

Target Domain ◽

Source Domain ◽

Learning Framework ◽

Common Space ◽

Adversarial Networks ◽

Shared Space ◽

Benchmark Datasets

With the recent explosive increase of digital data, image recognition and retrieval become a critical practical application. Hashing is an effective solution to this problem, due to its low storage requirement and high query speed. However, most of past works focus on hashing in a single (source) domain. Thus, the learned hash function may not adapt well in a new (target) domain that has a large distributional difference with the source domain. In this paper, we explore an end-to-end domain adaptive learning framework that simultaneously and precisely generates discriminative hash codes and classifies target domain images. Our method encodes two domains images into a semantic common space, followed by two independent generative adversarial networks arming at crosswise reconstructing two domains’ images, reducing domain disparity and improving alignment in the shared space. We evaluate our framework on four public benchmark datasets, all of which show that our method is superior to the other state-of-the-art methods on the tasks of object recognition and image retrieval.

Download Full-text