scholarly journals TRANSFER LEARNING BASED ON LOGISTIC REGRESSION

2015 ◽  
Vol XL-3/W3 ◽  
pp. 145-152
Author(s):  
A. Paul ◽  
F. Rottensteiner ◽  
C. Heipke
Keyword(s):  
Remote Sensing ◽  
Logistic Regression ◽  
Transfer Learning ◽  
Domain Adaptation ◽  
Real Data ◽  
Training Data ◽  
Model Parameters ◽  
Target Domain ◽  
Training Set ◽  
Data Set

In this paper we address the problem of classification of remote sensing images in the framework of transfer learning with a focus on domain adaptation. The main novel contribution is a method for transductive transfer learning in remote sensing on the basis of logistic regression. Logistic regression is a discriminative probabilistic classifier of low computational complexity, which can deal with multiclass problems. This research area deals with methods that solve problems in which labelled training data sets are assumed to be available only for a source domain, while classification is needed in the target domain with different, yet related characteristics. Classification takes place with a model of weight coefficients for hyperplanes which separate features in the transformed feature space. In term of logistic regression, our domain adaptation method adjusts the model parameters by iterative labelling of the target test data set. These labelled data features are iteratively added to the current training set which, at the beginning, only contains source features and, simultaneously, a number of source features are deleted from the current training set. Experimental results based on a test series with synthetic and real data constitutes a first proof-of-concept of the proposed method.

scholarly journals ITERATIVE RE-WEIGHTED INSTANCE TRANSFER FOR DOMAIN ADAPTATION

2016 ◽  
Vol III-3 ◽  
pp. 339-346 ◽  
Author(s):  
A. Paul ◽  
F. Rottensteiner ◽  
C. Heipke
Keyword(s):  
Domain Adaptation ◽  
Training Data ◽  
Aerial Images ◽  
Target Domain ◽  
Data Set ◽  
Source Domain ◽  
The Difference ◽  
A New Technique ◽  
Class Labels ◽  
Decision Boundaries

Domain adaptation techniques in transfer learning try to reduce the amount of training data required for classification by adapting a classifier trained on samples from a source domain to a new data set (target domain) where the features may have different distributions. In this paper, we propose a new technique for domain adaptation based on logistic regression. Starting with a classifier trained on training data from the source domain, we iteratively include target domain samples for which class labels have been obtained from the current state of the classifier, while at the same time removing source domain samples. In each iteration the classifier is re-trained, so that the decision boundaries are slowly transferred to the distribution of the target features. To make the transfer procedure more robust we introduce weights as a function of distance from the decision boundary and a new way of regularisation. Our methodology is evaluated using a benchmark data set consisting of aerial images and digital surface models. The experimental results show that in the majority of cases our domain adaptation approach can lead to an improvement of the classification accuracy without additional training data, but also indicate remaining problems if the difference in the feature distributions becomes too large.

scholarly journals A novel augmented deep transfer learning for classification of COVID-19 and other thoracic diseases from X-rays

2021 ◽  
Author(s):  
Fouzia Altaf ◽  
Syed M. S. Islam ◽  
Naeem Khalid Janjua
Keyword(s):  
Deep Learning ◽  
Medical Imaging ◽  
Transfer Learning ◽  
Medical Images ◽  
Training Data ◽  
Target Domain ◽  
Data Set ◽  
Learning Technique ◽  
Multi Class Classification ◽  
Effective Transfer

AbstractDeep learning has provided numerous breakthroughs in natural imaging tasks. However, its successful application to medical images is severely handicapped with the limited amount of annotated training data. Transfer learning is commonly adopted for the medical imaging tasks. However, a large covariant shift between the source domain of natural images and target domain of medical images results in poor transfer learning. Moreover, scarcity of annotated data for the medical imaging tasks causes further problems for effective transfer learning. To address these problems, we develop an augmented ensemble transfer learning technique that leads to significant performance gain over the conventional transfer learning. Our technique uses an ensemble of deep learning models, where the architecture of each network is modified with extra layers to account for dimensionality change between the images of source and target data domains. Moreover, the model is hierarchically tuned to the target domain with augmented training data. Along with the network ensemble, we also utilize an ensemble of dictionaries that are based on features extracted from the augmented models. The dictionary ensemble provides an additional performance boost to our method. We first establish the effectiveness of our technique with the challenging ChestXray-14 radiography data set. Our experimental results show more than 50% reduction in the error rate with our method as compared to the baseline transfer learning technique. We then apply our technique to a recent COVID-19 data set for binary and multi-class classification tasks. Our technique achieves 99.49% accuracy for the binary classification, and 99.24% for multi-class classification.

scholarly journals Unsupervised Adversarial Domain Adaptation with Error-Correcting Boundaries and Feature Adaption Metric for Remote-Sensing Scene Classification

2021 ◽  
Vol 13 (7) ◽  
pp. 1270
Author(s):  
Chenhui Ma ◽  
Dexuan Sha ◽  
Xiaodong Mu
Keyword(s):  
Remote Sensing ◽  
Domain Adaptation ◽  
Training Data ◽  
Scene Classification ◽  
Target Domain ◽  
Domain Specific ◽  
Invariant Features ◽  
Distribution Matching ◽  
Feature Adaptation ◽  
Public Datasets

Unsupervised domain adaptation (UDA) based on adversarial learning for remote-sensing scene classification has become a research hotspot because of the need to alleviating the lack of annotated training data. Existing methods train classifiers according to their ability to distinguish features from source or target domains. However, they suffer from the following two limitations: (1) the classifier is trained on source samples and forms a source-domain-specific boundary, which ignores features from the target domain and (2) semantically meaningful features are merely built from the adversary of a generator and a discriminator, which ignore selecting the domain invariant features. These issues limit the distribution matching performance of source and target domains, since each domain has its distinctive characteristic. To resolve these issues, we propose a framework with error-correcting boundaries and feature adaptation metric. Specifically, we design an error-correcting boundaries mechanism to build target-domain-specific classifier boundaries via multi-classifiers and error-correcting discrepancy loss, which significantly distinguish target samples and reduce their distinguished uncertainty. Then, we employ a feature adaptation metric structure to enhance the adaptation of ambiguous features via shallow layers of the backbone convolutional neural network and alignment loss, which automatically learns domain invariant features. The experimental results on four public datasets outperform other UDA methods of remote-sensing scene classification.

scholarly journals ITERATIVE RE-WEIGHTED INSTANCE TRANSFER FOR DOMAIN ADAPTATION

2016 ◽  
Vol III-3 ◽  
pp. 339-346 ◽  
Author(s):  
A. Paul ◽  
F. Rottensteiner ◽  
C. Heipke
Keyword(s):  
Domain Adaptation ◽  
Training Data ◽  
Aerial Images ◽  
Target Domain ◽  
Data Set ◽  
Source Domain ◽  
The Difference ◽  
A New Technique ◽  
Class Labels ◽  
Decision Boundaries

Domain adaptation techniques in transfer learning try to reduce the amount of training data required for classification by adapting a classifier trained on samples from a source domain to a new data set (target domain) where the features may have different distributions. In this paper, we propose a new technique for domain adaptation based on logistic regression. Starting with a classifier trained on training data from the source domain, we iteratively include target domain samples for which class labels have been obtained from the current state of the classifier, while at the same time removing source domain samples. In each iteration the classifier is re-trained, so that the decision boundaries are slowly transferred to the distribution of the target features. To make the transfer procedure more robust we introduce weights as a function of distance from the decision boundary and a new way of regularisation. Our methodology is evaluated using a benchmark data set consisting of aerial images and digital surface models. The experimental results show that in the majority of cases our domain adaptation approach can lead to an improvement of the classification accuracy without additional training data, but also indicate remaining problems if the difference in the feature distributions becomes too large.

scholarly journals A logistic regression model for predicting the occurrence of intense geomagnetic storms

2005 ◽  
Vol 23 (9) ◽  
pp. 2969-2974 ◽  
Author(s):  
N. Srivastava
Keyword(s):  
Logistic Regression ◽  
Regression Model ◽  
Logistic Regression Model ◽  
Geomagnetic Storms ◽  
Training Data ◽  
Validation Dataset ◽  
Model Parameters ◽  
Validation Data ◽  
Data Set ◽  
Logistic Regression Models

Abstract. A logistic regression model is implemented for predicting the occurrence of intense/super-intense geomagnetic storms. A binary dependent variable, indicating the occurrence of intense/super-intense geomagnetic storms, is regressed against a series of independent model variables that define a number of solar and interplanetary properties of geo-effective CMEs. The model parameters (regression coefficients) are estimated from a training data set which was extracted from a dataset of 64 geo-effective CMEs observed during 1996-2002. The trained model is validated by predicting the occurrence of geomagnetic storms from a validation dataset, also extracted from the same data set of 64 geo-effective CMEs, recorded during 1996-2002, but not used for training the model. The model predicts 78% of the geomagnetic storms from the validation data set. In addition, the model predicts 85% of the geomagnetic storms from the training data set. These results indicate that logistic regression models can be effectively used for predicting the occurrence of intense geomagnetic storms from a set of solar and interplanetary factors.

scholarly journals ADVERSARIAL DISCRIMINATIVE DOMAIN ADAPTATION FOR DEFORESTATION DETECTION

2021 ◽  
Vol V-3-2021 ◽  
pp. 151-158
Author(s):  
J. Noa ◽  
P. J. Soto ◽  
G. A. O. P. Costa ◽  
D. Wittich ◽  
R. Q. Feitosa ◽  
...  
Keyword(s):  
Domain Adaptation ◽  
Model Parameters ◽  
Training Methods ◽  
Set Domain ◽  
Target Domain ◽  
Training Set ◽  
Supervised Training ◽  
Sensing Applications ◽  
Remote Sensing Applications ◽  
Application Fields

Abstract. Although very efficient in a number of application fields, deep learning based models are known to demand large amounts of labeled data for training. Particularly for remote sensing applications, responding to that demand is generally expensive and time consuming. Moreover, supervised training methods tend to perform poorly when they are tested with a set of samples that does not match the general characteristics of the training set. Domain adaptation methods can be used to mitigate those problems, especially in applications where labeled data is only available for a particular region or epoch, i.e., for a source domain, but not for a target domain on which the model should be tested. In this work we introduce a domain adaptation approach based on representation matching for the deforestation detection task. The approach follows the Adversarial Discriminative Domain Adaptation (ADDA) framework, and we introduce a margin-based regularization constraint in the learning process that promotes a better convergence of the model parameters during training. The approach is evaluated using three different domains, which represent sites in different forest biomes. The experimental results show that the approach is successful in the adaptation of most of the domain combination scenarios, usually with considerable gains in relation to the baselines.

A Novel Epileptic Seizure Detection Algorithm Using Transfer K-Plane Clustering

2020 ◽  
Vol 10 (2) ◽  
pp. 484-488
Author(s):  
Lifang Peng ◽  
Bin Huang ◽  
Kefu Chen ◽  
Leyuan Zhou
Keyword(s):  
Transfer Learning ◽  
Epileptic Seizure ◽  
Clustering Algorithms ◽  
Training Data ◽  
Fuzzy Membership ◽  
Data Sets ◽  
Learning Mechanisms ◽  
Target Domain ◽  
Data Set ◽  
Clustering Effect

To recognize epileptic EEG signals, traditional clustering algorithms often need to satisfy three conditions to obtain better clustering results. The first condition is that the algorithm must not be sensitive to noise. The second condition is that the data set must be sufficient. The third condition is that the training data set and the testing data set must follow the same distribution. However, in actual applications, there are few data sets that are free of noise and have sufficient data volume. To address the effects of insufficient data sets and noise on clustering, this paper introduces fuzzy membership and transfer learning mechanisms based on K-plane clustering (KPC) and proposes a fuzzy KPC algorithm based on transfer learning (TFKPC). To improve the clustering effect, the TFKPC algorithm uses the knowledge summarized by the historical domain to guide the clustering process of the current (target) domain when the information is insufficient. In addition, the influence of noise on the clustering result is reduced by introducing fuzzy membership. Experiments show that the TFKPC algorithm proposed in this paper has a better clustering effect in the Epileptic Seizure Recognition Data Set than other comparison methods.

scholarly journals Low Resource Sequence Tagging with Weak Labels

2020 ◽  
Vol 34 (05) ◽  
pp. 8862-8869
Author(s):  
Edwin Simpson ◽  
Jonas Pfeiffer ◽  
Iryna Gurevych
Keyword(s):  
Transfer Learning ◽  
Domain Adaptation ◽  
Ensemble Methods ◽  
Training Data ◽  
Target Domain ◽  
Low Resource ◽  
Standard Data ◽  
High Resource ◽  
Crowdsourced Data ◽  
Resource Data

Current methods for sequence tagging depend on large quantities of domain-specific training data, limiting their use in new, user-defined tasks with few or no annotations. While crowdsourcing can be a cheap source of labels, it often introduces errors that degrade the performance of models trained on such crowdsourced data. Another solution is to use transfer learning to tackle low resource sequence labelling, but current approaches rely heavily on similar high resource datasets in different languages. In this paper, we propose a domain adaptation method using Bayesian sequence combination to exploit pre-trained models and unreliable crowdsourced data that does not require high resource data in a different language. Our method boosts performance by learning the relationship between each labeller and the target task and trains a sequence labeller on the target domain with little or no gold-standard data. We apply our approach to labelling diagnostic classes in medical and educational case studies, showing that the model achieves strong performance though zero-shot transfer learning and is more effective than alternative ensemble methods. Using NER and information extraction tasks, we show how our approach can train a model directly from crowdsourced labels, outperforming pipeline approaches that first aggregate the crowdsourced data, then train on the aggregated labels.

TrCSVM: a novel approach for the classification of melanoma skin cancer using transfer learning

2020 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Lokesh Singh ◽  
Rekh Ram Janghel ◽  
Satya Prakash Sahu
Keyword(s):  
Support Vector Machine ◽  
Skin Lesion ◽  
Transfer Learning ◽  
Domain Adaptation ◽  
Training Data ◽  
Support Vector ◽  
Second Phase ◽  
Target Domain ◽  
Content Type

PurposeThe study aims to cope with the problems confronted in the skin lesion datasets with less training data toward the classification of melanoma. The vital, challenging issue is the insufficiency of training data that occurred while classifying the lesions as melanoma and non-melanoma.Design/methodology/approachIn this work, a transfer learning (TL) framework Transfer Constituent Support Vector Machine (TrCSVM) is designed for melanoma classification based on feature-based domain adaptation (FBDA) leveraging the support vector machine (SVM) and Transfer AdaBoost (TrAdaBoost). The working of the framework is twofold: at first, SVM is utilized for domain adaptation for learning much transferrable representation between source and target domain. In the first phase, for homogeneous domain adaptation, it augments features by transforming the data from source and target (different but related) domains in a shared-subspace. In the second phase, for heterogeneous domain adaptation, it leverages knowledge by augmenting features from source to target (different and not related) domains to a shared-subspace. Second, TrAdaBoost is utilized to adjust the weights of wrongly classified data in the newly generated source and target datasets.FindingsThe experimental results empirically prove the superiority of TrCSVM than the state-of-the-art TL methods on less-sized datasets with an accuracy of 98.82%.Originality/valueExperiments are conducted on six skin lesion datasets and performance is compared based on accuracy, precision, sensitivity, and specificity. The effectiveness of TrCSVM is evaluated on ten other datasets towards testing its generalizing behavior. Its performance is also compared with two existing TL frameworks (TrResampling, TrAdaBoost) for the classification of melanoma.

scholarly journals UNSUPERVISED DOMAIN ADAPTATION USING A TEACHER-STUDENT NETWORK FOR CROSS-CITY CLASSIFICATION OF SENTINEL-2 IMAGES

2020 ◽  
Vol XLIII-B2-2020 ◽  
pp. 1569-1574
Author(s):  
J. Hu ◽  
L. Mou ◽  
X. X. Zhu
Keyword(s):  
Remote Sensing ◽  
Domain Adaptation ◽  
Training Data ◽  
Target Domain ◽  
Source Domain ◽  
Unsupervised Domain Adaptation ◽  
Teacher Student ◽  
The Mean ◽  
The City ◽  
Teacher Model

Abstract. A machine learning algorithm in remote sensing often fails in the inference of a data set which has a different geographic location than the training data. This is because data of different locations have different underlying distributions caused by complicated reasons, such as the climate and the culture. For a large scale or a global scale task, this issue becomes relevant since it is extremely expensive to collect training data over all regions of interest. Unsupervised domain adaptation is a potential solution for this issue. Its goal is to train an algorithm in a source domain and generalize it to a target domain without using any label from the target domain. Those domains can be associated to geographic locations in remote sensing. In this paper, we attempt to adapt the unsupervised domain adaptation strategy by using a teacher-student network, mean teacher model, to investigate a cross-city classification problem in remote sensing. The mean teacher model consists of two identical networks, a teacher network and a student network. The objective function is a combination of a classification loss and a consistent loss. The classification loss works within the source domain (a city) and aims at accomplishing the goal of classification. The consistent loss works within the target domain (another city) and aims at transferring the knowledge learned from the source to the target. In this paper, two cross-city scenarios are set up. First, we train the model with the data of the city Munich, Germany, and test it on the data of the city Moscow, Russia. The second one is carried out by switching the training and testing data. For comparison, the baseline algorithm is a ResNet-18 which is also chosen as the backbone for the teacher and student networks in the mean teacher model. With 10 independent runs, in the first scenario, the mean teacher model has a mean overall accuracy of 53.38% which is slightly higher than the mean overall accuracy of the baseline, 52.21%. However, in the second scenario, the mean teacher model has a mean overall accuracy of 62.71% which is 5% higher than the mean overall accuracy of the baseline, 57.76%. This work demonstrates that it is worthy to explore the potential of the mean teacher model to solve the domain adaptation issues in remote sensing.

Sign in / Sign up

Export Citation Format

Share Document