scholarly journals Adversarial domain adaptation for cross data source macromolecule in situ structural classification in cellular electron cryo-tomograms

2019 ◽  
Vol 35 (14) ◽  
pp. i260-i268 ◽  
Author(s):  
Ruogu Lin ◽  
Xiangrui Zeng ◽  
Kris Kitani ◽  
Min Xu
Keyword(s):  
Deep Learning ◽  
Domain Adaptation ◽  
Training Data ◽  
Supplementary Information ◽  
Target Domain ◽  
Structural Classification ◽  
Feature Extractor ◽  
Data Source ◽  
The Cross

Abstract Motivation Since 2017, an increasing amount of attention has been paid to the supervised deep learning-based macromolecule in situ structural classification (i.e. subtomogram classification) in cellular electron cryo-tomography (CECT) due to the substantially higher scalability of deep learning. However, the success of such supervised approach relies heavily on the availability of large amounts of labeled training data. For CECT, creating valid training data from the same data source as prediction data is usually laborious and computationally intensive. It would be beneficial to have training data from a separate data source where the annotation is readily available or can be performed in a high-throughput fashion. However, the cross data source prediction is often biased due to the different image intensity distributions (a.k.a. domain shift). Results We adapt a deep learning-based adversarial domain adaptation (3D-ADA) method to timely address the domain shift problem in CECT data analysis. 3D-ADA first uses a source domain feature extractor to extract discriminative features from the training data as the input to a classifier. Then it adversarially trains a target domain feature extractor to reduce the distribution differences of the extracted features between training and prediction data. As a result, the same classifier can be directly applied to the prediction data. We tested 3D-ADA on both experimental and realistically simulated subtomogram datasets under different imaging conditions. 3D-ADA stably improved the cross data source prediction, as well as outperformed two popular domain adaptation methods. Furthermore, we demonstrate that 3D-ADA can improve cross data source recovery of novel macromolecular structures. Availability and implementation https://github.com/xulabs/projects Supplementary information Supplementary data are available at Bioinformatics online.

scholarly journals Few shot domain adaptation for in situ macromolecule structural classification in cryoelectron tomograms

2020 ◽  
Author(s):  
Liangyong Yu ◽  
Ran Li ◽  
Xiangrui Zeng ◽  
Hongyi Wang ◽  
Jie Jin ◽  
...  
Keyword(s):  
Deep Learning ◽  
Large Scale ◽  
Spatial Organization ◽  
Domain Adaptation ◽  
Single Cells ◽  
Supplementary Information ◽  
Target Domain ◽  
Source Domain ◽  
Cellular Processes ◽  
Cross Domain

Abstract Motivation Cryoelectron tomography (cryo-ET) visualizes structure and spatial organization of macromolecules and their interactions with other subcellular components inside single cells in the close-to-native state at submolecular resolution. Such information is critical for the accurate understanding of cellular processes. However, subtomogram classification remains one of the major challenges for the systematic recognition and recovery of the macromolecule structures in cryo-ET because of imaging limits and data quantity. Recently, deep learning has significantly improved the throughput and accuracy of large-scale subtomogram classification. However, often it is difficult to get enough high-quality annotated subtomogram data for supervised training due to the enormous expense of labeling. To tackle this problem, it is beneficial to utilize another already annotated dataset to assist the training process. However, due to the discrepancy of image intensity distribution between source domain and target domain, the model trained on subtomograms in source domain may perform poorly in predicting subtomogram classes in the target domain. Results In this article, we adapt a few shot domain adaptation method for deep learning-based cross-domain subtomogram classification. The essential idea of our method consists of two parts: (i) take full advantage of the distribution of plentiful unlabeled target domain data, and (ii) exploit the correlation between the whole source domain dataset and few labeled target domain data. Experiments conducted on simulated and real datasets show that our method achieves significant improvement on cross domain subtomogram classification compared with baseline methods. Availability and implementation Software is available online https://github.com/xulabs/aitom. Supplementary information Supplementary data are available at Bioinformatics online.

scholarly journals Cross-Machine Fault Diagnosis with Semi-Supervised Discriminative Adversarial Domain Adaptation

Sensors ◽  
2020 ◽  
Vol 20 (13) ◽  
pp. 3753
Author(s):  
Xiaodong Wang ◽  
Feng Liu ◽  
Dongdong Zhao
Keyword(s):  
Fault Diagnosis ◽  
Working Conditions ◽  
Domain Adaptation ◽  
Rotating Machinery ◽  
Working Environment ◽  
Training Data ◽  
Target Domain ◽  
Machine Fault Diagnosis ◽  
Machine Fault ◽  
The Cross

Bearings are ubiquitous in rotating machinery and bearings in good working conditions are essential for the availability and safety of the machine. Various intelligent fault diagnosis models have been widely studied aiming to prevent system failures. These data-driven fault diagnosis models work well when training data and testing data are from the same distribution, which is not easy to sustain in industry since the working environment of rotating machinery is often subject to change. Recently, the domain adaptation methods for fault diagnosis between different working conditions have been extensively researched, which fully utilize the labeled data from the same machine under different working conditions to address this domain shift diploma. However, for a target machine with seldom occurred faulty data under any working conditions, the domain adaptation approaches between working conditions are not applicable. Hence, the cross-machine fault diagnosis tasks are recently proposed to utilize the labeled data from related but not identical machines. The larger domain shift between machines makes the cross-machine fault diagnosis a more challenging task. The large domain shift may cause the well-trained model on source domain deteriorates on target domain, and the ambiguous samples near the decision boundary are prone to be misclassified. In addition, the sparse faulty samples in target domain make a class-imbalanced scenario. To address the two issues, in this paper we propose a semi-supervised adversarial domain adaptation approach for cross-machine fault diagnosis which incorporates the virtual adversarial training and batch nuclear-norm maximization to make the fault diagnosis robust and discriminative. Experiments of transferring between three bearing datasets show that the proposed method is able to effectively learn a discriminative model given only a labeled faulty sample of each class in target domain. The research provides a feasible approach for knowledge transfer in fault diagnosis scenarios.

scholarly journals A Method for Vehicle Detection in High-Resolution Satellite Images that Uses a Region-Based Object Detector and Unsupervised Domain Adaptation

2020 ◽  
Vol 12 (3) ◽  
pp. 575 ◽  
Author(s):  
Yohei Koga ◽  
Hiroyuki Miyazaki ◽  
Ryosuke Shibasaki
Keyword(s):  
Deep Learning ◽  
Domain Adaptation ◽  
Vehicle Detection ◽  
Detection Performance ◽  
Training Data ◽  
Detection Accuracy ◽  
Target Area ◽  
Target Domain ◽  
Unsupervised Domain Adaptation ◽  
High Resolution Satellite Images

Recently, object detectors based on deep learning have become widely used for vehicle detection and contributed to drastic improvement in performance measures. However, deep learning requires much training data, and detection performance notably degrades when the target area of vehicle detection (the target domain) is different from the training data (the source domain). To address this problem, we propose an unsupervised domain adaptation (DA) method that does not require labeled training data, and thus can maintain detection performance in the target domain at a low cost. We applied Correlation alignment (CORAL) DA and adversarial DA to our region-based vehicle detector and improved the detection accuracy by over 10% in the target domain. We further improved adversarial DA by utilizing the reconstruction loss to facilitate learning semantic features. Our proposed method achieved slightly better performance than the accuracy achieved with the labeled training data of the target domain. We demonstrated that our improved DA method could achieve almost the same level of accuracy at a lower cost than non-DA methods with a sufficient amount of labeled training data of the target domain.

An intelligent fault diagnosis method based on domain adaptation for rolling bearings under variable load conditions

2021 ◽  
pp. 095440622110329
Author(s):  
Jianqun Zhang ◽  
Qing Zhang ◽  
Xianrong Qin ◽  
Yuantao Sun
Keyword(s):  
Feature Extraction ◽  
Fault Diagnosis ◽  
Domain Adaptation ◽  
Rolling Bearing ◽  
Training Data ◽  
Variable Load ◽  
K Nearest Neighbor ◽  
Target Domain ◽  
Bearing Faults ◽  
Load Conditions

To identify rolling bearing faults under variable load conditions, a method named DISA-KNN is proposed in this paper, which is based on the strategy of feature extraction-domain adaptation-classification. To be specific, the time-domain and frequency-domain indicators are used for feature extraction. Discriminative and domain invariant subspace alignment (DISA) is used to minimize the data distributions’ discrepancies between the training data (source domain) and testing data (target domain). K-nearest neighbor (KNN) is applied to identify rolling bearing faults. DISA-KNN’s validation is proved by the experimental signal collected under different load conditions. The identification accuracies obtained by the DISA-KNN method are more than 90% on four datasets, including one dataset with 99.5% accuracy. The strength of the proposed method is further highlighted by comparisons with the other 8 methods. These results reveal that the proposed method is promising for the rolling bearing fault diagnosis in real rotating machinery.

scholarly journals DeepIsoFun: a deep domain adaptation approach to predict isoform functions

2018 ◽  
Vol 35 (15) ◽  
pp. 2535-2544 ◽  
Author(s):  
Dipan Shaw ◽  
Hao Chen ◽  
Tao Jiang
Keyword(s):  
Network Architecture ◽  
Domain Adaptation ◽  
Multiple Instance Learning ◽  
Training Data ◽  
Supplementary Information ◽  
Operating Characteristics ◽  
Neural Network Architecture ◽  
Receiver Operating Characteristics Curve ◽  
Gene Functions ◽  
Better Than

AbstractMotivationIsoforms are mRNAs produced from the same gene locus by alternative splicing and may have different functions. Although gene functions have been studied extensively, little is known about the specific functions of isoforms. Recently, some computational approaches based on multiple instance learning have been proposed to predict isoform functions from annotated gene functions and expression data, but their performance is far from being desirable primarily due to the lack of labeled training data. To improve the performance on this problem, we propose a novel deep learning method, DeepIsoFun, that combines multiple instance learning with domain adaptation. The latter technique helps to transfer the knowledge of gene functions to the prediction of isoform functions and provides additional labeled training data. Our model is trained on a deep neural network architecture so that it can adapt to different expression distributions associated with different gene ontology terms.ResultsWe evaluated the performance of DeepIsoFun on three expression datasets of human and mouse collected from SRA studies at different times. On each dataset, DeepIsoFun performed significantly better than the existing methods. In terms of area under the receiver operating characteristics curve, our method acquired at least 26% improvement and in terms of area under the precision-recall curve, it acquired at least 10% improvement over the state-of-the-art methods. In addition, we also study the divergence of the functions predicted by our method for isoforms from the same gene and the overall correlation between expression similarity and the similarity of predicted functions.Availability and implementationhttps://github.com/dls03/DeepIsoFun/Supplementary informationSupplementary data are available at Bioinformatics online.

scholarly journals USING SEMANTICALLY PAIRED IMAGES TO IMPROVE DOMAIN ADAPTATION FOR THE SEMANTIC SEGMENTATION OF AERIAL IMAGES

2020 ◽  
Vol V-2-2020 ◽  
pp. 483-492
Author(s):  
D. Gritzner ◽  
J. Ostermann
Keyword(s):  
Time Window ◽  
Domain Adaptation ◽  
Geographical Area ◽  
Model Performance ◽  
Ground Truth ◽  
Semantic Segmentation ◽  
Training Data ◽  
Aerial Images ◽  
Target Domain ◽  
Training Examples

Abstract. Modern machine learning, especially deep learning, which is used in a variety of applications, requires a lot of labelled data for model training. Having an insufficient amount of training examples leads to models which do not generalize well to new input instances. This is a particular significant problem for tasks involving aerial images: often training data is only available for a limited geographical area and a narrow time window, thus leading to models which perform poorly in different regions, at different times of day, or during different seasons. Domain adaptation can mitigate this issue by using labelled source domain training examples and unlabeled target domain images to train a model which performs well on both domains. Modern adversarial domain adaptation approaches use unpaired data. We propose using pairs of semantically similar images, i.e., whose segmentations are accurate predictions of each other, for improved model performance. In this paper we show that, as an upper limit based on ground truth, using semantically paired aerial images during training almost always increases model performance with an average improvement of 4.2% accuracy and .036 mean intersection-over-union (mIoU). Using a practical estimate of semantic similarity, we still achieve improvements in more than half of all cases, with average improvements of 2.5% accuracy and .017 mIoU in those cases.

scholarly journals Multi-Source Distilling Domain Adaptation

2020 ◽  
Vol 34 (07) ◽  
pp. 12975-12983
Author(s):  
Sicheng Zhao ◽  
Guangzhi Wang ◽  
Shanghang Zhang ◽  
Yang Gu ◽  
Yaxian Li ◽  
...  
Keyword(s):  
Domain Adaptation ◽  
Feature Space ◽  
Wasserstein Distance ◽  
Training Data ◽  
Source Distribution ◽  
Single Source ◽  
Multiple Sources ◽  
Target Domain ◽  
Target Feature ◽  
Training Samples

Deep neural networks suffer from performance decay when there is domain shift between the labeled source domain and unlabeled target domain, which motivates the research on domain adaptation (DA). Conventional DA methods usually assume that the labeled data is sampled from a single source distribution. However, in practice, labeled data may be collected from multiple sources, while naive application of the single-source DA algorithms may lead to suboptimal solutions. In this paper, we propose a novel multi-source distilling domain adaptation (MDDA) network, which not only considers the different distances among multiple sources and the target, but also investigates the different similarities of the source samples to the target ones. Specifically, the proposed MDDA includes four stages: (1) pre-train the source classifiers separately using the training data from each source; (2) adversarially map the target into the feature space of each source respectively by minimizing the empirical Wasserstein distance between source and target; (3) select the source training samples that are closer to the target to fine-tune the source classifiers; and (4) classify each encoded target feature by corresponding source classifier, and aggregate different predictions using respective domain weight, which corresponds to the discrepancy between each source and target. Extensive experiments are conducted on public DA benchmarks, and the results demonstrate that the proposed MDDA significantly outperforms the state-of-the-art approaches. Our source code is released at: https://github.com/daoyuan98/MDDA.

scholarly journals Comprehensive evaluation of deep learning architectures for prediction of DNA/RNA sequence binding specificities

2019 ◽  
Vol 35 (14) ◽  
pp. i269-i277 ◽  
Author(s):  
Ameni Trabelsi ◽  
Mohamed Chaabane ◽  
Asa Ben-Hur
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Rna Binding ◽  
Binding Specificity ◽  
Training Data ◽  
Supplementary Information ◽  
Recurrent Networks ◽  
Dna And Rna ◽  
Learning Architectures ◽  
Dna And Rna Binding

Abstract Motivation Deep learning architectures have recently demonstrated their power in predicting DNA- and RNA-binding specificity. Existing methods fall into three classes: Some are based on convolutional neural networks (CNNs), others use recurrent neural networks (RNNs) and others rely on hybrid architectures combining CNNs and RNNs. However, based on existing studies the relative merit of the various architectures remains unclear. Results In this study we present a systematic exploration of deep learning architectures for predicting DNA- and RNA-binding specificity. For this purpose, we present deepRAM, an end-to-end deep learning tool that provides an implementation of a wide selection of architectures; its fully automatic model selection procedure allows us to perform a fair and unbiased comparison of deep learning architectures. We find that deeper more complex architectures provide a clear advantage with sufficient training data, and that hybrid CNN/RNN architectures outperform other methods in terms of accuracy. Our work provides guidelines that can assist the practitioner in choosing an appropriate network architecture, and provides insight on the difference between the models learned by convolutional and recurrent networks. In particular, we find that although recurrent networks improve model accuracy, this comes at the expense of a loss in the interpretability of the features learned by the model. Availability and implementation The source code for deepRAM is available at https://github.com/MedChaabane/deepRAM. Supplementary information Supplementary data are available at Bioinformatics online.

scholarly journals Distributional Correspondence Indexing for Cross-Lingual and Cross-Domain Sentiment Classification (Extended Abstract)

2018 ◽  
Author(s):  
Alejandro Moreo Fernández ◽  
Andrea Esuli ◽  
Fabrizio Sebastiani
Keyword(s):  
Domain Adaptation ◽  
State Of The Art ◽  
Sentiment Classification ◽  
Training Data ◽  
Target Domain ◽  
Source Domain ◽  
Machine Learning Methods ◽  
Cross Domain ◽  
Current State ◽  
Cross Lingual

Domain Adaptation (DA) techniques aim at enabling machine learning methods learn effective classifiers for a “target” domain when the only available training data belongs to a different “source” domain. In this extended abstract, we briefly describe our new DA method called Distributional Correspondence Indexing (DCI) for sentiment classification. DCI derives term representations in a vector space common to both domains where each dimension reflects its distributional correspondence to a pivot, i.e., to a highly predictive term that behaves similarly across domains. The experiments we have conducted show that DCI obtains better performance than current state-of-the-art techniques for cross-lingual and cross-domain sentiment classification.

scholarly journals ITERATIVE RE-WEIGHTED INSTANCE TRANSFER FOR DOMAIN ADAPTATION

2016 ◽  
Vol III-3 ◽  
pp. 339-346 ◽  
Author(s):  
A. Paul ◽  
F. Rottensteiner ◽  
C. Heipke
Keyword(s):  
Domain Adaptation ◽  
Training Data ◽  
Aerial Images ◽  
Target Domain ◽  
Data Set ◽  
Source Domain ◽  
The Difference ◽  
A New Technique ◽  
Class Labels ◽  
Decision Boundaries

Domain adaptation techniques in transfer learning try to reduce the amount of training data required for classification by adapting a classifier trained on samples from a source domain to a new data set (target domain) where the features may have different distributions. In this paper, we propose a new technique for domain adaptation based on logistic regression. Starting with a classifier trained on training data from the source domain, we iteratively include target domain samples for which class labels have been obtained from the current state of the classifier, while at the same time removing source domain samples. In each iteration the classifier is re-trained, so that the decision boundaries are slowly transferred to the distribution of the target features. To make the transfer procedure more robust we introduce weights as a function of distance from the decision boundary and a new way of regularisation. Our methodology is evaluated using a benchmark data set consisting of aerial images and digital surface models. The experimental results show that in the majority of cases our domain adaptation approach can lead to an improvement of the classification accuracy without additional training data, but also indicate remaining problems if the difference in the feature distributions becomes too large.

Sign in / Sign up

Export Citation Format

Share Document