Materials Representation and Transfer Learning for Multi-Property Prediction

The adoption of machine learning in materials science has rapidly transformed materials property prediction. Hurdles limiting full capitalization of recent advancements in machine learning include the limited development of methods to learn the underlying interactions of multiple elements, as well as the relationships among multiple properties, to facilitate property prediction in new composition spaces. To address these issues, we introduce the Hierarchical Correlation Learning for Multi-property Prediction (H-CLMP) framework that seamlessly integrates (i) prediction using only a material’s composition, (ii) learning and exploitation of correlations among target properties in multitarget regression, and (iii) leveraging training data from tangential domains via generative transfer learning. The model is demonstrated for prediction of spectral optical absorption of complex metal oxides spanning 69 3-cation metal oxide composition spaces. H-CLMP accurately predicts non-linear composition-property relationships in composition spaces for which no training data is available, which broadens the purview of machine learning to the discovery of materials with exceptional properties. This achievement results from the principled integration of latent embedding learning, property correlation learning, generative transfer learning, and attention models. The best performance is obtained using H-CLMP with Transfer learning (H-CLMP(T)) wherein a generative adversarial network is trained on computational density of states data and deployed in the target domain to augment prediction of optical absorption from composition. H-CLMP(T) aggregates multiple knowledge sources with a framework that is well-suited for multi-target regression across the physical sciences.

Download Full-text

Materials Representation and Transfer Learning for Multi-Property Prediction

10.26434/chemrxiv.14612307 ◽

2021 ◽

Author(s):

Shufeng Kong ◽

Dan Guevarra ◽

Carla P. Gomes ◽

John Gregoire

Keyword(s):

Machine Learning ◽

Optical Absorption ◽

Transfer Learning ◽

Materials Science ◽

Training Data ◽

Target Domain ◽

Generative Adversarial Network ◽

Property Prediction ◽

Adversarial Network ◽

Correlation Learning

Download Full-text

Autoencoder-based transfer learning in brain–computer interface for rehabilitation robot

International Journal of Advanced Robotic Systems ◽

10.1177/1729881419840860 ◽

2019 ◽

Vol 16 (2) ◽

pp. 172988141984086 ◽

Cited By ~ 4

Author(s):

Chuanqi Tan ◽

Fuchun Sun ◽

Bin Fang ◽

Tao Kong ◽

Wenchang Zhang

Keyword(s):

Transfer Learning ◽

Negative Transfer ◽

Brain Computer Interface ◽

Training Data ◽

Computer Interface ◽

Rehabilitation Robot ◽

Target Domain ◽

Source Domain ◽

Adversarial Network ◽

The Brain

The brain–computer interface-based rehabilitation robot has quickly become a very important research area due to its natural interaction. One of the most important problems in brain–computer interface is that large-scale annotated electroencephalography data sets required by advanced classifiers are almost impossible to acquire because biological data acquisition is challenging and quality annotation is costly. Transfer learning relaxes the hypothesis that the training data must be independent and identically distributed with the test data. It can be considered a powerful tool for solving the problem of insufficient training data. There are two basic issues with transfer learning, under transfer and negative transfer. We proposed a novel brain–computer interface framework by using autoencoder-based transfer learning, which includes three main components: an autoencoder framework, a joint adversarial network, and a regularized manifold constraint. The autoencoder framework automatically encodes and reconstructs data from source and target domains and forces the neural network to learn to represent these domains reliably. The joint adversarial network aims to force the network to learn to encode more appropriately for the source domain and target domain simultaneously, thereby overcoming the problem of under transfer. The regularized manifold constraint aims to avoid the problem of negative transfer by avoiding geometric manifold structure in the target domain being destroyed by the source domain. Experiments show that the brain–computer interface framework proposed by us can achieve better results than state-of-the-art approaches in electroencephalography signal classification tasks. This is helpful in aiding our rehabilitation robot to understand the intention of patients and can help patients to carry out rehabilitation exercises effectively.

Download Full-text

Spacecraft Intelligent Fault Diagnosis under Variable Working Conditions via Wasserstein Distance-Based Deep Adversarial Transfer Learning

International Journal of Aerospace Engineering ◽

10.1155/2021/6099818 ◽

2021 ◽

Vol 2021 ◽

pp. 1-16

Author(s):

Gang Xiang ◽

Kun Tian

Keyword(s):

Fault Diagnosis ◽

Transfer Learning ◽

Working Conditions ◽

Manufacturing Industry ◽

Probability Distributions ◽

Wasserstein Distance ◽

Target Domain ◽

Target Feature ◽

Generative Adversarial Network ◽

Adversarial Network

In recent years, deep learning methods which promote the accuracy and efficiency of fault diagnosis task without any extra requirement of artificial feature extraction have elicited the attention of researchers in the field of manufacturing industry as well as aerospace. However, the problems that data in source and target domains usually have different probability distributions because of different working conditions and there are insufficient labeled or even unlabeled data in target domain significantly deteriorate the performance and generalization of deep fault diagnosis models. To address these problems, we propose a novel Wasserstein Generative Adversarial Network with Gradient Penalty- (WGAN-GP-) based deep adversarial transfer learning (WDATL) model in this study, which exploits a domain critic to learn domain invariant feature representations by minimizing the Wasserstein distance between the source and target feature distributions through adversarial training. Moreover, an improved one-dimensional convolutional neural network- (CNN-) based feature extractor which utilizes exponential linear units (ELU) as activation functions and wide kernels is designed to automatically extract the latent features of raw time-series input data. Then, the fault model classifier trained in one working condition (source domain) with sufficient labeled samples could be generalized to diagnose data in other working conditions (target domain) with insufficient labeled samples. Experiments on two open datasets demonstrate that our proposed WDATL model outperforms most of the state-of-the-art approaches on transfer diagnosis tasks under diverse working circumstances.

Download Full-text

A Fault Diagnosis Method for Rolling Bearings Based on Parameter Transfer Learning under Imbalance Data Sets

Energies ◽

10.3390/en14040944 ◽

2021 ◽

Vol 14 (4) ◽

pp. 944

Author(s):

Cheng Peng ◽

Lingling Li ◽

Qing Chen ◽

Zhaohui Tang ◽

Weihua Gui ◽

...

Keyword(s):

Fault Diagnosis ◽

Transfer Learning ◽

Hot Spot ◽

Data Sets ◽

Target Domain ◽

Rolling Bearings ◽

Generative Adversarial Network ◽

Adversarial Network ◽

Recognition Ability ◽

Diagnosis Method

Fault diagnosis under the condition of data sets or samples with only a few fault labels has become a hot spot in the field of machinery fault diagnosis. To solve this problem, a fault diagnosis method based on deep transfer learning is proposed. Firstly, the discriminator of the generative adversarial network (GAN) is improved by enhancing its sparsity, and then adopts the adversarial mechanism to continuously optimize the recognition ability of the discriminator; finally, the parameter transfer learning (PTL) method is applied to transfer the trained discriminator to target domain to solve the fault diagnosis problem with only a small number of label samples. Experimental results show that this method has good fault diagnosis performance.

Download Full-text

PF-TL: Payload Feature-Based Transfer Learning for Dealing with the Lack of Training Data

Electronics ◽

10.3390/electronics10101148 ◽

2021 ◽

Vol 10 (10) ◽

pp. 1148

Author(s):

Ilok Jung ◽

Jongin Lim ◽

Huykang Kim

Keyword(s):

Machine Learning ◽

Intrusion Detection ◽

Transfer Learning ◽

Cyber Security ◽

Optimization Method ◽

Training Data ◽

Target Domain ◽

Feature Extraction Method ◽

Feature Based ◽

Hybrid Feature Extraction

The number of studies on applying machine learning to cyber security has increased over the past few years. These studies, however, are facing difficulties with making themselves usable in the real world, mainly due to the lack of training data and reusability of a created model. While transfer learning seems like a solution to these problems, the number of studies in the field of intrusion detection is still insufficient. Therefore, this study proposes payload feature-based transfer learning as a solution to the lack of training data when applying machine learning to intrusion detection by using the knowledge from an already known domain. Firstly, it expands the extracting range of information from header to payload to accurately deliver the information by using an effective hybrid feature extraction method. Secondly, this study provides an improved optimization method for the extracted features to create a labeled dataset for a target domain. This proposal was validated on publicly available datasets, using three distinctive scenarios, and the results confirmed its usability in practice by increasing the accuracy of the training data created from the transfer learning by 30%, compared to that of the non-transfer learning method. In addition, we showed that this approach can help in identifying previously unknown attacks and reusing models from different domains.

Download Full-text

ORGANIC (1).pdf

10.26434/chemrxiv.5309668.v1 ◽

2017 ◽

Author(s):

Benjamin Sanchez-Lengeling ◽

Carlos Outeiral ◽

Gabriel L. Guimaraes ◽

Alan Aspuru-Guzik

Keyword(s):

Machine Learning ◽

Learning Community ◽

Chemical Species ◽

Material Design ◽

Organic Photovoltaic ◽

Generative Adversarial Networks ◽

Generative Adversarial Network ◽

Adversarial Network ◽

Adversarial Networks ◽

Photovoltaic Material

Molecular discovery seeks to generate chemical species tailored to very specific needs. In this paper, we present ORGANIC, a framework based on Objective-Reinforced Generative Adversarial Networks (ORGAN), capable of producing a distribution over molecular space that matches with a certain set of desirable metrics. This methodology combines two successful techniques from the machine learning community: a Generative Adversarial Network (GAN), to create non-repetitive sensible molecular species, and Reinforcement Learning (RL), to bias this generative distribution towards certain attributes. We explore several applications, from optimization of random physicochemical properties to candidates for drug discovery and organic photovoltaic material design.

Download Full-text

Role of General Adversarial Networks in Mammogram Analysis: A Review

Current Medical Imaging Formerly Current Medical Imaging Reviews ◽

10.2174/1573405614666191115102318 ◽

2020 ◽

Vol 16 (7) ◽

pp. 863-877

Author(s):

Annapoorani Gopal ◽

Lathaselvi Gandhimaruthian ◽

Javid Ali

Keyword(s):

Breast Tumor ◽

Deep Neural Networks ◽

Training Data ◽

Learning Technology ◽

Breast Cancers ◽

Generative Adversarial Network ◽

Adversarial Network ◽

Adversarial Networks ◽

Tumor Extraction

The Deep Neural Networks have gained prominence in the biomedical domain, becoming the most commonly used networks after machine learning technology. Mammograms can be used to detect breast cancers with high precision with the help of Convolutional Neural Network (CNN) which is deep learning technology. An exhaustive labeled data is required to train the CNN from scratch. This can be overcome by deploying Generative Adversarial Network (GAN) which comparatively needs lesser training data during a mammogram screening. In the proposed study, the application of GANs in estimating breast density, high-resolution mammogram synthesis for clustered microcalcification analysis, effective segmentation of breast tumor, analysis of the shape of breast tumor, extraction of features and augmentation of the image during mammogram classification have been extensively reviewed.

Download Full-text

Classification of Space Objects by Using Deep Learning with Micro-Doppler Signature Images

Sensors ◽

10.3390/s21134365 ◽

2021 ◽

Vol 21 (13) ◽

pp. 4365

Author(s):

Kwangyong Jung ◽

Jae-In Lee ◽

Nammoon Kim ◽

Sunjin Oh ◽

Dong-Wook Seo

Keyword(s):

Transfer Learning ◽

Data Augmentation ◽

Doppler Frequency ◽

Extraction Methods ◽

Classification Performance ◽

Reconstruction Method ◽

Generative Adversarial Network ◽

Adversarial Network ◽

Space Objects ◽

Target Characteristics

Radar target classification is an important task in the missile defense system. State-of-the-art studies using micro-doppler frequency have been conducted to classify the space object targets. However, existing studies rely highly on feature extraction methods. Therefore, the generalization performance of the classifier is limited and there is room for improvement. Recently, to improve the classification performance, the popular approaches are to build a convolutional neural network (CNN) architecture with the help of transfer learning and use the generative adversarial network (GAN) to increase the training datasets. However, these methods still have drawbacks. First, they use only one feature to train the network. Therefore, the existing methods cannot guarantee that the classifier learns more robust target characteristics. Second, it is difficult to obtain large amounts of data that accurately mimic real-world target features by performing data augmentation via GAN instead of simulation. To mitigate the above problem, we propose a transfer learning-based parallel network with the spectrogram and the cadence velocity diagram (CVD) as the inputs. In addition, we obtain an EM simulation-based dataset. The radar-received signal is simulated according to a variety of dynamics using the concept of shooting and bouncing rays with relative aspect angles rather than the scattering center reconstruction method. Our proposed model is evaluated on our generated dataset. The proposed method achieved about 0.01 to 0.39% higher accuracy than the pre-trained networks with a single input feature.

Download Full-text

Estimating the phase volume fraction of multi-phase steel via unsupervised deep learning

Scientific Reports ◽

10.1038/s41598-021-85407-y ◽

2021 ◽

Vol 11 (1) ◽

Author(s):

Sung Wook Kim ◽

Seong-Hoon Kang ◽

Se-Jong Kim ◽

Seungchul Lee

Keyword(s):

High Performance ◽

Materials Science ◽

Volume Fraction ◽

Training Data ◽

Phase Fraction ◽

Phase Volume ◽

Generative Adversarial Network ◽

Phase Volume Fraction ◽

Original Dataset ◽

Multi Phase

AbstractAdvanced high strength steel (AHSS) is a steel of multi-phase microstructure that is processed under several conditions to meet the current high-performance requirements from the industry. Deep neural network (DNN) has emerged as a promising tool in materials science for the task of estimating the phase volume fraction of these steels. Despite its advantages, one of its major drawbacks is its requirement of a sufficient amount of training data with correct labels to the network. This often comes as a challenge in many areas where obtaining data and labeling it is extremely labor-intensive. To overcome this challenge, an unsupervised way of learning DNN, which does not require any manual labeling, is proposed. Information maximizing generative adversarial network (InfoGAN) is used to learn the underlying probability distribution of each phase and generate realistic sample points with class labels. Then, the generated data is used for training an MLP classifier, which in turn predicts the labels for the original dataset. The result shows a mean relative error of 4.53% at most, while it can be as low as 0.73%, which implies the estimated phase fraction closely matches the true phase fraction. This presents the high feasibility of using the proposed methodology for fast and precise estimation of phase volume fraction in both industry and academia.

Download Full-text

CrossGR

Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies ◽

10.1145/3448100 ◽

2021 ◽

Vol 5 (1) ◽

pp. 1-23

Author(s):

Xinyi Li ◽

Liqiong Chang ◽

Fangfang Song ◽

Ju Wang ◽

Xiaojiang Chen ◽

...

Keyword(s):

Gesture Recognition ◽

Low Cost ◽

User Involvement ◽

Recognition System ◽

Training Data ◽

Generative Adversarial Network ◽

Adversarial Network ◽

Target User ◽

Order Of Magnitude ◽

Training Examples

This paper focuses on a fundamental question in Wi-Fi-based gesture recognition: "Can we use the knowledge learned from some users to perform gesture recognition for others?". This problem is also known as cross-target recognition. It arises in many practical deployments of Wi-Fi-based gesture recognition where it is prohibitively expensive to collect training data from every single user. We present CrossGR, a low-cost cross-target gesture recognition system. As a departure from existing approaches, CrossGR does not require prior knowledge (such as who is currently performing a gesture) of the target user. Instead, CrossGR employs a deep neural network to extract user-agnostic but gesture-related Wi-Fi signal characteristics to perform gesture recognition. To provide sufficient training data to build an effective deep learning model, CrossGR employs a generative adversarial network to automatically generate many synthetic training data from a small set of real-world examples collected from a small number of users. Such a strategy allows CrossGR to minimize the user involvement and the associated cost in collecting training examples for building an accurate gesture recognition system. We evaluate CrossGR by applying it to perform gesture recognition across 10 users and 15 gestures. Experimental results show that CrossGR achieves an accuracy of over 82.6% (up to 99.75%). We demonstrate that CrossGR delivers comparable recognition accuracy, but uses an order of magnitude less training samples collected from the end-users when compared to state-of-the-art recognition systems.

Download Full-text