scholarly journals Deep learning framework for material design space exploration using active transfer learning and data augmentation

2021 ◽  
Vol 7 (1) ◽  
Author(s):  
Yongtae Kim ◽  
Youngsoo Kim ◽  
Charles Yang ◽  
Kundo Park ◽  
Grace X. Gu ◽  
...  
Keyword(s):  
Neural Network ◽  
Transfer Learning ◽  
Design Space Exploration ◽  
Predictive Power ◽  
Design Space ◽  
Data Augmentation ◽  
Generative Models ◽  
Training Dataset ◽  
Initial Training ◽  
Active Transfer

AbstractNeural network-based generative models have been actively investigated as an inverse design method for finding novel materials in a vast design space. However, the applicability of conventional generative models is limited because they cannot access data outside the range of training sets. Advanced generative models that were devised to overcome the limitation also suffer from the weak predictive power on the unseen domain. In this study, we propose a deep neural network-based forward design approach that enables an efficient search for superior materials far beyond the domain of the initial training set. This approach compensates for the weak predictive power of neural networks on an unseen domain through gradual updates of the neural network with active transfer learning and data augmentation methods. We demonstrate the potential of our framework with a grid composite optimization problem that has an astronomical number of possible design configurations. Results show that our proposed framework can provide excellent designs close to the global optima, even with the addition of a very small dataset corresponding to less than 0.5% of the initial training dataset size.

scholarly journals Deep Learning Framework for Material Design Space Exploration using Active Transfer Learning and Data Augmentation

2021 ◽  
Author(s):  
Yongtae Kim ◽  
Youngsoo Kim ◽  
Charles Yang ◽  
Kundo Park ◽  
Grace Gu ◽  
...  
Keyword(s):  
Neural Network ◽  
Transfer Learning ◽  
Design Space Exploration ◽  
Predictive Power ◽  
Design Space ◽  
Data Augmentation ◽  
Generative Models ◽  
Training Dataset ◽  
Initial Training ◽  
Active Transfer

Neural network-based generative models have been actively investigated as an inverse design method for finding novel materials in a vast design space. However, the applicability of conventional generative models is limited because they cannot access data outside the range of training sets. Advanced generative models that were devised to overcome the limitation also suffer from the weak predictive power on the unseen domain. In this study, we propose a deep neural network-based forward design approach that enables an efficient search for the superior materials far beyond the domain of the initial training set. This approach compensates for the weak predictive power of neural networks on an unseen domain through gradual updates of the neural network with active-transfer learning and data augmentation methods. We demonstrate the potential of our framework with a grid composite optimization problem that has an astronomical number of possible design configurations. Results show that our proposed framework can provide excellent designs close to the global optima, even with the addition of very small dataset corresponding to less than 0.5% of the initial training dataset size.

scholarly journals Ship Classification in High-Resolution SAR Images via Transfer Learning with Small Training Dataset

Sensors ◽  
2018 ◽  
Vol 19 (1) ◽  
pp. 63 ◽  
Author(s):  
Changchong Lu ◽  
Weihai Li
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Transfer Learning ◽  
Data Augmentation ◽  
Synthetic Aperture ◽  
Training Dataset ◽  
Classification Problems ◽  
Sar Images ◽  
Ship Classification ◽  
Small Dataset

Synthetic aperture radar (SAR) as an all-weather method of the remote sensing, now it has been an important tool in oceanographic observations, object tracking, etc. Due to advances in neural networks (NN), researchers started to study SAR ship classification problems with deep learning (DL) in recent years. However, the limited labeled SAR ship data become a bottleneck to train a neural network. In this paper, convolutional neural networks (CNNs) are applied to ship classification by using SAR images with the small datasets. To solve the problem of over-fitting which often appeared in training small dataset, we proposed a new method of data augmentation and combined it with transfer learning. Based on experiments and tests, the performance is evaluated. The results show that the types of the ships can be classified in high accuracies and reveal the effectiveness of our proposed method.

Molecular Generation Targeting Desired Electronic Properties via Deep Generative Models

2019 ◽  
Author(s):  
Qi Yuan ◽  
Alejandro Santana-Bonilla ◽  
Martijn Zwijnenburg ◽  
Kim Jelfs
Keyword(s):  
Neural Network ◽  
Electronic Properties ◽  
Transfer Learning ◽  
Recurrent Neural Network ◽  
Chemical Space ◽  
Generative Models ◽  
Molecular Features ◽  
Donor Acceptor ◽  
Homo Lumo ◽  
Training Sets

<p>The chemical space for novel electronic donor-acceptor oligomers with targeted properties was explored using deep generative models and transfer learning. A General Recurrent Neural Network model was trained from the ChEMBL database to generate chemically valid SMILES strings. The parameters of the General Recurrent Neural Network were fine-tuned via transfer learning using the electronic donor-acceptor database from the Computational Material Repository to generate novel donor-acceptor oligomers. Six different transfer learning models were developed with different subsets of the donor-acceptor database as training sets. We concluded that electronic properties such as HOMO-LUMO gaps and dipole moments of the training sets can be learned using the SMILES representation with deep generative models, and that the chemical space of the training sets can be efficiently explored. This approach identified approximately 1700 new molecules that have promising electronic properties (HOMO-LUMO gap <2 eV and dipole moment <2 Debye), 6-times more than in the original database. Amongst the molecular transformations, the deep generative model has learned how to produce novel molecules by trading off between selected atomic substitutions (such as halogenation or methylation) and molecular features such as the spatial extension of the oligomer. The method can be extended as a plausible source of new chemical combinations to effectively explore the chemical space for targeted properties.</p>

scholarly journals Effectiveness of transfer learning for enhancing tumor classification with a convolutional neural network on frozen sections

2020 ◽  
Vol 10 (1) ◽  
Author(s):  
Young-Gon Kim ◽  
Sungchul Kim ◽  
Cristina Eunbee Cho ◽  
In Hye Song ◽  
Hee Jin Lee ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Transfer Learning ◽  
Frozen Section ◽  
Medical Center ◽  
External Validation ◽  
Model Performance ◽  
Classification Model ◽  
Training Dataset

AbstractFast and accurate confirmation of metastasis on the frozen tissue section of intraoperative sentinel lymph node biopsy is an essential tool for critical surgical decisions. However, accurate diagnosis by pathologists is difficult within the time limitations. Training a robust and accurate deep learning model is also difficult owing to the limited number of frozen datasets with high quality labels. To overcome these issues, we validated the effectiveness of transfer learning from CAMELYON16 to improve performance of the convolutional neural network (CNN)-based classification model on our frozen dataset (N = 297) from Asan Medical Center (AMC). Among the 297 whole slide images (WSIs), 157 and 40 WSIs were used to train deep learning models with different dataset ratios at 2, 4, 8, 20, 40, and 100%. The remaining, i.e., 100 WSIs, were used to validate model performance in terms of patch- and slide-level classification. An additional 228 WSIs from Seoul National University Bundang Hospital (SNUBH) were used as an external validation. Three initial weights, i.e., scratch-based (random initialization), ImageNet-based, and CAMELYON16-based models were used to validate their effectiveness in external validation. In the patch-level classification results on the AMC dataset, CAMELYON16-based models trained with a small dataset (up to 40%, i.e., 62 WSIs) showed a significantly higher area under the curve (AUC) of 0.929 than those of the scratch- and ImageNet-based models at 0.897 and 0.919, respectively, while CAMELYON16-based and ImageNet-based models trained with 100% of the training dataset showed comparable AUCs at 0.944 and 0.943, respectively. For the external validation, CAMELYON16-based models showed higher AUCs than those of the scratch- and ImageNet-based models. Model performance for slide feasibility of the transfer learning to enhance model performance was validated in the case of frozen section datasets with limited numbers.

scholarly journals Voiceprint Identification for Limited Dataset Using the Deep Migration Hybrid Model Based on Transfer Learning

Sensors ◽  
2018 ◽  
Vol 18 (7) ◽  
pp. 2399 ◽  
Author(s):  
Cunwei Sun ◽  
Yuxin Yang ◽  
Chang Wen ◽  
Kai Xie ◽  
Fangqing Wen
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Transfer Learning ◽  
Hybrid Model ◽  
Data Augmentation ◽  
Small Sample ◽  
Restricted Boltzmann Machine ◽  
Small Samples ◽  
Boltzmann Machine ◽  
Training Time

The convolutional neural network (CNN) has made great strides in the area of voiceprint recognition; but it needs a huge number of data samples to train a deep neural network. In practice, it is too difficult to get a large number of training samples, and it cannot achieve a better convergence state due to the limited dataset. In order to solve this question, a new method using a deep migration hybrid model is put forward, which makes it easier to realize voiceprint recognition for small samples. Firstly, it uses Transfer Learning to transfer the trained network from the big sample voiceprint dataset to our limited voiceprint dataset for the further training. Fully-connected layers of a pre-training model are replaced by restricted Boltzmann machine layers. Secondly, the approach of Data Augmentation is adopted to increase the number of voiceprint datasets. Finally, we introduce fast batch normalization algorithms to improve the speed of the network convergence and shorten the training time. Our new voiceprint recognition approach uses the TLCNN-RBM (convolutional neural network mixed restricted Boltzmann machine based on transfer learning) model, which is the deep migration hybrid model that is used to achieve an average accuracy of over 97%, which is higher than that when using either CNN or the TL-CNN network (convolutional neural network based on transfer learning). Thus, an effective method for a small sample of voiceprint recognition has been provided.

scholarly journals Design Space Exploration of Neural Network Activation Function Circuits

2019 ◽  
Vol 38 (10) ◽  
pp. 1974-1978 ◽  
Author(s):  
Tao Yang ◽  
Yadong Wei ◽  
Zhijun Tu ◽  
Haolun Zeng ◽  
Michel A. Kinsy ◽  
...  
Keyword(s):  
Neural Network ◽  
Design Space Exploration ◽  
Design Space ◽  
Space Exploration ◽  
Activation Function ◽  
Network Activation

Tomato pest classification using deep convolutional neural network with transfer learning, fine tuning and scratch learning

2021 ◽  
pp. 1-10
Author(s):  
Gayatri Pattnaik ◽  
Vimal K. Shrivastava ◽  
K. Parvathi
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Transfer Learning ◽  
Data Augmentation ◽  
State Of The Art ◽  
Deep Convolutional Neural Network ◽  
Fine Tuning ◽  
Tomato Plants ◽  
Random Weights

Pests are major threat to economic growth of a country. Application of pesticide is the easiest way to control the pest infection. However, excessive utilization of pesticide is hazardous to environment. The recent advances in deep learning have paved the way for early detection and improved classification of pest in tomato plants which will benefit the farmers. This paper presents a comprehensive analysis of 11 state-of-the-art deep convolutional neural network (CNN) models with three configurations: transfers learning, fine-tuning and scratch learning. The training in transfer learning and fine tuning initiates from pre-trained weights whereas random weights are used in case of scratch learning. In addition, the concept of data augmentation has been explored to improve the performance. Our dataset consists of 859 tomato pest images from 10 categories. The results demonstrate that the highest classification accuracy of 94.87% has been achieved in the transfer learning approach by DenseNet201 model with data augmentation.

Sign in / Sign up

Export Citation Format

Share Document