The importance of transfer learning in seismic modeling and imaging

Geophysics ◽  
2019 ◽  
Vol 84 (6) ◽  
pp. A47-A52 ◽  
Author(s):  
Ali Siahkoohi ◽  
Mathias Louboutin ◽  
Felix J. Herrmann
Keyword(s):  
Neural Network ◽  
Transfer Learning ◽  
Low Cost ◽  
Fine Tuning ◽  
Numerical Dispersion ◽  
High Fidelity ◽  
Velocity Models ◽  
Current Survey ◽  
Exploration Seismology ◽  
Fine Tune

Accurate forward modeling is essential for solving inverse problems in exploration seismology. Unfortunately, it is often not possible to afford being physically or numerically accurate. To overcome this conundrum, we make use of raw and processed data from nearby surveys. We have used these data, consisting of shot records or velocity models, to pretrain a neural network to correct for the effects of, for instance, the free surface or numerical dispersion, both of which can be considered as proxies for incomplete or inaccurate physics. Given this pretrained neural network, we apply transfer learning to fine-tune this pretrained neural network so it performs well on its task of mapping low-cost, but low-fidelity, solutions to high-fidelity solutions for the current survey. As long as we can limit ourselves during fine-tuning to using only a small fraction of high-fidelity data, we gain processing the current survey while using information from nearby surveys. We examined this principle by removing surface-related multiples and ghosts from shot records and the effects of numerical dispersion from migrated images and wave simulations.

scholarly journals Fine-tuning of a generative neural network for designing multi-target compounds

2021 ◽  
Author(s):  
Thomas Blaschke ◽  
Jürgen Bajorath
Keyword(s):  
Neural Network ◽  
Small Molecules ◽  
De Novo ◽  
Pharmaceutical Research ◽  
Generative Models ◽  
Fine Tuning ◽  
Data Sets ◽  
Single Target ◽  
Fine Tune ◽  
Target Activity

AbstractExploring the origin of multi-target activity of small molecules and designing new multi-target compounds are highly topical issues in pharmaceutical research. We have investigated the ability of a generative neural network to create multi-target compounds. Data sets of experimentally confirmed multi-target, single-target, and consistently inactive compounds were extracted from public screening data considering positive and negative assay results. These data sets were used to fine-tune the REINVENT generative model via transfer learning to systematically recognize multi-target compounds, distinguish them from single-target or inactive compounds, and construct new multi-target compounds. During fine-tuning, the model showed a clear tendency to increasingly generate multi-target compounds and structural analogs. Our findings indicate that generative models can be adopted for de novo multi-target compound design.

scholarly journals Cloud based mobile solution for early detection of Skin Cancer using Artificial Intelligence

2021 ◽  
pp. 312-324
Author(s):  
Pawan Sonawane ◽  
Sahel Shardhul ◽  
Raju Mendhe
Keyword(s):  
Neural Network ◽  
Artificial Intelligence ◽  
Skin Cancer ◽  
Early Detection ◽  
Transfer Learning ◽  
Network Model ◽  
Neural Network Model ◽  
Mobile Application ◽  
Low Cost ◽  
Skin Tumors

The vast majority of skin cancer deaths are from melanoma, with about 1.04 million cases annually. Early detection of the same can be immensely helpful in order to try to cure it. But most of the diagnosis procedures are either extremely expensive or not available to a vast majority, as these centers are concentrated in urban regions only. Thus, there is a need for an application that can perform a quick, efficient, and low-cost diagnosis. Our solution proposes to build a server less mobile application on the AWS cloud that takes the images of potential skin tumors and classifies it as either Malignant or Benign. The classification would be carried out using a trained Convolution Neural Network model and Transfer learning (Inception v3). Several experiments will be performed based on Morphology and Color of the tumor to identify ideal parameters.

Tomato pest classification using deep convolutional neural network with transfer learning, fine tuning and scratch learning

2021 ◽  
pp. 1-10
Author(s):  
Gayatri Pattnaik ◽  
Vimal K. Shrivastava ◽  
K. Parvathi
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Transfer Learning ◽  
Data Augmentation ◽  
State Of The Art ◽  
Deep Convolutional Neural Network ◽  
Fine Tuning ◽  
Tomato Plants ◽  
Random Weights

Pests are major threat to economic growth of a country. Application of pesticide is the easiest way to control the pest infection. However, excessive utilization of pesticide is hazardous to environment. The recent advances in deep learning have paved the way for early detection and improved classification of pest in tomato plants which will benefit the farmers. This paper presents a comprehensive analysis of 11 state-of-the-art deep convolutional neural network (CNN) models with three configurations: transfers learning, fine-tuning and scratch learning. The training in transfer learning and fine tuning initiates from pre-trained weights whereas random weights are used in case of scratch learning. In addition, the concept of data augmentation has been explored to improve the performance. Our dataset consists of 859 tomato pest images from 10 categories. The results demonstrate that the highest classification accuracy of 94.87% has been achieved in the transfer learning approach by DenseNet201 model with data augmentation.

A Study of the Influence of Data Complexity and Similarity on Soft Biometrics Classification Performance in a Transfer Learning Scenario

2021 ◽  
Vol 18 (2) ◽  
pp. 56-65
Author(s):  
Marcelo Romero ◽  
◽  
Matheus Gutoski ◽  
Leandro Takeshi Hattori ◽  
Manassés Ribeiro ◽  
...  
Keyword(s):  
Transfer Learning ◽  
Classification Performance ◽  
Fine Tuning ◽  
Similarity Metrics ◽  
Soft Biometrics ◽  
Final Layer ◽  
The One ◽  
Fully Connected ◽  
The Relationship ◽  
Fine Tune

Transfer learning is a paradigm that consists in training and testing classifiers with datasets drawn from distinct distributions. This technique allows to solve a particular problem using a model that was trained for another purpose. In the recent years, this practice has become very popular due to the increase of public available pre-trained models that can be fine-tuned to be applied in different scenarios. However, the relationship between the datasets used for training the model and the test data is usually not addressed, specially where the fine-tuning process is done only for the fully connected layers of a Convolutional Neural Network with pre-trained weights. This work presents a study regarding the relationship between the datasets used in a transfer learning process in terms of the performance achieved by models complexities and similarities. For this purpose, we fine-tune the final layer of Convolutional Neural Networks with pre-trained weights using diverse soft biometrics datasets. An evaluation of the performances of the models, when tested with datasets that are different from the one used for training the model, is presented. Complexity and similarity metrics are also used to perform the evaluation.

scholarly journals Is One Teacher Model Enough to Transfer Knowledge to a Student Model?

Algorithms ◽  
2021 ◽  
Vol 14 (11) ◽  
pp. 334
Author(s):  
Nicola Landro ◽  
Ignazio Gallo ◽  
Riccardo La Grassa
Keyword(s):  
Neural Network ◽  
Transfer Learning ◽  
Learning Problem ◽  
Learning Techniques ◽  
Starting Point ◽  
Learning Technique ◽  
Interesting Approach ◽  
Classification Tasks ◽  
Learned Features ◽  
Fine Tune

Nowadays, the transfer learning technique can be successfully applied in the deep learning field through techniques that fine-tune the CNN’s starting point so it may learn over a huge dataset such as ImageNet and continue to learn on a fixed dataset to achieve better performance. In this paper, we designed a transfer learning methodology that combines the learned features of different teachers to a student network in an end-to-end model, improving the performance of the student network in classification tasks over different datasets. In addition to this, we tried to answer the following questions which are in any case directly related to the transfer learning problem addressed here. Is it possible to improve the performance of a small neural network by using the knowledge gained from a more powerful neural network? Can a deep neural network outperform the teacher using transfer learning? Experimental results suggest that neural networks can transfer their learning to student networks using our proposed architecture, designed to bring to light a new interesting approach for transfer learning techniques. Finally, we provide details of the code and the experimental settings.

Transfer Learning and Textual Analysis of Accounting Disclosures: Applying Big Data Methods to Small(er) Data Sets

2021 ◽  
Author(s):  
Federico Siano ◽  
Peter Wysocki
Keyword(s):  
Big Data ◽  
Transfer Learning ◽  
Textual Analysis ◽  
Low Cost ◽  
Language Model ◽  
Fine Tuning ◽  
Superior Performance ◽  
Data Sets ◽  
Quarterly Earnings ◽  
Accounting Disclosures

We introduce and apply machine transfer learning methods to analyze accounting disclosures. We use the examples of the new BERT language model and sentiment analysis of quarterly earnings disclosures to demonstrate the key transfer learning concepts of: (i) pre-training on generic "Big Data", (ii) fine-tuning on small accounting datasets, and (iii) using a language model that captures context rather than stand-alone words. Overall, we show that this new approach is easy to implement, uses widely-available and low-cost computing resources, and has superior performance relative to existing textual analysis tools in accounting. We conclude with suggestions for opportunities to apply transfer learning to address important accounting research questions.

scholarly journals Deep and Wide Transfer Learning with Kernel Matching for Pooling Data from Electroencephalography and Psychological Questionnaires

Sensors ◽  
2021 ◽  
Vol 21 (15) ◽  
pp. 5105
Author(s):  
Diego Fabian Collazos-Huertas ◽  
Luisa Fernanda Velasquez-Martinez ◽  
Hernan Dario Perez-Nastar ◽  
Andres Marino Alvarez-Meza ◽  
German Castellanos-Dominguez
Keyword(s):  
Neural Network ◽  
Transfer Learning ◽  
Motor Skills ◽  
Self Regulation ◽  
Fine Tuning ◽  
Single Subject ◽  
Target Network ◽  
Electroencephalogram Eeg ◽  
Psychological Questionnaires ◽  
Target Sets

Motor imagery (MI) promotes motor learning and encourages brain–computer interface systems that entail electroencephalogram (EEG) decoding. However, a long period of training is required to master brain rhythms’ self-regulation, resulting in users with MI inefficiency. We introduce a parameter-based approach of cross-subject transfer-learning to improve the performances of poor-performing individuals in MI-based BCI systems, pooling data from labeled EEG measurements and psychological questionnaires via kernel-embedding. To this end, a Deep and Wide neural network for MI classification is implemented to pre-train the network from the source domain. Then, the parameter layers are transferred to initialize the target network within a fine-tuning procedure to recompute the Multilayer Perceptron-based accuracy. To perform data-fusion combining categorical features with the real-valued features, we implement stepwise kernel-matching via Gaussian-embedding. Finally, the paired source–target sets are selected for evaluation purposes according to the inefficiency-based clustering by subjects to consider their influence on BCI motor skills, exploring two choosing strategies of the best-performing subjects (source space): single-subject and multiple-subjects. Validation results achieved for discriminant MI tasks demonstrate that the introduced Deep and Wide neural network presents competitive performance of accuracy even after the inclusion of questionnaire data.

scholarly journals Transfer learning using AlexNet Convolutional Neural Network for Face Recognition

2020 ◽  
Vol 9 (11) ◽  
pp. 285-294
Keyword(s):  
Neural Network ◽  
Face Recognition ◽  
Transfer Learning ◽  
Data Augmentation ◽  
Recognition System ◽  
Training Data ◽  
Fine Tuning ◽  
Data Sets ◽  
Learning Method ◽  
Face Recognition System

This research is aimed to achieve high-precision accuracy and for face recognition system. Convolution Neural Network is one of the Deep Learning approaches and has demonstrated excellent performance in many fields, including image recognition of a large amount of training data (such as ImageNet). In fact, hardware limitations and insufficient training data-sets are the challenges of getting high performance. Therefore, in this work the Deep Transfer Learning method using AlexNet pre-trained CNN is proposed to improve the performance of the face-recognition system even for a smaller number of images. The transfer learning method is used to fine-tuning on the last layer of AlexNet CNN model for new classification tasks. The data augmentation (DA) technique also proposed to minimize the over-fitting problem during Deep transfer learning training and to improve accuracy. The results proved the improvement in over-fitting and in performance after using the data augmentation technique. All the experiments were tested on UTeMFD, GTFD, and CASIA-Face V5 small data-sets. As a result, the proposed system achieved a high accuracy as 100% on UTeMFD, 96.67% on GTFD, and 95.60% on CASIA-Face V5 in less than 0.05 seconds of recognition time.

scholarly journals Deep learning prediction of mild cognitive impairment conversion to Alzheimer’s disease at 3 years after diagnosis using longitudinal and whole-brain 3D MRI

2021 ◽  
Vol 7 ◽  
pp. e560
Author(s):  
Ethan Ocasio ◽  
Tim Q. Duong
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Cognitive Impairment ◽  
Mild Cognitive Impairment ◽  
Longitudinal Data ◽  
Convolutional Neural Network ◽  
Transfer Learning ◽  
Fine Tuning ◽  
3D Mri ◽  
Whole Brain

Background While there is no cure for Alzheimer’s disease (AD), early diagnosis and accurate prognosis of AD may enable or encourage lifestyle changes, neurocognitive enrichment, and interventions to slow the rate of cognitive decline. The goal of our study was to develop and evaluate a novel deep learning algorithm to predict mild cognitive impairment (MCI) to AD conversion at three years after diagnosis using longitudinal and whole-brain 3D MRI. Methods This retrospective study consisted of 320 normal cognition (NC), 554 MCI, and 237 AD patients. Longitudinal data include T1-weighted 3D MRI obtained at initial presentation with diagnosis of MCI and at 12-month follow up. Whole-brain 3D MRI volumes were used without a priori segmentation of regional structural volumes or cortical thicknesses. MRIs of the AD and NC cohort were used to train a deep learning classification model to obtain weights to be applied via transfer learning for prediction of MCI patient conversion to AD at three years post-diagnosis. Two (zero-shot and fine tuning) transfer learning methods were evaluated. Three different convolutional neural network (CNN) architectures (sequential, residual bottleneck, and wide residual) were compared. Data were split into 75% and 25% for training and testing, respectively, with 4-fold cross validation. Prediction accuracy was evaluated using balanced accuracy. Heatmaps were generated. Results The sequential convolutional approach yielded slightly better performance than the residual-based architecture, the zero-shot transfer learning approach yielded better performance than fine tuning, and CNN using longitudinal data performed better than CNN using a single timepoint MRI in predicting MCI conversion to AD. The best CNN model for predicting MCI conversion to AD at three years after diagnosis yielded a balanced accuracy of 0.793. Heatmaps of the prediction model showed regions most relevant to the network including the lateral ventricles, periventricular white matter and cortical gray matter. Conclusions This is the first convolutional neural network model using longitudinal and whole-brain 3D MRIs without extracting regional brain volumes or cortical thicknesses to predict future MCI to AD conversion at 3 years after diagnosis. This approach could lead to early prediction of patients who are likely to progress to AD and thus may lead to better management of the disease.

scholarly journals Pengenalan Jamur Yang Dapat Dikonsumsi Menggunakan Metode Transfer Learning Pada Convolutional Neural Network

2021 ◽  
Vol 5 (2) ◽  
pp. 81-91
Author(s):  
Elok Iedfitra Haksoro ◽  
Abas Setiawan
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Transfer Learning ◽  
Morphological Characteristics ◽  
Learning Rate ◽  
Fine Tuning ◽  
Training Stage ◽  
Long Time ◽  
Accuracy Result ◽  
Poisonous Mushrooms

Not all mushrooms are edible because some are poisonous. The edible or poisonous mushrooms can be identified by paying attention to the morphological characteristics of mushrooms, such as shape, color, and texture. There is an issue: some poisonous mushrooms have morphological features that are very similar to edible mushrooms. It can lead to the misidentification of mushrooms. This work aims to recognize edible or poisonous mushrooms using a Deep Learning approach, typically Convolutional Neural Networks. Because the training process will take a long time, Transfer Learning was applied to accelerate the learning process. Transfer learning uses an existing model as a base model in our neural network by transferring information from the related domain. There are Four base models are used, namely MobileNets, MobileNetV2, ResNet50, and VGG19. Each base model will be subjected to several experimental scenarios, such as setting the different learning rate values for pre-training and fine-tuning. The results show that the Convolutional Neural Network with transfer learning method can recognize edible or poisonous mushrooms with more than 86% accuracy. Moreover, the best accuracy result is 92.19% obtained from the base model of MobileNetsV2 with a learning rate of 0,00001 at the pre-training stage and 0,0001 at the fine-tuning stage.

Sign in / Sign up

Export Citation Format

Share Document