Heck reaction prediction using a transformer model based on a transfer learning strategy

2020 ◽  
Vol 56 (65) ◽  
pp. 9368-9371 ◽  
Author(s):  
Ling Wang ◽  
Chengyun Zhang ◽  
Renren Bai ◽  
Jianjun Li ◽  
Hongliang Duan
Keyword(s):  
Transfer Learning ◽  
Heck Reaction ◽  
Learning Strategy ◽  
Chemical Data ◽  
Proof Of Concept ◽  
Reaction Prediction ◽  
Model Based ◽  
Transformer Model

A proof-of-concept methodology for addressing small amounts of chemical data using transfer learning is presented.

scholarly journals Data Augmentation and Transfer Learning Strategies for Reaction Prediction in Low Chemical Data Regimes

2020 ◽  
Author(s):  
Yun Zhang ◽  
Ling Wang ◽  
Xinqiao Wang ◽  
Chengyun Zhang ◽  
Jiamin Ge ◽  
...  
Keyword(s):  
Deep Learning ◽  
Learning Strategies ◽  
Transfer Learning ◽  
Data Augmentation ◽  
Learning Strategy ◽  
Predictive Performance ◽  
Learning Model ◽  
Training Data ◽  
Reaction Prediction ◽  
Transformer Model

<p><b>Abstract:</b> Effective and rapid deep learning method to predict chemical reactions contributes to the research and development of organic chemistry and drug discovery. Despite the outstanding capability of deep learning in retrosynthesis and forward synthesis, predictions based on small chemical datasets generally result in low accuracy due to an insufficiency of reaction examples. Here, we introduce a new state art of method, which integrates transfer learning with transformer model to predict the outcomes of the Baeyer-Villiger reaction which is a representative small dataset reaction. The results demonstrate that introducing transfer learning strategy markedly improves the top-1 accuracy of the transformer-transfer learning model (81.8%) over that of the transformer-baseline model (58.4%). Moreover, we further introduce data augmentation to the input reaction SMILES, which allows for better performance and improves the accuracy of the transformer-transfer learning model (86.7%). In summary, both transfer learning and data augmentation methods significantly improve the predictive performance of transformer model, which are powerful methods used in chemistry field to eliminate the restriction of limited training data.</p>

scholarly journals Data Augmentation and Transfer Learning Strategies for Reaction Prediction in Low Chemical Data Regimes

2020 ◽  
Author(s):  
Yun Zhang ◽  
Ling Wang ◽  
Xinqiao Wang ◽  
Chengyun Zhang ◽  
Jiamin Ge ◽  
...  
Keyword(s):  
Deep Learning ◽  
Learning Strategies ◽  
Transfer Learning ◽  
Data Augmentation ◽  
Learning Strategy ◽  
Predictive Performance ◽  
Learning Model ◽  
Training Data ◽  
Reaction Prediction ◽  
Transformer Model

<p><b>Abstract:</b> Effective and rapid deep learning method to predict chemical reactions contributes to the research and development of organic chemistry and drug discovery. Despite the outstanding capability of deep learning in retrosynthesis and forward synthesis, predictions based on small chemical datasets generally result in low accuracy due to an insufficiency of reaction examples. Here, we introduce a new state art of method, which integrates transfer learning with transformer model to predict the outcomes of the Baeyer-Villiger reaction which is a representative small dataset reaction. The results demonstrate that introducing transfer learning strategy markedly improves the top-1 accuracy of the transformer-transfer learning model (81.8%) over that of the transformer-baseline model (58.4%). Moreover, we further introduce data augmentation to the input reaction SMILES, which allows for better performance and improves the accuracy of the transformer-transfer learning model (86.7%). In summary, both transfer learning and data augmentation methods significantly improve the predictive performance of transformer model, which are powerful methods used in chemistry field to eliminate the restriction of limited training data.</p>

Reproducing the Invention of a Named Reaction: Predicting Unseen Chemical Reactions via Zero-Shot Learning

2021 ◽  
Author(s):  
An Su ◽  
Ling Wang ◽  
Xinqiao Wang ◽  
Chengyun Zhang ◽  
Hongliang Duan
Keyword(s):  
Chemical Reactions ◽  
Suzuki Reaction ◽  
Auxiliary Information ◽  
Coupling Reaction ◽  
Proof Of Concept ◽  
Reaction Prediction ◽  
Training Samples ◽  
Wide Range ◽  
The One ◽  
Transformer Model

<div> The study focuses on the proof-of-concept that the human invention of a named reaction can be reproduced by the zero-shot learning version of transformer.</div><div>While state-of-art reaction prediction machine learning models can predict chemical reactions through the transfer learning of thousands of training samples with the same reaction types as the ones to predict, how to prepare the models to predict truly "unseen" reactions remains a question. We aim to equip the transformer model with the ability to predict unseen reactions following the concept of "zero-shot learning". To find what kind of auxiliary information is needed, we reproduce the human invention of the Chan-Lam coupling reaction where the inventor was inspired by two existing reactions---Suzuki reaction and Barton's bismuth arylation reaction. After training with the samples from these two reactions as well as the USPTO dataset, the transformer model can pre-dict the Chan-Lam coupling reaction with 55.7% top-1 accuracy which is a huge im-provement comparing to 17.2% from the model trained with the USPTO dataset only. Our model also mimics the later stage of this history where the initial case of Chan-Lam coupling reaction was generalized to a wide range of reactants and reagents via the "one-shot learning" approach. The results of this study show that having existing reactions as auxiliary information can help the transformer predict unseen reactions and providing just one or few samples of the unseen reaction can boost the model's gener-alization ability.<br></div>

Reproducing the Invention of a Named Reaction: Predicting Unseen Chemical Reactions via Zero-Shot Learning

2021 ◽  
Author(s):  
An Su ◽  
Ling Wang ◽  
Xinqiao Wang ◽  
Chengyun Zhang ◽  
Hongliang Duan
Keyword(s):  
Chemical Reactions ◽  
Suzuki Reaction ◽  
Auxiliary Information ◽  
Coupling Reaction ◽  
Proof Of Concept ◽  
Reaction Prediction ◽  
Training Samples ◽  
Wide Range ◽  
The One ◽  
Transformer Model

<div> The study focuses on the proof-of-concept that the human invention of a named reaction can be reproduced by the zero-shot learning version of transformer.</div><div>While state-of-art reaction prediction machine learning models can predict chemical reactions through the transfer learning of thousands of training samples with the same reaction types as the ones to predict, how to prepare the models to predict truly "unseen" reactions remains a question. We aim to equip the transformer model with the ability to predict unseen reactions following the concept of "zero-shot learning". To find what kind of auxiliary information is needed, we reproduce the human invention of the Chan-Lam coupling reaction where the inventor was inspired by two existing reactions---Suzuki reaction and Barton's bismuth arylation reaction. After training with the samples from these two reactions as well as the USPTO dataset, the transformer model can pre-dict the Chan-Lam coupling reaction with 55.7% top-1 accuracy which is a huge im-provement comparing to 17.2% from the model trained with the USPTO dataset only. Our model also mimics the later stage of this history where the initial case of Chan-Lam coupling reaction was generalized to a wide range of reactants and reagents via the "one-shot learning" approach. The results of this study show that having existing reactions as auxiliary information can help the transformer predict unseen reactions and providing just one or few samples of the unseen reaction can boost the model's gener-alization ability.<br></div>

Data augmentation and transfer learning strategies for reaction prediction in low chemical data regimes

2021 ◽  
Author(s):  
Yun Zhang ◽  
Ling Wang ◽  
Xinqiao Wang ◽  
Chengyun Zhang ◽  
Jiamin Ge ◽  
...  
Keyword(s):  
Organic Chemistry ◽  
Deep Learning ◽  
Drug Discovery ◽  
Research And Development ◽  
Learning Strategies ◽  
Transfer Learning ◽  
Chemical Reactions ◽  
Data Augmentation ◽  
Learning Method ◽  
Reaction Prediction

An effective and rapid deep learning method to predict chemical reactions contributes to the research and development of organic chemistry and drug discovery.

scholarly journals Deep Transfer Learning for Improved Detection of Keratoconus using Corneal Topographic Maps

2021 ◽  
Author(s):  
Ali H. Al-Timemy ◽  
Nebras H. Ghaeb ◽  
Zahraa M. Mosa ◽  
Javier Escudero
Keyword(s):  
Transfer Learning ◽  
Learning Strategy ◽  
Clinical Decision ◽  
Topographic Maps ◽  
Computer Assisted ◽  
Clinical Knowledge ◽  
Average Accuracy ◽  
Assisted Diagnosis ◽  
Improved Accuracy ◽  
Fine Tune

Abstract Clinical keratoconus (KCN) detection is a challenging and time-consuming task. In the diagnosis process, ophthalmologists must revise demographic and clinical ophthalmic examinations. The latter include slit-lamb, corneal topographic maps, and Pentacam indices (PI). We propose an Ensemble of Deep Transfer Learning (EDTL) based on corneal topographic maps. We consider four pretrained networks, SqueezeNet (SqN), AlexNet (AN), ShuffleNet (SfN), and MobileNet-v2 (MN), and fine-tune them on a dataset of KCN and normal cases, each including four topographic maps. We also consider a PI classifier. Then, our EDTL method combines the output probabilities of each of the five classifiers to obtain a decision based on the fusion of probabilities. Individually, the classifier based on PI achieved 93.1% accuracy, whereas the deep classifiers reached classification accuracies over 90% only in isolated cases. Overall, the average accuracy of the deep networks over the four corneal maps ranged from 86% (SfN) to 89.9% (AN). The classifier ensemble increased the accuracy of the deep classifiers based on corneal maps to values ranging (92.2% to 93.1%) for SqN and (93.1% to 94.8%) for AN. Including in the ensemble-specific combinations of corneal maps’ classifiers and PI increased the accuracy to 98.3%. Moreover, visualization of first learner filters in the networks and Grad-CAMs confirmed that the networks had learned relevant clinical features. This study shows the potential of creating ensembles of deep classifiers fine-tuned with a transfer learning strategy as it resulted in an improved accuracy while showing learnable filters and Grad-CAMs that agree with clinical knowledge. This is a step further towards the potential clinical deployment of an improved computer-assisted diagnosis system for KCN detection to help ophthalmologists to confirm the clinical decision and to perform fast and accurate KCN treatment.

scholarly journals BeautyNet: Joint Multiscale CNN and Transfer Learning Method for Unconstrained Facial Beauty Prediction

2019 ◽  
Vol 2019 ◽  
pp. 1-14 ◽  
Author(s):  
Yikui Zhai ◽  
He Cao ◽  
Wenbo Deng ◽  
Junying Gan ◽  
Vincenzo Piuri ◽  
...  
Keyword(s):  
Transfer Learning ◽  
Classification Accuracy ◽  
Learning Strategy ◽  
State Of The Art ◽  
Activation Function ◽  
Training Data ◽  
Fine Grained ◽  
Pattern Recognition Problem ◽  
Face Features ◽  
Facial Beauty

Because of the lack of discriminative face representations and scarcity of labeled training data, facial beauty prediction (FBP), which aims at assessing facial attractiveness automatically, has become a challenging pattern recognition problem. Inspired by recent promising work on fine-grained image classification using the multiscale architecture to extend the diversity of deep features, BeautyNet for unconstrained facial beauty prediction is proposed in this paper. Firstly, a multiscale network is adopted to improve the discriminative of face features. Secondly, to alleviate the computational burden of the multiscale architecture, MFM (max-feature-map) is utilized as an activation function which can not only lighten the network and speed network convergence but also benefit the performance. Finally, transfer learning strategy is introduced here to mitigate the overfitting phenomenon which is caused by the scarcity of labeled facial beauty samples and improves the proposed BeautyNet’s performance. Extensive experiments performed on LSFBD demonstrate that the proposed scheme outperforms the state-of-the-art methods, which can achieve 67.48% classification accuracy.

Robust Finger Vein Recognition Based on Deep CNN with Spatial Attention and Bias Field Correction

2021 ◽  
Vol 30 (01) ◽  
pp. 2140005
Author(s):  
Zhe Huang ◽  
Chengan Guo
Keyword(s):  
Spatial Attention ◽  
Transfer Learning ◽  
Learning Strategy ◽  
Bias Field ◽  
The Public ◽  
Finger Vein ◽  
Bias Field Correction ◽  
Training Samples ◽  
Vein Recognition ◽  
Finger Vein Recognition

As one of the biometric information based authentication technologies, finger vein recognition has received increasing attention due to its safety and convenience. However, it is still a challenging task to design an efficient and robust finger vein recognition system because of the low quality of the finger vein images, lack of sufficient number of training samples with image-level annotated information and no pixel-level finger vein texture labels in the public available finger vein databases. In this paper, we propose a novel CNN-based finger vein recognition approach with bias field correction, spatial attention mechanism and a multistage transfer learning strategy to cope with the difficulties mentioned above. In the proposed method, the bias field correction module is to remove the unbalanced bias field of the original images by using a two-dimensional polynomial fitting algorithm, the spatial attention module is to enhance the informative vein texture regions while suppressing the other less informative regions, and the multistage transfer learning strategy is to solve the problem caused by insufficient training for CNN-based model due to lack of labeled training samples in the public finger vein databases. Moreover, several measures, including a label smoothing scheme and data augmentation, are exploited to improve the performance of the proposed method. Extensive experiments have been conducted in the work on three public databases, and the results show that the proposed approach outperforms the existing state-of-the-art methods.

Sign in / Sign up

Export Citation Format

Share Document