DrugEx v2: De Novo Design of Drug Molecule by Pareto-based Multi-Objective Reinforcement Learning in Polypharmacology

In polypharmacology, ideal drugs are required to bind to multiple specific targets to enhance efficacy or to reduce resistance formation. Although deep learning has achieved breakthrough in drug discovery, most of its applications only focus on a single drug target to generate drug-like active molecules in spite of the reality that drug molecules often interact with more than one target which can have desired (polypharmacology) or undesired (toxicity) effects. In a previous study we proposed a new method named DrugEx that integrates an exploration strategy into RNN-based reinforcement learning to improve the diversity of the generated molecules. Here, we extended our DrugEx algorithm with multi-objective optimization to generate drug molecules towards more than one specific target (two adenosine receptors, A1AR and A2AAR, and the potassium ion channel hERG in this study). In our model, we applied an RNN as the agent and machine learning predictors as the environment, both of which were pre-trained in advance and then interplayed under the reinforcement learning framework. The concept of evolutionary algorithms was merged into our method such that crossover and mutation operations were implemented by the same deep learning model as the agent. During the training loop, the agent generates a batch of SMILES-based molecules. Subsequently scores for all objectives provided by the environment are used for constructing Pareto ranks of the generated molecules with non-dominated sorting and Tanimoto-based crowding distance algorithms. Here, we adopted GPU acceleration to speed up the process of Pareto optimization. The final reward of each molecule is calculated based on the Pareto ranking with the ranking selection algorithm. The agent is trained under the guidance of the reward to make sure it can generate more desired molecules after convergence of the training process. All in all we demonstrate generation of compounds with a diverse predicted selectivity profile toward multiple targets, offering the potential of high efficacy and lower toxicity.

Download Full-text

DrugEx v2: De Novo Design of Drug Molecule by Pareto-based Multi-Objective Reinforcement Learning in Polypharmacology

10.26434/chemrxiv.14474127.v2 ◽

2021 ◽

Author(s):

Xuhan Liu ◽

Kai Ye ◽

Herman Van Vlijmen ◽

Michael T. M. Emmerich ◽

Adriaan P. IJzerman ◽

...

Keyword(s):

Deep Learning ◽

Reinforcement Learning ◽

Drug Target ◽

De Novo ◽

Multi Objective ◽

Learning Framework ◽

Drug Molecules ◽

Pareto Ranking ◽

Speed Up ◽

Deep Learning Model

In polypharmacology, ideal drugs are required to bind to multiple specific targets to enhance efficacy or to reduce resistance formation. Although deep learning has achieved breakthrough in drug discovery, most of its applications only focus on a single drug target to generate drug-like active molecules in spite of the reality that drug molecules often interact with more than one target which can have desired (polypharmacology) or undesired (toxicity) effects. In a previous study we proposed a new method named DrugEx that integrates an exploration strategy into RNN-based reinforcement learning to improve the diversity of the generated molecules. Here, we extended our DrugEx algorithm with multi-objective optimization to generate drug molecules towards more than one specific target (two adenosine receptors, A1AR and A2AAR, and the potassium ion channel hERG in this study). In our model, we applied an RNN as the agent and machine learning predictors as the environment, both of which were pre-trained in advance and then interplayed under the reinforcement learning framework. The concept of evolutionary algorithms was merged into our method such that crossover and mutation operations were implemented by the same deep learning model as the agent. During the training loop, the agent generates a batch of SMILES-based molecules. Subsequently scores for all objectives provided by the environment are used for constructing Pareto ranks of the generated molecules with non-dominated sorting and Tanimoto-based crowding distance algorithms. Here, we adopted GPU acceleration to speed up the process of Pareto optimization. The final reward of each molecule is calculated based on the Pareto ranking with the ranking selection algorithm. The agent is trained under the guidance of the reward to make sure it can generate more desired molecules after convergence of the training process. All in all we demonstrate generation of compounds with a diverse predicted selectivity profile toward multiple targets, offering the potential of high efficacy and lower toxicity.

Download Full-text

DrugEx v2: de novo design of drug molecules by Pareto-based multi-objective reinforcement learning in polypharmacology

Journal of Cheminformatics ◽

10.1186/s13321-021-00561-9 ◽

2021 ◽

Vol 13 (1) ◽

Author(s):

Xuhan Liu ◽

Kai Ye ◽

Herman W. T. van Vlijmen ◽

Michael T. M. Emmerich ◽

Adriaan P. IJzerman ◽

...

Keyword(s):

Deep Learning ◽

Reinforcement Learning ◽

De Novo ◽

De Novo Design ◽

Sorting Algorithm ◽

Multiple Targets ◽

Multi Objective ◽

Learning Framework ◽

Drug Molecules ◽

Pareto Ranking

AbstractIn polypharmacology drugs are required to bind to multiple specific targets, for example to enhance efficacy or to reduce resistance formation. Although deep learning has achieved a breakthrough in de novo design in drug discovery, most of its applications only focus on a single drug target to generate drug-like active molecules. However, in reality drug molecules often interact with more than one target which can have desired (polypharmacology) or undesired (toxicity) effects. In a previous study we proposed a new method named DrugEx that integrates an exploration strategy into RNN-based reinforcement learning to improve the diversity of the generated molecules. Here, we extended our DrugEx algorithm with multi-objective optimization to generate drug-like molecules towards multiple targets or one specific target while avoiding off-targets (the two adenosine receptors, A1AR and A2AAR, and the potassium ion channel hERG in this study). In our model, we applied an RNN as the agent and machine learning predictors as the environment. Both the agent and the environment were pre-trained in advance and then interplayed under a reinforcement learning framework. The concept of evolutionary algorithms was merged into our method such that crossover and mutation operations were implemented by the same deep learning model as the agent. During the training loop, the agent generates a batch of SMILES-based molecules. Subsequently scores for all objectives provided by the environment are used to construct Pareto ranks of the generated molecules. For this ranking a non-dominated sorting algorithm and a Tanimoto-based crowding distance algorithm using chemical fingerprints are applied. Here, we adopted GPU acceleration to speed up the process of Pareto optimization. The final reward of each molecule is calculated based on the Pareto ranking with the ranking selection algorithm. The agent is trained under the guidance of the reward to make sure it can generate desired molecules after convergence of the training process. All in all we demonstrate generation of compounds with a diverse predicted selectivity profile towards multiple targets, offering the potential of high efficacy and low toxicity.

Download Full-text

BEHRT-HF: an interpretable transformer-based, deep learning model for prediction of incident heart failure

European Heart Journal ◽

10.1093/ehjci/ehaa946.3553 ◽

2020 ◽

Vol 41 (Supplement_2) ◽

Author(s):

S Rao ◽

Y Li ◽

R Ramakrishnan ◽

A Hassaine ◽

D Canoy ◽

...

Keyword(s):

Heart Failure ◽

Deep Learning ◽

State Of The Art ◽

Failure Prediction ◽

Predictive Performance ◽

Learning Model ◽

Learning Framework ◽

Incident Heart Failure ◽

Ablation Study ◽

Deep Learning Model

Abstract Background/Introduction Predicting incident heart failure has been challenging. Deep learning models when applied to rich electronic health records (EHR) offer some theoretical advantages. However, empirical evidence for their superior performance is limited and they remain commonly uninterpretable, hampering their wider use in medical practice. Purpose We developed a deep learning framework for more accurate and yet interpretable prediction of incident heart failure. Methods We used longitudinally linked EHR from practices across England, involving 100,071 patients, 13% of whom had been diagnosed with incident heart failure during follow-up. We investigated the predictive performance of a novel transformer deep learning model, “Transformer for Heart Failure” (BEHRT-HF), and validated it using both an external held-out dataset and an internal five-fold cross-validation mechanism using area under receiver operating characteristic (AUROC) and area under the precision recall curve (AUPRC). Predictor groups included all outpatient and inpatient diagnoses within their temporal context, medications, age, and calendar year for each encounter. By treating diagnoses as anchors, we alternatively removed different modalities (ablation study) to understand the importance of individual modalities to the performance of incident heart failure prediction. Using perturbation-based techniques, we investigated the importance of associations between selected predictors and heart failure to improve model interpretability. Results BEHRT-HF achieved high accuracy with AUROC 0.932 and AUPRC 0.695 for external validation, and AUROC 0.933 (95% CI: 0.928, 0.938) and AUPRC 0.700 (95% CI: 0.682, 0.718) for internal validation. Compared to the state-of-the-art recurrent deep learning model, RETAIN-EX, BEHRT-HF outperformed it by 0.079 and 0.030 in terms of AUPRC and AUROC. Ablation study showed that medications were strong predictors, and calendar year was more important than age. Utilising perturbation, we identified and ranked the intensity of associations between diagnoses and heart failure. For instance, the method showed that established risk factors including myocardial infarction, atrial fibrillation and flutter, and hypertension all strongly associated with the heart failure prediction. Additionally, when population was stratified into different age groups, incident occurrence of a given disease had generally a higher contribution to heart failure prediction in younger ages than when diagnosed later in life. Conclusions Our state-of-the-art deep learning framework outperforms the predictive performance of existing models whilst enabling a data-driven way of exploring the relative contribution of a range of risk factors in the context of other temporal information. Funding Acknowledgement Type of funding source: Private grant(s) and/or Sponsorship. Main funding source(s): National Institute for Health Research, Oxford Martin School, Oxford Biomedical Research Centre

Download Full-text

Adaptive Compression for Online Computer Vision: An Edge Reinforcement Learning Approach

ACM Transactions on Multimedia Computing Communications and Applications ◽

10.1145/3447878 ◽

2021 ◽

Vol 17 (4) ◽

pp. 1-23

Author(s):

Zhaoliang He ◽

Hongshan Li ◽

Zhi Wang ◽

Shutao Xia ◽

Wenwu Zhu

Keyword(s):

Computer Vision ◽

Deep Learning ◽

Reinforcement Learning ◽

Quality Level ◽

Image Size ◽

Standard Quality ◽

Encapsulation Method ◽

Cloud Servers ◽

Model Structures ◽

Deep Learning Model

With the growth of computer vision-based applications, an explosive amount of images have been uploaded to cloud servers that host such online computer vision algorithms, usually in the form of deep learning models. JPEG has been used as the de facto compression and encapsulation method for images. However, standard JPEG configuration does not always perform well for compressing images that are to be processed by a deep learning model—for example, the standard quality level of JPEG leads to 50% of size overhead (compared with the best quality level selection) on ImageNet under the same inference accuracy in popular computer vision models (e.g., InceptionNet and ResNet). Knowing this, designing a better JPEG configuration for online computer vision-based services is still extremely challenging. First, cloud-based computer vision models are usually a black box to end-users; thus, it is challenging to design JPEG configuration without knowing their model structures. Second, the “optimal” JPEG configuration is not fixed; instead, it is determined by confounding factors, including the characteristics of the input images and the model, the expected accuracy and image size, and so forth. In this article, we propose a reinforcement learning (RL)-based adaptive JPEG configuration framework, AdaCompress. In particular, we design an edge (i.e., user-side) RL agent that learns the optimal compression quality level to achieve an expected inference accuracy and upload image size, only from the online inference results, without knowing details of the model structures. Furthermore, we design an explore-exploit mechanism to let the framework fast switch an agent when it detects a performance degradation, mainly due to the input change (e.g., images captured across daytime and night). Our evaluation experiments using real-world online computer vision-based APIs from Amazon Rekognition, Face++, and Baidu Vision show that our approach outperforms existing baselines by reducing the size of images by one-half to one-third while the overall classification accuracy only decreases slightly. Meanwhile, AdaCompress adaptively re-trains or re-loads the RL agent promptly to maintain the performance.

Download Full-text

A novel ensemble deep learning model with dynamic error correction and multi-objective ensemble pruning for time series forecasting

Information Sciences ◽

10.1016/j.ins.2020.08.053 ◽

2021 ◽

Vol 544 ◽

pp. 427-445

Author(s):

Shuai Zhang ◽

Yong Chen ◽

Wenyu Zhang ◽

Ruijun Feng

Keyword(s):

Time Series ◽

Deep Learning ◽

Error Correction ◽

Dynamic Error ◽

Learning Model ◽

Time Series Forecasting ◽

Ensemble Pruning ◽

Multi Objective ◽

Deep Learning Model

Download Full-text

Analysis of COVID-19 Infections on a CT Image Using DeepSense Model

Frontiers in Public Health ◽

10.3389/fpubh.2020.599550 ◽

2020 ◽

Vol 8 ◽

Author(s):

Adil Khadidos ◽

Alaa O. Khadidos ◽

Srihari Kannan ◽

Yuvaraj Natarajan ◽

Sachi Nandan Mohanty ◽

...

Keyword(s):

Neural Network ◽

Deep Learning ◽

Input Data ◽

Training Data ◽

Learning Framework ◽

Learning Classifiers ◽

Tomography Image ◽

Mining Model ◽

Deep Learning Model

In this paper, a data mining model on a hybrid deep learning framework is designed to diagnose the medical conditions of patients infected with the coronavirus disease 2019 (COVID-19) virus. The hybrid deep learning model is designed as a combination of convolutional neural network (CNN) and recurrent neural network (RNN) and named as DeepSense method. It is designed as a series of layers to extract and classify the related features of COVID-19 infections from the lungs. The computerized tomography image is used as an input data, and hence, the classifier is designed to ease the process of classification on learning the multidimensional input data using the Expert Hidden layers. The validation of the model is conducted against the medical image datasets to predict the infections using deep learning classifiers. The results show that the DeepSense classifier offers accuracy in an improved manner than the conventional deep and machine learning classifiers. The proposed method is validated against three different datasets, where the training data are compared with 70%, 80%, and 90% training data. It specifically provides the quality of the diagnostic method adopted for the prediction of COVID-19 infections in a patient.

Download Full-text

PredMP: a web server for de novo prediction and visualization of membrane proteins

Bioinformatics ◽

10.1093/bioinformatics/bty684 ◽

2018 ◽

Vol 35 (4) ◽

pp. 691-693 ◽

Cited By ~ 9

Author(s):

Sheng Wang ◽

Shiyang Fei ◽

Zongan Wang ◽

Yu Li ◽

Jinbo Xu ◽

...

Keyword(s):

Deep Learning ◽

Lipid Bilayer ◽

3D Model ◽

De Novo ◽

3D Structure ◽

Supplementary Information ◽

Model Generation ◽

Transmembrane Topology ◽

Distance Restraints ◽

Deep Learning Model

Abstract Motivation PredMP is the first web service, to our knowledge, that aims at de novo prediction of the membrane protein (MP) 3D structure followed by the embedding of the MP into the lipid bilayer for visualization. Our approach is based on a high-throughput Deep Transfer Learning (DTL) method that first predicts MP contacts by learning from non-MPs and then predicts the 3D model of the MP using the predicted contacts as distance restraints. This algorithm is derived from our previous Deep Learning (DL) method originally developed for soluble protein contact prediction, which has been officially ranked No. 1 in CASP12. The DTL framework in our approach overcomes the challenge that there are only a limited number of solved MP structures for training the deep learning model. There are three modules in the PredMP server: (i) The DTL framework followed by the contact-assisted folding protocol has already been implemented in RaptorX-Contact, which serves as the key module for 3D model generation; (ii) The 1D annotation module, implemented in RaptorX-Property, is used to predict the secondary structure and disordered regions; and (iii) the visualization module to display the predicted MPs embedded in the lipid bilayer guided by the predicted transmembrane topology. Results Tested on 510 non-redundant MPs, our server predicts correct folds for ∼290 MPs, which significantly outperforms existing methods. Tested on a blind and live benchmark CAMEO from September 2016 to January 2018, PredMP can successfully model all 10 MPs belonging to the hard category. Availability and implementation PredMP is freely accessed on the web at http://www.predmp.com. Supplementary information Supplementary data are available at Bioinformatics online.

Download Full-text

Personalized deep learning of individual immunopeptidomes to identify neoantigens for cancer vaccines

10.1101/620468 ◽

2019 ◽

Cited By ~ 1

Author(s):

Ngoc Hieu Tran ◽

Rui Qiao ◽

Lei Xin ◽

Xin Chen ◽

Baozhen Shan ◽

...

Keyword(s):

Mass Spectrometry ◽

Deep Learning ◽

De Novo ◽

Mass Spectrometry Data ◽

Main Role ◽

Accurate Identification ◽

Identification Rate ◽

Cell Responses ◽

First Time ◽

Deep Learning Model

AbstractTumor-specific neoantigens play the main role for developing personal vaccines in cancer immunotherapy. We propose, for the first time, a personalized de novo sequencing workflow to identify HLA-I and HLA-II neoantigens directly and solely from mass spectrometry data. Our workflow trains a personal deep learning model on the immunopeptidome of an individual patient and then uses it to predict mutated neoantigens of that patient. This personalized learning and mass spectrometry-based approach enables comprehensive and accurate identification of neoantigens. We applied the workflow to datasets of five melanoma patients and substantially improved the accuracy and identification rate of de novo HLA peptides by 14.3% and 38.9%, respectively. This subsequently led to the identification of 10,440 HLA-I and 1,585 HLA-II new peptides that were not presented in existing databases. Most importantly, our workflow successfully discovered 17 neoantigens of both HLA-I and HLA-II, including those with validated T cell responses and those novel neoantigens that had not been reported in previous studies.

Download Full-text

Expedited Tensor Program Compilation Based on LightGBM

Journal of Physics Conference Series ◽

10.1088/1742-6596/2078/1/012019 ◽

2021 ◽

Vol 2078 (1) ◽

pp. 012019

Author(s):

Gonghan Liu ◽

Yue Li ◽

Xiaoling Wang

Keyword(s):

Deep Learning ◽

Cost Model ◽

Effective Solution ◽

Learning Framework ◽

Model Based ◽

Deep Networks ◽

Speed Up ◽

Real Hardware

Abstract If the traditional deep learning framework needs to support a new operator, it usually needs to be highly optimized by experts or hardware vendors to be usable in practice, which is inefficient. The deep learning compiler has proved to be an effective solution to this problem, but it still suffers from unbearably long overall optimization time. In this paper, aiming at the XGBoost cost model in Ansor, we train a cost model based on LightGBM algorithm, which accelerates the optimization time without compromising the accuracy. Experimentation with real hardware shows that our algorithm provides 1.8× speed up in optimization over XGBoost, while also improving inference time of the deep networks by 6.1 %.

Download Full-text