PharmaNet: Pharmaceutical discovery with deep recurrent neural networks

AbstractThe discovery and development of novel pharmaceuticals is an area of active research mainly due to the large investments required and long payback times. As of 2016, the development of a novel drug candidate required up to $ USD 2.6 billion in investment for only 10% rate of approval by the FDA. To help decreasing the costs associated with the process, a number of in silico approaches have been developed with relatively low success due to limited predicting performance. Here, we introduced a machine learning-based algorithm as an alternative for a more accurate search of new pharmacological candidates, which takes advantage of Recurrent Neural Networks (RNN) for active molecule prediction within large databases. Our approach, termed PharmaNet was implemented here to search for ligands against specific cell receptors within 102 targets of the DUD-E database, which contains 22886 active molecules. PharmaNet comprises three main phases. First, a SMILES representation of the molecule is converted into a raw molecular image. Second, a convolutional encoder processes the data to obtain a fingerprint molecular image that is finally analyzed by a Recurrent Neural Network (RNN). This approach enables precise predictions of the molecules’ target on the basis of the feature extraction, the sequence analysis and the relevant information filtered out throughout the process. Molecule Target prediction is a highly unbalanced detection problem and therefore, we propose that an adequate evaluation metric of performance is the area under the Normalized Average Precision (NAP) curve. PharmaNet largely surpasses the previous state-of-the-art method with 95.8% in the Receiver Operating Characteristic curve (ROC-AUC) and 58.9% in the NAP curve. We obtained a perfect performance for human farnesyl pyrophosphate synthase (FPPS), which is a potential target for antimicrobial and anticancer treatments. We decided to test PharmaNet for activity prediction against FPPS by searching in the CHEMBL data set. We obtained [3] potential inhibitors that were further validated through both molecular docking and in silico toxicity prediction. Most importantly, one of this candidates, CHEMBL2007613, was predicted as a potential antiviral due to its involvement on the PCDH17 pathway, which has been reported to be related to viral infections.

Download Full-text

PharmaNet: Pharmaceutical discovery with deep recurrent neural networks

PLoS ONE ◽

10.1371/journal.pone.0241728 ◽

2021 ◽

Vol 16 (4) ◽

pp. e0241728

Author(s):

Paola Ruiz Puentes ◽

Natalia Valderrama ◽

Cristina González ◽

Laura Daza ◽

Carolina Muñoz-Camargo ◽

...

Keyword(s):

Neural Networks ◽

Recurrent Neural Networks ◽

In Silico ◽

Viral Infections ◽

Characteristic Curve ◽

Relevant Information ◽

Drug Candidate ◽

Specific Cell ◽

Data Set ◽

Molecular Image

The discovery and development of novel pharmaceuticals is an area of active research mainly due to the large investments required and long payback times. As of 2016, the development of a novel drug candidate required up to $ USD 2.6 billion in investment for only 10% rate of approval by the FDA. To help decreasing the costs associated with the process, a number of in silico approaches have been developed with relatively low success due to limited predicting performance. Here, we introduced a machine learning-based algorithm as an alternative for a more accurate search of new pharmacological candidates, which takes advantage of Recurrent Neural Networks (RNN) for active molecule prediction within large databases. Our approach, termed PharmaNet was implemented here to search for ligands against specific cell receptors within 102 targets of the DUD-E database, which contains 22886 active molecules. PharmaNet comprises three main phases. First, a SMILES representation of the molecule is converted into a raw molecular image. Second, a convolutional encoder processes the data to obtain a fingerprint molecular image that is finally analyzed by a Recurrent Neural Network (RNN). This approach enables precise predictions of the molecules’ target on the basis of the feature extraction, the sequence analysis and the relevant information filtered out throughout the process. Molecule Target prediction is a highly unbalanced detection problem and therefore, we propose that an adequate evaluation metric of performance is the area under the Normalized Average Precision (NAP) curve. PharmaNet largely surpasses the previous state-of-the-art method with 97.7% in the Receiver Operating Characteristic curve (ROC-AUC) and 65.5% in the NAP curve. We obtained a perfect performance for human farnesyl pyrophosphate synthase (FPPS), which is a potential target for antimicrobial and anticancer treatments. We decided to test PharmaNet for activity prediction against FPPS by searching in the CHEMBL data set. We obtained three (3) potential inhibitors that were further validated through both molecular docking and in silico toxicity prediction. Most importantly, one of this candidates, CHEMBL2007613, was predicted as a potential antiviral due to its involvement on the PCDH17 pathway, which has been reported to be related to viral infections.

Download Full-text

MONITORING AND CLASSIFYING THE STATE OF HARD DISKS USING RECURRENT NEURAL NETWORKS

Kontrol Diagnostika ◽

10.14489/td.2021.10.pp.036-043 ◽

2021 ◽

pp. 36-43

Author(s):

L. A. Demidova ◽

A. V. Filatov

Keyword(s):

Neural Networks ◽

Recurrent Neural Networks ◽

Experimental Studies ◽

Machine Learning Algorithms ◽

Classification Model ◽

Dense Layer ◽

Hard Disks ◽

Destructive Testing ◽

Data Set ◽

State Classification

The article considers an approach to solving the problem of monitoring and classifying the states of hard disks, which is solved on a regular basis, within the framework of the concept of non-destructive testing. It is proposed to solve this problem by developing a classification model using machine learning algorithms, in particular, using recurrent neural networks with Simple RNN, LSTM and GRU architectures. To develop a classification model, a data set based on the values of SMART sensors installed on hard disks it used. It represents a group of multidimensional time series. At the same time, the structure of the classification model contains two layers of a neural network with one of the recurrent architectures, as well as a Dropout layer and a Dense layer. The results of experimental studies confirming the advantages of LSTM and GRU architectures as part of hard disk state classification models are presented.

Download Full-text

Recurrent Neural Networks for Predicting Outcomes after Liver Transplantation: Representing Temporal Sequence of Clinical Observations

Methods of Information in Medicine ◽

10.1055/s-0038-1634197 ◽

2001 ◽

Vol 40 (05) ◽

pp. 386-391 ◽

Cited By ~ 10

Author(s):

H. R. Doyle ◽

B. Parmanto

Keyword(s):

Neural Networks ◽

Time Series ◽

Liver Transplant ◽

Recurrent Neural Networks ◽

Current Model ◽

Time Series Prediction ◽

Data Set ◽

Test Set ◽

Clinical Observations ◽

Learning Set

Summary Objectives: This paper investigates a version of recurrent neural network with the backpropagation through time (BPTT) algorithm for predicting liver transplant graft failure based on a time series sequence of clinical observations. The objective is to improve upon the current approaches to liver transplant outcome prediction by developing a more complete model that takes into account not only the preoperative risk assessment, but also the early postoperative history. Methods: A 6-fold cross-validation procedure was used to measure the performance of the networks. The data set was divided into a learning set and a test set by maintaining the same proportion of positive and negative cases in the original set. The effects of network complexity on overfitting were investigated by constructing two types of networks with different numbers of hidden units. For each type of network, 10 individual networks were trained on the learning set and used to form a committee. The performance of the networks was measured exhaustively with respect to both the entire training and test sets. Results: The networks were capable of learning the time series problem and achieved good performances of 90% correct classification on the learning set and 78% on the test set. The prediction accuracy increases as more information becomes progressively available after the operation with the daily improvement of 10% on the learning set and 5% on the test set. Conclusions: Recurrent neural networks trained with BPTT algorithm are capable of learning to represent temporal behavior of the time series prediction task. This model is an improvement upon the current model that does not take into account postoperative temporal information.

Download Full-text

Classification of adaptor proteins using recurrent neural networks and PSSM profiles

10.26686/wgtn.14315876.v1 ◽

2021 ◽

Author(s):

NQ Khanh Le ◽

QH Nguyen ◽

X Chen ◽

S Rahardja ◽

Binh Nguyen

Keyword(s):

Neural Networks ◽

Computational Biology ◽

Recurrent Neural Networks ◽

Protein Function ◽

Intracellular Signaling ◽

Protein Function Prediction ◽

Characteristic Curve ◽

Adaptor Proteins ◽

The Neural Networks ◽

Prediction Problems

© 2019 The Author(s). Background: Adaptor proteins are carrier proteins that play a crucial role in signal transduction. They commonly consist of several modular domains, each having its own binding activity and operating by forming complexes with other intracellular-signaling molecules. Many studies determined that the adaptor proteins had been implicated in a variety of human diseases. Therefore, creating a precise model to predict the function of adaptor proteins is one of the vital tasks in bioinformatics and computational biology. Few computational biology studies have been conducted to predict the protein functions, and in most of those studies, position specific scoring matrix (PSSM) profiles had been used as the features to be fed into the neural networks. However, the neural networks could not reach the optimal result because the sequential information in PSSMs has been lost. This study proposes an innovative approach by incorporating recurrent neural networks (RNNs) and PSSM profiles to resolve this problem. Results: Compared to other state-of-the-art methods which had been applied successfully in other problems, our method achieves enhancement in all of the common measurement metrics. The area under the receiver operating characteristic curve (AUC) metric in prediction of adaptor proteins in the cross-validation and independent datasets are 0.893 and 0.853, respectively. Conclusions: This study opens a research path that can promote the use of RNNs and PSSM profiles in bioinformatics and computational biology. Our approach is reproducible by scientists that aim to improve the performance results of different protein function prediction problems. Our source code and datasets are available at https://github.com/ngphubinh/adaptors.

Download Full-text

In Silico Prediction of siRNA Ionizable-Lipid Nanoparticles in vivo Efficacy: Machine Learning Modeling Based on Formulation and Molecular Descriptors

10.20944/preprints202108.0254.v1 ◽

2021 ◽

Author(s):

Abdelkader A Metwally ◽

Amira A Nayel ◽

Rania M Hathout

Keyword(s):

Machine Learning ◽

Neural Networks ◽

Artificial Neural Networks ◽

In Silico ◽

Molecular Descriptors ◽

Lipid Nanoparticles ◽

Data Set ◽

In Vivo Efficacy ◽

Artificial Neural

In silico prediction of the in vivo efficacy of siRNA ionizable-lipid nanoparticles is desirable yet never achieved before. This study aims to computationally predict siRNA nanoparticles in vivo efficacy, which saves time and resources. A data set containing 120 entries was prepared by combining molecular descriptors of the ionizable lipids together with two nanoparticles formulation characteristics. Input descriptor combinations were selected by an evolutionary algorithm. Artificial neural networks, support vector machines and partial least squares regression were used for QSAR modeling. Depending on how the data set is split, two training sets and two external validation sets were prepared. Training and validation sets contained 90 and 30 entries respectively. The results showed the successful predictions of validation set log(dose) with R2val = 0.86 – 0.89 and 0.75 – 80 for validation sets one and two respectively. Artificial neural networks resulted in the best R2val for both validation sets. For predictions that have high bias, improvement of R2val from 0.47 to 0.96 was achieved by selecting the training set lipids lying within the applicability domain. In conclusion, in vivo performance of siRNA nanoparticles was successfully predicted by combining cheminformatics with machine learning techniques.

Download Full-text

Building Energy Consumption Raw Data Forecasting Using Data Cleaning and Deep Recurrent Neural Networks

Buildings ◽

10.3390/buildings9090204 ◽

2019 ◽

Vol 9 (9) ◽

pp. 204 ◽

Cited By ~ 5

Author(s):

Yang ◽

Tan ◽

Santamouris ◽

Lee

Keyword(s):

Neural Network ◽

Neural Networks ◽

Energy Consumption ◽

Missing Data ◽

Recurrent Neural Networks ◽

Building Energy ◽

Building Energy Consumption ◽

Data Set ◽

Raw Data ◽

Deep Recurrent Neural Network

With the rising focus on building energy big data analysis, there lacks a framework for raw data preprocessing to answer the question of how to handle the missing data in the raw data set. This study presents a methodology and framework for building energy consumption raw data forecasting. A case building is used to forecast the energy consumption by using deep recurrent neural networks. Four different methodologies to impute missing data in the raw data set are compared and implemented. The question of sensitivity of gap size and available data percentage on the imputation accuracy was tested. The cleaned data were then used for building energy forecasting. While the existing studies explored only the use of small recurrent networks of 2 layers and less, the question of whether a deep network of more than 2 layers would be performing better for building energy consumption forecasting should be explored. In addition, the problem of overfitting has been cited as a significant problem in using deep networks. In this study, the deep recurrent neural network is then used to explore the use of deeper networks and their regularization in the context of an energy load forecasting task. The results show a mean absolute error of 2.1 can be achieved through the 2*32 gated neural network model. In applying regularization methods to overcome model overfitting, the study found that weights regularization did indeed delay the onset of overfitting.

Download Full-text

Evaluation of Recurrent Neural Network and its Variants for Intrusion Detection System (IDS)

International Journal of Information System Modeling and Design ◽

10.4018/ijismd.2017070103 ◽

2017 ◽

Vol 8 (3) ◽

pp. 43-63 ◽

Cited By ~ 17

Author(s):

R Vinayakumar ◽

K.P. Soman ◽

Prabaharan Poornachandran

Keyword(s):

Machine Learning ◽

Neural Networks ◽

Intrusion Detection ◽

Recurrent Neural Networks ◽

Machine Learning Algorithms ◽

Time Range ◽

Sequence Information ◽

Time Lags ◽

Data Set ◽

Arbitrary Length

This article describes how sequential data modeling is a relevant task in Cybersecurity. Sequences are attributed temporal characteristics either explicitly or implicitly. Recurrent neural networks (RNNs) are a subset of artificial neural networks (ANNs) which have appeared as a powerful, principle approach to learn dynamic temporal behaviors in an arbitrary length of large-scale sequence data. Furthermore, stacked recurrent neural networks (S-RNNs) have the potential to learn complex temporal behaviors quickly, including sparse representations. To leverage this, the authors model network traffic as a time series, particularly transmission control protocol / internet protocol (TCP/IP) packets in a predefined time range with a supervised learning method, using millions of known good and bad network connections. To find out the best architecture, the authors complete a comprehensive review of various RNN architectures with its network parameters and network structures. Ideally, as a test bed, they use the existing benchmark Defense Advanced Research Projects Agency / Knowledge Discovery and Data Mining (DARPA) / (KDD) Cup ‘99' intrusion detection (ID) contest data set to show the efficacy of these various RNN architectures. All the experiments of deep learning architectures are run up to 1000 epochs with a learning rate in the range [0.01-0.5] on a GPU-enabled TensorFlow and experiments of traditional machine learning algorithms are done using Scikit-learn. Experiments of families of RNN architecture achieved a low false positive rate in comparison to the traditional machine learning classifiers. The primary reason is that RNN architectures are able to store information for long-term dependencies over time-lags and to adjust with successive connection sequence information. In addition, the effectiveness of RNN architectures are shown for the UNSW-NB15 data set.

Download Full-text

A Review of Recurrent Neural Networks: LSTM Cells and Network Architectures

Neural Computation ◽

10.1162/neco_a_01199 ◽

2019 ◽

Vol 31 (7) ◽

pp. 1235-1270 ◽

Cited By ~ 54

Author(s):

Yong Yu ◽

Xiaosheng Si ◽

Changhua Hu ◽

Jianxun Zhang

Keyword(s):

Neural Networks ◽

Recurrent Neural Networks ◽

Short Term Memory ◽

Cell Structure ◽

Relevant Information ◽

Future Research ◽

Sequential Data ◽

Learning Capacity ◽

Research Areas ◽

Almost All

Recurrent neural networks (RNNs) have been widely adopted in research areas concerned with sequential data, such as text, audio, and video. However, RNNs consisting of sigma cells or tanh cells are unable to learn the relevant information of input data when the input gap is large. By introducing gate functions into the cell structure, the long short-term memory (LSTM) could handle the problem of long-term dependencies well. Since its introduction, almost all the exciting results based on RNNs have been achieved by the LSTM. The LSTM has become the focus of deep learning. We review the LSTM cell and its variants to explore the learning capacity of the LSTM cell. Furthermore, the LSTM networks are divided into two broad categories: LSTM-dominated networks and integrated LSTM networks. In addition, their various applications are discussed. Finally, future research directions are presented for LSTM networks.

Download Full-text

Feature Explanations in Recurrent Neural Networks for Predicting Risk of Mortality in Intensive Care Patients

Journal of Personalized Medicine ◽

10.3390/jpm11090934 ◽

2021 ◽

Vol 11 (9) ◽

pp. 934

Author(s):

Thanakron Na Pattalung ◽

Thammasin Ingviya ◽

Sitthichok Chaichulee

Keyword(s):

Neural Networks ◽

Critical Care ◽

Intensive Care ◽

Recurrent Neural Networks ◽

Characteristic Curve ◽

Vital Signs ◽

Care Staff ◽

Intensive Care Patients ◽

Clinical Observations ◽

Risk Of Mortality

Critical care staff are presented with a large amount of data, which made it difficult to systematically evaluate. Early detection of patients whose condition is deteriorating could reduce mortality, improve treatment outcomes, and allow a better use of healthcare resources. In this study, we propose a data-driven framework for predicting the risk of mortality that combines high-accuracy recurrent neural networks with interpretable explanations. Our model processes time-series of vital signs and laboratory observations to predict the probability of a patient’s mortality in the intensive care unit (ICU). We investigated our approach on three public critical care databases: Multiparameter Intelligent Monitoring in Intensive Care III (MIMIC-III), MIMIC-IV, and eICU. Our models achieved an area under the receiver operating characteristic curve (AUC) of 0.87–0.91. Our approach was not only able to provide the predicted mortality risk but also to recognize and explain the historical contributions of the associated factors to the prediction. The explanations provided by our model were consistent with the literature. Patients may benefit from early intervention if their clinical observations in the ICU are continuously monitored in real time.

Download Full-text

Inversion of vehicle-induced signals based on seismic interferometry and recurrent neural networks

Geophysics ◽

10.1190/geo2020-0498.1 ◽

2021 ◽

pp. 1-44

Author(s):

Lu Liu ◽

Yujin Liu ◽

Tao Li ◽

Yi He ◽

Yue Du ◽

...

Keyword(s):

Neural Networks ◽

Surface Wave ◽

Real Time ◽

Shear Wave ◽

Recurrent Neural Networks ◽

Field Data ◽

Seismic Interferometry ◽

Data Set ◽

Wave Velocities ◽

Shear Wave Velocities

Vehicle-induced vibrations provide useful signals for passive seismic exploration. Such signals are repeatable and environmentally friendly, and hence can provide an economical way to analyze subsurface structures. We propose a new workflow to monitor the roads or railways by producing 1-D subsurface shear-wave velocities in real time. This workflow consists of two steps: seismic interferometry and recurrent neural networks (RNN). Seismic interferometry can efficiently retrieve the surface waves by crosscorrelating the vehicle-induced vibrations. The RNN is designed to first encode the picked dispersion curve into a fixed-length vector and then decode the vector into 1-D shear-wave velocities. To simulate the railway vibrations, we first analyze the time-dependent characteristic of the high-speed-train source and verify its mathematical expression by comparing the frequency spectrum of real data and the synthetic one. We then introduce the RNN-based surface-wave dispersion inversion method and validate the designed network structure using the 3D SEG/EAGE overthrust model. Finally, seismic interferometry and RNN-based surface-wave inversion are applied to a synthetic train-induced data set, a 33-minute field record of railway vibrations and a 76-minute field data of road vibrations, respectively. Both of the synthetic and field data tests show that the proposed workflow can be a feasible and cost-effective tool for real-time monitoring of the subsurface media along roads and railways.

Download Full-text