scholarly journals Interpreting Neural Networks for Biological Sequences by Learning Stochastic Masks

2021 ◽  
Author(s):  
Johannes Linder ◽  
Alyssa La Fleur ◽  
Zibo Chen ◽  
Ajasja Ljubetič ◽  
David Baker ◽  
...  
Keyword(s):  
Neural Networks ◽  
Language Processing ◽  
Protein Interactions ◽  
De Novo ◽  
Regulatory Elements ◽  
Structural Determinants ◽  
Model Interpretation ◽  
Scoring Matrices ◽  
Non Linear ◽  
Linear Interactions

AbstractSequence-based neural networks can learn to make accurate predictions from large biological datasets, but model interpretation remains challenging. Many existing feature attribution methods are optimized for continuous rather than discrete input patterns and assess individual feature importance in isolation, making them ill-suited for interpreting non-linear interactions in molecular sequences. Building on work in computer vision and natural language processing, we developed an approach based on deep generative modeling - Scrambler networks - wherein the most salient sequence positions are identified with learned input masks. Scramblers learn to generate Position-Specific Scoring Matrices (PSSMs) where unimportant nucleotides or residues are ‘scrambled’ by raising their entropy. We apply Scramblers to interpret the effects of genetic variants, uncover non-linear interactions between cis-regulatory elements, explain binding specificity for protein-protein interactions, and identify structural determinants of de novo designed proteins. We show that interpretation based on a generative model allows for efficient attribution across large datasets and results in high-quality explanations, often outperforming state-of-the-art methods.

scholarly journals De novo prediction of RNA-protein interactions with Graph Neural Networks

2021 ◽  
Author(s):  
Viplove Arora ◽  
Guido Sanguinetti
Keyword(s):  
Neural Networks ◽  
Protein Interactions ◽  
Large Scale ◽  
Rna Binding ◽  
De Novo ◽  
Rna Binding Proteins ◽  
Large Data ◽  
Data Sets ◽  
Graph Neural Networks ◽  
Post Transcriptional Regulation

RNA-binding proteins (RBPs) are key co- and post-transcriptional regulators of gene expression, playing a crucial role in many biological processes. Experimental methods like CLIP-seq have enabled the identification of transcriptome-wide RNA-protein interactions for select proteins, however the time and resource intensive nature of these technologies call for the development of computational methods to complement their predictions. Here we leverage recent, large-scale CLIP-seq experiments to construct a de novo predictor of RNA-protein interactions based on graph neural networks (GNN). We show that the GNN method allows not only to predict missing links in a RNA-protein network, but to predict the entire complement of targets of previously unassayed proteins, and even to reconstruct the entire network of RNA-protein interactions in different conditions based on minimal information. Our results demonstrate the potential of machine learning methods to extract useful information on post-transcriptional regulation from large data sets.

scholarly journals Neural Language Modeling for Molecule Generation

2021 ◽  
Author(s):  
Sanjar Adilov
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Deep Learning ◽  
Natural Language Processing ◽  
Drug Design ◽  
Natural Language ◽  
Language Processing ◽  
De Novo ◽  
Language Modeling ◽  
Machine Learning Methods

Generative neural networks have shown promising results in <i>de novo</i> drug design. Recent studies suggest that one of the efficient ways to produce novel molecules matching target properties is to model SMILES sequences using deep learning in a way similar to language modeling in natural language processing. In this paper, we present a survey of various machine learning methods for SMILES-based language modeling and propose our benchmarking results on a standardized subset of ChEMBL database.

scholarly journals Neural Language Modeling for Molecule Generation

2021 ◽  
Author(s):  
Sanjar Adilov
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Deep Learning ◽  
Natural Language Processing ◽  
Drug Design ◽  
Natural Language ◽  
Language Processing ◽  
De Novo ◽  
Language Modeling ◽  
Machine Learning Methods

Generative neural networks have shown promising results in <i>de novo</i> drug design. Recent studies suggest that one of the efficient ways to produce novel molecules matching target properties is to model SMILES sequences using deep learning in a way similar to language modeling in natural language processing. In this paper, we present a survey of various machine learning methods for SMILES-based language modeling and propose our benchmarking results on a standardized subset of ChEMBL database.

scholarly journals Non-linear Neurons with Human-like Apical Dendrite Activations

2020 ◽  
Author(s):  
Mariana-Iuliana Georgescu ◽  
Radu Tudor Ionescu ◽  
Nicolae-Catalin Ristea ◽  
Nicu Sebe
Keyword(s):  
Neural Networks ◽  
Language Processing ◽  
Apical Dendrite ◽  
Activation Function ◽  
Data Sets ◽  
Network Architectures ◽  
Logical Function ◽  
Non Linear ◽  
Hidden Layer ◽  
Decision Boundaries

<pre>In order to classify linearly non-separable data, neurons are typically organized into multi-layer neural networks that are equipped with at least one hidden layer. Inspired by some recent discoveries in neuroscience, we propose a new neuron model along with a novel activation function enabling learning of non-linear decision boundaries using a single neuron. We show that a standard neuron followed by the novel apical dendrite activation (ADA) can learn the XOR logical function with 100% accuracy. Furthermore, we conduct experiments on three benchmark data sets from computer vision and natural language processing, i.e. Fashion-MNIST, UTKFace and MOROCO, showing that the ADA and the leaky ADA functions provide superior results to Rectified Liner Units (ReLU) and leaky ReLU, for various neural network architectures, e.g. 1-hidden layer or 2-hidden layers multi-layer perceptrons (MLPs) and convolutional neural networks (CNNs) such as LeNet, VGG, ResNet and Character-level CNN. We also obtain further improvements when we change the standard model of the neuron with our pyramidal neuron with apical dendrite activations (PyNADA).<br></pre>

scholarly journals Words to Matter: De novo Architected Materials Design Using Transformer Neural Networks

2021 ◽  
Vol 8 ◽  
Author(s):  
Zhenze Yang ◽  
Markus J. Buehler
Keyword(s):  
Neural Networks ◽  
Natural Language ◽  
Language Processing ◽  
De Novo ◽  
Three Dimensional ◽  
Neural Nets ◽  
Element Analysis ◽  
Text Input ◽  
Material Specimen ◽  
Three Dimensional Models

Transformer neural networks have become widely used in a variety of AI applications, enabling significant advances in Natural Language Processing (NLP) and computer vision. Here we demonstrate the use of transformer neural networks in the de novo design of architected materials using a unique approach based on text input that enables the design to be directed by descriptive text, such as “a regular lattice of steel”. Since transformer neural nets enable the conversion of data from distinct forms into one another, including text into images, such methods have the potential to be used as a natural-language-driven tool to develop complex materials designs. In this study we use the Contrastive Language-Image Pre-Training (CLIP) and VQGAN neural networks in an iterative process to generate images that reflect text prompt driven materials designs. We then use the resulting images to generate three-dimensional models that can be realized using additive manufacturing, resulting in physical samples of these text-based materials. We present several such word-to-matter examples, and analyze 3D printed material specimen through associated additional finite element analysis, especially focused on mechanical properties including mechanism design. As an emerging new field, such language-based design approaches can have profound impact, including the use of transformer neural nets to generate machine code for 3D printing, optimization of processing conditions, and other end-to-end design environments that intersect directly with human language.

scholarly journals Transmol: Repurposing Language Model for Molecular Generation

2021 ◽  
Author(s):  
Rustam Zhumagambetov ◽  
Vsevolod A. Peshkov ◽  
Siamac Fazli
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Image Processing ◽  
Language Processing ◽  
Scientific Community ◽  
De Novo ◽  
Language Model ◽  
The Core ◽  
Molecular Libraries ◽  
Core Metrics

Recent advances in convolutional neural networks have inspired the application of deep learning to other disciplines. Even though image processing and natural language processing have turned out to be the most successful, there are many other areas that have benefited, like computational chemistry in general and drug design in particular. From 2018 the scientific community has seen a surge of methodologies related to the generation of diverse molecular libraries using machine learning. However, no algorithm used an attention mechanisms for <i>de novo</i> molecular generation. Here we employ a variant of transformers, a recent NLP architecture, for this purpose. We have achieved a statistically significant increase in some of the core metrics of the MOSES benchmark. Furthermore, a novel way of generating libraries fusing two molecules as seeds has been described.

scholarly journals Unified AI framework to uncover deep interrelationships between gene expression and Alzheimer’s disease neuropathologies

2020 ◽  
Author(s):  
Nicasia Beebe-Wang ◽  
Safiye Celik ◽  
Ethan Weinberger ◽  
Pascal Sturmfels ◽  
Philip L. De Jager ◽  
...  
Keyword(s):  
Gene Expression ◽  
Neural Networks ◽  
Deep Neural Networks ◽  
Joint Analysis ◽  
Linear Relationships ◽  
Non Linear ◽  
Consistent Manner ◽  
Complex Relationships ◽  
Expression Program ◽  
Linear Interactions

ABSTRACTDeep neural networks offer a promising approach for capturing complex, non-linear relationships among variables. Because they require immense sample sizes, their potential has yet to be fully tapped for understanding complex relationships between gene expression and human phenotypes. Encouragingly, a growing number of diseases are being studied through consortium efforts. Here we introduce a new analysis framework, namely MD-AD (Multi-task Deep learning for Alzheimer’s Disease neuropathology), which leverages an unexpected synergy between deep neural networks and multi-cohort settings. In these settings, true joint analysis can be stymied using conventional statistical methods, which (1) require “harmonized” phenotypes (i.e., measured in a highly consistent manner) and (2) tend to capture cohort-level variations, obscuring the subtler true disease signals. Instead, MD-AD incorporates multiple related phenotypes sparsely measured across cohorts, and learns complex, non-linear interactions between genes and phenotypes not discovered using conventional expression data analysis methods (e.g., component analysis and module detection), enabling the model to capture subtler signals than cohort-level variations. Applied to the largest available collection of brain samples (N=1,758), we demonstrate that MD-AD learns a truly generalizable relationship between gene expression program and AD-related neuropathology. The learned program generalizes in several important ways, including recapitulation of the disease progress in animal models and across tissue types, and we show that such generalizability is not achieved by previous statistical paradigms. Its ability to identify genes with high non-linear relevance to neuropathology enabled us to identify a sex-specific relationship between neuropathology and immune response across microglia, providing a nuanced context for association between inflammatory genes and AD.

scholarly journals Transmol: Repurposing Language Model for Molecular Generation

2021 ◽  
Author(s):  
Rustam Zhumagambetov ◽  
Vsevolod A. Peshkov ◽  
Siamac Fazli
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Image Processing ◽  
Language Processing ◽  
Scientific Community ◽  
De Novo ◽  
Language Model ◽  
The Core ◽  
Molecular Libraries ◽  
Core Metrics

Recent advances in convolutional neural networks have inspired the application of deep learning to other disciplines. Even though image processing and natural language processing have turned out to be the most successful, there are many other areas that have benefited, like computational chemistry in general and drug design in particular. From 2018 the scientific community has seen a surge of methodologies related to the generation of diverse molecular libraries using machine learning. However, no algorithm used an attention mechanisms for <i>de novo</i> molecular generation. Here we employ a variant of transformers, a recent NLP architecture, for this purpose. We have achieved a statistically significant increase in some of the core metrics of the MOSES benchmark. Furthermore, a novel way of generating libraries fusing two molecules as seeds has been described.

scholarly journals Non-linear Neurons with Human-like Apical Dendrite Activations

2020 ◽  
Author(s):  
Mariana-Iuliana Georgescu ◽  
Radu Tudor Ionescu ◽  
Nicolae-Catalin Ristea ◽  
Nicu Sebe
Keyword(s):  
Neural Networks ◽  
Language Processing ◽  
Apical Dendrite ◽  
Activation Function ◽  
Data Sets ◽  
Network Architectures ◽  
Logical Function ◽  
Non Linear ◽  
Hidden Layer ◽  
Decision Boundaries

<pre>In order to classify linearly non-separable data, neurons are typically organized into multi-layer neural networks that are equipped with at least one hidden layer. Inspired by some recent discoveries in neuroscience, we propose a new neuron model along with a novel activation function enabling learning of non-linear decision boundaries using a single neuron. We show that a standard neuron followed by the novel apical dendrite activation (ADA) can learn the XOR logical function with 100% accuracy. Furthermore, we conduct experiments on three benchmark data sets from computer vision and natural language processing, i.e. Fashion-MNIST, UTKFace and MOROCO, showing that the ADA and the leaky ADA functions provide superior results to Rectified Liner Units (ReLU) and leaky ReLU, for various neural network architectures, e.g. 1-hidden layer or 2-hidden layers multi-layer perceptrons (MLPs) and convolutional neural networks (CNNs) such as LeNet, VGG, ResNet and Character-level CNN. We also obtain further improvements when we change the standard model of the neuron with our pyramidal neuron with apical dendrite activations (PyNADA).<br></pre>

Target-Templated de novo Design of Macrocyclic D-/L-Peptides: Inhibitors of the PD-1/PD-L1 Interaction

2020 ◽  
Author(s):  
Salvador Guardiola ◽  
Monica Varese ◽  
Xavier Roig ◽  
Jesús Garcia ◽  
Ernest Giralt
Keyword(s):  
Protein Interactions ◽  
Cyclic Peptides ◽  
General Framework ◽  
Large Scale ◽  
De Novo ◽  
Inhibitory Effect ◽  
Original Text ◽  
Protein Protein Interactions ◽  
Retraction Notice ◽  
Pharmaceutical Properties

<p>NOTE: This preprint has been retracted by consensus from all authors. See the retraction notice in place above; the original text can be found under "Version 1", accessible from the version selector above.</p><p><br></p><p>------------------------------------------------------------------------</p><p><br></p><p>Peptides, together with antibodies, are among the most potent biochemical tools to modulate challenging protein-protein interactions. However, current structure-based methods are largely limited to natural peptides and are not suitable for designing target-specific binders with improved pharmaceutical properties, such as macrocyclic peptides. Here we report a general framework that leverages the computational power of Rosetta for large-scale backbone sampling and energy scoring, followed by side-chain composition, to design heterochiral cyclic peptides that bind to a protein surface of interest. To showcase the applicability of our approach, we identified two peptides (PD-<i>i</i>3 and PD-<i>i</i>6) that target PD-1, a key immune checkpoint, and work as protein ligand decoys. A comprehensive biophysical evaluation confirmed their binding mechanism to PD-1 and their inhibitory effect on the PD-1/PD-L1 interaction. Finally, elucidation of their solution structures by NMR served as validation of our <i>de novo </i>design approach. We anticipate that our results will provide a general framework for designing target-specific drug-like peptides.<i></i></p>

Sign in / Sign up

Export Citation Format

Share Document