scholarly journals Efficient generative modeling of protein sequences using simple autoregressive models

2021 ◽  
Author(s):  
Jeanne Trinquier ◽  
Guido Uguzzoni ◽  
Andrea Pagnani ◽  
Francesco Zamponi ◽  
Martin Weigt
Keyword(s):  
Protein Design ◽  
Sequence Space ◽  
Sequence Data ◽  
Computational Cost ◽  
Specific Protein ◽  
Generative Models ◽  
Amino Acid Sequences ◽  
Autoregressive Models ◽  
Generative Modeling ◽  
Model Probability

Generative models emerge as promising candidates for novel sequence-data driven approaches to protein design, and for the extraction of structural and functional information about proteins deeply hidden in rapidly growing sequence databases. Here we propose simple autoregressive models as highly accurate but computationally extremely efficient generative sequence models. We show that they perform similarly to existing approaches based on Boltzmann machines or deep generative models, but at a substantially lower computational cost. Furthermore, the simple structure of our models has distinctive mathematical advantages, which translate into an improved applicability in sequence generation and evaluation. Using these models, we can easily estimate both the model probability of a given sequence, and the size of the functional sequence space related to a specific protein family. In the case of response regulators, we find a huge number of ca. 1068 sequences, which nevertheless constitute only the astronomically small fraction 10-80 of all amino-acid sequences of the same length. These findings illustrate the potential and the difficulty in exploring sequence space via generative sequence models.

scholarly journals Efficient generative modeling of protein sequences using simple autoregressive models

2021 ◽  
Vol 12 (1) ◽  
Author(s):  
Jeanne Trinquier ◽  
Guido Uguzzoni ◽  
Andrea Pagnani ◽  
Francesco Zamponi ◽  
Martin Weigt
Keyword(s):  
Protein Design ◽  
Sequence Space ◽  
Sequence Data ◽  
Computational Cost ◽  
Specific Protein ◽  
Generative Models ◽  
Amino Acid Sequences ◽  
Autoregressive Models ◽  
Computationally Efficient ◽  
Generative Modeling

AbstractGenerative models emerge as promising candidates for novel sequence-data driven approaches to protein design, and for the extraction of structural and functional information about proteins deeply hidden in rapidly growing sequence databases. Here we propose simple autoregressive models as highly accurate but computationally efficient generative sequence models. We show that they perform similarly to existing approaches based on Boltzmann machines or deep generative models, but at a substantially lower computational cost (by a factor between 102 and 103). Furthermore, the simple structure of our models has distinctive mathematical advantages, which translate into an improved applicability in sequence generation and evaluation. Within these models, we can easily estimate both the probability of a given sequence, and, using the model’s entropy, the size of the functional sequence space related to a specific protein family. In the example of response regulators, we find a huge number of ca. 1068 possible sequences, which nevertheless constitute only the astronomically small fraction 10−80 of all amino-acid sequences of the same length. These findings illustrate the potential and the difficulty in exploring sequence space via generative sequence models.

scholarly journals De Novo Protein Design for Novel Folds using Guided Conditional Wasserstein Generative Adversarial Networks (gcWGAN)

2019 ◽  
Author(s):  
Mostafa Karimi ◽  
Shaowen Zhu ◽  
Yue Cao ◽  
Yang Shen
Keyword(s):  
Protein Design ◽  
Sequence Space ◽  
De Novo ◽  
Sequence Data ◽  
Generative Models ◽  
Current Data ◽  
Data Driven ◽  
Supplementary Information ◽  
Generative Adversarial Networks ◽  
Sequence Structure

AbstractMotivationFacing data quickly accumulating on protein sequence and structure, this study is addressing the following question: to what extent could current data alone reveal deep insights into the sequence-structure relationship, such that new sequences can be designed accordingly for novel structure folds?ResultsWe have developed novel deep generative models, constructed low-dimensional and generalizable representation of fold space, exploited sequence data with and without paired structures, and developed ultra-fast fold predictor as an oracle providing feedback. The resulting semi-supervised gcWGAN is assessed with the oracle over 100 novel folds not in the training set and found to generate more yields and cover 3.6 times more target folds compared to a competing data-driven method (cVAE). Assessed with structure predictor over representative novel folds (including one not even part of basis folds), gcWGAN designs are found to have comparable or better fold accuracy yet much more sequence diversity and novelty than cVAE. gcWGAN explores uncharted sequence space to design proteins by learning from current sequence-structure data. The ultra fast data-driven model can be a powerful addition to principle-driven design methods through generating seed designs or tailoring sequence space.AvailabilityData and source codes will be available upon [email protected] informationSupplementary data are available at Bioinformatics online.

scholarly journals AC2: An Efficient Protein Sequence Compression Tool Using Artificial Neural Networks and Cache-Hash Models

Entropy ◽  
2021 ◽  
Vol 23 (5) ◽  
pp. 530
Author(s):  
Milton Silva ◽  
Diogo Pratas ◽  
Armando J. Pinho
Keyword(s):  
Protein Sequence ◽  
Sequence Data ◽  
Specific Protein ◽  
General Purpose ◽  
Amino Acid Sequences ◽  
Input Size ◽  
Protein Sequence Data ◽  
Analysis Application ◽  
Straightforward Solution ◽  
Human Coronaviruses

Recently, the scientific community has witnessed a substantial increase in the generation of protein sequence data, triggering emergent challenges of increasing importance, namely efficient storage and improved data analysis. For both applications, data compression is a straightforward solution. However, in the literature, the number of specific protein sequence compressors is relatively low. Moreover, these specialized compressors marginally improve the compression ratio over the best general-purpose compressors. In this paper, we present AC2, a new lossless data compressor for protein (or amino acid) sequences. AC2 uses a neural network to mix experts with a stacked generalization approach and individual cache-hash memory models to the highest-context orders. Compared to the previous compressor (AC), we show gains of 2–9% and 6–7% in reference-free and reference-based modes, respectively. These gains come at the cost of three times slower computations. AC2 also improves memory usage against AC, with requirements about seven times lower, without being affected by the sequences’ input size. As an analysis application, we use AC2 to measure the similarity between each SARS-CoV-2 protein sequence with each viral protein sequence from the whole UniProt database. The results consistently show higher similarity to the pangolin coronavirus, followed by the bat and human coronaviruses, contributing with critical results to a current controversial subject. AC2 is available for free download under GPLv3 license.

scholarly journals Generating functional protein variants with variational autoencoders

2021 ◽  
Vol 17 (2) ◽  
pp. e1008736
Author(s):  
Alex Hawkins-Hooker ◽  
Florence Depardieu ◽  
Sebastien Baur ◽  
Guillaume Couairon ◽  
Arthur Chen ◽  
...  
Keyword(s):  
Protein Design ◽  
Protein Sequence ◽  
Rational Design ◽  
Sequence Data ◽  
3D Structure ◽  
Generative Models ◽  
Functional Protein ◽  
Long Distance ◽  
Functional Variants ◽  
Protein Sequence Data

The vast expansion of protein sequence databases provides an opportunity for new protein design approaches which seek to learn the sequence-function relationship directly from natural sequence variation. Deep generative models trained on protein sequence data have been shown to learn biologically meaningful representations helpful for a variety of downstream tasks, but their potential for direct use in the design of novel proteins remains largely unexplored. Here we show that variational autoencoders trained on a dataset of almost 70000 luciferase-like oxidoreductases can be used to generate novel, functional variants of the luxA bacterial luciferase. We propose separate VAE models to work with aligned sequence input (MSA VAE) and raw sequence input (AR-VAE), and offer evidence that while both are able to reproduce patterns of amino acid usage characteristic of the family, the MSA VAE is better able to capture long-distance dependencies reflecting the influence of 3D structure. To confirm the practical utility of the models, we used them to generate variants of luxA whose luminescence activity was validated experimentally. We further showed that conditional variants of both models could be used to increase the solubility of luxA without disrupting function. Altogether 6/12 of the variants generated using the unconditional AR-VAE and 9/11 generated using the unconditional MSA VAE retained measurable luminescence, together with all 23 of the less distant variants generated by conditional versions of the models; the most distant functional variant contained 35 differences relative to the nearest training set sequence. These results demonstrate the feasibility of using deep generative models to explore the space of possible protein sequences and generate useful variants, providing a method complementary to rational design and directed evolution approaches.

scholarly journals Conditional Generative Modeling for De Novo Protein Design with Hierarchical Functions

2021 ◽  
Author(s):  
Tim Kucera ◽  
Matteo Togninalli ◽  
Laetitia Meng-Papaxanthos
Keyword(s):  
Protein Design ◽  
De Novo ◽  
General Purpose ◽  
Generative Models ◽  
Generative Adversarial Network ◽  
Sequence Generation ◽  
Adversarial Network ◽  
Evaluation Scheme ◽  
Generative Modeling ◽  
Experimental Protocols

Motivation: Protein Design has become increasingly important for medical and biotechnological applications. Because of the complex mechanisms underlying protein formation, the creation of a novel protein requires tedious and time-consuming computational or experimental protocols. At the same time, Machine Learning has enabled to solve complex problems by leveraging the large amounts of available data, more recently with great improvements on the domain of generative modeling. Yet, generative models have mainly been applied to specific sub-problems of protein design. Results: Here we approach the problem of general purpose Protein Design conditioned on functional labels of the hierarchical Gene Ontology. Since a canonical way to evaluate generative models in this domain is missing, we devise an evaluation scheme of several biologically and statistically inspired metrics. We then develop the conditional generative adversarial network ProteoGAN and show that it outperforms several classic and more recent deep learning baselines for protein sequence generation. We further give insights into the model by analysing hyperparameters and ablation baselines. Lastly, we hypothesize that a functionally conditional model could create proteins with novel functions by combining labels and provide first steps into this direction of research.

scholarly journals The primary divisions of life: a phylogenomic approach employing composition-heterogeneous methods

2009 ◽  
Vol 364 (1527) ◽  
pp. 2197-2207 ◽  
Author(s):  
Peter G. Foster ◽  
Cymon J. Cox ◽  
T. Martin Embley
Keyword(s):  
Sequence Data ◽  
Phylogenetic Signal ◽  
Amino Acid Sequences ◽  
Rate Variation ◽  
Molecular Sequence Data ◽  
Protein Coding ◽  
Molecular Sequence ◽  
Heterogeneous Models ◽  
Matrix Heterogeneity ◽  
Rna Genes

The three-domains tree, which depicts eukaryotes and archaebacteria as monophyletic sister groups, is the dominant model for early eukaryotic evolution. By contrast, the ‘eocyte hypothesis’, where eukaryotes are proposed to have originated from within the archaebacteria as sister to the Crenarchaeota (also called the eocytes), has been largely neglected in the literature. We have investigated support for these two competing hypotheses from molecular sequence data using methods that attempt to accommodate the across-site compositional heterogeneity and across-tree compositional and rate matrix heterogeneity that are manifest features of these data. When ribosomal RNA genes were analysed using standard methods that do not adequately model these kinds of heterogeneity, the three-domains tree was supported. However, this support was eroded or lost when composition-heterogeneous models were used, with concomitant increase in support for the eocyte tree for eukaryotic origins. Analysis of combined amino acid sequences from 41 protein-coding genes supported the eocyte tree, whether or not composition-heterogeneous models were used. The possible effects of substitutional saturation of our data were examined using simulation; these results suggested that saturation is delayed by among-site rate variation in the sequences, and that phylogenetic signal for ancient relationships is plausibly present in these data.

scholarly journals Techniques for the verification of minimal phylogenetic trees illustrated with ten mammalian haemoglobin sequences

1980 ◽  
Vol 187 (1) ◽  
pp. 65-74 ◽  
Author(s):  
D Penny ◽  
M D Hendy ◽  
L R Foulds
Keyword(s):  
Amino Acid ◽  
Phylogenetic Tree ◽  
Protein Sequence ◽  
Phylogenetic Trees ◽  
Sequence Data ◽  
Protein Sequences ◽  
Nucleotide Sequences ◽  
Amino Acid Sequences ◽  
Minimal Tree ◽  
Protein Sequence Data

We have recently reported a method to identify the shortest possible phylogenetic tree for a set of protein sequences [Foulds Hendy & Penny (1979) J. Mol. Evol. 13. 127–150; Foulds, Penny & Hendy (1979) J. Mol. Evol. 13, 151–166]. The present paper discusses issues that arise during the construction of minimal phylogenetic trees from protein-sequence data. The conversion of the data from amino acid sequences into nucleotide sequences is shown to be advantageous. A new variation of a method for constructing a minimal tree is presented. Our previous methods have involved first constructing a tree and then either proving that it is minimal or transforming it into a minimal tree. The approach presented in the present paper progressively builds up a tree, taxon by taxon. We illustrate this approach by using it to construct a minimal tree for ten mammalian haemoglobin alpha-chain sequences. Finally we define a measure of the complexity of the data and illustrate a method to derive a directed phylogenetic tree from the minimal tree.

scholarly journals Molecular characterization of the coat protein gene revealed considerable diversity of viral species complex in Garlic (Allium sativum L.)

2020 ◽  
Author(s):  
Abel Debebe Mitiku ◽  
Dawit Tesfaye Degefu ◽  
Adane Abraham ◽  
Desta Mejan ◽  
Pauline Asami ◽  
...  
Keyword(s):  
Amino Acid ◽  
Coat Protein ◽  
Species Complex ◽  
Sequence Data ◽  
Amino Acid Sequences ◽  
Viral Gene ◽  
Planting Material ◽  
Virus C ◽  
Garlic Virus ◽  
Material Exchange

AbstractGarlic is one of the most crucial Allium vegetables used as seasoning of foods. It has a lot of benefits from the medicinal and nutritional point of view; however, its production is highly constrained by both biotic and abiotic challenges. Among these, viral infections are the most prevalent factors affecting crop productivity around the globe. This experiment was conducted on eleven selected garlic accessions and three improved varieties collected from different garlic growing agro-climatic regions of Ethiopia. This study aimed to identify and characterize the isolated garlic virus using the coat protein (CP) gene and further determine their phylogenetic relatedness. RNA was extracted from fresh young leaves, thirteen days old seedlings, which showed yellowing, mosaic, and stunting symptoms. Pairwise molecular diversity for CP nucleotide and amino acid sequences were calculated using MEGA5. Maximum Likelihood tree of CP nucleotide sequence data of Allexivirus and Potyvirus were conducted using PhyML, while a neighbor-joining tree was constructed for the amino acid sequence data using MEGA5. From the result, five garlic viruses were identified viz. Garlic virus C (78.6 %), Garlic virus D (64.3 %), Garlic virus X (78.6 %), Onion yellow dwarf virus (OYDV) (100%), and Leek yellow stripe virus (LYSV) (78.6 %). The study revealed the presence of complex mixtures of viruses with 42.9 % of the samples had co-infected with a species complex of Garlic virus C, Garlic virus D, Garlic virus X, OYDV, and LYSV. Pairwise comparisons of the isolated Potyviruses and Allexiviruses species revealed high identity with that of the known members of their respected species. As an exception, less within species identity was observed among Garlic virus C isolates as compared with that of the known members of the species. Finally, our results highlighted the need for stepping up a working framework to establish virus-free garlic planting material exchange in the country which could result in the reduction of viral gene flow across the country.Author SummaryGarlic viruses are the most devastating disease since garlic is the most vulnerable crop due to their vegetative nature of propagation. Currently, the garlic viruses are the aforementioned production constraint in Ethiopia. However, so far very little is known on the identification, diversity, and dissemination of garlic infecting viruses in the country. Here we explore the prevalence, genetic diversity, and the presence of mixed infection of garlic viruses in Ethiopia using next generation sequencing platform. Analysis of nucleotide and amino acid sequences of coat protein genes from infected samples revealed the association of three species from Allexivirus and two species from Potyvirus in a complex mixture. Ultimately the article concludes there is high time to set up a working framework to establish garlic free planting material exchange platform which could result in a reduction of viral gene flow across the country.

scholarly journals Deep learning predicts short non-coding RNA functions from only raw sequence data

2020 ◽  
Vol 16 (11) ◽  
pp. e1008415
Author(s):  
Teresa Maria Rosaria Noviello ◽  
Francesco Ceccarelli ◽  
Michele Ceccarelli ◽  
Luigi Cerulo
Keyword(s):  
Secondary Structure ◽  
Sequence Data ◽  
Computational Cost ◽  
Large Data ◽  
Sequence Information ◽  
Structure Information ◽  
Non Coding Rna ◽  
A Genome ◽  
Data Volume ◽  
Biological Functionality

Small non-coding RNAs (ncRNAs) are short non-coding sequences involved in gene regulation in many biological processes and diseases. The lack of a complete comprehension of their biological functionality, especially in a genome-wide scenario, has demanded new computational approaches to annotate their roles. It is widely known that secondary structure is determinant to know RNA function and machine learning based approaches have been successfully proven to predict RNA function from secondary structure information. Here we show that RNA function can be predicted with good accuracy from a lightweight representation of sequence information without the necessity of computing secondary structure features which is computationally expensive. This finding appears to go against the dogma of secondary structure being a key determinant of function in RNA. Compared to recent secondary structure based methods, the proposed solution is more robust to sequence boundary noise and reduces drastically the computational cost allowing for large data volume annotations. Scripts and datasets to reproduce the results of experiments proposed in this study are available at: https://github.com/bioinformatics-sannio/ncrna-deep.

scholarly journals Learning from the Reliability Paradox: How Theoretically Informed Generative Models Can Advance the Social, Behavioral, and Brain Sciences

2020 ◽  
Author(s):  
Nathaniel Haines ◽  
Peter D. Kvam ◽  
Louis H. Irving ◽  
Colin Smith ◽  
Theodore P. Beauchaine ◽  
...  
Keyword(s):  
Individual Differences ◽  
Theory Development ◽  
Generative Models ◽  
Parameter Estimates ◽  
Group Level ◽  
Retest Reliability ◽  
Individual Level ◽  
Generative Modeling ◽  
Behavioral Tasks ◽  
Test Retest Reliability

Behavioral tasks (e.g., Stroop task) that produce replicable group-level effects (e.g., Stroop effect) often fail to reliably capture individual differences between participants (e.g., low test-retest reliability). This “reliability paradox” has led many researchers to conclude that most behavioral tasks cannot be used to develop and advance theories of individual differences. However, these conclusions are derived from statistical models that provide only superficial summary descriptions of behavioral data, thereby ignoring theoretically-relevant data-generating mechanisms that underly individual-level behavior. More generally, such descriptive methods lack the flexibility to test and develop increasingly complex theories of individual differences. To resolve this theory-description gap, we present generative modeling approaches, which involve using background knowledge to specify how behavior is generated at the individual level, and in turn how the distributions of individual-level mechanisms are characterized at the group level—all in a single joint model. Generative modeling shifts our focus away from estimating descriptive statistical “effects” toward estimating psychologically meaningful parameters, while simultaneously accounting for measurement error that would otherwise attenuate individual difference correlations. Using simulations and empirical data from the Implicit Association Test and Stroop, Flanker, Posner Cueing, and Delay Discounting tasks, we demonstrate how generative models yield (1) higher test-retest reliability estimates, and (2) more theoretically informative parameter estimates relative to traditional statistical approaches. Our results reclaim optimism regarding the utility of behavioral paradigms for testing and advancing theories of individual differences, and emphasize the importance of formally specifying and checking model assumptions to reduce theory-description gaps and facilitate principled theory development.

Sign in / Sign up

Export Citation Format

Share Document