scholarly journals The Power of Universal Contextualized Protein Embeddings in Cross-species Protein Function Prediction

2021 ◽  
Vol 17 ◽  
pp. 117693432110626
Author(s):  
Irene van den Bent ◽  
Stavros Makrodimitris ◽  
Marcel Reinders
Keyword(s):  
Protein Function ◽  
Contextual Information ◽  
Protein Function Prediction ◽  
Protein Sequences ◽  
Learning Task ◽  
Function Prediction ◽  
Training Data ◽  
Language Models ◽  
Molecular Function ◽  
Test Species

Computationally annotating proteins with a molecular function is a difficult problem that is made even harder due to the limited amount of available labeled protein training data. Unsupervised protein embeddings partly circumvent this limitation by learning a universal protein representation from many unlabeled sequences. Such embeddings incorporate contextual information of amino acids, thereby modeling the underlying principles of protein sequences insensitive to the context of species. We used an existing pre-trained protein embedding method and subjected its molecular function prediction performance to detailed characterization, first to advance the understanding of protein language models, and second to determine areas of improvement. Then, we applied the model in a transfer learning task by training a function predictor based on the embeddings of annotated protein sequences of one training species and making predictions on the proteins of several test species with varying evolutionary distance. We show that this approach successfully generalizes knowledge about protein function from one eukaryotic species to various other species, outperforming both an alignment-based and a supervised-learning-based baseline. This implies that such a method could be effective for molecular function prediction in inadequately annotated species from understudied taxonomic kingdoms.

scholarly journals The power of universal contextualised protein embeddings in cross-species protein function prediction

2021 ◽  
Author(s):  
Irene van den Bent ◽  
Stavros Makrodimitris ◽  
Marcel Reinders
Keyword(s):  
Amino Acid ◽  
Protein Function ◽  
Prediction Models ◽  
Protein Sequences ◽  
Function Prediction ◽  
Training Data ◽  
Language Models ◽  
Molecular Function ◽  
Test Species ◽  
Protein Functions

AbstractComputationally annotating proteins with a molecular function is a difficult problem that is made even harder due to the limited amount of available labelled protein training data. A recently published supervised molecular function predicting model partly circumvents this limitation by making its predictions based on the universal (i.e. task-agnostic) contextualised protein embeddings from the deep pre-trained unsupervised protein language model SeqVec. SeqVec embeddings incorporate contextual information of amino acids, thereby modelling the underlying principles of protein sequences insensitive to the context of species.We applied the existing SeqVec-based molecular function prediction model in a transfer learning task by training the model on annotated protein sequences of one training species and making predictions on the proteins of several test species with varying evolutionary distance. We show that this approach successfully generalises knowledge about protein function from one eukaryotic species to various other species, proving itself an effective method for molecular function prediction in inadequately annotated species from understudied taxonomic kingdoms. Furthermore, we submitted the performance of our SeqVec-based prediction models to detailed characterisation, first to advance the understanding of protein language models and second to determine areas of improvement.Author summaryProteins are diverse molecules that regulate all processes in biology. The field of synthetic biology aims to understand these protein functions to solve problems in medicine, manufacturing, and agriculture. Unfortunately, for many proteins only their amino acid sequence is known whereas their function remains unknown. Only a few species have been well-studied such as mouse, human and yeast. Hence, we need to increase knowledge on protein functions. Doing so is, however, complicated as determining protein functions experimentally is time-consuming, expensive, and technically limited. Computationally predicting protein functions offers a faster and more scalable approach but is hampered as it requires much data to design accurate function prediction algorithms. Here, we show that it is possible to computationally generalize knowledge on protein function from one well-studied training species to another test species. Additionally, we show that the quality of these protein function predictions depends on how structurally similar the proteins are between the species. Advantageously, the predictors require only the annotations of proteins from the training species and mere amino acid sequences of test species which may particularly benefit the function prediction of species from understudied taxonomic kingdoms such as the Plantae, Protozoa and Chromista.

scholarly journals Unsupervised protein embeddings outperform hand-crafted sequence and structure features at predicting molecular function

2020 ◽  
Author(s):  
Amelia Villegas-Morcillo ◽  
Stavros Makrodimitris ◽  
Roeland C H J van Ham ◽  
Angel M Gomez ◽  
Victoria Sanchez ◽  
...  
Keyword(s):  
Protein Function ◽  
Prediction Models ◽  
Protein Function Prediction ◽  
3D Structure ◽  
Function Prediction ◽  
Feature Representation ◽  
Training Data ◽  
Supplementary Information ◽  
Molecular Function ◽  
Structure Information

Abstract Motivation Protein function prediction is a difficult bioinformatics problem. Many recent methods use deep neural networks to learn complex sequence representations and predict function from these. Deep supervised models require a lot of labeled training data which are not available for this task. However, a very large amount of protein sequences without functional labels is available. Results We applied an existing deep sequence model that had been pretrained in an unsupervised setting on the supervised task of protein molecular function prediction. We found that this complex feature representation is effective for this task, outperforming hand-crafted features such as one-hot encoding of amino acids, k-mer counts, secondary structure and backbone angles. Also, it partly negates the need for complex prediction models, as a two-layer perceptron was enough to achieve competitive performance in the third Critical Assessment of Functional Annotation benchmark. We also show that combining this sequence representation with protein 3D structure information does not lead to performance improvement, hinting that 3D structure is also potentially learned during the unsupervised pretraining. Availability and implementation Implementations of all used models can be found at https://github.com/stamakro/GCN-for-Structure-and-Function. Supplementary information Supplementary data are available at Bioinformatics online.

scholarly journals DeepGOZero: Improving protein function prediction from sequence and zero-shot learning based on ontology axioms

2022 ◽  
Author(s):  
Maxat Kulmanov ◽  
Robert Hoehndorf
Keyword(s):  
Machine Learning ◽  
Protein Function ◽  
Protein Function Prediction ◽  
Prediction Method ◽  
Function Prediction ◽  
Training Data ◽  
Large Set ◽  
Theoretic Approach ◽  
Machine Learning Model ◽  
Protein Functions

Motivation: Protein functions are often described using the Gene Ontology (GO) which is an ontology consisting of over 50,000 classes and a large set of formal axioms. Predicting the functions of proteins is one of the key challenges in computational biology and a variety of machine learning methods have been developed for this purpose. However, these methods usually require significant amount of training data and cannot make predictions for GO classes which have only few or no experimental annotations. Results: We developed DeepGOZero, a machine learning model which improves predictions for functions with no or only a small number of annotations. To achieve this goal, we rely on a model-theoretic approach for learning ontology embeddings and combine it with neural networks for protein function prediction. DeepGOZero can exploit formal axioms in the GO to make zero-shot predictions, i.e., predict protein functions even if not a single protein in the training phase was associated with that function. Furthermore, the zero-shot prediction method employed by DeepGOZero is generic and can be applied whenever associations with ontology classes need to be predicted. Availability: http://github.com/bio-ontology-research-group/deepgozero

scholarly journals Neural network and random forest models in protein function prediction

2019 ◽  
Author(s):  
Kai Hakala ◽  
Suwisa Kaewphan ◽  
Jari Björne ◽  
Farrokh Mehryary ◽  
Hans Moen ◽  
...  
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Random Forest ◽  
Protein Function ◽  
Protein Function Prediction ◽  
Protein Sequences ◽  
Function Prediction ◽  
Learning System ◽  
Large Set ◽  
Competitive Performance

AbstractOver the past decade, the demand for automated protein function prediction has increased due to the volume of newly sequenced proteins. In this paper, we address the function prediction task by developing an ensemble system automatically assigning Gene Ontology (GO) terms to the given input protein sequence.We develop an ensemble system which combines the GO predictions made by random forest (RF) and neural network (NN) classifiers. Both RF and NN models rely on features derived from BLAST sequence alignments, taxonomy and protein signature analysis tools. In addition, we report on experiments with a NN model that directly analyzes the amino acid sequence as its sole input, using a convolutional layer. The Swiss-Prot database is used as the training and evaluation data.In the CAFA3 evaluation, which relies on experimental verification of the functional predictions, our submitted ensemble model demonstrates competitive performance ranking among top-10 best-performing systems out of over 100 submitted systems. In this paper, we evaluate and further improve the CAFA3-submitted system. Our machine learning models together with the data pre-processing and feature generation tools are publicly available as an open source software athttps://github.com/TurkuNLP/CAFA3Author summaryUnderstanding the role and function of proteins in biological processes is fundamental for new biological discoveries. Whereas modern sequencing methods have led to a rapid growth of protein databases, the function of these sequences is often unknown and expensive to determine experimentally. This has spurred a lot of interest in predictive modelling of protein functions.We develop a machine learning system for annotating protein sequences with functional definitions selected from a vast set of predefined functions. The approach is based on a combination of neural network and random forest classifiers with features covering structural and taxonomic properties and sequence similarity. The system is thoroughly evaluated on a large set of manually curated functional annotations and shows competitive performance in comparison to other suggested approaches. We also analyze the predictions for different functional annotation and taxonomy categories and measure the importance of different features for the task. This analysis reveals that the system is particularly efficient for bacterial protein sequences.

scholarly journals Accurate Protein Function Prediction via Graph Attention Networks with Predicted Structure Information

2021 ◽  
Author(s):  
Boqiao Lai ◽  
Jinbo Xu
Keyword(s):  
Protein Function ◽  
Structure Prediction ◽  
Protein Function Prediction ◽  
Function Prediction ◽  
Language Models ◽  
Structure Information ◽  
Functional Annotations ◽  
Residue Contact ◽  
Sequence Identity ◽  
Contact Graphs

Experimental protein function annotation does not scale with the fast-growing sequence databases. Only a tiny fraction (<0.1%) of protein sequences in UniProtKB has experimentally determined functional annotations. Computational methods may predict protein function in a high-throughput way, but its accuracy is not very satisfactory. Based upon recent breakthroughs in protein structure prediction and protein language models, we develop GAT-GO, a graph attention network (GAT) method that may substantially improve protein function prediction by leveraging predicted inter-residue contact graphs and protein sequence embedding. Our experimental results show that GAT-GO greatly outperforms the latest sequence- and structure-based deep learning methods. On the PDB-mmseqs testset where the train and test proteins share <15% sequence identity, GAT-GO yields Fmax(maximum F-score) 0.508, 0.416, 0.501, and AUPRC(area under the precision-recall curve) 0.427, 0.253, 0.411 for the MFO, BPO, CCO ontology domains, respectively, much better than homology-based method BLAST (Fmax 0.117,0.121,0.207 and AUPRC 0.120, 0.120, 0.163). On the PDB-cdhit testset where the training and test proteins share higher sequence identity, GAT-GO obtains Fmax 0.637, 0.501, 0.542 for the MFO, BPO, CCO ontology domains, respectively, and AUPRC 0.662, 0.384, 0.481, significantly exceeding the just-published graph convolution method DeepFRI, which has Fmax 0.542, 0.425, 0.424 and AUPRC 0.313, 0.159, 0.193.

scholarly journals Unsupervised protein embeddings outperform hand-crafted sequence and structure features at predicting molecular function

2020 ◽  
Author(s):  
Amelia Villegas-Morcillo ◽  
Stavros Makrodimitris ◽  
Roeland C.H.J. van Ham ◽  
Angel M. Gomez ◽  
Victoria Sanchez ◽  
...  
Keyword(s):  
Protein Function ◽  
Prediction Models ◽  
Protein Function Prediction ◽  
3D Structure ◽  
Structure And Function ◽  
Function Prediction ◽  
Training Data ◽  
Supplementary Information ◽  
Dimensional Structure ◽  
And Function

AbstractMotivationProtein function prediction is a difficult bioinformatics problem. Many recent methods use deep neural networks to learn complex sequence representations and predict function from these. Deep supervised models require a lot of labeled training data which are not available for this task. However, a very large amount of protein sequences without functional labels is available.ResultsWe applied an existing deep sequence model that had been pre-trained in an unsupervised setting on the supervised task of protein function prediction. We found that this complex feature representation is effective for this task, outperforming hand-crafted features such as one-hot encoding of amino acids, k-mer counts, secondary structure and backbone angles. Also, it partly negates the need for deep prediction models, as a two-layer perceptron was enough to achieve state-of-the-art performance in the third Critical Assessment of Functional Annotation benchmark. We also show that combining this sequence representation with protein 3D structure information does not lead to performance improvement, hinting that three-dimensional structure is also potentially learned during the unsupervised pre-training.AvailabilityImplementations of all used models can be found at https://github.com/stamakro/GCN-for-Structure-and-Function.Contactameliavm@ugr.esSupplementary informationSupplementary data are available online.

scholarly journals Embeddings from deep learning transfer GO annotations beyond homology

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Maria Littmann ◽  
Michael Heinzinger ◽  
Christian Dallago ◽  
Tobias Olenyi ◽  
Burkhard Rost
Keyword(s):  
Protein Function ◽  
Protein Sequences ◽  
Language Models ◽  
Evolutionary Information ◽  
Pairwise Sequence Identity ◽  
Intrinsically Disordered ◽  
Intrinsically Disordered Regions ◽  
Sequence Identity ◽  
Experimental Function ◽  
Go Terms

AbstractKnowing protein function is crucial to advance molecular and medical biology, yet experimental function annotations through the Gene Ontology (GO) exist for fewer than 0.5% of all known proteins. Computational methods bridge this sequence-annotation gap typically through homology-based annotation transfer by identifying sequence-similar proteins with known function or through prediction methods using evolutionary information. Here, we propose predicting GO terms through annotation transfer based on proximity of proteins in the SeqVec embedding rather than in sequence space. These embeddings originate from deep learned language models (LMs) for protein sequences (SeqVec) transferring the knowledge gained from predicting the next amino acid in 33 million protein sequences. Replicating the conditions of CAFA3, our method reaches an Fmax of 37 ± 2%, 50 ± 3%, and 57 ± 2% for BPO, MFO, and CCO, respectively. Numerically, this appears close to the top ten CAFA3 methods. When restricting the annotation transfer to proteins with < 20% pairwise sequence identity to the query, performance drops (Fmax BPO 33 ± 2%, MFO 43 ± 3%, CCO 53 ± 2%); this still outperforms naïve sequence-based transfer. Preliminary results from CAFA4 appear to confirm these findings. Overall, this new concept is likely to change the annotation of proteins, in particular for proteins from smaller families or proteins with intrinsically disordered regions.

Sign in / Sign up

Export Citation Format

Share Document