Protein model accuracy estimation empowered by deep learning and inter-residue distance prediction in CASP14

AbstractThe inter-residue contact prediction and deep learning showed the promise to improve the estimation of protein model accuracy (EMA) in the 13th Critical Assessment of Protein Structure Prediction (CASP13). To further leverage the improved inter-residue distance predictions to enhance EMA, during the 2020 CASP14 experiment, we integrated several new inter-residue distance features with the existing model quality assessment features in several deep learning methods to predict the quality of protein structural models. According to the evaluation of performance in selecting the best model from the models of CASP14 targets, our three multi-model predictors of estimating model accuracy (MULTICOM-CONSTRUCT, MULTICOM-AI, and MULTICOM-CLUSTER) achieve the averaged loss of 0.073, 0.079, and 0.081, respectively, in terms of the global distance test score (GDT-TS). The three methods are ranked first, second, and third out of all 68 CASP14 predictors. MULTICOM-DEEP, the single-model predictor of estimating model accuracy (EMA), is ranked within top 10 among all the single-model EMA methods according to GDT-TS score loss. The results demonstrate that inter-residue distance features are valuable inputs for deep learning to predict the quality of protein structural models. However, larger training datasets and better ways of leveraging inter-residue distance information are needed to fully explore its potentials.

Download Full-text

Protein model accuracy estimation empowered by deep learning and inter-residue distance prediction in CASP14

10.1101/2021.01.31.428975 ◽

2021 ◽

Author(s):

Xiao Chen ◽

Jian Liu ◽

Zhiye Guo ◽

Tianqi Wu ◽

Jie Hou ◽

...

Keyword(s):

Deep Learning ◽

Structure Prediction ◽

Structural Models ◽

Single Model ◽

Model Accuracy ◽

Model Quality ◽

Residue Contact ◽

Contact Distance ◽

Protein Model ◽

Contact Predictions

AbstractThe inter-residue contact prediction and deep learning showed the promise to improve the estimation of protein model accuracy (EMA) in the 13th Critical Assessment of Protein Structure Prediction (CASP13). During the 2020 CASP14 experiment, we developed and tested several EMA predictors that used deep learning with the new features based on inter-residue distance/contact predictions as well as the existing model quality features. The average global distance test (GDT-TS) score loss of ranking CASP14 structural models by three multi-model MULTICOM EMA predictors (MULTICOM-CONSTRUCT, MULTICOM-AI, and MULTICOM-CLUSTER) is 0.073, 0.079, and 0.081, respectively, which are ranked first, second, and third places out of 68 CASP14 EMA predictors. The single-model EMA predictor (MULTICOM-DEEP) is ranked 10th place among all the single-model EMA methods in terms of GDT_TS score loss. The results show that deep learning and contact/distance predictions are useful in ranking and selecting protein structural models.

Download Full-text

Protein Model Accuracy Estimation Empowered by Deep Learning and Inter-residue Distance Prediction in CASP14

10.21203/rs.3.rs-228012/v1 ◽

2021 ◽

Author(s):

Xiao Chen ◽

Jian Liu ◽

Zhiye Guo ◽

Tianqi Wu ◽

Jie Hou ◽

...

Keyword(s):

Deep Learning ◽

Structure Prediction ◽

Structural Models ◽

Single Model ◽

Model Accuracy ◽

Model Quality ◽

Residue Contact ◽

Contact Distance ◽

Protein Model ◽

Contact Predictions

Abstract The inter-residue contact prediction and deep learning showed the promise to improve the estimation of protein model accuracy (CASP13) in the 13th Critical Assessment of Protein Structure Prediction (CASP13). During the 2020 CASP14 experiment, we developed and tested several EMA predictors that used deep learning with the new features based on inter-residue distance/contact predictions as well as the existing model quality features. The average global distance test (GDT-TS) score loss of ranking CASP14 structural models by three multi-model MULTICOM EMA predictors (MULTICOM-CONSTRUCT, MULTICOM-AI, and MULTICOM-CLUSTER) is 0.073, 0.079, and 0.081, respectively, which are ranked first, second, and third places out of 68 CASP14 EMA predictors. The single-model EMA predictor (MULTICOM-DEEP) is ranked 10th place among all the single-model EMA methods in terms of GDT-TS score loss. The results show that deep learning and contact/distance predictions are useful in ranking and selecting protein structural models.

Download Full-text

DISTEMA: distance map-based estimation of single protein model accuracy with attentive 2D convolutional neural network

10.1101/2021.03.29.437573 ◽

2021 ◽

Author(s):

Xiao Chen ◽

Jianling Cheng

Keyword(s):

Neural Network ◽

Deep Learning ◽

Convolutional Neural Network ◽

Structural Models ◽

Model Quality ◽

Distance Map ◽

Protein Model ◽

Single Protein ◽

Ranking Loss

AbstractBackgroundEstimation of the accuracy (quality) of protein structural models is important for both prediction and use of protein structural models. Deep learning methods have been used to integrate protein structure features to predict the quality of protein models. Inter-residue distances are key information for predicting protein’s tertiary structures and therefore have good potentials to predict the quality of protein structural models. However, few methods have been developed to fully take advantage of predicted inter-residue distance maps to estimate the accuracy of a single protein structural model.ResultWe developed an attentive 2D convolutional neural network (CNN) with channel-wise attention to take only a raw difference map between the inter-residue distance map calculated from a single protein model and the distance map predicted from the protein sequence as input to predict the quality of the model. The network comprises multiple convolutional layers, batch normalization layers, dense layers, and Squeeze-and-Excitation blocks with attention to automatically extract features relevant to protein model quality from the raw input without using any expert-curated features. We evaluated DISTEMA’s capability of selecting the best models for CASP13 targets in terms of ranking loss of GDT-TS score. The ranking loss of DISTEMA is 0.079, lower than several state-of-the-art single-model quality assessment methods. The work demonstrates that using raw inter-residue distance information alone with deep learning can predict the quality of protein structural models reasonably well.

Download Full-text

A single-model quality assessment method for poor quality protein structure

10.21203/rs.3.rs-17080/v1 ◽

2020 ◽

Author(s):

Jianquan Ouyang ◽

Ningqiao Huang ◽

Yunqi Jiang

Keyword(s):

Protein Structure ◽

Quality Assessment ◽

Structure Prediction ◽

Assessment Method ◽

Poor Quality ◽

Single Model ◽

Model Quality ◽

Model Quality Assessment ◽

Quality Assessment Method

Abstract Quality assessment of protein tertiary structure prediction models, in which structures of the best quality are selected from decoys, is a major challenge in protein structure prediction, and is crucial to determine a model’s utility and potential applications. Estimating the quality of a single model predicts the model’s quality based on the single model itself. In general, the Pearson correlation value of the quality assessment method increases in tandem with an increase in the quality of the model pool. However, there is no consensus regarding the best method to select a few good models from the poor quality model pool. In this work, we introduce a novel single-model quality assessment method for poor quality models that uses simple linear combinations of six features. We perform weighted search and linear regression on a large dataset of models from the 12th Critical Assessment of Protein Structure Prediction (CASP12) and benchmark the results on CASP13 models. We demonstrate that our method achieves outstanding performance on poor quality models.

Download Full-text

QDeep: distance-based protein model quality estimation by residue-level ensemble error classifications using stacked deep residual neural networks

Bioinformatics ◽

10.1093/bioinformatics/btaa455 ◽

2020 ◽

Vol 36 (Supplement_1) ◽

pp. i285-i291 ◽

Cited By ~ 1

Author(s):

Md Hossain Shuvo ◽

Sutanu Bhattacharya ◽

Debswapna Bhattacharya

Keyword(s):

Neural Networks ◽

Protein Structure ◽

Protein Structure Prediction ◽

Structure Prediction ◽

Residue Level ◽

Supplementary Information ◽

Model Quality ◽

Quality Estimation ◽

Distance Information ◽

Protein Model

Abstract Motivation Protein model quality estimation, in many ways, informs protein structure prediction. Despite their tight coupling, existing model quality estimation methods do not leverage inter-residue distance information or the latest technological breakthrough in deep learning that has recently revolutionized protein structure prediction. Results We present a new distance-based single-model quality estimation method called QDeep by harnessing the power of stacked deep residual neural networks (ResNets). Our method first employs stacked deep ResNets to perform residue-level ensemble error classifications at multiple predefined error thresholds, and then combines the predictions from the individual error classifiers for estimating the quality of a protein structural model. Experimental results show that our method consistently outperforms existing state-of-the-art methods including ProQ2, ProQ3, ProQ3D, ProQ4, 3DCNN, MESHI, and VoroMQA in multiple independent test datasets across a wide-range of accuracy measures; and that predicted distance information significantly contributes to the improved performance of QDeep. Availability and implementation https://github.com/Bhattacharya-Lab/QDeep. Supplementary information Supplementary data are available at Bioinformatics online.

Download Full-text

Deep convolutional neural networks for predicting the quality of single protein structural models

10.1101/590620 ◽

2019 ◽

Cited By ~ 3

Author(s):

Jie Hou ◽

Renzhi Cao ◽

Jianlin Cheng

Keyword(s):

Structure Prediction ◽

Structural Model ◽

Structural Models ◽

Structural Features ◽

Fixed Number ◽

Deep Convolutional Neural Networks ◽

Global Quality ◽

Protein Model ◽

Single Protein

AbstractPredicting the global quality and local (residual-specific) quality of a single protein structural model is important for protein structure prediction and application. In this work, we developed a deep one-dimensional convolutional neural network (1DCNN) that predicts the absolute local quality of a single protein model as well as two 1DCNNs to predict both local and global quality simultaneously through a novel multi-task learning framework. The networks accept sequential and structural features (i.e. amino acid sequence, agreement of secondary structure and solvent accessibilities, residual disorder properties and Rosetta energies) of a protein model of any size as input to predict its quality, which is different from existing methods using a fixed number of hand-crafted features as input. Our three methods (InteractQA-net, JointQA-net and LocalQA-net) were trained on the structural models of the single-domain protein targets of CASP8, 9, 10 and evaluated on the models of CASP11 and CASP12 targets. The results show that the performance of our deep learning methods is comparable to the state-of-the-art quality assessment methods. Our study also demonstrates that combining local and global quality predictions together improves the global quality prediction accuracy. The source code and executable of our methods are available at:https://github.com/multicom-toolbox/DeepCovQA

Download Full-text

QDeep: distance-based protein model quality estimation by residue-level ensemble error classifications using stacked deep residual neural networks

10.1101/2020.01.31.928622 ◽

2020 ◽

Author(s):

Md Hossain Shuvo ◽

Sutanu Bhattacharya ◽

Debswapna Bhattacharya

Keyword(s):

Neural Networks ◽

Protein Structure ◽

Protein Structure Prediction ◽

Structure Prediction ◽

Residue Level ◽

Estimation Methods ◽

Model Quality ◽

Quality Estimation ◽

Distance Information ◽

Protein Model

AbstractMotivationProtein model quality estimation, in many ways, informs protein structure prediction. Despite their tight coupling, existing model quality estimation methods do not leverage inter-residue distance information or the latest technological breakthrough in deep learning that has recently revolutionized protein structure prediction.ResultsWe present a new distance-based single-model quality estimation method called QDeep by harnessing the power of stacked deep residual neural networks (ResNets). Our method first employs stacked deep ResNets to perform residue-level ensemble error classifications at multiple predefined error thresholds, and then combines the predictions from the individual error classifiers for estimating the quality of a protein structural model. Experimental results show that our method consistently out-performs existing state-of-the-art methods including ProQ2, ProQ3, ProQ3D, ProQ4, 3DCNN, MESHI, and VoroMQA in multiple independent test datasets across a wide-range of accuracy measures; and that predicted distance information significantly contributes to the improved performance of QDeep.Availabilityhttps://github.com/Bhattacharya-Lab/[email protected]

Download Full-text

A single-model quality assessment method for poor quality protein structure

10.21203/rs.3.rs-17080/v2 ◽

2020 ◽

Author(s):

Jianquan Ouyang ◽

Ningqiao Huang ◽

Yunqi Jiang

Keyword(s):

Protein Structure ◽

Quality Assessment ◽

Structure Prediction ◽

Assessment Method ◽

Poor Quality ◽

Single Model ◽

Model Quality ◽

Model Quality Assessment ◽

Quality Assessment Method

Abstract Background: Quality assessment of protein tertiary structure prediction models, in which structures of the best quality are selected from decoys, is a major challenge in protein structure prediction, and is crucial to determine a model’s utility and potential applications. Estimating the quality of a single model predicts the model’s quality based on the single model itself. In general, the Pearson correlation value of the quality assessment method increases in tandem with an increase in the quality of the model pool. However, there is no consensus regarding the best method to select a few good models from the poor quality model pool.Results: We introduce a novel single-model quality assessment method for poor quality models that uses simple linear combinations of six features. We perform weighted search and linear regression on a large dataset of models from the 12th Critical Assessment of Protein Structure Prediction (CASP12) and benchmark the results on CASP13 models. We demonstrate that our method achieves outstanding performance on poor quality models.Conclusions: According to results of poor protein structure assessment based on six features, contact prediction and relying on fewer prediction features can improve selection accuracy.

Download Full-text

Improved protein model quality assessments by changing the target function

10.1101/270678 ◽

2018 ◽

Author(s):

Karolis Uziela ◽

David Menéndez Hurtado ◽

Nanjiang Shu ◽

Björn Wallner ◽

Arne Elofsson

Keyword(s):

Structure Prediction ◽

Target Function ◽

Local Environment ◽

Global Model ◽

Model Quality ◽

Quality Estimation ◽

Protein Model ◽

Local Quality ◽

Quality Assessments

AbstractProtein modeling quality is an important part of protein structure prediction. We have for more than a decade developed a set of methods for this problem. We have used various types of description of the protein and different machine learning methodologies. However, common to all these methods has been the target function used for training. The target function in ProQ describes the local quality of a residue in a protein model. In all versions of ProQ the target function has been the S-score. However, other quality estimation functions also exist, which can be divided into superposition- and contact-based methods. The superposition-based methods, such as S-score, are based on a rigid body superposition of a protein model and the native structure, while the contact-based methods compare the local environment of each residue. Here, we examine the effects of retraining our latest predictor, ProQ3D, using identical inputs but different target functions. We find that the c ntact-based methods are easier to predict and that predictors trained on these measures provide some advantages when it comes to identifying the best model. One possible reason for this is that contact based methods are better at estimating the quality of multi-domain targets. However, training on the S-score gives the best correlation with the GDT_TS score, which is commonly used in CASP to score the global model quality. To take the advantage of both of these features we provide an updated version of ProQ3D that predicts local and global model quality estimates based on different quality estimates.

Download Full-text

Methods for estimation of model accuracy in CASP12

10.1101/143925 ◽

2017 ◽

Cited By ~ 2

Author(s):

Arne Elofsson ◽

Keehyoung Joo ◽

Chen Keasar ◽

Jooyoung Lee ◽

Ali H. A. Maghrabi ◽

...

Keyword(s):

3D Models ◽

Estimation Methods ◽

Consensus Methods ◽

Single Model ◽

Model Accuracy ◽

Model Quality ◽

Quality Estimation ◽

Accuracy Estimation ◽

Local Quality

AbstractMethods for reliably estimating the quality of 3D models of proteins are essential drivers for the wide adoption and serious acceptance of protein structure predictions by life scientists. In this paper, the most successful groups in CASP12 describe their latest methods for Estimates of Model Accuracy (EMA). We show that pure single model accuracy estimation methods has shown clear progress since CASP11; the three top methods (MESHI, ProQ3, SVMQA) all perform better than the top method of CASP11 (ProQ2). The pure single model accuracy estimation methods outperform quasi-single (ModFOLD6 variations) and consensus methods (Pcons, ModFOLDclust2, Pcomb-domain and Wallner) in model selection, but are still not as good as those methods in absolute model quality estimation and predictions of local quality. Finally, we show that when using contact based model quality measures (CAD, 1DDT) the single model quality methods perform relatively better.

Download Full-text