scholarly journals Analysis of deep learning methods for blind protein contact prediction in CASP12

2017 ◽  
Author(s):  
Sheng Wang ◽  
Siqi Sun ◽  
Jinbo Xu

AbstractHere we present the results of protein contact prediction achieved in CASP12 by our RaptorX-Contact server, which is an early implementation of our deep learning method for contact prediction. On a set of 38 free-modeling target domains with a median family size of around 58 effective sequences, our server obtained an average top L/5 long- and medium-range contact accuracy of 47% and 44%, respectively (L=length). A more advanced implementation has an average accuracy of 59% and 57%, respectively. Our deep learning method formulates contact prediction as an image pixel-level labeling problem and simultaneously predicts all residue pairs of a protein using a combination of two deep residual neural networks, taking as input the residue conservation information, predicted secondary structure and solvent accessibility, contact potential, and co-evolution information. Our approach differs from existing methods mainly in (1) formulating contact prediction as a pixel-level image labeling problem instead of an image-level classification problem; (2) simultaneously predicting all contacts of an individual protein to make effective use of contact occurrence patterns; and (3) integrating both 1D and 2D deep convolutional neural networks to effectively learn complex sequence-structure relationship including high-order residue correlation. This paper discusses the RaptorX-Contact pipeline, both contact prediction and contact-based folding results, and finally the strength and weakness of our method.


2016 ◽  
Author(s):  
Sheng Wang ◽  
Siqi Sun ◽  
Zhen Li ◽  
Renyu Zhang ◽  
Jinbo Xu

AbstractMotivationProtein contacts contain key information for the understanding of protein structure and function and thus, contact prediction from sequence is an important problem. Recently exciting progress has been made on this problem, but the predicted contacts for proteins without many sequence homologs is still of low quality and not extremely useful for de novo structure prediction.MethodThis paper presents a new deep learning method that predicts contacts by integrating both evolutionary coupling (EC) and sequence conservation information through an ultra-deep neural network formed by two deep residual neural networks. The first residual network conducts a series of 1-dimensional convolutional transformation of sequential features; the second residual network conducts a series of 2-dimensional convolutional transformation of pairwise information including output of the first residual network, EC information and pairwise potential. By using very deep residual networks, we can model contact occurring patterns and very complex sequence-structure relationship and thus, obtain high-quality contact prediction regardless of how many sequence homologs are available for proteins in question.ResultsOur method greatly outperforms existing methods and leads to much more accurate contact-assisted folding. Tested on 105 CASP11 targets, 76 past CAMEO hard targets, and 398 membrane proteins, the average top L long-range prediction accuracy obtained our method, one representative EC method CCMpred and the CASP11 winner MetaPSICOV is 0.47, 0.21 and 0.30, respectively; the average top L/10 long-range accuracy of our method, CCMpred and MetaPSICOV is 0.77, 0.47 and 0.59, respectively. Ab initio folding using our predicted contacts as restraints but without any force fields can yield correct folds (i.e., TMscore>0.6) for 203 of the 579 test proteins, while that using MetaPSICOV- and CCMpred-predicted contacts can do so for only 79 and 62 of them, respectively. Our contact-assisted models also have much better quality than template-based models especially for membrane proteins. The 3D models built from our contact prediction have TMscore>0.5 for 208 of the 398 membrane proteins, while those from homology modeling have TMscore>0.5 for only 10 of them. Further, even if trained by only non-membrane proteins, our deep learning method works very well on membrane protein contact prediction. In the recent blind CAMEO benchmark, our fully-automated web server implementing this method successfully folded 5 targets with a new fold and only 0.3L-2.3L effective sequence homologs, including one β protein of 182 residues, one α+β protein of 125 residues, one α protein of 140 residues and one α protein of 217 residues and one α/β of 260 residues.Availability:http://raptorx.uchicago.edu/ContactMap/Author SummaryProtein contact prediction and contact-assisted folding has made good progress due to direct evolutionary coupling analysis (DCA). However, DCA is effective on only some proteins with a very large number of sequence homologs. To further improve contact prediction, we borrow ideas from deep learning, which has recently revolutionized object recognition, speech recognition and the GO game. Our deep learning method can model complex sequence-structure relationship and high-order correlation (i.e., contact occurring patterns) and thus, improve contact prediction accuracy greatly. Our test results show that our method greatly outperforms the state-of-the-art methods regardless how many sequence homologs are available for a protein in question. Ab initio folding guided by our predicted contacts may fold many more test proteins than the other contact predictors. Our contact-assisted 3D models also have much better quality than homology models built from the training proteins, especially for membrane proteins. One interesting finding is that even trained with only soluble proteins, our method performs very well on membrane proteins. Recent blind test in CAMEO confirms that our method can fold large proteins with a new fold and only a small number of sequence homologs.



2019 ◽  
Vol 277 ◽  
pp. 02024 ◽  
Author(s):  
Lincan Li ◽  
Tong Jia ◽  
Tianqi Meng ◽  
Yizhe Liu

In this paper, an accurate two-stage deep learning method is proposed to detect vulnerable plaques in ultrasonic images of cardiovascular. Firstly, a Fully Convonutional Neural Network (FCN) named U-Net is used to segment the original Intravascular Optical Coherence Tomography (IVOCT) cardiovascular images. We experiment on different threshold values to find the best threshold for removing noise and background in the original images. Secondly, a modified Faster RCNN is adopted to do precise detection. The modified Faster R-CNN utilize six-scale anchors (122,162,322,642,1282,2562) instead of the conventional one scale or three scale approaches. First, we present three problems in cardiovascular vulnerable plaque diagnosis, then we demonstrate how our method solve these problems. The proposed method in this paper apply deep convolutional neural networks to the whole diagnostic procedure. Test results show the Recall rate, Precision rate, IoU (Intersection-over-Union) rate and Total score are 0.94, 0.885, 0.913 and 0.913 respectively, higher than the 1st team of CCCV2017 Cardiovascular OCT Vulnerable Plaque Detection Challenge. AP of the designed Faster RCNN is 83.4%, higher than conventional approaches which use one-scale or three-scale anchors. These results demonstrate the superior performance of our proposed method and the power of deep learning approaches in diagnose cardiovascular vulnerable plaques.



2021 ◽  
Vol 13 (7) ◽  
pp. 1236
Author(s):  
Yuanjun Shu ◽  
Wei Li ◽  
Menglong Yang ◽  
Peng Cheng ◽  
Songchen Han

Convolutional neural networks (CNNs) have been widely used in change detection of synthetic aperture radar (SAR) images and have been proven to have better precision than traditional methods. A two-stage patch-based deep learning method with a label updating strategy is proposed in this paper. The initial label and mask are generated at the pre-classification stage. Then a two-stage updating strategy is applied to gradually recover changed areas. At the first stage, diversity of training data is gradually restored. The output of the designed CNN network is further processed to generate a new label and a new mask for the following learning iteration. As the diversity of data is ensured after the first stage, pixels within uncertain areas can be easily classified at the second stage. Experiment results on several representative datasets show the effectiveness of our proposed method compared with several existing competitive methods.



Author(s):  
Bo Wang ◽  
Xiaoting Yu ◽  
Chengeng Huang ◽  
Qinghong Sheng ◽  
Yuanyuan Wang ◽  
...  

The excellent feature extraction ability of deep convolutional neural networks (DCNNs) has been demonstrated in many image processing tasks, by which image classification can achieve high accuracy with only raw input images. However, the specific image features that influence the classification results are not readily determinable and what lies behind the predictions is unclear. This study proposes a method combining the Sobel and Canny operators and an Inception module for ship classification. The Sobel and Canny operators obtain enhanced edge features from the input images. A convolutional layer is replaced with the Inception module, which can automatically select the proper convolution kernel for ship objects in different image regions. The principle is that the high-level features abstracted by the DCNN, and the features obtained by multi-convolution concatenation of the Inception module must ultimately derive from the edge information of the preprocessing input images. This indicates that the classification results are based on the input edge features, which indirectly interpret the classification results to some extent. Experimental results show that the combination of the edge features and the Inception module improves DCNN ship classification performance. The original model with the raw dataset has an average accuracy of 88.72%, while when using enhanced edge features as input, it achieves the best performance of 90.54% among all models. The model that replaces the fifth convolutional layer with the Inception module has the best performance of 89.50%. It performs close to VGG-16 on the raw dataset and is significantly better than other deep neural networks. The results validate the functionality and feasibility of the idea posited.



Author(s):  
Sheng Shen ◽  
M. K. Sadoughi ◽  
Xiangyi Chen ◽  
Mingyi Hong ◽  
Chao Hu

Over the past two decades, safety and reliability of lithium-ion (Li-ion) rechargeable batteries have been receiving a considerable amount of attention from both industry and academia. To guarantee safe and reliable operation of a Li-ion battery pack and build failure resilience in the pack, battery management systems (BMSs) should possess the capability to monitor, in real time, the state of health (SOH) of the individual cells in the pack. This paper presents a deep learning method, named deep convolutional neural networks, for cell-level SOH assessment based on the capacity, voltage, and current measurements during a charge cycle. The unique features of deep convolutional neural networks include the local connectivity and shared weights, which enable the model to estimate battery capacity accurately using the measurements during charge. To our knowledge, this is the first attempt to apply deep learning to online SOH assessment of Li-ion battery. 10-year daily cycling data from implantable Li-ion cells are used to verify the performance of the proposed method. Compared with traditional machine learning methods such as relevance vector machine and shallow neural networks, the proposed method is demonstrated to produce higher accuracy and robustness in capacity estimation.



2021 ◽  
Vol 2 (01) ◽  
pp. 41-51
Author(s):  
Jwan Saeed ◽  
Subhi Zeebaree

Skin cancer is among the primary cancer types that manifest due to various dermatological disorders, which may be further classified into several types based on morphological features, color, structure, and texture. The mortality rate of patients who have skin cancer is contingent on preliminary and rapid detection and diagnosis of malignant skin cancer cells. Limitations in current dermoscopic images, including shadow, artifact, and noise, affect image quality, which may hamper detection effort. Attempts to overcome these challenges have been made by analyzing the images using deep learning neural networks to perform skin cancer detection. In this paper, the authors review the state-of-the-art in authoritative deep learning concepts pertinent to skin cancer detection and classification.



2018 ◽  
Author(s):  
Νικόλαος Πασσαλής

Οι πρόσφατες εξελίξεις στον τομέα της Βαθιάς Μάθησης (Deep Learning) παρείχαν ισχυρά εργαλεία ανάλυσης δεδομένων. Παρόλα αυτά, η μεγάλη υπολογιστική πολυπλοκότητα των μεθόδων Βαθιάς Μάθησης περιορίζει σημαντικά τη δυνατότητα εφαρμογής τους, ειδικά όταν οι διαθέσιμοι υπολογιστικοί πόροι είναι περιορισμένοι. Επιπλέον, η ευελιξία πολλών μεθόδων βαθιάς μάθησης περιορίζεται σημαντικά από την αδυναμία τους να συνδυαστούν αποτελεσματικά με κλασικές μεθόδους Μηχανικής Μάθησης. Η κύρια στόχευση της παρούσας διδακτορικής διατριβής είναι η ανάπτυξη μεθόδων Βαθιάς Μάθησης οι οποίες θα μπορούν να χρησιμοποιηθούν αποτελεσματικά για την επίλυση διαφόρων προβλημάτων ανάλυσης δεδομένων (κατηγοριοποίηση, ομαδοποίηση, παλινδρόμηση, κτλ.) με τη χρήση διαφορετικών δεδομένων (εικόνα, βίντεο, κείμενο, χρονοσειρές), ενώ ταυτόχρονα αντιμετωπίζουν αποτελεσματικά τα παραπάνω προβλήματα. Για τον σκοπό αυτό, πρώτα αναπτύχθηκε μία νευρωνική επέκταση του μοντέλου του Σάκου Χαρακτηριστικών (Bag-of-Features), η οποία συνδυάστηκε με πολλούς διαφορετικούς εξαγωγείς χαρακτηριστικών (feature extractors), συμπεριλαμβανομένων Βαθιών Συνελικτικών Νευρωνικών Δικτύων (Deep Convolutional Neural Networks). Αυτό επέτρεψε τη σημαντική αύξηση και της ακρίβειας των δικτύων, όσο και της αντοχής τους σε μεταβολές στην κατανομή εισόδου, καθώς και τη μείωση του πλήθους των παραμέτρων που απαιτούνται σε σύγκριση με ανταγωνιστικές μεθόδους. Στη συνέχεια, προτάθηκε μία μέθοδος μάθησης αναπαραστάσεων η οποία είναι ικανή να παράγει αναπαραστάσεις προσαρμοσμένες για το πρόβλημα της ανάκτησης πληροφορίας, αυξάνοντας σημαντικά την επίδοση των αναπαραστάσεων στα αντίστοιχα προβλήματα. Έπειτα, προτάθηκε μία ευέλικτη και αποδοτική μέθοδος μεταφοράς γνώσης (knowledge transfer), η οποία είναι σε θέση να ‘‘αποστάξει’’ τη γνώση από ένα μεγάλο και περίπλοκο νευρωνικό δίκτυο σε ένα γρηγορότερο και μικρότερο. Η αποτελεσματικότητα της προτεινόμενης μεθόδου διαπιστώθηκε με τη χρήση πολλών διαφορετικών πρωτοκόλλων αξιολόγησης. Επίσης, διαπιστώθηκε ότι το πρόβλημα μείωσης διάστασης (dimensionality reduction) μπορεί να εκφραστεί ως ένα πρόβλημα μεταφοράς γνώσης από μία κατάλληλα ορισμένη Συνάρτηση Πυκνότητας Πιθανότητας (Probability Density Function, PDF) σε ένα μοντέλο Μηχανικής Μάθησης με τη χρήση της μεθόδου που περιεγράφηκε προηγουμένως. Έτσι είναι εφικτό να οριστεί ένα γενικό πλαίσιο (framework) μείωσης διάστασης, το οποίο επίσης συνδυάστηκε με μοντέλα Βαθιάς Μάθησης, ώστε να εξάγει αναπαραστάσεις βελτιστοποιημένες για προβλήματα ομαδοποίησης. Τέλος, αναπτύχθηκε μία βιβλιοθήκη ανοικτού κώδικα η οποία υλοποιεί την παραπάνω μέθοδο μείωσης διάστασης, καθώς και μία μέθοδο σταθεροποίησης της σύγκλισης στοχαστικών τεχνικών βελτιστοποίησης αρχιτεκτονικών Βαθιάς Μάθησης.



Entropy ◽  
2019 ◽  
Vol 21 (5) ◽  
pp. 456 ◽  
Author(s):  
Hao Cheng ◽  
Dongze Lian ◽  
Shenghua Gao ◽  
Yanlin Geng

Inspired by the pioneering work of the information bottleneck (IB) principle for Deep Neural Networks’ (DNNs) analysis, we thoroughly study the relationship among the model accuracy, I ( X ; T ) and I ( T ; Y ) , where I ( X ; T ) and I ( T ; Y ) are the mutual information of DNN’s output T with input X and label Y. Then, we design an information plane-based framework to evaluate the capability of DNNs (including CNNs) for image classification. Instead of each hidden layer’s output, our framework focuses on the model output T. We successfully apply our framework to many application scenarios arising in deep learning and image classification problems, such as image classification with unbalanced data distribution, model selection, and transfer learning. The experimental results verify the effectiveness of the information plane-based framework: Our framework may facilitate a quick model selection and determine the number of samples needed for each class in the unbalanced classification problem. Furthermore, the framework explains the efficiency of transfer learning in the deep learning area.



Sign in / Sign up

Export Citation Format

Share Document