scholarly journals Graph convolutional network for protein sequence sampling and prediction using structural data

2021 ◽  
Author(s):  
Gabriel Andres Orellana ◽  
Javier Caceres-Delpiano ◽  
Roberto Ibañez ◽  
Leonardo Álvarez
Keyword(s):  
Protein Sequence ◽  
Structural Data ◽  
Target Structure ◽  
Convolutional Network ◽  
Test Set ◽  
Sequence Prediction ◽  
Trained Classifier ◽  
Proposed Model ◽  
Additional Question ◽  
Backbone Structure

The increasing integration between protein engineering and machine learning has led to many interesting results. A problem still to solve is to evaluate the likelihood that a sequence will fold into a target structure. This problem can be also viewed as sequence prediction from a known structure.In the current work, we propose improvements in the recent architecture of Geometric Vector Perceptrons in order to optimize the sampling of sequences from a known backbone structure. The proposed model differs from the original in that there is: (i) no updating in the vectorial embedding, only in the scalar one, (ii) only one layer of decoding. The first aspect improves the accuracy of the model and reduces the use of memory, the second allows for training of the model with several tasks without incurring data leakage.We treat the trained classifier as an Energy-Based Model and sample sequences by sampling amino acids in a non-autoreggresive manner in the empty positions of the sequence using energy-guided criteria and followed by Monte Carlo optimization.We improve the median identity of samples from 40.2% to 44.7%.An additional question worth investigating is whether sampled and original sequences fold into similar structures independent of their identity. We chose proteins in our test set whose sampled sequences show low identity (under 30%) but for which our model predicted favorable energies. We used trRosetta server and observed that the predicted structures for sampled sequences highly resemble the predicted structures for original sequences, with an average TM score of 0.848.

A gating context-aware text classification model with BERT and graph convolutional networks

2021 ◽  
pp. 1-13
Author(s):  
Weiqi Gao ◽  
Hao Huang
Keyword(s):  
Text Classification ◽  
Structural Data ◽  
Classification Model ◽  
Context Aware ◽  
Convolutional Network ◽  
Convolutional Networks ◽  
Gating Mechanism ◽  
Document Frequency ◽  
Proposed Model ◽  
Benchmark Datasets

Graph convolutional networks (GCNs), which are capable of effectively processing graph-structural data, have been successfully applied in text classification task. Existing studies on GCN based text classification model largely concerns with the utilization of word co-occurrence and Term Frequency-Inverse Document Frequency (TF–IDF) information for graph construction, which to some extent ignore the context information of the texts. To solve this problem, we propose a gating context-aware text classification model with Bidirectional Encoder Representations from Transformers (BERT) and graph convolutional network, named as Gating Context GCN (GC-GCN). More specifically, we integrates the graph embedding with BERT embedding by using a GCN with gating mechanism enables the acquisition of context coding. We carry out text classification experiments to show the effectiveness of the proposed model. Experimental results shown our model has respectively obtained 0.19%, 0.57%, 1.05% and 1.17% improvements over the Text-GCN baseline on the 20NG, R8, R52, and Ohsumed benchmark datasets. Furthermore, to overcome the problem that word co-occurrence and TF–IDF are not suitable for graph construction for short texts, Euclidean distance is used to combine with word co-occurrence and TF–IDF information. We obtain an improvement by 1.38% on the MR dataset compared to Text-GCN baseline.

scholarly journals Neural networks to learn protein sequence-function relationships from deep mutational scanning data

2020 ◽  
Author(s):  
Sam Gelman ◽  
Philip A. Romero ◽  
Anthony Gitter
Keyword(s):  
Protein Structure ◽  
Protein Sequence ◽  
Internal Representation ◽  
Superior Performance ◽  
Network Architectures ◽  
Convolutional Network ◽  
Learning Framework ◽  
And Function ◽  
Multiple Neural Network ◽  
Function Mapping

ABSTRACTThe mapping from protein sequence to function is highly complex, making it challenging to predict how sequence changes will affect a protein’s behavior and properties. We present a supervised deep learning framework to learn the sequence-function mapping from deep mutational scanning data and make predictions for new, uncharacterized sequence variants. We test multiple neural network architectures, including a graph convolutional network that incorporates protein structure, to explore how a network’s internal representation affects its ability to learn the sequence-function mapping. Our supervised learning approach displays superior performance over physics-based and unsupervised prediction methods. We find networks that capture nonlinear interactions and share parameters across sequence positions are important for learning the relationship between sequence and function. Further analysis of the trained models reveals the networks’ ability to learn biologically meaningful information about protein structure and mechanism. Our software is available from https://github.com/gitter-lab/nn4dms.

scholarly journals Graph-Driven Generative Models for Heterogeneous Multi-Task Learning

2020 ◽  
Vol 34 (01) ◽  
pp. 979-988
Author(s):  
Wenlin Wang ◽  
Hongteng Xu ◽  
Zhe Gan ◽  
Bai Li ◽  
Guoyin Wang ◽  
...  
Keyword(s):  
Generative Models ◽  
Healthcare Applications ◽  
Convolutional Network ◽  
Learning Tasks ◽  
Proposed Model ◽  
Heterogeneous Learning ◽  
Clinical Topic ◽  
Generative Processes ◽  
Admission Type ◽  
Uniform Manner

We propose a novel graph-driven generative model, that unifies multiple heterogeneous learning tasks into the same framework. The proposed model is based on the fact that heterogeneous learning tasks, which correspond to different generative processes, often rely on data with a shared graph structure. Accordingly, our model combines a graph convolutional network (GCN) with multiple variational autoencoders, thus embedding the nodes of the graph (i.e., samples for the tasks) in a uniform manner, while specializing their organization and usage to different tasks. With a focus on healthcare applications (tasks), including clinical topic modeling, procedure recommendation and admission-type prediction, we demonstrate that our method successfully leverages information across different tasks, boosting performance in all tasks and outperforming existing state-of-the-art approaches.

scholarly journals Revisiting Graph Based Collaborative Filtering: A Linear Residual Graph Convolutional Network Approach

2020 ◽  
Vol 34 (01) ◽  
pp. 27-34 ◽  
Author(s):  
Lei Chen ◽  
Le Wu ◽  
Richang Hong ◽  
Kun Zhang ◽  
Meng Wang
Keyword(s):  
Collaborative Filtering ◽  
Representation Learning ◽  
Superior Performance ◽  
Convolutional Network ◽  
Convolutional Networks ◽  
Proposed Model ◽  
Non Linear ◽  
Efficiency And Effectiveness ◽  
Residual Graph ◽  
Interaction Modeling

Graph Convolutional Networks~(GCNs) are state-of-the-art graph based representation learning models by iteratively stacking multiple layers of convolution aggregation operations and non-linear activation operations. Recently, in Collaborative Filtering~(CF) based Recommender Systems~(RS), by treating the user-item interaction behavior as a bipartite graph, some researchers model higher-layer collaborative signals with GCNs. These GCN based recommender models show superior performance compared to traditional works. However, these models suffer from training difficulty with non-linear activations for large user-item graphs. Besides, most GCN based models could not model deeper layers due to the over smoothing effect with the graph convolution operation. In this paper, we revisit GCN based CF models from two aspects. First, we empirically show that removing non-linearities would enhance recommendation performance, which is consistent with the theories in simple graph convolutional networks. Second, we propose a residual network structure that is specifically designed for CF with user-item interaction modeling, which alleviates the over smoothing problem in graph convolution aggregation operation with sparse user-item interaction data. The proposed model is a linear model and it is easy to train, scale to large datasets, and yield better efficiency and effectiveness on two real datasets. We publish the source code at https://github.com/newlei/LR-GCCF.

scholarly journals FAOD-Net: A Fast AOD-Net for Dehazing Single Image

2020 ◽  
Vol 2020 ◽  
pp. 1-11 ◽  
Author(s):  
Wen Qian ◽  
Chao Zhou ◽  
Dengyin Zhang
Keyword(s):  
Quality Evaluation ◽  
Deep Neural Networks ◽  
Reference Image ◽  
Context Information ◽  
Global Information ◽  
Single Image ◽  
Training Set ◽  
Test Set ◽  
Full Reference ◽  
Proposed Model

In this paper, we present an extremely computation-efficient model called FAOD-Net for dehazing single image. FAOD-Net is based on a streamlined architecture that uses depthwise separable convolutions to build lightweight deep neural networks. Moreover, the pyramid pooling module is added in FAOD-Net to aggregate the context information of different regions of the image, thereby improving the ability of the network model to obtain the global information of the foggy image. To get the best FAOD-Net, we use the RESIDE training set to train our proposed model. In addition, we have carried out extensive experiments on the RESIDE test set. We use full-reference and no-reference image quality evaluation indicators to measure the effect of dehazing. Experimental results show that the proposed algorithm has satisfactory results in terms of defogging quality and speed.

scholarly journals A Spatiotemporal Convolutional Network for Multi-Behavior Recognition of Pigs

Sensors ◽  
2020 ◽  
Vol 20 (8) ◽  
pp. 2381
Author(s):  
Dan Li ◽  
Kaifeng Zhang ◽  
Zhenbo Li ◽  
Yifei Chen
Keyword(s):  
Network Architecture ◽  
Statistical Data ◽  
State Of The Art ◽  
Behavior Recognition ◽  
Convolutional Network ◽  
Test Set ◽  
Time Consumption ◽  
And Behavior ◽  
Human Eyes

The statistical data of different kinds of behaviors of pigs can reflect their health status. However, the traditional behavior statistics of pigs were obtained and then recorded from the videos through human eyes. In order to reduce labor and time consumption, this paper proposed a pig behavior recognition network with a spatiotemporal convolutional network based on the SlowFast network architecture for behavior classification of five categories. Firstly, a pig behavior recognition video dataset (PBVD-5) was built by cutting short clips from 3-month non-stop shooting videos, which was composed of five categories of pig’s behavior: feeding, lying, motoring, scratching and mounting. Subsequently, a SlowFast network based spatiotemporal convolutional network for the pig’s multi-behavior recognition (PMB-SCN) was proposed. The results of the networks with variant architectures of the PMB-SCN were implemented and the optimal architecture was compared with the state-of-the-art single stream 3D convolutional network in our dataset. Our 3D pig behavior recognition network showed a top-1 accuracy of 97.63% and a views accuracy of 96.35% on the test set of PBVD and a top-1 accuracy of 91.87% and a views accuracy of 84.47% on a new test set collected from a completely different pigsty. The experimental results showed that this network provided remarkable ability of generalization and possibility for the subsequent pig detection and behavior recognition simultaneously.

scholarly journals Sentence Compression Using BERT and Graph Convolutional Networks

2021 ◽  
Vol 11 (21) ◽  
pp. 9910
Author(s):  
Yo-Han Park ◽  
Gyong-Ho Lee ◽  
Yong-Seok Choi ◽  
Kong-Joo Lee
Keyword(s):  
Language Processing ◽  
Large Scale ◽  
Convolutional Network ◽  
Convolutional Networks ◽  
Compression Model ◽  
Sentence Compression ◽  
Dependency Tree ◽  
Proposed Model ◽  
Syntactic Information ◽  
Input Sentence

Sentence compression is a natural language-processing task that produces a short paraphrase of an input sentence by deleting words from the input sentence while ensuring grammatical correctness and preserving meaningful core information. This study introduces a graph convolutional network (GCN) into a sentence compression task to encode syntactic information, such as dependency trees. As we upgrade the GCN to activate a directed edge, the compression model with the GCN layers can distinguish between parent and child nodes in a dependency tree when aggregating adjacent nodes. Furthermore, by increasing the number of GCN layers, the model can gradually collect high-order information of a dependency tree when propagating node information through the layers. We implement a sentence compression model for Korean and English, respectively. This model consists of three components: pre-trained BERT model, GCN layers, and a scoring layer. The scoring layer can determine whether a word should remain in a compressed sentence by relying on the word vector containing contextual and syntactic information encoded by BERT and GCN layers. To train and evaluate the proposed model, we used the Google sentence compression dataset for English and a Korean sentence compression corpus containing about 140,000 sentence pairs for Korean. The experimental results demonstrate that the proposed model achieves state-of-the-art performance for English. To the best of our knowledge, this sentence compression model based on the deep learning model trained with a large-scale corpus is the first attempt for Korean.

scholarly journals Identifying Human Daily Activity Types with Time-Aware Interactions

2020 ◽  
Vol 10 (24) ◽  
pp. 8922
Author(s):  
Renyao Chen ◽  
Hong Yao ◽  
Runjia Li ◽  
Xiaojun Kang ◽  
Shengwen Li ◽  
...  
Keyword(s):  
Social Media ◽  
Human Activity ◽  
Model Performance ◽  
Heterogeneous Data ◽  
Accurate Identification ◽  
Convolutional Network ◽  
Proposed Model ◽  
Crowdsourced Data ◽  
Activity Types ◽  
Media Data

Human activities embedded in crowdsourced data, such as social media trajectory, represent individual daily styles and patterns, which are valuable in many applications. However, the accurate identification of human activity types (HATs) from social media is challenging, possibly because interactions between posts and users at different time are overlooked. To fill this gap, we propose a novel model that introduces the interactions hidden in social media and synthesizes Graph Convolutional Network (GCN) for identifying HAT. The model first characterizes interactions among words, posts, dates, and users, and then derives a Time Gated Human Activity Graph Convolutional Network (TG-HAGCN) to predict the HATs of social media trajectory. To examine the proposed model performance, we built a new dataset including interactions between post content, post time, and users from the open Yelp dataset. Experimental results show that exploiting interactions hidden in social media to recognize HATs achieves state-of-the-art performance with high accuracy. The study indicates that interactions among social media promotes ability of machine learning on social media data mining and intelligent applications, and offers a reference solution for how to fuse multi-type heterogeneous data in social media.

Sign in / Sign up

Export Citation Format

Share Document