scholarly journals Ramanujan bipartite graph products for efficient block sparse neural networks

2021 ◽  
Author(s):  
Dharma Teja Vooturi ◽  
Girish Varma ◽  
Kishore Kothapalli
Keyword(s):  
Neural Networks ◽  
Bipartite Graph ◽  
Graph Products

scholarly journals TSI-GNN: Extending Graph Neural Networks to Handle Missing Data in Temporal Settings

2021 ◽  
Vol 4 ◽  
Author(s):  
David Gordon ◽  
Panayiotis Petousis ◽  
Henry Zheng ◽  
Davina Zamanzadeh ◽  
Alex A.T. Bui
Keyword(s):  
Neural Networks ◽  
Missing Data ◽  
Bipartite Graph ◽  
Message Passing ◽  
Representation Learning ◽  
Temporal Information ◽  
Graph Representation ◽  
Sequence Information ◽  
Novel Approach ◽  
Graph Neural Networks

We present a novel approach for imputing missing data that incorporates temporal information into bipartite graphs through an extension of graph representation learning. Missing data is abundant in several domains, particularly when observations are made over time. Most imputation methods make strong assumptions about the distribution of the data. While novel methods may relax some assumptions, they may not consider temporality. Moreover, when such methods are extended to handle time, they may not generalize without retraining. We propose using a joint bipartite graph approach to incorporate temporal sequence information. Specifically, the observation nodes and edges with temporal information are used in message passing to learn node and edge embeddings and to inform the imputation task. Our proposed method, temporal setting imputation using graph neural networks (TSI-GNN), captures sequence information that can then be used within an aggregation function of a graph neural network. To the best of our knowledge, this is the first effort to use a joint bipartite graph approach that captures sequence information to handle missing data. We use several benchmark datasets to test the performance of our method against a variety of conditions, comparing to both classic and contemporary methods. We further provide insight to manage the size of the generated TSI-GNN model. Through our analysis we show that incorporating temporal information into a bipartite graph improves the representation at the 30% and 60% missing rate, specifically when using a nonlinear model for downstream prediction tasks in regularly sampled datasets and is competitive with existing temporal methods under different scenarios.

scholarly journals Distilling Knowledge from Well-Informed Soft Labels for Neural Relation Extraction

2020 ◽  
Vol 34 (05) ◽  
pp. 9620-9627 ◽  
Author(s):  
Zhenyu Zhang ◽  
Xiaobo Shu ◽  
Bowen Yu ◽  
Tingwen Liu ◽  
Jiapeng Zhao ◽  
...  
Keyword(s):  
Neural Networks ◽  
Bipartite Graph ◽  
Prior Knowledge ◽  
Semantic Information ◽  
Relation Extraction ◽  
Important Task ◽  
Experimental Results ◽  
Plain Text ◽  
Knowledge Distillation ◽  
The Rich

Extracting relations from plain text is an important task with wide application. Most existing methods formulate it as a supervised problem and utilize one-hot hard labels as the sole target in training, neglecting the rich semantic information among relations. In this paper, we aim to explore the supervision with soft labels in relation extraction, which makes it possible to integrate prior knowledge. Specifically, a bipartite graph is first devised to discover type constraints between entities and relations based on the entire corpus. Then, we combine such type constraints with neural networks to achieve a knowledgeable model. Furthermore, this model is regarded as teacher to generate well-informed soft labels and guide the optimization of a student network via knowledge distillation. Besides, a multi-aspect attention mechanism is introduced to help student mine latent information from text. In this way, the enhanced student inherits the dark knowledge (e.g., type constraints and relevance among relations) from teacher, and directly serves the testing scenarios without any extra constraints. We conduct extensive experiments on the TACRED and SemEval datasets, the experimental results justify the effectiveness of our approach.

scholarly journals Signed Bipartite Graph Neural Networks

2021 ◽  
Author(s):  
Junjie Huang ◽  
Huawei Shen ◽  
Qi Cao ◽  
Shuchang Tao ◽  
Xueqi Cheng
Keyword(s):  
Neural Networks ◽  
Bipartite Graph ◽  
Graph Neural Networks

scholarly journals An Effective Backward Filter Pruning Algorithm Using K1,n Bipartite Graph-Based Clustering and the Decreasing Pruning Rate Approach

2021 ◽  
Author(s):  
Kuo-Liang Chung ◽  
Yu-Lun Chang
Keyword(s):  
Neural Networks ◽  
Bipartite Graph ◽  
Binary Search ◽  
The Other ◽  
Floating Point ◽  
Clustering Method ◽  
Pruning Algorithm ◽  
Similar Accuracy ◽  
Graph Based Clustering ◽  
Pruning Methods

Setting a fixed pruning rate and/or specified threshold for pruning filters in convolutional layers has been widely used to reduce the number of parameters required in the convolutional neural networks (CNN) model. However, it fails to fully prune redundant filters for different layers whose redundant filters vary with layers. To overcome this disadvantage, we propose a new backward filter pruning algorithm using a sorted bipartite graph- and binary search-based (SBGBS-based) clustering and decreasing pruning rate (DPR) approach. We first represent each filter of the last layer by a bipartite graph 𝐾1 𝑛, with one root mean set and one 𝑛-weight set, where 𝑛 denotes the number of weights in the filter. Next, according to the accuracy loss tolerance, an SBGBS-based clustering method is used to partition all filters into clusters as maximal as possible. Then, for each cluster, we retain the filter corresponding to the bipartite graph with the median root mean among 𝑛 root means in the cluster, but we discard the other filters in the same cluster. Following the DPR approach, we repeat the above SBGBS-based filtering pruning approach to the backward layer until all layers are processed. Based on the CIFAR-10 and MNIST datasets, the proposed filter pruning algorithm has been deployed into VGG-16, AlexNet, LeNet, and ResNet. With similar accuracy, the thorough experimental results have demonstrated the substantial parameters and floating-point operations reduction merits of our filter pruning algorithm relative to the existing filter pruning methods.

scholarly journals Argumentation Reasoning with Graph Neural Networks for Reddit Conversation Analysis

2021 ◽  
Author(s):  
Teresa Alsinet ◽  
Josep Argelich ◽  
Ramón Béjar ◽  
Daniel Gibert ◽  
Jordi Planes ◽  
...  
Keyword(s):  
Neural Networks ◽  
Bipartite Graph ◽  
Computational Models ◽  
Automated Analysis ◽  
Acceptable Error ◽  
Reasoning System ◽  
Number Of Layers ◽  
Winning Set ◽  
Graph Neural Networks ◽  
Online Debate

The automated analysis of different trends in online debating forums is an interesting tool for sampling the agreement between citizens in different topics. In these online debating forums, users post different comments and answers to previous comments of other users. In previous work, we have defined computational models to measure different values in these online debating forums. A main ingredient in these models has been the identification of the set of winning posts trough an argumentation problem that characterizes this winning set trough a particular argumentation acceptance semantics. In the argumentation problem we first associate the online debate to analyze as a debate tree. Then, comments are divided in two groups, the ones that agree with the root comment of the debate, and the ones that disagree with it, and we extract a bipartite graph where the unique edges are the disagree edges between comments of the two different groups. Once we compute the set of winning posts, we compute the different measures we are interested to get from the debate, as functions defined over the bipartite graph and the set of winning posts. In this work, we propose to explore the use of graph neural networks to solve the problem of computing these measures, using as input the debate tree, instead of our previous argumentation reasoning system that works with the bipartite graph. We focus on the particular online debate forum Reddit, and on the computation of a measure of the polarization in the debate. Our results over a set of Reddit debates, show that graph neural networks can be used with them to compute the polarization measure with an acceptable error, even if the number of layers of the network is bounded by a constant.

scholarly journals An Effective Backward Filter Pruning Algorithm Using K1,n Bipartite Graph-Based Clustering and the Decreasing Pruning Rate Approach

2021 ◽  
Author(s):  
Kuo-Liang Chung ◽  
Yu-Lun Chang
Keyword(s):  
Neural Networks ◽  
Bipartite Graph ◽  
Binary Search ◽  
The Other ◽  
Floating Point ◽  
Clustering Method ◽  
Pruning Algorithm ◽  
Similar Accuracy ◽  
Graph Based Clustering ◽  
Pruning Methods

Setting a fixed pruning rate and/or specified threshold for pruning filters in convolutional layers has been widely used to reduce the number of parameters required in the convolutional neural networks (CNN) model. However, it fails to fully prune redundant filters for different layers whose redundant filters vary with layers. To overcome this disadvantage, we propose a new backward filter pruning algorithm using a sorted bipartite graph- and binary search-based (SBGBS-based) clustering and decreasing pruning rate (DPR) approach. We first represent each filter of the last layer by a bipartite graph 𝐾1 𝑛, with one root mean set and one 𝑛-weight set, where 𝑛 denotes the number of weights in the filter. Next, according to the accuracy loss tolerance, an SBGBS-based clustering method is used to partition all filters into clusters as maximal as possible. Then, for each cluster, we retain the filter corresponding to the bipartite graph with the median root mean among 𝑛 root means in the cluster, but we discard the other filters in the same cluster. Following the DPR approach, we repeat the above SBGBS-based filtering pruning approach to the backward layer until all layers are processed. Based on the CIFAR-10 and MNIST datasets, the proposed filter pruning algorithm has been deployed into VGG-16, AlexNet, LeNet, and ResNet. With similar accuracy, the thorough experimental results have demonstrated the substantial parameters and floating-point operations reduction merits of our filter pruning algorithm relative to the existing filter pruning methods.

scholarly journals Improving Knowledge Tracing via Pre-training Question Embeddings

2020 ◽  
Author(s):  
Yunfei Liu ◽  
Yang Yang ◽  
Xianyu Chen ◽  
Jian Shen ◽  
Haifeng Zhang ◽  
...  
Keyword(s):  
Neural Networks ◽  
Bipartite Graph ◽  
State Of The Art ◽  
Side Information ◽  
Knowledge Tracing ◽  
Question Difficulty

Knowledge tracing (KT) defines the task of predicting whether students can correctly answer questions based on their historical response. Although much research has been devoted to exploiting the question information, plentiful advanced information among questions and skills hasn't been well extracted, making it challenging for previous work to perform adequately. In this paper, we demonstrate that large gains on KT can be realized by pre-training embeddings for each question on abundant side information, followed by training deep KT models on the obtained embeddings. To be specific, the side information includes question difficulty and three kinds of relations contained in a bipartite graph between questions and skills. To pre-train the question embeddings, we propose to use product-based neural networks to recover the side information. As a result, adopting the pre-trained embeddings in existing deep KT models significantly outperforms state-of-the-art baselines on three common KT datasets.

Sign in / Sign up

Export Citation Format

Share Document