scholarly journals Tensor Graph Convolutional Networks for Text Classification

2020 ◽  
Vol 34 (05) ◽  
pp. 8409-8416
Author(s):  
Xien Liu ◽  
Xinxin You ◽  
Xiao Zhang ◽  
Ji Wu ◽  
Ping Lv
Keyword(s):  
Neural Networks ◽  
Text Classification ◽  
Contextual Information ◽  
Classification Problem ◽  
Heterogeneous Information ◽  
Convolutional Networks ◽  
Benchmark Datasets ◽  
Graph Propagation ◽  
Single Graph ◽  
New Framework

Compared to sequential learning models, graph-based neural networks exhibit some excellent properties, such as ability capturing global information. In this paper, we investigate graph-based neural networks for text classification problem. A new framework TensorGCN (tensor graph convolutional networks), is presented for this task. A text graph tensor is firstly constructed to describe semantic, syntactic, and sequential contextual information. Then, two kinds of propagation learning perform on the text graph tensor. The first is intra-graph propagation used for aggregating information from neighborhood nodes in a single graph. The second is inter-graph propagation used for harmonizing heterogeneous information between graphs. Extensive experiments are conducted on benchmark datasets, and the results illustrate the effectiveness of our proposed framework. Our proposed TensorGCN presents an effective way to harmonize and integrate heterogeneous information from different kinds of graphs.

scholarly journals A Sequential Graph Neural Network for Short Text Classification

Algorithms ◽  
2021 ◽  
Vol 14 (12) ◽  
pp. 352
Author(s):  
Ke Zhao ◽  
Lan Huang ◽  
Rui Song ◽  
Qiang Shen ◽  
Hao Xu
Keyword(s):  
Language Processing ◽  
Text Classification ◽  
Short Term Memory ◽  
Contextual Information ◽  
Extended Model ◽  
Short Text ◽  
Convolutional Networks ◽  
Word Representation ◽  
Benchmark Datasets ◽  
Sequential Information

Short text classification is an important problem of natural language processing (NLP), and graph neural networks (GNNs) have been successfully used to solve different NLP problems. However, few studies employ GNN for short text classification, and most of the existing graph-based models ignore sequential information (e.g., word orders) in each document. In this work, we propose an improved sequence-based feature propagation scheme, which fully uses word representation and document-level word interaction and overcomes the limitations of textual features in short texts. On this basis, we utilize this propagation scheme to construct a lightweight model, sequential GNN (SGNN), and its extended model, ESGNN. Specifically, we build individual graphs for each document in the short text corpus based on word co-occurrence and use a bidirectional long short-term memory network (Bi-LSTM) to extract the sequential features of each document; therefore, word nodes in the document graph retain contextual information. Furthermore, two different simplified graph convolutional networks (GCNs) are used to learn word representations based on their local structures. Finally, word nodes combined with sequential information and local information are incorporated as the document representation. Extensive experiments on seven benchmark datasets demonstrate the effectiveness of our method.

scholarly journals METHODS FOR INCREASING THE CLASSIFICATION ACCURACY BASED ON MODIFICATIONS OF THE BASIC ARCHITECTURE OF CONVOLUTIONAL NEURAL NETWORKS

ScienceRise ◽  
2020 ◽  
pp. 10-16
Author(s):  
Svitlana Shapovalova ◽  
Yurii Moskalenko
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Classification Accuracy ◽  
Malignant Tumors ◽  
Classification Problem ◽  
Video Memory ◽  
Backbone Networks ◽  
Convolutional Networks ◽  
High Recognition Accuracy ◽  
Technological Product

Object of research: basic architectures of deep learning neural networks. Investigated problem: insufficient accuracy of solving the classification problem based on the basic architectures of deep learning neural networks. An increase in accuracy requires a significant complication of the architecture, which, in turn, leads to an increase in the required computing resources, as well as the consumption of video memory and the cost of learning/output time. Therefore, the problem arises of determining such methods for modifying basic architectures that improve the classification accuracy and require insignificant additional computing resources. Main scientific results: based on the analysis of existing methods for improving the classification accuracy on the convolutional networks of basic architectures, it is determined what is most effective: scaling the ScanNet architecture, learning the ensemble of TreeNet models, integrating several CBNet backbone networks. For computational experiments, these modifications of the basic architectures are implemented, as well as their combinations: ScanNet + TreeNet, ScanNet + CBNet. The effectiveness of these methods in comparison with basic architectures has been proven when solving the problem of recognizing malignant tumors with diagnostic images – SIIM-ISIC Melanoma Classification, the train/test set of which is presented on the Kaggle platform. The accuracy value for the area under the ROC curve metric has increased from 0.94489 (basic architecture network) to 0.96317 (network with ScanNet + CBNet modifications). At the same time, the output compared to the basic architecture (EfficientNet-b5) increased from 440 to 490 seconds, and the consumption of video memory increased from 8 to 9.2 gigabytes, which is acceptable. Innovative technological product: methods for achieving high recognition accuracy from a diagnostic signal based on deep learning neural networks of basic architectures. Scope of application of the innovative technological product: automatic diagnostics systems in the following areas: medicine, seismology, astronomy (classification by images) onboard control systems and systems for monitoring transport and vehicle flows or visitors (recognition of scenes with camera frames).

scholarly journals Graph Convolutional Networks for Text Classification

2019 ◽  
Vol 33 ◽  
pp. 7370-7377 ◽  
Author(s):  
Liang Yao ◽  
Chengsheng Mao ◽  
Yuan Luo
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Language Processing ◽  
Text Classification ◽  
State Of The Art ◽  
Classical Problem ◽  
Experimental Results ◽  
Training Data ◽  
Convolutional Networks ◽  
Single Text

Text classification is an important and classical problem in natural language processing. There have been a number of studies that applied convolutional neural networks (convolution on regular grid, e.g., sequence) to classification. However, only a limited number of studies have explored the more flexible graph convolutional neural networks (convolution on non-grid, e.g., arbitrary graph) for the task. In this work, we propose to use graph convolutional networks for text classification. We build a single text graph for a corpus based on word co-occurrence and document word relations, then learn a Text Graph Convolutional Network (Text GCN) for the corpus. Our Text GCN is initialized with one-hot representation for word and document, it then jointly learns the embeddings for both words and documents, as supervised by the known class labels for documents. Our experimental results on multiple benchmark datasets demonstrate that a vanilla Text GCN without any external word embeddings or knowledge outperforms state-of-the-art methods for text classification. On the other hand, Text GCN also learns predictive word and document embeddings. In addition, experimental results show that the improvement of Text GCN over state-of-the-art comparison methods become more prominent as we lower the percentage of training data, suggesting the robustness of Text GCN to less training data in text classification.

A gating context-aware text classification model with BERT and graph convolutional networks

2021 ◽  
pp. 1-13
Author(s):  
Weiqi Gao ◽  
Hao Huang
Keyword(s):  
Text Classification ◽  
Structural Data ◽  
Classification Model ◽  
Context Aware ◽  
Convolutional Network ◽  
Convolutional Networks ◽  
Gating Mechanism ◽  
Document Frequency ◽  
Proposed Model ◽  
Benchmark Datasets

Graph convolutional networks (GCNs), which are capable of effectively processing graph-structural data, have been successfully applied in text classification task. Existing studies on GCN based text classification model largely concerns with the utilization of word co-occurrence and Term Frequency-Inverse Document Frequency (TF–IDF) information for graph construction, which to some extent ignore the context information of the texts. To solve this problem, we propose a gating context-aware text classification model with Bidirectional Encoder Representations from Transformers (BERT) and graph convolutional network, named as Gating Context GCN (GC-GCN). More specifically, we integrates the graph embedding with BERT embedding by using a GCN with gating mechanism enables the acquisition of context coding. We carry out text classification experiments to show the effectiveness of the proposed model. Experimental results shown our model has respectively obtained 0.19%, 0.57%, 1.05% and 1.17% improvements over the Text-GCN baseline on the 20NG, R8, R52, and Ohsumed benchmark datasets. Furthermore, to overcome the problem that word co-occurrence and TF–IDF are not suitable for graph construction for short texts, Euclidean distance is used to combine with word co-occurrence and TF–IDF information. We obtain an improvement by 1.38% on the MR dataset compared to Text-GCN baseline.

scholarly journals Graph Convolutional Networks by Architecture Search for PolSAR Image Classification

2021 ◽  
Vol 13 (7) ◽  
pp. 1404
Author(s):  
Hongying Liu ◽  
Derong Xu ◽  
Tianwen Zhu ◽  
Fanhua Shang ◽  
Yuanyuan Liu ◽  
...  
Keyword(s):  
Neural Networks ◽  
Large Scale ◽  
Spatial Relations ◽  
Classification Problem ◽  
Search Method ◽  
Sample Distribution ◽  
Convolutional Networks ◽  
Training Samples ◽  
Graph Neural Networks

Classification of polarimetric synthetic aperture radar (PolSAR) images has achieved good results due to the excellent fitting ability of neural networks with a large number of training samples. However, the performance of most convolutional neural networks (CNNs) degrades dramatically when only a few labeled training samples are available. As one well-known class of semi-supervised learning methods, graph convolutional networks (GCNs) have gained much attention recently to address the classification problem with only a few labeled samples. As the number of layers grows in the network, the parameters dramatically increase. It is challenging to determine an optimal architecture manually. In this paper, we propose a neural architecture search method based GCN (ASGCN) for the classification of PolSAR images. We construct a novel graph whose nodes combines both the physical features and spatial relations between pixels or samples to represent the image. Then we build a new searching space whose components are empirically selected from some graph neural networks for architecture search and develop the differentiable architecture search method to construction our ASGCN. Moreover, to address the training of large-scale images, we present a new weighted mini-batch algorithm to reduce the computing memory consumption and ensure the balance of sample distribution, and also analyze and compare with other similar training strategies. Experiments on several real-world PolSAR datasets show that our method has improved the overall accuracy as much as 3.76% than state-of-the-art methods.

scholarly journals Hybrid Quantum-Classical Neural Networks for Text Classification

2021 ◽  
Author(s):  
Dhruv Baronia
Keyword(s):  
Neural Networks ◽  
Quantum Computing ◽  
Text Classification ◽  
Classical Model ◽  
Classification Problem ◽  
Quantum Circuit ◽  
Hybrid Models ◽  
Proof Of Concept ◽  
Intermediate Scale

Quantum Computing presents an interesting paradigm where it can possibly offer certain improvements and additions to a classical network while training. This method is particularly prevalent in the current Noisy Intermediate-Scale Quantum era, where we can test these theories using libraries such as Pennylane in conjunction with robust ML frameworks such as TensorFlow. This paper presents a proof-of-concept for the same, using a hybrid quantum-classical model to solve a text classification problem on the IMDB Movie Sentiment Dataset. These hybrid models utilize precalculated embeddings and dense layers alongside a variational quantum circuit layer. We created 4 such models, utilizing various kinds of embeddings, namely NNLM-128, NNLM-50, Swivel and USE, using TFHub and Pennylane. We also trained classical versions of these models, without the variational quantum layer to evaluate the performances. All models were trained on the same data, keeping the parameters constant.

scholarly journals Hybrid Quantum-Classical Neural Networks for Text Classification

2021 ◽  
Author(s):  
Dhruv Baronia
Keyword(s):  
Neural Networks ◽  
Quantum Computing ◽  
Text Classification ◽  
Classical Model ◽  
Classification Problem ◽  
Quantum Circuit ◽  
Hybrid Models ◽  
Proof Of Concept ◽  
Intermediate Scale

Quantum Computing presents an interesting paradigm where it can possibly offer certain improvements and additions to a classical network while training. This method is particularly prevalent in the current Noisy Intermediate-Scale Quantum era, where we can test these theories using libraries such as Pennylane in conjunction with robust ML frameworks such as TensorFlow. This paper presents a proof-of-concept for the same, using a hybrid quantum-classical model to solve a text classification problem on the IMDB Movie Sentiment Dataset. These hybrid models utilize precalculated embeddings and dense layers alongside a variational quantum circuit layer. We created 4 such models, utilizing various kinds of embeddings, namely NNLM-128, NNLM-50, Swivel and USE, using TFHub and Pennylane. We also trained classical versions of these models, without the variational quantum layer to evaluate the performances. All models were trained on the same data, keeping the parameters constant.

Video Classification Using 3D Convolutional Neural Network

2021 ◽  
pp. 1-18
Author(s):  
K. Jairam Naik ◽  
Annukriti Soni
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Spatial Information ◽  
Three Dimensional ◽  
Classification Problem ◽  
Video Classification ◽  
Convolutional Networks ◽  
3D Cnn ◽  
Deep Learning Neural Network

Since video includes both temporal and spatial features, it has become a fascinating classification problem. Each frame within a video holds important information called spatial information, as does the context of that frame relative to the frames before it in time called temporal information. Several methods have been invented for video classification, but each one is suffering from its own drawback. One of such method is called convolutional neural networks (CNN) model. It is a category of deep learning neural network model that can turn directly on the underdone inputs. However, such models are recently limited to handling two-dimensional inputs only. This chapter implements a three-dimensional convolutional neural networks (CNN) model for video classification to analyse the classification accuracy gained using the 3D CNN model. The 3D convolutional networks are preferred for video classification since they inherently apply convolutions in the 3D space.

Sign in / Sign up

Export Citation Format

Share Document