Negative-supervised capsule graph neural network for few-shot text classification

2021 ◽  
pp. 1-13
Author(s):  
Ling Ding ◽  
Xiaojun Chen ◽  
Yang Xiang
Keyword(s):  
Neural Network ◽  
Text Classification ◽  
Routing Algorithm ◽  
Information Aggregation ◽  
Dynamic Routing ◽  
Experimental Results ◽  
Text Data ◽  
Support Set ◽  
Different Types ◽  
Benchmark Datasets

Few-shot text classification aims to learn a classifier from very few labeled text data. Existing studies on this topic mainly adopt prototypical networks and focus on interactive information between support set and query instances to learn generalized class prototypes. However, in the process of encoding, these methods only pay attention to the matching information between support set and query instances, and ignore much useful information about intra-class similarity and inter-class dissimilarity between all support samples. Therefore, in this paper we propose a negative-supervised capsule graph neural network (NSCGNN) which explicitly takes use of the similarity and dissimilarity between samples to make the text representations of the same type closer with each other and the ones of different types farther away, leading to representative and discriminative class prototypes. We firstly construct a graph to obtain text representations in the form of node capsules, where both intra-cluster similarity and inter-cluster dissimilarity between all samples are explored with information aggregation and negative supervision. Then, in order to induce generalized class prototypes based on those node capsules obtained from graph neural network, the dynamic routing algorithm is utilized in our model. Experimental results demonstrate the effectiveness of our proposed NSCGNN model, which outperforms existing few-shot approaches on three benchmark datasets.

scholarly journals A Robust Text Classifier Based on Denoising Deep Neural Network in the Analysis of Big Data

2017 ◽  
Vol 2017 ◽  
pp. 1-10 ◽  
Author(s):  
Wulamu Aziguli ◽  
Yuanyu Zhang ◽  
Yonghong Xie ◽  
Dezheng Zhang ◽  
Xiong Luo ◽  
...  
Keyword(s):  
Neural Network ◽  
Big Data ◽  
Language Processing ◽  
Text Classification ◽  
Deep Neural Network ◽  
Big Data Analytics ◽  
Research Area ◽  
Text Data ◽  
Computational Performance ◽  
Benchmark Datasets

Text classification has always been an interesting issue in the research area of natural language processing (NLP). While entering the era of big data, a good text classifier is critical to achieving NLP for scientific big data analytics. With the ever-increasing size of text data, it has posed important challenges in developing effective algorithm for text classification. Given the success of deep neural network (DNN) in analyzing big data, this article proposes a novel text classifier using DNN, in an effort to improve the computational performance of addressing big text data with hybrid outliers. Specifically, through the use of denoising autoencoder (DAE) and restricted Boltzmann machine (RBM), our proposed method, named denoising deep neural network (DDNN), is able to achieve significant improvement with better performance of antinoise and feature extraction, compared to the traditional text classification algorithms. The simulations on benchmark datasets verify the effectiveness and robustness of our proposed text classifier.

scholarly journals Sentiment Classification Using Convolutional Neural Networks

2019 ◽  
Vol 9 (11) ◽  
pp. 2347 ◽  
Author(s):  
Hannah Kim ◽  
Young-Seob Jeong
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Text Classification ◽  
State Of The Art ◽  
Sentiment Classification ◽  
Learning Models ◽  
Text Data ◽  
Textual Data ◽  
Better Than

As the number of textual data is exponentially increasing, it becomes more important to develop models to analyze the text data automatically. The texts may contain various labels such as gender, age, country, sentiment, and so forth. Using such labels may bring benefits to some industrial fields, so many studies of text classification have appeared. Recently, the Convolutional Neural Network (CNN) has been adopted for the task of text classification and has shown quite successful results. In this paper, we propose convolutional neural networks for the task of sentiment classification. Through experiments with three well-known datasets, we show that employing consecutive convolutional layers is effective for relatively longer texts, and our networks are better than other state-of-the-art deep learning models.

scholarly journals Evaluation of CNN, Alexnet and GoogleNet for Fruit Recognition

2018 ◽  
Vol 12 (2) ◽  
pp. 468 ◽  
Author(s):  
Nur Azida Muhammad ◽  
Amelina Ab Nasir ◽  
Zaidah Ibrahim ◽  
Nurbaity Sabri
Keyword(s):  
Neural Network ◽  
Computer Vision ◽  
Convolutional Neural Network ◽  
Recognition Accuracy ◽  
Weather Condition ◽  
Experimental Results ◽  
Human Intervention ◽  
Recognition Result ◽  
Different Types

Fruit recognition is useful for automatic fruit harvesting. Fruit recognition application can reduce or minimize human intervention during fruit harvesting operation. However, in computer vision, fruit recognition is very challenging because of similar shapes, colors and textures among various fruits. Illuminations changes due to weather condition also leads to a challenging task for fruit recognition. Thus, this paper tends to investigate the performance of basic Convolutional Neural Network (CNN), Alexnet and Googlenet in recognizing nine different types of fruits from a publicly available dataset.  The experimental results indicate that all these techniques produce excellent recognition accuracy, but basic CNN achieves the fastest recognition result compared with Alexnet and Googlenet.

scholarly journals Improved Dynamic Routing Algorithm for Information Aggregation

2021 ◽  
Vol 5 (1) ◽  
Author(s):  
Gongbin Chen ◽  
◽  
Wei Xiang ◽  
Yansong Deng ◽  
◽  
...  
Keyword(s):  
Routing Algorithm ◽  
Information Aggregation ◽  
Dynamic Routing ◽  
Attention Mechanism ◽  
Aggregation Method ◽  
Text Encoding ◽  
Natural Logarithm ◽  
Aggregate Information ◽  
Routing Policy ◽  
Series Of Experiments

Information aggregation is an essential component of text encoding, but it has been paid less attention. The pooling-based (max or average pooling) aggregation method is a bottom-up and passive aggregation method, and loses a lot of important information. Recently, attention mechanism and dynamic routing policy are separately used to aggregate information, but their aggregation capabilities can be further improved. In this paper, we proposed an novel aggregation method combining attention mechanism and dynamic routing, which can strengthen the ability of information aggregation and improve the quality of text encoding. Then, a novel Leaky Natural Logarithm (LNL) squash function is designed to alleviate the “saturation” problem of the squash function of the original dynamic routing. Layer Normalization is added to the dynamic routing policy for speeding up routing convergence as well. A series of experiments are conducted on five text classification benchmarks. Experimental results show that our method outperforms other aggregating methods.

scholarly journals An External Knowledge Enhanced Graph-based Neural Network for Sentence Ordering

2021 ◽  
Vol 70 ◽  
pp. 545-566
Author(s):  
Yongjing Yin ◽  
Shaopeng Lai ◽  
Linfeng Song ◽  
Chulun Zhou ◽  
Xianpei Han ◽  
...  
Keyword(s):  
Neural Network ◽  
State Of The Art ◽  
Recurrent Network ◽  
Experimental Results ◽  
Graph Representation ◽  
External Knowledge ◽  
Text Coherence ◽  
Depth Analysis ◽  
Benchmark Datasets ◽  
Modeling Task

As an important text coherence modeling task, sentence ordering aims to coherently organize a given set of unordered sentences. To achieve this goal, the most important step is to effectively capture and exploit global dependencies among these sentences. In this paper, we propose a novel and flexible external knowledge enhanced graph-based neural network for sentence ordering. Specifically, we first represent the input sentences as a graph, where various kinds of relations (i.e., entity-entity, sentence-sentence and entity-sentence) are exploited to make the graph representation more expressive and less noisy. Then, we introduce graph recurrent network to learn semantic representations of the sentences. To demonstrate the effectiveness of our model, we conduct experiments on several benchmark datasets. The experimental results and in-depth analysis show our model significantly outperforms the existing state-of-the-art models.

scholarly journals Order-Free Learning Alleviating Exposure Bias in Multi-Label Classification

2020 ◽  
Vol 34 (04) ◽  
pp. 6038-6045
Author(s):  
Che-Ping Tsai ◽  
Hung-Yi Lee
Keyword(s):  
Neural Network ◽  
Recurrent Neural Network ◽  
Experimental Results ◽  
Prediction Problem ◽  
Generalization Capability ◽  
Training Set ◽  
Large Margin ◽  
Sequence Prediction ◽  
Benchmark Datasets ◽  
New Framework

Multi-label classification (MLC) assigns multiple labels to each sample. Prior studies show that MLC can be transformed to a sequence prediction problem with a recurrent neural network (RNN) decoder to model the label dependency. However, training a RNN decoder requires a predefined order of labels, which is not directly available in the MLC specification. Besides, RNN thus trained tends to overfit the label combinations in the training set and have difficulty generating unseen label sequences. In this paper, we propose a new framework for MLC which does not rely on a predefined label order and thus alleviates exposure bias. The experimental results on three multi-label classification benchmark datasets show that our method outperforms competitive baselines by a large margin. We also find the proposed approach has a higher probability of generating label combinations not seen during training than the baseline models. The result shows that the proposed approach has better generalization capability.

The effects of globalisation techniques on feature selection for text classification

2020 ◽  
pp. 016555152093089
Author(s):  
Bekir Parlak ◽  
Alper Kursat Uysal
Keyword(s):  
Feature Selection ◽  
Text Classification ◽  
High Volume ◽  
Experimental Results ◽  
Support Vector ◽  
Svm Classifier ◽  
Chi Square ◽  
Discriminative Feature ◽  
Benchmark Datasets ◽  
Different Characteristics

Text classification (TC) is very important and critical task in the 21th century as there exist high volume of electronic data on the Internet. In TC, textual data are characterised by a huge number of highly sparse features/terms. A typical TC consists of many steps and one of the most important steps is undoubtedly feature selection (FS). In this study, we have comprehensively investigated the effects of various globalisation techniques on local feature selection (LFS) methods using datasets with different characteristics such as multi-class unbalanced (MCU), multi-class balanced (MCB), binary-class unbalanced (BCU) and binary-class balanced (BCB). The globalisation techniques used in this study are summation (SUM), weighted-sum (AVG), and maximum (MAX). To investigate the effect of globalisation techniques, we used three LFS methods named as Discriminative Feature Selection (DFSS), odds ratio (OR) and chi-square (CHI2). In the experiments, we have utilised four different benchmark datasets named as Reuters-21578, 20Newsgroup., Enron1, and Polarity in addition to Support Vector Machines (SVM) and Decision Tree (DT) classifiers. According to the experimental results, the most successful globalisation technique is AVG while all situations are taken into account. The experimental results indicate that DFSS method is more successful than OR and CHI2 methods on datasets with MCU and MCB characteristics. However, CHI2 method seems more accurate than OR and DFSS methods on datasets with BCU and BCB characteristics. Also, SVM classifier performed better than DT classifier in most cases.

scholarly journals Research on News Text Classification Based on Deep Learning Convolutional Neural Network

2021 ◽  
Vol 2021 ◽  
pp. 1-6
Author(s):  
Yunlong Zhu
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Text Classification ◽  
Information Gain ◽  
Basic Structure ◽  
Text Data ◽  
Hash Algorithm ◽  
Low Efficiency ◽  
Viable System

Aiming at the problems of low classification accuracy and low efficiency of existing news text classification methods, a new method of news text classification based on deep learning convolutional neural network is proposed. Determine the weight of the news text data through the VSM (Viable System Model) vector space model, calculate the information gain of mutual information, and determine the characteristics of the news text data; on this basis, use the hash algorithm to encode the news text data to calculate any news. The spacing between the text data realizes the feature preprocessing of the news text data; this article analyzes the basic structure of the deep learning convolutional neural network, uses the convolutional layer in the convolutional neural network to determine the change value of the convolution kernel, trains the news text data, builds a news text classifier of deep learning convolutional neural network, and completes news text classification. The experimental results show that the deep learning convolutional neural network can improve the accuracy and speed of news text classification, which is feasible.

Accurate and Transferable Multitask Prediction of Chemical Properties with an Atoms-in-Molecule Neural Network

2018 ◽  
Author(s):  
Roman Zubatyuk ◽  
Justin S. Smith ◽  
Jerzy Leszczynski ◽  
Olexandr Isayev
Keyword(s):  
Neural Network ◽  
Molecular System ◽  
Computational Cost ◽  
Chemical Properties ◽  
The State ◽  
Molecular Properties ◽  
Training Data ◽  
Dft Methods ◽  
Benchmark Datasets ◽  
Quantum Phenomena

<p>Atomic and molecular properties could be evaluated from the fundamental Schrodinger’s equation and therefore represent different modalities of the same quantum phenomena. Here we present AIMNet, a modular and chemically inspired deep neural network potential. We used AIMNet with multitarget training to learn multiple modalities of the state of the atom in a molecular system. The resulting model shows on several benchmark datasets the state-of-the-art accuracy, comparable to the results of orders of magnitude more expensive DFT methods. It can simultaneously predict several atomic and molecular properties without an increase in computational cost. With AIMNet we show a new dimension of transferability: the ability to learn new targets utilizing multimodal information from previous training. The model can learn implicit solvation energy (like SMD) utilizing only a fraction of original training data, and archive MAD error of 1.1 kcal/mol compared to experimental solvation free energies in MNSol database.</p>

Sign in / Sign up

Export Citation Format

Share Document