A Neural N-Gram Network for Text Classification

2018 ◽  
Vol 22 (3) ◽  
pp. 380-386 ◽  
Author(s):  
Zhenguo Yan ◽  
◽  
Yue Wu
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Text Classification ◽  
Word Order ◽  
Input Data ◽  
State Of The Art ◽  
Poor Performance ◽  
Proposed Model ◽  
Classification Tasks ◽  
N Gram

Convolutional Neural Networks (CNNs) effectively extract local features from input data. However, CNN based on word embedding and convolution layers displays poor performance in text classification tasks when compared with traditional baseline methods. We address this problem and propose a model named NNGN that simplifies the convolution layer in the CNN by replacing it with a pooling layer that extracts n-gram embedding in a simpler way and obtains document representations via linear computation. We implement two settings in our model to extract n-gram features. In the first setting, which we refer to as seq-NNGN, we consider word order within each n-gram. In the second setting, BoW-NNGN, we do not consider word order. We compare the performance of these settings in different classification tasks with those of other models. The experimental results show that our proposed model achieves better performance than state-of-the-art models.

scholarly journals Sentiment Classification Using Convolutional Neural Networks

2019 ◽  
Vol 9 (11) ◽  
pp. 2347 ◽  
Author(s):  
Hannah Kim ◽  
Young-Seob Jeong
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Text Classification ◽  
State Of The Art ◽  
Sentiment Classification ◽  
Learning Models ◽  
Text Data ◽  
Textual Data ◽  
Better Than

As the number of textual data is exponentially increasing, it becomes more important to develop models to analyze the text data automatically. The texts may contain various labels such as gender, age, country, sentiment, and so forth. Using such labels may bring benefits to some industrial fields, so many studies of text classification have appeared. Recently, the Convolutional Neural Network (CNN) has been adopted for the task of text classification and has shown quite successful results. In this paper, we propose convolutional neural networks for the task of sentiment classification. Through experiments with three well-known datasets, we show that employing consecutive convolutional layers is effective for relatively longer texts, and our networks are better than other state-of-the-art deep learning models.

scholarly journals Incorporating Context-Relevant Knowledge into Convolutional Neural Networks for Short Text Classification

2019 ◽  
Vol 33 ◽  
pp. 10067-10068 ◽  
Author(s):  
Jingyun Xu ◽  
Yi Cai
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Convolutional Neural Network ◽  
Convolutional Neural Networks ◽  
Text Classification ◽  
Classification Methods ◽  
Short Text ◽  
Proposed Model ◽  
High Level ◽  
Context Features

Some text classification methods don’t work well on short texts due to the data sparsity. What’s more, they don’t fully exploit context-relevant knowledge. In order to tackle these problems, we propose a neural network to incorporate context-relevant knowledge into a convolutional neural network for short text classification. Our model consists of two modules. The first module utilizes two layers to extract concept and context features respectively and then employs an attention layer to extract those context-relevant concepts. The second module utilizes a convolutional neural network to extract high-level features from the word and the contextrelevant concept features. The experimental results on three datasets show that our proposed model outperforms the stateof-the-art models.

scholarly journals HierarchyNet: Hierarchical CNN-Based Urban Building Classification

2020 ◽  
Vol 12 (22) ◽  
pp. 3794
Author(s):  
Salma Taoufiq ◽  
Balázs Nagy ◽  
Csaba Benedek
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
State Of The Art ◽  
Feedback Signal ◽  
Hierarchical Network ◽  
Urban Buildings ◽  
Virtual City ◽  
Classification Tasks ◽  
Coarse To Fine

Automatic building categorization and analysis are particularly relevant for smart city applications and cultural heritage programs. Taking a picture of the facade of a building and instantly obtaining information about it can enable the automation of processes in urban planning, virtual city tours, and digital archiving of cultural artifacts. In this paper, we go beyond traditional convolutional neural networks (CNNs) for image classification and propose the HierarchyNet: a new hierarchical network for the classification of urban buildings from all across the globe into different main and subcategories from images of their facades. We introduce a coarse-to-fine hierarchy on the dataset and the model learns to simultaneously extract features and classify across both levels of hierarchy. We propose a new multiplicative layer, which is able to improve the accuracy of the finer prediction by considering the feedback signal of the coarse layers. We have quantitatively evaluated the proposed approach both on our proposed building datasets, as well as on various benchmark databases to demonstrate that the model is able to efficiently learn hierarchical information. The HierarchyNet model is able to outperform the state-of-the-art convolutional neural networks in urban building classification as well as in other multi-label classification tasks while using significantly fewer parameters.

Hybrid pooling with wavelets for convolutional neural networks

2022 ◽  
pp. 1-10
Author(s):  
Daniel Trevino-Sanchez ◽  
Vicente Alarcon-Aquino
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
State Of The Art ◽  
Computational Cost ◽  
Relevant Information ◽  
Accuracy Improvement ◽  
Proposed Model ◽  
Benchmark Datasets ◽  
Augmentation Techniques ◽  
High Computational Cost

The need to detect and classify objects correctly is a constant challenge, being able to recognize them at different scales and scenarios, sometimes cropped or badly lit is not an easy task. Convolutional neural networks (CNN) have become a widely applied technique since they are completely trainable and suitable to extract features. However, the growing number of convolutional neural networks applications constantly pushes their accuracy improvement. Initially, those improvements involved the use of large datasets, augmentation techniques, and complex algorithms. These methods may have a high computational cost. Nevertheless, feature extraction is known to be the heart of the problem. As a result, other approaches combine different technologies to extract better features to improve the accuracy without the need of more powerful hardware resources. In this paper, we propose a hybrid pooling method that incorporates multiresolution analysis within the CNN layers to reduce the feature map size without losing details. To prevent relevant information from losing during the downsampling process an existing pooling method is combined with wavelet transform technique, keeping those details "alive" and enriching other stages of the CNN. Achieving better quality characteristics improves CNN accuracy. To validate this study, ten pooling methods, including the proposed model, are tested using four benchmark datasets. The results are compared with four of the evaluated methods, which are also considered as the state-of-the-art.

scholarly journals Graph Convolutional Networks for Text Classification

2019 ◽  
Vol 33 ◽  
pp. 7370-7377 ◽  
Author(s):  
Liang Yao ◽  
Chengsheng Mao ◽  
Yuan Luo
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Language Processing ◽  
Text Classification ◽  
State Of The Art ◽  
Classical Problem ◽  
Experimental Results ◽  
Training Data ◽  
Convolutional Networks ◽  
Single Text

Text classification is an important and classical problem in natural language processing. There have been a number of studies that applied convolutional neural networks (convolution on regular grid, e.g., sequence) to classification. However, only a limited number of studies have explored the more flexible graph convolutional neural networks (convolution on non-grid, e.g., arbitrary graph) for the task. In this work, we propose to use graph convolutional networks for text classification. We build a single text graph for a corpus based on word co-occurrence and document word relations, then learn a Text Graph Convolutional Network (Text GCN) for the corpus. Our Text GCN is initialized with one-hot representation for word and document, it then jointly learns the embeddings for both words and documents, as supervised by the known class labels for documents. Our experimental results on multiple benchmark datasets demonstrate that a vanilla Text GCN without any external word embeddings or knowledge outperforms state-of-the-art methods for text classification. On the other hand, Text GCN also learns predictive word and document embeddings. In addition, experimental results show that the improvement of Text GCN over state-of-the-art comparison methods become more prominent as we lower the percentage of training data, suggesting the robustness of Text GCN to less training data in text classification.

scholarly journals A Novel Electricity Theft Detection Scheme Based on Text Convolutional Neural Networks

Energies ◽  
2020 ◽  
Vol 13 (21) ◽  
pp. 5758
Author(s):  
Xiaofeng Feng ◽  
Hengyu Hui ◽  
Ziyang Liang ◽  
Wenchong Guo ◽  
Huakun Que ◽  
...  
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Data Augmentation ◽  
State Of The Art ◽  
Electricity Consumption ◽  
Temporal Data ◽  
Detection Scheme ◽  
Fine Grained ◽  
Electricity Theft ◽  
Proposed Model

Electricity theft decreases electricity revenues and brings risks to power usage’s safety, which has been increasingly challenging nowadays. As the mainstream in the relevant studies, the state-of-the-art data-driven approaches mainly detect electricity theft events from the perspective of the correlations between different daily or weekly loads, which is relatively inadequate to extract features from hours or more of fine-grained temporal data. In view of the above deficiencies, we propose a novel electricity theft detection scheme based on text convolutional neural networks (TextCNN). Specifically, we convert electricity consumption measurements over a horizon of interest into a two-dimensional time-series containing the intraday electricity features. Based on the data structure, the proposed method can accurately capture various periodical features of electricity consumption. Moreover, a data augmentation method is proposed to cope with the imbalance of electricity theft data. Extensive experimental results based on realistic Chinese and Irish datasets indicate that the proposed model achieves a better performance compared with other existing methods.

scholarly journals A pseudo-softmax function for hardware-based high speed image classification

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Gian Carlo Cardarilli ◽  
Luca Di Nunzio ◽  
Rocco Fazzolari ◽  
Daniele Giardino ◽  
Alberto Nannarelli ◽  
...  
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
High Speed ◽  
Hardware Implementation ◽  
State Of The Art ◽  
Approximation Error ◽  
Hardware Accelerators ◽  
Selection Policy ◽  
Approximation Errors ◽  
Classification Tasks

AbstractIn this work a novel architecture, named pseudo-softmax, to compute an approximated form of the softmax function is presented. This architecture can be fruitfully used in the last layer of Neural Networks and Convolutional Neural Networks for classification tasks, and in Reinforcement Learning hardware accelerators to compute the Boltzmann action-selection policy. The proposed pseudo-softmax design, intended for efficient hardware implementation, exploits the typical integer quantization of hardware-based Neural Networks obtaining an accurate approximation of the result. In the paper, a detailed description of the architecture is given and an extensive analysis of the approximation error is performed by using both custom stimuli and real-world Convolutional Neural Networks inputs. The implementation results, based on CMOS standard-cell technology, compared to state-of-the-art architectures show reduced approximation errors.

scholarly journals Transformable Convolutional Neural Network for Text Classification

2018 ◽  
Author(s):  
Liqiang Xiao ◽  
Honglun Zhang ◽  
Wenqing Chen ◽  
Yongkun Wang ◽  
Yaohui Jin
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Natural Language Processing ◽  
Language Processing ◽  
Text Classification ◽  
State Of The Art ◽  
Feature Transformation ◽  
Sampling Locations ◽  
N Gram ◽  
Complex Features

Convolutional neural networks (CNNs) have shown their promising performance for natural language processing tasks, which extract n-grams as features to represent the input. However, n-gram based CNNs are inherently limited to fixed geometric structure and cannot proactively adapt to the transformations of features. In this paper, we propose two modules to provide CNNs with the flexibility for complex features and the adaptability for transformation, namely, transformable convolution and transformable pooling. Our method fuses dynamic and static deviations to redistribute the sampling locations, which can capture both current and global transformations. Our modules can be easily integrated by other models to generate new transformable networks. We test proposed modules on two state-of-the-art models, and the results demonstrate that our modules can effectively adapt to the feature transformation in text classification.

scholarly journals Adaptive Region Embedding for Text Classification

2019 ◽  
Vol 33 ◽  
pp. 7314-7321
Author(s):  
Liuyu Xiang ◽  
Xiaoming Jin ◽  
Lan Yi ◽  
Guiguang Ding
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Text Classification ◽  
State Of The Art ◽  
Context Information ◽  
Great Success ◽  
Recurrent Networks ◽  
Learning Models ◽  
Benchmark Datasets

Deep learning models such as convolutional neural networks and recurrent networks are widely applied in text classification. In spite of their great success, most deep learning models neglect the importance of modeling context information, which is crucial to understanding texts. In this work, we propose the Adaptive Region Embedding to learn context representation to improve text classification. Specifically, a metanetwork is learned to generate a context matrix for each region, and each word interacts with its corresponding context matrix to produce the regional representation for further classification. Compared to previous models that are designed to capture context information, our model contains less parameters and is more flexible. We extensively evaluate our method on 8 benchmark datasets for text classification. The experimental results prove that our method achieves state-of-the-art performances and effectively avoids word ambiguity.

scholarly journals VisNet: Deep Convolutional Neural Networks for Forecasting Atmospheric Visibility

Sensors ◽  
2019 ◽  
Vol 19 (6) ◽  
pp. 1343 ◽  
Author(s):  
Akmaljon Palvanov ◽  
Young Cho
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
State Of The Art ◽  
Weather Conditions ◽  
Input Image ◽  
Deep Convolutional Neural Networks ◽  
Low Contrast ◽  
Proposed Model ◽  
Number Of Classes ◽  
Visibility Estimation

Visibility is a complex phenomenon inspired by emissions and air pollutants or by factors, including sunlight, humidity, temperature, and time, which decrease the clarity of what is visible through the atmosphere. This paper provides a detailed overview of the state-of-the-art contributions in relation to visibility estimation under various foggy weather conditions. We propose VisNet, which is a new approach based on deep integrated convolutional neural networks for the estimation of visibility distances from camera imagery. The implemented network uses three streams of deep integrated convolutional neural networks, which are connected in parallel. In addition, we have collected the largest dataset with three million outdoor images and exact visibility values for this study. To evaluate the model’s performance fairly and objectively, the model is trained on three image datasets with different visibility ranges, each with a different number of classes. Moreover, our proposed model, VisNet, evaluated under dissimilar fog density scenarios, uses a diverse set of images. Prior to feeding the network, each input image is filtered in the frequency domain to remove low-level features, and a spectral filter is applied to each input for the extraction of low-contrast regions. Compared to the previous methods, our approach achieves the highest performance in terms of classification based on three different datasets. Furthermore, our VisNet considerably outperforms not only the classical methods, but also state-of-the-art models of visibility estimation.

Sign in / Sign up

Export Citation Format

Share Document