Deep Generative Model with Supervised Latent Space for Text Classification

Recent advances in applying deep neural networks to Bayesian Modelling, sparked resurgence of in- terest in Variational Methods. Notably, the main contribution in this area is Reparametrization Trick introduced in [1] and [2]. VAE model [1], is unsupervised and therefore its application to classification is not optimal. In this work, we research the possibility to extend the model to supervised case. We first start with the model known as Supervised Variational Autoencoder that is researched in the literature in various forms [3] and [4]. We then modify objective function in such a way, that latent space can be better fitted to multiclass problem. Finally, we introduce a new method that uses information about classes to modify latent space, so it even better reflects differences between classes. All of this, will use only two dimensions. We will show, that mainstream classifiers applied to such a space, achieve significantly better performance than if applied to original datasets and VAE generated data. We also show, how our novel approach can be used to calculate better classification score, and how it can be used to generate data for a given class.

Download Full-text

Representing Deep Neural Networks Latent Space Geometries with Graphs

Algorithms ◽

10.3390/a14020039 ◽

2021 ◽

Vol 14 (2) ◽

pp. 39

Author(s):

Carlos Lassance ◽

Vincent Gripon ◽

Antonio Ortega

Keyword(s):

Machine Learning ◽

Neural Networks ◽

Deep Learning ◽

Objective Function ◽

Learning Process ◽

Deep Neural Networks ◽

State Of The Art ◽

The Core ◽

Learning Tasks ◽

Latent Space

Deep Learning (DL) has attracted a lot of attention for its ability to reach state-of-the-art performance in many machine learning tasks. The core principle of DL methods consists of training composite architectures in an end-to-end fashion, where inputs are associated with outputs trained to optimize an objective function. Because of their compositional nature, DL architectures naturally exhibit several intermediate representations of the inputs, which belong to so-called latent spaces. When treated individually, these intermediate representations are most of the time unconstrained during the learning process, as it is unclear which properties should be favored. However, when processing a batch of inputs concurrently, the corresponding set of intermediate representations exhibit relations (what we call a geometry) on which desired properties can be sought. In this work, we show that it is possible to introduce constraints on these latent geometries to address various problems. In more detail, we propose to represent geometries by constructing similarity graphs from the intermediate representations obtained when processing a batch of inputs. By constraining these Latent Geometry Graphs (LGGs), we address the three following problems: (i) reproducing the behavior of a teacher architecture is achieved by mimicking its geometry, (ii) designing efficient embeddings for classification is achieved by targeting specific geometries, and (iii) robustness to deviations on inputs is achieved via enforcing smooth variation of geometry between consecutive latent spaces. Using standard vision benchmarks, we demonstrate the ability of the proposed geometry-based methods in solving the considered problems.

Download Full-text

Improving the accuracy using pre-trained word embeddings on deep neural networks for Turkish text classification

Physica A Statistical Mechanics and its Applications ◽

10.1016/j.physa.2019.123288 ◽

2020 ◽

Vol 541 ◽

pp. 123288 ◽

Cited By ~ 3

Author(s):

Murat Aydoğan ◽

Ali Karci

Keyword(s):

Neural Networks ◽

Text Classification ◽

Deep Neural Networks ◽

Word Embeddings ◽

Turkish Text

Download Full-text

Multitask Learning for Text Classification with Deep Neural Networks

Research and Development in Intelligent Systems XXXIII ◽

10.1007/978-3-319-47175-4_8 ◽

2016 ◽

pp. 119-133 ◽

Cited By ~ 1

Author(s):

Hossein Ghodrati Noushahr ◽

Samad Ahmadi

Keyword(s):

Neural Networks ◽

Text Classification ◽

Deep Neural Networks ◽

Multitask Learning

Download Full-text

Universal Rules for Fooling Deep Neural Networks based Text Classification

2019 IEEE Congress on Evolutionary Computation (CEC) ◽

10.1109/cec.2019.8790213 ◽

2019 ◽

Cited By ~ 4

Author(s):

Di Li ◽

Danilo Vasconcellos Vargas ◽

Sakurai Kouichi

Keyword(s):

Neural Networks ◽

Text Classification ◽

Deep Neural Networks

Download Full-text

Single Cortical Neurons as Deep Artificial Neural Networks

10.1101/613141 ◽

2019 ◽

Cited By ~ 4

Author(s):

David Beniaguev ◽

Idan Segev ◽

Michael London

Keyword(s):

Neural Networks ◽

Cortical Neurons ◽

Deep Neural Networks ◽

Pyramidal Neurons ◽

Classical Architecture ◽

Novel Approach ◽

Cortical Pyramidal Neurons ◽

Spatio Temporal ◽

Cortical Pyramidal Cell

AbstractWe introduce a novel approach to study neurons as sophisticated I/O information processing units by utilizing recent advances in the field of machine learning. We trained deep neural networks (DNNs) to mimic the I/O behavior of a detailed nonlinear model of a layer 5 cortical pyramidal cell, receiving rich spatio-temporal patterns of input synapse activations. A Temporally Convolutional DNN (TCN) with seven layers was required to accurately, and very efficiently, capture the I/O of this neuron at the millisecond resolution. This complexity primarily arises from local NMDA-based nonlinear dendritic conductances. The weight matrices of the DNN provide new insights into the I/O function of cortical pyramidal neurons, and the approach presented can provide a systematic characterization of the functional complexity of different neuron types. Our results demonstrate that cortical neurons can be conceptualized as multi-layered “deep” processing units, implying that the cortical networks they form have a non-classical architecture and are potentially more computationally powerful than previously assumed.

Download Full-text

CrowdTC: Crowd-powered Learning for Text Classification

ACM Transactions on Knowledge Discovery from Data ◽

10.1145/3457216 ◽

2021 ◽

Vol 16 (1) ◽

pp. 1-23

Author(s):

Keyu Yang ◽

Yunjun Gao ◽

Lei Liang ◽

Song Bian ◽

Lu Chen ◽

...

Keyword(s):

Neural Network ◽

Neural Networks ◽

Text Classification ◽

Deep Neural Networks ◽

Semantic Information ◽

Human Beings ◽

Hybrid Neural Network ◽

Public Datasets ◽

Almost All ◽

The Cost

Text classification is a fundamental task in content analysis. Nowadays, deep learning has demonstrated promising performance in text classification compared with shallow models. However, almost all the existing models do not take advantage of the wisdom of human beings to help text classification. Human beings are more intelligent and capable than machine learning models in terms of understanding and capturing the implicit semantic information from text. In this article, we try to take guidance from human beings to classify text. We propose Crowd-powered learning for Text Classification (CrowdTC for short). We design and post the questions on a crowdsourcing platform to extract keywords in text. Sampling and clustering techniques are utilized to reduce the cost of crowdsourcing. Also, we present an attention-based neural network and a hybrid neural network to incorporate the extracted keywords as human guidance into deep neural networks. Extensive experiments on public datasets confirm that CrowdTC improves the text classification accuracy of neural networks by using the crowd-powered keyword guidance.

Download Full-text

A Novel Approach to Encrypt Data Using Deep Neural Networks

Lecture Notes in Networks and Systems - Advances in Distributed Computing and Machine Learning ◽

10.1007/978-981-16-4807-6_44 ◽

2022 ◽

pp. 467-475

Author(s):

Aditya Oke ◽

Abhishek Kushwaha ◽

Manan Agrawal ◽

Utkrisht Sahai ◽

Rahul Raman

Keyword(s):

Neural Networks ◽

Deep Neural Networks ◽

Novel Approach ◽

Encrypt Data

Download Full-text

Heart Disease Prediction Using Deep Neural Networks: A Novel Approach

10.1007/978-981-16-2422-3_56 ◽

2021 ◽

pp. 725-736

Author(s):

Kondeth Fathima ◽

E. R. Vimina

Keyword(s):

Neural Networks ◽

Heart Disease ◽

Deep Neural Networks ◽

Disease Prediction ◽

Novel Approach

Download Full-text

Cloud Computing Virtual Machine Workload Prediction Method Based on Variational Autoencoder

International Journal of Systems and Software Security and Protection ◽

10.4018/ijsssp.2021070103 ◽

2021 ◽

Vol 12 (2) ◽

pp. 33-45

Author(s):

Fargana J. Abdullayeva

Keyword(s):

Neural Networks ◽

Cloud Computing ◽

Deep Neural Networks ◽

Virtual Machines ◽

Prediction Method ◽

Data Reconstruction ◽

Cloud Infrastructure ◽

Workload Prediction ◽

Variational Autoencoder ◽

Probability Criterion

The paper proposes a method for predicting the workload of virtual machines in the cloud infrastructure. Reconstruction probabilities of variational autoencoders were used to provide the prediction. Reconstruction probability is a probability criterion that considers the variability in the distribution of variables. In the proposed approach, the values of the reconstruction probabilities of the variational autoencoder show the workload level of the virtual machines. The results of the experiments showed that variational autoencoders gave better results in predicting the workload of virtual machines compared to simple deep neural networks. The generative characteristics of the variational autoencoders determine the workload level by the data reconstruction.

Download Full-text