Self-supervised graph convolutional clustering by preserving latent distribution

2021 ◽  
Vol 437 ◽  
pp. 218-226
Author(s):  
Shiwen Kou ◽  
Wei Xia ◽  
Xiangdong Zhang ◽  
Quanxue Gao ◽  
Xinbo Gao
Keyword(s):  
Latent Distribution

scholarly journals A Survey on Variational Autoencoders from a Green AI Perspective

2021 ◽  
Vol 2 (4) ◽  
Author(s):  
Andrea Asperti ◽  
Davide Evangelista ◽  
Elena Loli Piccolomini
Keyword(s):  
Architectural Design ◽  
Mathematical Formulation ◽  
Representation Learning ◽  
Generative Models ◽  
Model Description ◽  
Energetic Efficiency ◽  
Detailed Model ◽  
Latent Distribution ◽  
Quantitative Results ◽  
Generation Problem

AbstractVariational Autoencoders (VAEs) are powerful generative models that merge elements from statistics and information theory with the flexibility offered by deep neural networks to efficiently solve the generation problem for high-dimensional data. The key insight of VAEs is to learn the latent distribution of data in such a way that new meaningful samples can be generated from it. This approach led to tremendous research and variations in the architectural design of VAEs, nourishing the recent field of research known as unsupervised representation learning. In this article, we provide a comparative evaluation of some of the most successful, recent variations of VAEs. We particularly focus the analysis on the energetic efficiency of the different models, in the spirit of the so-called Green AI, aiming both to reduce the carbon footprint and the financial cost of generative techniques. For each architecture, we provide its mathematical formulation, the ideas underlying its design, a detailed model description, a running implementation and quantitative results.

scholarly journals Parametric UMAP Embeddings for Representation and Semisupervised Learning

2021 ◽  
pp. 1-27
Author(s):  
Tim Sainburg ◽  
Leland McInnes ◽  
Timothy Q. Gentner
Keyword(s):  
Algebraic Topology ◽  
Gradient Descent ◽  
Graphical Representation ◽  
Semisupervised Learning ◽  
Stochastic Gradient Descent ◽  
Second Step ◽  
Data Set ◽  
Parametric Relationship ◽  
Latent Distribution ◽  
Low Dimensional

Abstract UMAP is a nonparametric graph-based dimensionality reduction algorithm using applied Riemannian geometry and algebraic topology to find low-dimensional embeddings of structured data. The UMAP algorithm consists of two steps: (1) computing a graphical representation of a data set (fuzzy simplicial complex) and (2) through stochastic gradient descent, optimizing a low-dimensional embedding of the graph. Here, we extend the second step of UMAP to a parametric optimization over neural network weights, learning a parametric relationship between data and embedding. We first demonstrate that parametric UMAP performs comparably to its nonparametric counterpart while conferring the benefit of a learned parametric mapping (e.g., fast online embeddings for new data). We then explore UMAP as a regularization, constraining the latent distribution of autoencoders, parametrically varying global structure preservation, and improving classifier accuracy for semisupervised learning by capturing structure in unlabeled data.

Dual-View Conditional Variational Auto-Encoder for Emotional Dialogue Generation

2022 ◽  
Vol 21 (3) ◽  
pp. 1-18
Author(s):  
Mei Li ◽  
Jiajun Zhang ◽  
Xiang Lu ◽  
Chengqing Zong
Keyword(s):  
Relevant Information ◽  
Generation Process ◽  
Complementary Information ◽  
Emotional Information ◽  
Content Relevance ◽  
Latent Distribution ◽  
The Given

Emotional dialogue generation aims to generate appropriate responses that are content relevant with the query and emotion consistent with the given emotion tag. Previous work mainly focuses on incorporating emotion information into the sequence to sequence or conditional variational auto-encoder (CVAE) models, and they usually utilize the given emotion tag as a conditional feature to influence the response generation process. However, emotion tag as a feature cannot well guarantee the emotion consistency between the response and the given emotion tag. In this article, we propose a novel Dual-View CVAE model to explicitly model the content relevance and emotion consistency jointly. These two views gather the emotional information and the content-relevant information from the latent distribution of responses, respectively. We jointly model the dual-view via VAE to get richer and complementary information. Extensive experiments on both English and Chinese emotion dialogue datasets demonstrate the effectiveness of our proposed Dual-View CVAE model, which significantly outperforms the strong baseline models in both aspects of content relevance and emotion consistency.

scholarly journals Modelling and forecasting art movements with CGANs

2020 ◽  
Vol 7 (4) ◽  
pp. 191569
Author(s):  
Edoardo Lisi ◽  
Mohammad Malekzadeh ◽  
Hamed Haddadi ◽  
F. Din-Houn Lau ◽  
Seth Flaxman
Keyword(s):  
Generative Adversarial Networks ◽  
Vector Autoregressive ◽  
Adversarial Networks ◽  
Latent Distribution ◽  
Small Set ◽  
One Step ◽  
Temporal Aspect ◽  
Novel Method ◽  
The Mean ◽  
Modelling And Forecasting

Conditional generative adversarial networks (CGANs) are a recent and popular method for generating samples from a probability distribution conditioned on latent information. The latent information often comes in the form of a discrete label from a small set. We propose a novel method for training CGANs which allows us to condition on a sequence of continuous latent distributions f (1) , …, f ( K ) . This training allows CGANs to generate samples from a sequence of distributions. We apply our method to paintings from a sequence of artistic movements, where each movement is considered to be its own distribution. Exploiting the temporal aspect of the data, a vector autoregressive (VAR) model is fitted to the means of the latent distributions that we learn, and used for one-step-ahead forecasting, to predict the latent distribution of a future art movement f ( K +1) . Realizations from this distribution can be used by the CGAN to generate ‘future’ paintings. In experiments, this novel methodology generates accurate predictions of the evolution of art. The training set consists of a large dataset of past paintings. While there is no agreement on exactly what current art period we find ourselves in, we test on plausible candidate sets of present art, and show that the mean distance to our predictions is small.

scholarly journals Variational Autoencoder-Based Multiple Image Captioning Using a Caption Attention Map

2019 ◽  
Vol 9 (13) ◽  
pp. 2699 ◽  
Author(s):  
Boeun Kim ◽  
Saim Shin ◽  
Hyedong Jung
Keyword(s):  
Generative Models ◽  
Research Topic ◽  
Video Data ◽  
Image Feature ◽  
Image Captioning ◽  
Multiple Image ◽  
Visually Impaired People ◽  
Proposed Model ◽  
Variational Autoencoder ◽  
Latent Distribution

Image captioning is a promising research topic that is applicable to services that search for desired content in a large amount of video data and a situation explanation service for visually impaired people. Previous research on image captioning has been focused on generating one caption per image. However, to increase usability in applications, it is necessary to generate several different captions that contain various representations for an image. We propose a method to generate multiple captions using a variational autoencoder, which is one of the generative models. Because an image feature plays an important role when generating captions, a method to extract a Caption Attention Map (CAM) of the image is proposed, and CAMs are projected to a latent distribution. In addition, methods for the evaluation of multiple image captioning tasks are proposed that have not yet been actively researched. The proposed model outperforms in the aspect of diversity compared with the base model when the accuracy is comparable. Moreover, it is verified that the model using CAM generates detailed captions describing various content in the image.

scholarly journals The Assignment of Scores Procedure for Ordinal Categorical Data

2014 ◽  
Vol 2014 ◽  
pp. 1-7 ◽  
Author(s):  
Han-Ching Chen ◽  
Nae-Sheng Wang
Keyword(s):  
Social Sciences ◽  
Categorical Data ◽  
Ordinal Data ◽  
Interval Data ◽  
Score System ◽  
Ordered Categorical Data ◽  
The Social ◽  
Latent Distribution ◽  
Ordered Categorical

Ordinal data are the most frequently encountered type of data in the social sciences. Many statistical methods can be used to process such data. One common method is to assign scores to the data, convert them into interval data, and further perform statistical analysis. There are several authors who have recently developed assigning score methods to assign scores to ordered categorical data. This paper proposes an approach that defines an assigning score system for an ordinal categorical variable based on underlying continuous latent distribution with interpretation by using three case study examples. The results show that the proposed score system is well for skewed ordinal categorical data.

Scale Shrinkage and the Estimation of Latent Distribution Parameters

1988 ◽  
Vol 13 (3) ◽  
pp. 227-241 ◽  
Author(s):  
Gregory Camilli
Keyword(s):  
Measurement Errors ◽  
Empirical Bayes ◽  
Large Degree ◽  
Estimation Errors ◽  
Ability Estimates ◽  
Population Distributions ◽  
Latent Distribution ◽  
Scale Scores ◽  
Distribution Parameters ◽  
True Ability

In this paper the phenomenon of scale shrinkage is examined. Specifically, emphasis is placed on the pattern of decreasing variances in IRT scale scores from fall to spring within a grade. It is concluded that certain situations exist in which scale shrinkage is predictable with unidimensional tests. It depends, to a large degree, on the match between item difficulties and the level of examinee ability. As the mismatch increases, so do distortions of scale because of systematic estimation errors (bias), measurement errors, and unobtainable ability estimates. These problems exist for all observed or estimated scores; however, it is shown in this paper that questions concerning the population distributions of true ability can potentially be addressed with empirical Bayes techniques.

scholarly journals Latent Distribution Preserving Deep Subspace Clustering

2019 ◽  
Author(s):  
Lei Zhou ◽  
Xiao Bai ◽  
Dong Wang ◽  
Xianglong Liu ◽  
Jun Zhou ◽  
...  
Keyword(s):  
Linear Subspace ◽  
State Of The Art ◽  
Subspace Clustering ◽  
Real Data ◽  
High Dimensional ◽  
Clustering Methods ◽  
Real World Data ◽  
Latent Distribution ◽  
Computer Vision Applications ◽  
Strong Capacity

Subspace clustering is a useful technique for many computer vision applications in which the intrinsic dimension of high-dimensional data is smaller than the ambient dimension. Traditional subspace clustering methods often rely on the self-expressiveness property, which has proven effective for linear subspace clustering. However, they perform unsatisfactorily on real data with complex nonlinear subspaces. More recently, deep autoencoder based subspace clustering methods have achieved success owning to the more powerful representation extracted by the autoencoder network. Unfortunately, these methods only considering the reconstruction of original input data can hardly guarantee the latent representation for the data distributed in subspaces, which inevitably limits the performance in practice. In this paper, we propose a novel deep subspace clustering method based on a latent distribution-preserving autoencoder, which introduces a distribution consistency loss to guide the learning of distribution-preserving latent representation, and consequently enables strong capacity of characterizing the real-world data for subspace clustering. Experimental results on several public databases show that our method achieves significant improvement compared with the state-of-the-art subspace clustering methods.

Sign in / Sign up

Export Citation Format

Share Document