scholarly journals Unsupervised Abstractive Opinion Summarization by Generating Sentences with Tree-Structured Topic Guidance

2021 ◽  
Vol 9 ◽  
pp. 945-961
Author(s):  
Masaru Isonuma ◽  
Junichiro Mori ◽  
Danushka Bollegala ◽  
Ichiro Sakata
Keyword(s):  
Gaussian Mixture ◽  
Word Embedding ◽  
Experimental Results ◽  
Mixture Component ◽  
Gaussian Component ◽  
Summarization Method ◽  
Opinion Summarization ◽  
Variational Autoencoder ◽  
Topic Distribution ◽  
Abstractive Summarization

Abstract This paper presents a novel unsupervised abstractive summarization method for opinionated texts. While the basic variational autoencoder-based models assume a unimodal Gaussian prior for the latent code of sentences, we alternate it with a recursive Gaussian mixture, where each mixture component corresponds to the latent code of a topic sentence and is mixed by a tree-structured topic distribution. By decoding each Gaussian component, we generate sentences with tree-structured topic guidance, where the root sentence conveys generic content, and the leaf sentences describe specific topics. Experimental results demonstrate that the generated topic sentences are appropriate as a summary of opinionated texts, which are more informative and cover more input contents than those generated by the recent unsupervised summarization model (Bražinskas et al., 2020). Furthermore, we demonstrate that the variance of latent Gaussians represents the granularity of sentences, analogous to Gaussian word embedding (Vilnis and McCallum, 2015).

scholarly journals Resisting Adversarial Attacks Using Gaussian Mixture Variational Autoencoders

2019 ◽  
Vol 33 ◽  
pp. 541-548 ◽  
Author(s):  
Partha Ghosh ◽  
Arpan Losalka ◽  
Michael J. Black
Keyword(s):  
Neural Networks ◽  
Latent Variable ◽  
Deep Neural Networks ◽  
Gaussian Mixture ◽  
Mixture Component ◽  
Unified Framework ◽  
Single Class ◽  
Variational Autoencoder ◽  
Practical Challenge ◽  
Mixture Prior

Susceptibility of deep neural networks to adversarial attacks poses a major theoretical and practical challenge. All efforts to harden classifiers against such attacks have seen limited success till now. Two distinct categories of samples against which deep neural networks are vulnerable, “adversarial samples” and “fooling samples”, have been tackled separately so far due to the difficulty posed when considered together. In this work, we show how one can defend against them both under a unified framework. Our model has the form of a variational autoencoder with a Gaussian mixture prior on the latent variable, such that each mixture component corresponds to a single class. We show how selective classification can be performed using this model, thereby causing the adversarial objective to entail a conflict. The proposed method leads to the rejection of adversarial samples instead of misclassification, while maintaining high precision and recall on test data. It also inherently provides a way of learning a selective classifier in a semi-supervised scenario, which can similarly resist adversarial attacks. We further show how one can reclassify the detected adversarial samples by iterative optimization.1

Generating realistic Arabic handwriting dataset

2019 ◽  
Vol 8 (4) ◽  
pp. 460
Author(s):  
Mahmoud I. Abdalla ◽  
Mohsen A. Rashwan ◽  
Mohamed A. Elserafy
Keyword(s):  
Word Recognition ◽  
Word List ◽  
Holistic Approach ◽  
Gaussian Mixture ◽  
Normalization Method ◽  
Gaussian Component ◽  
Efficient System ◽  
Mixture Regression ◽  
Word Generation ◽  
Input Text

During the previous year's holistic approach showing satisfactory results to solve ‎the ‎problem of Arabic handwriting word  recognition instead of word letters ‎‎segmentation.‎ ‎In this paper, we present an efficient system for ‎ generation realistic Arabic handwriting dataset from ASCII input ‎text. We carefully selected simple word list that contains most Arabic ‎letters normal and ligature connection cases. To improve the ‎performance of new letters reproduction we developed our ‎normalization method that adapt its clustering action according to ‎created Arabic letters families. We enhanced  Gaussian Mixture ‎Model process to learn letters template by detecting the ‎number and position of Gaussian component by implementing ‎Ramer-Douglas-Peucker‎ algorithm which improve the new letters ‎shapes reproduced by using and Gaussian Mixture Regression. ‎‎We learn the translation distance between word-part to achieve ‎real handwriting word generation shape.‎ Using combination of LSTM and CTC layer as a recognizer to validate the ‎efficiency of our approach in generating new realistic Arabic handwriting words inherit user handwriting style as shown by the experimental results.‎ 

Strategies to Enhance the Performance of Gaussian Mixture Model Fitting for Uncertainty Quantification by Conditioning to Production Data

2021 ◽  
Author(s):  
Guohua Gao ◽  
Jeroen Vink ◽  
Fredrik Saaf ◽  
Terence Wells
Keyword(s):  
Gaussian Mixture Model ◽  
Mixture Model ◽  
History Matching ◽  
Gaussian Mixture ◽  
Model Parameters ◽  
Gaussian Component ◽  
Large Set ◽  
Cpu Time ◽  
Fitting Method ◽  
New Strategies

Abstract When formulating history matching within the Bayesian framework, we may quantify the uncertainty of model parameters and production forecasts using conditional realizations sampled from the posterior probability density function (PDF). It is quite challenging to sample such a posterior PDF. Some methods e.g., Markov chain Monte Carlo (MCMC), are very expensive (e.g., MCMC) while others are cheaper but may generate biased samples. In this paper, we propose an unconstrained Gaussian Mixture Model (GMM) fitting method to approximate the posterior PDF and investigate new strategies to further enhance its performance. To reduce the CPU time of handling bound constraints, we reformulate the GMM fitting formulation such that an unconstrained optimization algorithm can be applied to find the optimal solution of unknown GMM parameters. To obtain a sufficiently accurate GMM approximation with the lowest number of Gaussian components, we generate random initial guesses, remove components with very small or very large mixture weights after each GMM fitting iteration and prevent their reappearance using a dedicated filter. To prevent overfitting, we only add a new Gaussian component if the quality of the GMM approximation on a (large) set of blind-test data sufficiently improves. The unconstrained GMM fitting method with the new strategies proposed in this paper is validated using nonlinear toy problems and then applied to a synthetic history matching example. It can construct a GMM approximation of the posterior PDF that is comparable to the MCMC method, and it is significantly more efficient than the constrained GMM fitting formulation, e.g., reducing the CPU time by a factor of 800 to 7300 for problems we tested, which makes it quite attractive for large scale history matching problems.

scholarly journals A Hierarchical End-to-End Model for Jointly Improving Text Summarization and Sentiment Classification

2018 ◽  
Author(s):  
Shuming Ma ◽  
Xu Sun ◽  
Junyang Lin ◽  
Xuancheng Ren
Keyword(s):  
Hierarchical Structure ◽  
Online Reviews ◽  
Text Summarization ◽  
Sentiment Classification ◽  
Experimental Results ◽  
Joint Learning ◽  
End To End ◽  
Abstractive Summarization ◽  
Main Ideas ◽  
Different Levels

Text summarization and sentiment classification both aim to capture the main ideas of the text but at different levels. Text summarization is to describe the text within a few sentences, while sentiment classification can be regarded as a special type of summarization which ``summarizes'' the text into a even more abstract fashion, i.e., a sentiment class. Based on this idea, we propose a hierarchical end-to-end model for joint learning of text summarization and sentiment classification, where the sentiment classification label is treated as the further ``summarization'' of the text summarization output. Hence, the sentiment classification layer is put upon the text summarization layer, and a hierarchical structure is derived. Experimental results on Amazon online reviews datasets show that our model achieves better performance than the strong baseline systems on both abstractive summarization and sentiment classification.

scholarly journals Speaker Recognition System Based on Wavelet Features and Gaussian Mixture Models

2019 ◽  
Vol 9 (1) ◽  
pp. 5363-5367
Keyword(s):  
Mixture Models ◽  
Speaker Recognition ◽  
Gaussian Mixture Models ◽  
Gaussian Mixture ◽  
Recognition System ◽  
Gaussian Component ◽  
Discrete Wavelet ◽  
Signal Features ◽  
Wavelet Features ◽  
Gmm Classifier

Identification of a person’s voice from the different voices is known as speaker recognition. The speech signals of individuals are selected by means of speaker recognition or identification. In this work, an efficient method for speaker recognition is made by using Discrete Wavelet Transform (DWT) features and Gaussian Mixture Models (GMM) for classification is presented. The input speech signal features are decomposed by DWT into subband coefficients. The DWT subband coefficient features are the input for the classification. Classification is made by GMM classifier at 4, 8, 16 and 32 Gaussian component levels. Results show a better accuracy of 96.18% speaker signals using DWT features and GMM classifier

scholarly journals Weakly Supervised Disentanglement by Pairwise Similarities

2020 ◽  
Vol 34 (04) ◽  
pp. 3495-3502 ◽  
Author(s):  
Junxiang Chen ◽  
Kayhan Batmanghelich
Keyword(s):  
Real World ◽  
Latent Variables ◽  
Generative Models ◽  
Experimental Results ◽  
New Method ◽  
Weak Supervision ◽  
Real World Problem ◽  
Variational Autoencoder ◽  
Weakly Supervised

Recently, researches related to unsupervised disentanglement learning with deep generative models have gained substantial popularity. However, without introducing supervision, there is no guarantee that the factors of interest can be successfully recovered (Locatello et al. 2018). Motivated by a real-world problem, we propose a setting where the user introduces weak supervision by providing similarities between instances based on a factor to be disentangled. The similarity is provided as either a binary (yes/no) or real-valued label describing whether a pair of instances are similar or not. We propose a new method for weakly supervised disentanglement of latent variables within the framework of Variational Autoencoder. Experimental results demonstrate that utilizing weak supervision improves the performance of the disentanglement method substantially.

scholarly journals Non-I.I.D. Multi-Instance Learning for Predicting Instance and Bag Labels with Variational Auto-Encoder

2021 ◽  
Author(s):  
Weijia Zhang
Keyword(s):  
Medical Imaging ◽  
Supervised Learning ◽  
Real World ◽  
State Of The Art ◽  
Experimental Results ◽  
Weakly Supervised Learning ◽  
Variational Autoencoder ◽  
Label Prediction ◽  
Weakly Supervised ◽  
Better Than

Multi-instance learning is a type of weakly supervised learning. It deals with tasks where the data is a set of bags and each bag is a set of instances. Only the bag labels are observed whereas the labels for the instances are unknown. An important advantage of multi-instance learning is that by representing objects as a bag of instances, it is able to preserve the inherent dependencies among parts of the objects. Unfortunately, most existing algorithms assume all instances to be identically and independently distributed, which violates real-world scenarios since the instances within a bag are rarely independent. In this work, we propose the Multi-Instance Variational Autoencoder (MIVAE) algorithm which explicitly models the dependencies among the instances for predicting both bag labels and instance labels. Experimental results on several multi-instance benchmarks and end-to-end medical imaging datasets demonstrate that MIVAE performs better than state-of-the-art algorithms for both instance label and bag label prediction tasks.

scholarly journals A Task Learning Mechanism for the Telerobots

2019 ◽  
Vol 16 (02) ◽  
pp. 1950009 ◽  
Author(s):  
Jing Luo ◽  
Chenguang Yang ◽  
Qiang Li ◽  
Min Wang
Keyword(s):  
Gaussian Mixture Model ◽  
Mixture Model ◽  
Gaussian Mixture ◽  
Experimental Results ◽  
Complex Tasks ◽  
Learning Mechanism ◽  
Learning Framework ◽  
Task Learning

Telerobotic systems have attracted growing attention because of their superiority in the dangerous or unknown interaction tasks. It is very challenging to exploit such systems to implement complex tasks in an autonomous way. In this paper, we propose a task learning framework to represent the manipulation skill demonstrated by a remotely controlled robot. Gaussian mixture model is utilized to encode and parametrize the smooth task trajectory according to the observations from the demonstrations. After encoding the demonstrated trajectory, a new task trajectory is generated based on the variability information of the learned model. Experimental results have demonstrated the feasibility of the proposed method.

A Gaussian Mixture Variational Autoencoder-Based Approach for Designing Phononic Bandgap Metamaterials

2021 ◽  
Author(s):  
Zihan Wang ◽  
Weikang Xian ◽  
M. Ridha Baccouche ◽  
Horst Lanzerath ◽  
Ying Li ◽  
...  
Keyword(s):  
Learning Strategies ◽  
Pareto Frontier ◽  
Feature Learning ◽  
Gaussian Mixture ◽  
Structural Features ◽  
Frequency Range ◽  
Bandgap Width ◽  
Performance Space ◽  
Variational Autoencoder ◽  
Gp Model

Abstract Phononic bandgap metamaterials, which consist of periodic cellular structures, are capable of absorbing energy within a certain frequency range. Designing metamaterials that trap waves across a wide wave frequency range is still a challenging task. In this study, we proposed a deep feature learning-based framework to design cellular metamaterial structures considering two design objectives: bandgap width and stiffness. A Gaussian mixture variational autoencoder (GM-VAE) is employed to extract structural features and a Gaussian Process (GP) model is employed to enable property-driven structure optimization. By comparing the GM-VAE and a regular variational autoencoder (VAE), we demonstrate that (i) GM-VAE has the advantage of learning capability, and (ii) GM-VAE discovers a more diversified design set (in terms of the distribution in the performance space) in the unsupervised learning-based generative design. Two supervised learning strategies, building independent single-response GP models for each output and building an all-in-one multi-response GP model for all outputs, are employed and compared to establish the relationship between the latent features and the properties of interest. Multi-objective design optimization is conducted to obtain the Pareto frontier with respect to bandgap width and stiffness. The effectiveness of the proposed design framework is validated by comparing the performances of newly discovered designs with existing designs. The caveats to designing phonic bandgap metamaterials are summarized.

Sign in / Sign up

Export Citation Format

Share Document