A Variational Method for Learning Sparse and Overcomplete Representations

2001 ◽  
Vol 13 (11) ◽  
pp. 2517-2532 ◽  
Author(s):  
Mark Girolami
Keyword(s):  
Lower Bound ◽  
Expectation Maximization ◽  
Prior Distribution ◽  
Expectation Maximization Algorithm ◽  
Variational Approximation ◽  
Heavy Tailed Distributions ◽  
Data Representations ◽  
Heavy Tailed ◽  
Sparse Prior ◽  
Basis Vectors

An expectation-maximization algorithm for learning sparse and overcomplete data representations is presented. The proposed algorithm exploits a variational approximation to a range of heavy-tailed distributions whose limit is the Laplacian. A rigorous lower bound on the sparse prior distribution is derived, which enables the analytic marginalization of a lower bound on the data likelihood. This lower bound enables the development of an expectation-maximization algorithm for learning the overcomplete basis vectors and inferring the most probable basis coefficients.

scholarly journals Multiple scaled contaminated normal distribution and its application in clustering

2019 ◽  
pp. 1471082X1989093 ◽  
Author(s):  
Antonio Punzo ◽  
Cristina Tortora
Keyword(s):  
Expectation Maximization ◽  
Heavy Tails ◽  
Expectation Maximization Algorithm ◽  
Simulated Data ◽  
Principal Component ◽  
Multivariate Normal ◽  
Degree Of Contamination ◽  
Robust Estimates ◽  
Directional Detection ◽  
Heavy Tailed

The multivariate contaminated normal (MCN) distribution represents a simple heavy-tailed generalization of the multivariate normal (MN) distribution to model elliptical contoured scatters in the presence of mild outliers (also referred to as ‘bad’ points herein) and automatically detect bad points. The price of these advantages is two additional parameters: proportion of good observations and degree of contamination. However, in a multivariate setting, only one proportion of good observations and only one degree of contamination may be limiting. To overcome this limitation, we propose a multiple scaled contaminated normal (MSCN) distribution. Among its parameters, we have an orthogonal matrix Γ. In the space spanned by the vectors (principal components) of Γ, there is a proportion of good observations and a degree of contamination for each component. Moreover, each observation has a posterior probability of being good with respect to each principal component. Thanks to this probability, the method provides directional robust estimates of the parameters of the nested MN and automatic directional detection of bad points. The term ‘directional’ is added to specify that the method works separately for each principal component. Mixtures of MSCN distributions are also proposed, and an expectation-maximization algorithm is used for parameter estimation. Real and simulated data are considered to show the usefulness of our mixture with respect to well-established mixtures of symmetric distributions with heavy tails.

scholarly journals Penalized Maximum Likelihood Method to a Class of Skewness Data Analysis

2014 ◽  
Vol 2014 ◽  
pp. 1-7
Author(s):  
Xuedong Chen ◽  
Qianying Zeng ◽  
Qiankun Song
Keyword(s):  
Linear Regression ◽  
Regression Models ◽  
Expectation Maximization Algorithm ◽  
Simulated Data ◽  
Likelihood Method ◽  
Skew Normal Distribution ◽  
Linear Regression Models ◽  
Heavy Tailed Distributions ◽  
Random Term ◽  
Heavy Tailed

An extension of some standard likelihood and variable selection criteria based on procedures of linear regression models under the skew-normal distribution or the skew-tdistribution is developed. This novel class of models provides a useful generalization of symmetrical linear regression models, since the random term distributions cover both symmetric as well as asymmetric and heavy-tailed distributions. A generalized expectation-maximization algorithm is developed for computing thel1penalized estimator. Efficacy of the proposed methodology and algorithm is demonstrated by simulated data.

Ordered subset expectation maximization algorithm for image reconstruction applied to zero-flow or low-flow mimicking brain phantom

2005 ◽  
Vol 25 (1_suppl) ◽  
pp. S678-S678
Author(s):  
Yasuhiro Akazawa ◽  
Yasuhiro Katsura ◽  
Ryohei Matsuura ◽  
Piao Rishu ◽  
Ansar M D Ashik ◽  
...  
Keyword(s):  
Image Reconstruction ◽  
Expectation Maximization ◽  
Expectation Maximization Algorithm ◽  
Low Flow ◽  
Brain Phantom ◽  
Zero Flow

Probability and Random Processes

2018 ◽  
Author(s):  
Stefan Thurner ◽  
Rudolf Hanel ◽  
Peter Klimekl
Keyword(s):  
Random Processes ◽  
Distribution Functions ◽  
Basic Logic ◽  
Heavy Tailed Distributions ◽  
Random Components ◽  
Classical Distribution ◽  
Heavy Tailed ◽  
Basic Facts ◽  
Stochastic Events ◽  
Stochastic Event

Phenomena, systems, and processes are rarely purely deterministic, but contain stochastic,probabilistic, or random components. For that reason, a probabilistic descriptionof most phenomena is necessary. Probability theory provides us with the tools for thistask. Here, we provide a crash course on the most important notions of probabilityand random processes, such as odds, probability, expectation, variance, and so on. Wedescribe the most elementary stochastic event—the trial—and develop the notion of urnmodels. We discuss basic facts about random variables and the elementary operationsthat can be performed on them. We learn how to compose simple stochastic processesfrom elementary stochastic events, and discuss random processes as temporal sequencesof trials, such as Bernoulli and Markov processes. We touch upon the basic logic ofBayesian reasoning. We discuss a number of classical distribution functions, includingpower laws and other fat- or heavy-tailed distributions.

Sign in / Sign up

Export Citation Format

Share Document