scholarly journals Iterative reconstruction of rank-one matrices in noise

2018 ◽  
Vol 7 (3) ◽  
pp. 531-562 ◽  
Author(s):  
Alyson K Fletcher ◽  
Sundeep Rangan
Keyword(s):  
Gaussian Noise ◽  
Probabilistic Model ◽  
Message Passing ◽  
Evolution Equations ◽  
Belief Propagation ◽  
Equivalent Model ◽  
The Matrix ◽  
Rank One ◽  
Estimation Problems ◽  
Loopy Belief Propagation

Abstract We consider the problem of estimating a rank-one matrix in Gaussian noise under a probabilistic model for the left and right factors of the matrix. The probabilistic model can impose constraints on the factors including sparsity and positivity that arise commonly in learning problems. We propose a family of algorithms that reduce the problem to a sequence of scalar estimation computations. These algorithms are similar to approximate message-passing techniques based on Gaussian approximations of loopy belief propagation that have been used recently in compressed sensing. Leveraging analysis methods by Bayati and Montanari, we show that the asymptotic behavior of the algorithm is described by a simple scalar equivalent model, where the distribution of the estimates at each iteration is identical to certain scalar estimates of the variables in Gaussian noise. Moreover, the effective Gaussian noise level is described by a set of state evolution equations. The proposed approach to deriving algorithms thus provides a computationally simple and general method for rank-one estimation problems with a precise analysis in certain high-dimensional settings.

THE DESIGN AND TESTING OF A FIRST-ORDER LOGIC-BASED STOCHASTIC MODELING LANGUAGE

2006 ◽  
Vol 15 (06) ◽  
pp. 979-1005 ◽  
Author(s):  
DANIEL J. PLESS ◽  
CHAYAN CHAKRABARTI ◽  
ROSHAN RAMMOHAN ◽  
GEORGE F. LUGER
Keyword(s):  
Stochastic Modeling ◽  
Message Passing ◽  
Belief Propagation ◽  
Markov Models ◽  
Propositional Calculus ◽  
Expressive Power ◽  
Digital Circuit ◽  
First Order ◽  
Representational Power ◽  
Loopy Belief Propagation

We have created a logic-based, Turing-complete language for stochastic modeling. Since the inference scheme for this language is based on a variant of Pearl's loopy belief propagation algorithm, we call it Loopy Logic. Traditional Bayesian networks have limited expressive power, basically constrained to finite domains as in the propositional calculus. Our language contains variables that can capture general classes of situations, events and relationships. A first-order language is also able to reason about potentially infinite classes and situations using constructs such as hidden Markov models(HMMs). Our language uses an Expectation-Maximization (EM) type learning of parameters. This has a natural fit with the Loopy Belief Propagation used for inference since both can be viewed as iterative message passing algorithms. We present the syntax and theoretical foundations for our Loopy Logic language. We then demonstrate three examples of stochastic modeling and diagnosis that explore the representational power of the language. A mechanical fault detection example displays how Loopy Logic can model time-series processes using an HMM variant. A digital circuit example exhibits the probabilistic modeling capabilities, and finally, a parameter fitting example demonstrates the power for learning unknown stochastic values.

scholarly journals Threshold Computation for Spatially Coupled Turbo-Like Codes on the AWGN Channel

Entropy ◽  
2021 ◽  
Vol 23 (2) ◽  
pp. 240
Author(s):  
Muhammad Umar Farooq ◽  
Alexandre Graell i Amat ◽  
Michael Lentmaier
Keyword(s):  
Gaussian Noise ◽  
Belief Propagation ◽  
Additive White Gaussian Noise ◽  
Maximum A Posteriori ◽  
Prediction Methods ◽  
Threshold Analysis ◽  
A Posteriori ◽  
Awgn Channel ◽  
Efficient Prediction ◽  
Binary Erasure Channel

In this paper, we perform a belief propagation (BP) decoding threshold analysis of spatially coupled (SC) turbo-like codes (TCs) (SC-TCs) on the additive white Gaussian noise (AWGN) channel. We review Monte-Carlo density evolution (MC-DE) and efficient prediction methods, which determine the BP thresholds of SC-TCs over the AWGN channel. We demonstrate that instead of performing time-consuming MC-DE computations, the BP threshold of SC-TCs over the AWGN channel can be predicted very efficiently from their binary erasure channel (BEC) thresholds. From threshold results, we conjecture that the similarity of MC-DE and predicted thresholds is related to the threshold saturation capability as well as capacity-approaching maximum a posteriori (MAP) performance of an SC-TC ensemble.

scholarly journals Cooperative Localization for Mobile Networks: A Distributed Belief Propagation–Mean Field Message Passing Algorithm

2016 ◽  
Vol 23 (6) ◽  
pp. 828-832 ◽  
Author(s):  
Burak Cakmak ◽  
Daniel N. Urup ◽  
Florian Meyer ◽  
Troels Pedersen ◽  
Bernard H. Fleury ◽  
...  
Keyword(s):  
Mobile Networks ◽  
Message Passing ◽  
Belief Propagation ◽  
Mean Field ◽  
Cooperative Localization ◽  
Message Passing Algorithm

scholarly journals Constraint Satisfaction by Survey Propagation

2005 ◽  
Author(s):  
Alfredo Braunstein ◽  
Marc Mézard
Keyword(s):  
Constraint Satisfaction ◽  
Message Passing ◽  
Statistical Physics ◽  
Belief Propagation ◽  
Solution Space ◽  
Error Correcting Codes ◽  
Constraint Satisfaction Problems ◽  
Probabilistic Methods ◽  
Survey Propagation ◽  
Algorithmic Strategy

Methods and analyses from statistical physics are of use not only in studying the performance of algorithms, but also in developing efficient algorithms. Here, we consider survey propagation (SP), a new approach for solving typical instances of random constraint satisfaction problems. SP has proven successful in solving random k-satisfiability (k -SAT) and random graph q-coloring (q-COL) in the “hard SAT” region of parameter space [79, 395, 397, 412], relatively close to the SAT/UNSAT phase transition discussed in the previous chapter. In this chapter we discuss the SP equations, and suggest a theoretical framework for the method [429] that applies to a wide class of discrete constraint satisfaction problems. We propose a way of deriving the equations that sheds light on the capabilities of the algorithm, and illustrates the differences with other well-known iterative probabilistic methods. Our approach takes into account the clustered structure of the solution space described in chapter 3, and involves adding an additional “joker” value that variables can be assigned. Within clusters, a variable can be frozen to some value, meaning that the variable always takes the same value for all solutions (satisfying assignments) within the cluster. Alternatively, it can be unfrozen, meaning that it fluctuates from solution to solution within the cluster. As we will discuss, the SP equations manage to describe the fluctuations by assigning joker values to unfrozen variables. The overall algorithmic strategy is iterative and decomposable in two elementary steps. The first step is to evaluate the marginal probabilities of frozen variables using the SP message-passing procedure. The second step, or decimation step, is to use this information to fix the values of some variables and simplify the problem. The notion of message passing will be illustrated throughout the chapter by comparing it with a simpler procedure known as belief propagation (mentioned in ch. 3 in the context of error correcting codes) in which no assumptions are made about the structure of the solution space. The chapter is organized as follows. In section 2 we provide the general formalism, defining constraint satisfaction problems as well as the key concepts of factor graphs and cavities, using the concrete examples of satisfiability and graph coloring.

scholarly journals Exploiting symmetries for scaling loopy belief propagation and relational training

2013 ◽  
Vol 92 (1) ◽  
pp. 91-132 ◽  
Author(s):  
Babak Ahmadi ◽  
Kristian Kersting ◽  
Martin Mladenov ◽  
Sriraam Natarajan
Keyword(s):  
Belief Propagation ◽  
Loopy Belief Propagation ◽  
Relational Training
Sign in / Sign up

Export Citation Format

Share Document