scholarly journals A Spectral Approach to Analysing Belief Propagation for 3-Colouring

2009 ◽  
Vol 18 (6) ◽  
pp. 881-912 ◽  
Author(s):  
AMIN COJA-OGHLAN ◽  
ELCHANAN MOSSEL ◽  
DAN VILENCHIK
Keyword(s):  
Graphical Models ◽  
Message Passing ◽  
Graphical Model ◽  
Belief Propagation ◽  
Graph Colouring ◽  
Rigorous Result ◽  
Theoretical Understanding ◽  
Message Passing Algorithm ◽  
Survey Propagation ◽  
Graphical Structure

Belief propagation (BP) is a message-passing algorithm that computes the exact marginal distributions at every vertex of a graphical model without cycles. While BP is designed to work correctly on trees, it is routinely applied to general graphical models that may contain cycles, in which case neither convergence, nor correctness in the case of convergence is guaranteed. Nonetheless, BP has gained popularity as it seems to remain effective in many cases of interest, even when the underlying graph is ‘far’ from being a tree. However, the theoretical understanding of BP (and its new relative survey propagation) when applied to CSPs is poor.Contributing to the rigorous understanding of BP, in this paper we relate the convergence of BP to spectral properties of the graph. This encompasses a result for random graphs with a ‘planted’ solution; thus, we obtain the first rigorous result on BP for graph colouring in the case of a complex graphical structure (as opposed to trees). In particular, the analysis shows how belief propagation breaks the symmetry between the 3! possible permutations of the colour classes.

scholarly journals Cooperative Localization for Mobile Networks: A Distributed Belief Propagation–Mean Field Message Passing Algorithm

2016 ◽  
Vol 23 (6) ◽  
pp. 828-832 ◽  
Author(s):  
Burak Cakmak ◽  
Daniel N. Urup ◽  
Florian Meyer ◽  
Troels Pedersen ◽  
Bernard H. Fleury ◽  
...  
Keyword(s):  
Mobile Networks ◽  
Message Passing ◽  
Belief Propagation ◽  
Mean Field ◽  
Cooperative Localization ◽  
Message Passing Algorithm

scholarly journals Constraint Satisfaction by Survey Propagation

2005 ◽  
Author(s):  
Alfredo Braunstein ◽  
Marc Mézard
Keyword(s):  
Constraint Satisfaction ◽  
Message Passing ◽  
Statistical Physics ◽  
Belief Propagation ◽  
Solution Space ◽  
Error Correcting Codes ◽  
Constraint Satisfaction Problems ◽  
Probabilistic Methods ◽  
Survey Propagation ◽  
Algorithmic Strategy

Methods and analyses from statistical physics are of use not only in studying the performance of algorithms, but also in developing efficient algorithms. Here, we consider survey propagation (SP), a new approach for solving typical instances of random constraint satisfaction problems. SP has proven successful in solving random k-satisfiability (k -SAT) and random graph q-coloring (q-COL) in the “hard SAT” region of parameter space [79, 395, 397, 412], relatively close to the SAT/UNSAT phase transition discussed in the previous chapter. In this chapter we discuss the SP equations, and suggest a theoretical framework for the method [429] that applies to a wide class of discrete constraint satisfaction problems. We propose a way of deriving the equations that sheds light on the capabilities of the algorithm, and illustrates the differences with other well-known iterative probabilistic methods. Our approach takes into account the clustered structure of the solution space described in chapter 3, and involves adding an additional “joker” value that variables can be assigned. Within clusters, a variable can be frozen to some value, meaning that the variable always takes the same value for all solutions (satisfying assignments) within the cluster. Alternatively, it can be unfrozen, meaning that it fluctuates from solution to solution within the cluster. As we will discuss, the SP equations manage to describe the fluctuations by assigning joker values to unfrozen variables. The overall algorithmic strategy is iterative and decomposable in two elementary steps. The first step is to evaluate the marginal probabilities of frozen variables using the SP message-passing procedure. The second step, or decimation step, is to use this information to fix the values of some variables and simplify the problem. The notion of message passing will be illustrated throughout the chapter by comparing it with a simpler procedure known as belief propagation (mentioned in ch. 3 in the context of error correcting codes) in which no assumptions are made about the structure of the solution space. The chapter is organized as follows. In section 2 we provide the general formalism, defining constraint satisfaction problems as well as the key concepts of factor graphs and cavities, using the concrete examples of satisfiability and graph coloring.

Efficient nonparametric belief propagation for pose estimation and manipulation of articulated objects

2019 ◽  
Vol 4 (30) ◽  
pp. eaaw4523 ◽  
Author(s):  
Karthik Desingh ◽  
Shiyang Lu ◽  
Anthony Opipari ◽  
Odest Chadwicke Jenkins
Keyword(s):  
Pose Estimation ◽  
Message Passing ◽  
Belief Propagation ◽  
Sensor Data ◽  
Depth Sensor ◽  
Continuous Space ◽  
Articulated Objects ◽  
Message Passing Algorithm ◽  
Wide Range ◽  
Object Part

Robots working in human environments often encounter a wide range of articulated objects, such as tools, cabinets, and other jointed objects. Such articulated objects can take an infinite number of possible poses, as a point in a potentially high-dimensional continuous space. A robot must perceive this continuous pose to manipulate the object to a desired pose. This problem of perception and manipulation of articulated objects remains a challenge due to its high dimensionality and multimodal uncertainty. Here, we describe a factored approach to estimate the poses of articulated objects using an efficient approach to nonparametric belief propagation. We consider inputs as geometrical models with articulation constraints and observed RGBD (red, green, blue, and depth) sensor data. The described framework produces object-part pose beliefs iteratively. The problem is formulated as a pairwise Markov random field (MRF), where each hidden node (continuous pose variable) is an observed object-part’s pose and the edges denote the articulation constraints between the parts. We describe articulated pose estimation by a “pull” message passing algorithm for nonparametric belief propagation (PMPNBP) and evaluate its convergence properties over scenes with articulated objects. Robot experiments are provided to demonstrate the necessity of maintaining beliefs to perform goal-driven manipulation tasks.

scholarly journals Single-Trace Attacks on Keccak

2020 ◽  
pp. 243-268 ◽  
Author(s):  
Matthias J. Kannwischer ◽  
Peter Pessl ◽  
Robert Primas
Keyword(s):  
Template Matching ◽  
Message Passing ◽  
Graphical Model ◽  
Side Channel ◽  
Key Recovery ◽  
Single Observation ◽  
Message Passing Algorithm ◽  
Post Quantum Cryptography ◽  
Single Trace ◽  
Differential Attacks

Since its selection as the winner of the SHA-3 competition, Keccak, with all its variants, has found a large number of applications. It is, for instance, a common building block in schemes submitted to NIST’s post-quantum cryptography project. In many of these applications, Keccak processes ephemeral secrets. In such a setting, side-channel adversaries are limited to a single observation, meaning that differential attacks are inherently prevented. If, however, such a single trace of Keccak can already be sufficient for key recovery has so far been unknown. In this paper, we change the above by presenting the first single-trace attack targeting Keccak. Our method is based on soft-analytical side-channel attacks and, thus, combines template matching with message passing in a graphical model of the attacked algorithm. As a straight-forward model of Keccak does not yield satisfactory results, we describe several optimizations for the modeling and the message-passing algorithm. Their combination allows attaining high attack performance in terms of both success rate as well as computational runtime. We evaluate our attack assuming generic software (microcontroller) targets and thus use simulations in the generic noisy Hamming-weight leakage model. Hence, we assume relatively modest profiling capabilities of the adversary. Nonetheless, the attack can reliably recover secrets in a large number of evaluated scenarios at realistic noise levels. Consequently, we demonstrate the need for countermeasures even in settings where DPA is not a threat.

Extended message passing algorithm for inference in loopy Gaussian graphical models

2004 ◽  
Vol 2 (2) ◽  
pp. 153-169 ◽  
Author(s):  
K.H. Plarre ◽  
P.R. Kumar
Keyword(s):  
Graphical Models ◽  
Message Passing ◽  
Gaussian Graphical Models ◽  
Message Passing Algorithm

scholarly journals An Efficient Implementation of Track-Oriented Multiple Hypothesis Tracker Using Graphical Model Approaches

2017 ◽  
Vol 2017 ◽  
pp. 1-11 ◽  
Author(s):  
Jinping Sun ◽  
Qing Li ◽  
Xuwang Zhang ◽  
Wei Sun
Keyword(s):  
Message Passing ◽  
Graphical Model ◽  
Belief Propagation ◽  
Data Association ◽  
Independent Set ◽  
Graph Representation ◽  
Maximum Weight ◽  
Hypothesis Formation ◽  
Maximum Weight Independent Set ◽  
Multiple Hypothesis

The multiple hypothesis tracker (MHT) is currently the preferred method for addressing data association problem in multitarget tracking (MTT) application. MHT seeks the most likely global hypothesis by enumerating all possible associations over time, which is equal to calculating maximum a posteriori (MAP) estimate over the report data. Despite being a well-studied method, MHT remains challenging mostly because of the computational complexity of data association. In this paper, we describe an efficient method for solving the data association problem using graphical model approaches. The proposed method uses the graph representation to model the global hypothesis formation and subsequently applies an efficient message passing algorithm to obtain the MAP solution. Specifically, the graph representation of data association problem is formulated as a maximum weight independent set problem (MWISP), which translates the best global hypothesis formation into finding the maximum weight independent set on the graph. Then, a max-product belief propagation (MPBP) inference algorithm is applied to seek the most likely global hypotheses with the purpose of avoiding a brute force hypothesis enumeration procedure. The simulation results show that the proposed MPBP-MHT method can achieve better tracking performance than other algorithms in challenging tracking situations.

scholarly journals Duality of graphical models and tensor networks

2018 ◽  
Vol 8 (2) ◽  
pp. 273-288 ◽  
Author(s):  
Elina Robeva ◽  
Anna Seigal
Keyword(s):  
Graphical Models ◽  
Graphical Model ◽  
Belief Propagation ◽  
Discrete Variables ◽  
Network Algorithms ◽  
Research Areas ◽  
Tensor Networks ◽  
Tensor Network

Abstract In this article we show the duality between tensor networks and undirected graphical models with discrete variables. We study tensor networks on hypergraphs, which we call tensor hypernetworks. We show that the tensor hypernetwork on a hypergraph exactly corresponds to the graphical model given by the dual hypergraph. We translate various notions under duality. For example, marginalization in a graphical model is dual to contraction in the tensor network. Algorithms also translate under duality. We show that belief propagation corresponds to a known algorithm for tensor network contraction. This article is a reminder that the research areas of graphical models and tensor networks can benefit from interaction.

Combined Belief Propagation-Mean Field Message Passing Algorithm for Dirichlet Process Mixtures

2019 ◽  
Vol 26 (7) ◽  
pp. 1041-1045
Author(s):  
Xinhua Lu ◽  
Chuanzong Zhang ◽  
Zhongyong Wang
Keyword(s):  
Dirichlet Process ◽  
Message Passing ◽  
Belief Propagation ◽  
Mean Field ◽  
Message Passing Algorithm ◽  
Dirichlet Process Mixtures

scholarly journals Learning General Latent-Variable Graphical Models with Predictive Belief Propagation

2020 ◽  
Vol 34 (04) ◽  
pp. 6118-6126
Author(s):  
Borui Wang ◽  
Geoffrey Gordon
Keyword(s):  
Em Algorithm ◽  
Graphical Models ◽  
Message Passing ◽  
Latent Variable ◽  
Belief Propagation ◽  
Learning Problems ◽  
Local Optima ◽  
Learning Framework ◽  
The Em Algorithm ◽  
Spectral Algorithm

Learning general latent-variable probabilistic graphical models is a key theoretical challenge in machine learning and artificial intelligence. All previous methods, including the EM algorithm and the spectral algorithms, face severe limitations that largely restrict their applicability and affect their performance. In order to overcome these limitations, in this paper we introduce a novel formulation of message-passing inference over junction trees named predictive belief propagation, and propose a new learning and inference algorithm for general latent-variable graphical models based on this formulation. Our proposed algorithm reduces the hard parameter learning problem into a sequence of supervised learning problems, and unifies the learning of different kinds of latent graphical models into a single learning framework, which is local-optima-free and statistically consistent. We then give a proof of the correctness of our algorithm and show in experiments on both synthetic and real datasets that our algorithm significantly outperforms both the EM algorithm and the spectral algorithm while also being orders of magnitude faster to compute.

scholarly journals Recovering a hidden community beyond the Kesten–Stigum threshold in O(|E|log*|V|) time

2018 ◽  
Vol 55 (2) ◽  
pp. 325-352 ◽  
Author(s):  
Bruce Hajek ◽  
Yihong Wu ◽  
Jiaming Xu
Keyword(s):  
Message Passing ◽  
Belief Propagation ◽  
Linear Time ◽  
Signal To Noise Ratio ◽  
Bp Algorithm ◽  
Local Algorithm ◽  
Message Passing Algorithm ◽  
Edge Probability ◽  
Effective Signal ◽  
Total Time Complexity

Abstract Community detection is considered for a stochastic block model graph of n vertices, with K vertices in the planted community, edge probability p for pairs of vertices both in the community, and edge probability q for other pairs of vertices. The main focus of the paper is on weak recovery of the community based on the graph G, with o(K) misclassified vertices on average, in the sublinear regime n1-o(1) ≤ K ≤ o(n). A critical parameter is the effective signal-to-noise ratio λ = K2(p - q)2 / ((n - K)q), with λ = 1 corresponding to the Kesten–Stigum threshold. We show that a belief propagation (BP) algorithm achieves weak recovery if λ > 1 / e, beyond the Kesten–Stigum threshold by a factor of 1 / e. The BP algorithm only needs to run for log*n + O(1) iterations, with the total time complexity O(|E|log*n), where log*n is the iterated logarithm of n. Conversely, if λ ≤ 1 / e, no local algorithm can asymptotically outperform trivial random guessing. Furthermore, a linear message-passing algorithm that corresponds to applying a power iteration to the nonbacktracking matrix of the graph is shown to attain weak recovery if and only if λ > 1. In addition, the BP algorithm can be combined with a linear-time voting procedure to achieve the information limit of exact recovery (correctly classify all vertices with high probability) for all K ≥ (n / logn) (ρBP + o(1)), where ρBP is a function of p / q.

Sign in / Sign up

Export Citation Format

Share Document