scholarly journals Constraint Satisfaction by Survey Propagation

2005 ◽  
Author(s):  
Alfredo Braunstein ◽  
Marc Mézard
Keyword(s):  
Constraint Satisfaction ◽  
Message Passing ◽  
Statistical Physics ◽  
Belief Propagation ◽  
Solution Space ◽  
Error Correcting Codes ◽  
Constraint Satisfaction Problems ◽  
Probabilistic Methods ◽  
Survey Propagation ◽  
Algorithmic Strategy

Methods and analyses from statistical physics are of use not only in studying the performance of algorithms, but also in developing efficient algorithms. Here, we consider survey propagation (SP), a new approach for solving typical instances of random constraint satisfaction problems. SP has proven successful in solving random k-satisfiability (k -SAT) and random graph q-coloring (q-COL) in the “hard SAT” region of parameter space [79, 395, 397, 412], relatively close to the SAT/UNSAT phase transition discussed in the previous chapter. In this chapter we discuss the SP equations, and suggest a theoretical framework for the method [429] that applies to a wide class of discrete constraint satisfaction problems. We propose a way of deriving the equations that sheds light on the capabilities of the algorithm, and illustrates the differences with other well-known iterative probabilistic methods. Our approach takes into account the clustered structure of the solution space described in chapter 3, and involves adding an additional “joker” value that variables can be assigned. Within clusters, a variable can be frozen to some value, meaning that the variable always takes the same value for all solutions (satisfying assignments) within the cluster. Alternatively, it can be unfrozen, meaning that it fluctuates from solution to solution within the cluster. As we will discuss, the SP equations manage to describe the fluctuations by assigning joker values to unfrozen variables. The overall algorithmic strategy is iterative and decomposable in two elementary steps. The first step is to evaluate the marginal probabilities of frozen variables using the SP message-passing procedure. The second step, or decimation step, is to use this information to fix the values of some variables and simplify the problem. The notion of message passing will be illustrated throughout the chapter by comparing it with a simpler procedure known as belief propagation (mentioned in ch. 3 in the context of error correcting codes) in which no assumptions are made about the structure of the solution space. The chapter is organized as follows. In section 2 we provide the general formalism, defining constraint satisfaction problems as well as the key concepts of factor graphs and cavities, using the concrete examples of satisfiability and graph coloring.

scholarly journals A Residual-Based Message Passing Algorithm for Constraint Satisfaction Problems

2022 ◽  
Author(s):  
Chun-Yan Zhao ◽  
Yan-Rong Fu ◽  
Jin-Hua Zhao
Keyword(s):  
Constraint Satisfaction ◽  
Message Passing ◽  
Optimization Problems ◽  
Algorithm Design ◽  
Success Probability ◽  
Computational Cost ◽  
Solution Space ◽  
Constraint Satisfaction Problems ◽  
Threshold Phenomena ◽  
Message Passing Algorithms

Abstract Message passing algorithms, whose iterative nature captures well complicated interactions among interconnected variables in complex systems and extracts information from the fixed point of iterated messages, provide a powerful toolkit in tackling hard computational tasks in optimization, inference, and learning problems. In the context of constraint satisfaction problems (CSPs), when a control parameter (such as constraint density) is tuned, multiple threshold phenomena emerge, signaling fundamental structural transitions in their solution space. Finding solutions around these transition points is exceedingly challenging for algorithm design, where message passing algorithms suffer from a large message fluctuation far from convergence. Here we introduce a residual-based updating step into message passing algorithms, in which messages varying large between consecutive steps are given a high priority in updating process. For the specific example of model RB, a typical prototype of random CSPs with growing domains, we show that our algorithm improves the convergence of message updating and increases the success probability in finding solutions around the satisfiability threshold with a low computational cost. Our approach to message passing algorithms should be of value for exploring their power in developing algorithms to find ground-state solutions and understand the detailed structure of solution space of hard optimization problems.

scholarly journals Optimal errors and phase transitions in high-dimensional generalized linear models

2019 ◽  
Vol 116 (12) ◽  
pp. 5451-5460 ◽  
Author(s):  
Jean Barbier ◽  
Florent Krzakala ◽  
Nicolas Macris ◽  
Léo Miolane ◽  
Lenka Zdeborová
Keyword(s):  
Phase Transitions ◽  
Generalized Linear Models ◽  
Message Passing ◽  
Statistical Physics ◽  
Linear Models ◽  
Optimal Estimation ◽  
Error Correcting Codes ◽  
Data Matrix ◽  
High Dimensional ◽  
Special Cases

Generalized linear models (GLMs) are used in high-dimensional machine learning, statistics, communications, and signal processing. In this paper we analyze GLMs when the data matrix is random, as relevant in problems such as compressed sensing, error-correcting codes, or benchmark models in neural networks. We evaluate the mutual information (or “free entropy”) from which we deduce the Bayes-optimal estimation and generalization errors. Our analysis applies to the high-dimensional limit where both the number of samples and the dimension are large and their ratio is fixed. Nonrigorous predictions for the optimal errors existed for special cases of GLMs, e.g., for the perceptron, in the field of statistical physics based on the so-called replica method. Our present paper rigorously establishes those decades-old conjectures and brings forward their algorithmic interpretation in terms of performance of the generalized approximate message-passing algorithm. Furthermore, we tightly characterize, for many learning problems, regions of parameters for which this algorithm achieves the optimal performance and locate the associated sharp phase transitions separating learnable and nonlearnable regions. We believe that this random version of GLMs can serve as a challenging benchmark for multipurpose algorithms.

scholarly journals CRITICALITY AND HETEROGENEITY IN THE SOLUTION SPACE OF RANDOM CONSTRAINT SATISFACTION PROBLEMS

2010 ◽  
Vol 24 (18) ◽  
pp. 3479-3487 ◽  
Author(s):  
HAIJUN ZHOU
Keyword(s):  
Constraint Satisfaction ◽  
Spin Glasses ◽  
Solution Space ◽  
Threshold Value ◽  
Critical Value ◽  
Constraint Satisfaction Problems ◽  
Model Systems ◽  
Dynamical Heterogeneity ◽  
Ergodicity Breaking ◽  
Random Constraint Satisfaction Problems

Random constraint satisfaction problems are interesting model systems for spin-glasses and glassy dynamics studies. As the constraint density of such a system reaches certain threshold value, its solution space may split into extremely many clusters. In this work we argue that this ergodicity-breaking transition is preceded by a homogeneity-breaking transition. For random K-SAT and K-XORSAT, we show that many solution communities start to form in the solution space as the constraint density reaches a critical value αcm, with each community containing a set of solutions that are more similar with each other than with the outsider solutions. At αcm the solution space is in a critical state. The connection of these results to the onset of dynamical heterogeneity in lattice glass models is discussed.

scholarly journals Universality of the SAT-UNSAT (jamming) threshold in non-convex continuous constraint satisfaction problems

2017 ◽  
Vol 2 (3) ◽  
Author(s):  
Silvio Franz ◽  
Giorgio Parisi ◽  
Maxime Sevelev ◽  
Pierfrancesco Urbani ◽  
Francesco Zamponi
Keyword(s):  
Constraint Satisfaction ◽  
Disordered Systems ◽  
Statistical Physics ◽  
Degrees Of Freedom ◽  
Universality Class ◽  
Constraint Satisfaction Problems ◽  
Worst Case ◽  
Average Case ◽  
Soft Spheres ◽  
Scaling Solution

Random constraint satisfaction problems (CSP) have been studied extensively using statistical physics techniques. They provide a benchmark to study average case scenarios instead of the worst case one. The interplay between statistical physics of disordered systems and computer science has brought new light into the realm of computational complexity theory, by introducing the notion of clustering of solutions, related to replica symmetry breaking. However, the class of problems in which clustering has been studied often involve discrete degrees of freedom: standard random CSPs are random (aka disordered Ising models) or random coloring problems (aka disordered Potts models). In this work we consider instead problems that involve continuous degrees of freedom. The simplest prototype of these problems is the perceptron. Here we discuss in detail the full phase diagram of the model. In the regions of parameter space where the problem is non-convex, leading to multiple disconnected clusters of solutions, the solution is critical at the SAT/UNSAT threshold and lies in the same universality class of the jamming transition of soft spheres. We show how the critical behavior at the satisfiability threshold emerges, and we compute the critical exponents associated to the approach to the transition from both the SAT and UNSAT phase. We conjecture that there is a large universality class of non-convex continuous CSPs whose SAT-UNSAT threshold is described by the same scaling solution.

scholarly journals A Spectral Approach to Analysing Belief Propagation for 3-Colouring

2009 ◽  
Vol 18 (6) ◽  
pp. 881-912 ◽  
Author(s):  
AMIN COJA-OGHLAN ◽  
ELCHANAN MOSSEL ◽  
DAN VILENCHIK
Keyword(s):  
Graphical Models ◽  
Message Passing ◽  
Graphical Model ◽  
Belief Propagation ◽  
Graph Colouring ◽  
Rigorous Result ◽  
Theoretical Understanding ◽  
Message Passing Algorithm ◽  
Survey Propagation ◽  
Graphical Structure

Belief propagation (BP) is a message-passing algorithm that computes the exact marginal distributions at every vertex of a graphical model without cycles. While BP is designed to work correctly on trees, it is routinely applied to general graphical models that may contain cycles, in which case neither convergence, nor correctness in the case of convergence is guaranteed. Nonetheless, BP has gained popularity as it seems to remain effective in many cases of interest, even when the underlying graph is ‘far’ from being a tree. However, the theoretical understanding of BP (and its new relative survey propagation) when applied to CSPs is poor.Contributing to the rigorous understanding of BP, in this paper we relate the convergence of BP to spectral properties of the graph. This encompasses a result for random graphs with a ‘planted’ solution; thus, we obtain the first rigorous result on BP for graph colouring in the case of a complex graphical structure (as opposed to trees). In particular, the analysis shows how belief propagation breaks the symmetry between the 3! possible permutations of the colour classes.

Sign in / Sign up

Export Citation Format

Share Document