scholarly journals Duality of graphical models and tensor networks

2018 ◽  
Vol 8 (2) ◽  
pp. 273-288 ◽  
Author(s):  
Elina Robeva ◽  
Anna Seigal
Keyword(s):  
Graphical Models ◽  
Graphical Model ◽  
Belief Propagation ◽  
Discrete Variables ◽  
Network Algorithms ◽  
Research Areas ◽  
Tensor Networks ◽  
Tensor Network

Abstract In this article we show the duality between tensor networks and undirected graphical models with discrete variables. We study tensor networks on hypergraphs, which we call tensor hypernetworks. We show that the tensor hypernetwork on a hypergraph exactly corresponds to the graphical model given by the dual hypergraph. We translate various notions under duality. For example, marginalization in a graphical model is dual to contraction in the tensor network. Algorithms also translate under duality. We show that belief propagation corresponds to a known algorithm for tensor network contraction. This article is a reminder that the research areas of graphical models and tensor networks can benefit from interaction.

scholarly journals On the geometry of tensor network states

2012 ◽  
Vol 12 (3&4) ◽  
pp. 346-354
Author(s):  
Joseph M. Landsburg ◽  
Yang Qi ◽  
Ke Ye
Keyword(s):  
Graphical Models ◽  
Matrix Multiplication ◽  
Algebraic Statistics ◽  
Geometric Complexity ◽  
Network State ◽  
Geometric Complexity Theory ◽  
Tensor Networks ◽  
Tensor Network ◽  
Network States ◽  
Tensor Network States

We answer a question of L. Grasedyck that arose in quantum information theory, showing that the limit of tensors in a space of tensor network states need not be a tensor network state. We also give geometric descriptions of spaces of tensor networks states corresponding to trees and loops. Grasedyck's question has a surprising connection to the area of Geometric Complexity Theory, in that the result is equivalent to the statement that the boundary of the Mulmuley-Sohoni type variety associated to matrix multiplication is strictly larger than the projections of matrix multiplication (and re-expressions of matrix multiplication and its projections after changes of bases). Tensor Network States are also related to graphical models in algebraic statistics.

scholarly journals A Spectral Approach to Analysing Belief Propagation for 3-Colouring

2009 ◽  
Vol 18 (6) ◽  
pp. 881-912 ◽  
Author(s):  
AMIN COJA-OGHLAN ◽  
ELCHANAN MOSSEL ◽  
DAN VILENCHIK
Keyword(s):  
Graphical Models ◽  
Message Passing ◽  
Graphical Model ◽  
Belief Propagation ◽  
Graph Colouring ◽  
Rigorous Result ◽  
Theoretical Understanding ◽  
Message Passing Algorithm ◽  
Survey Propagation ◽  
Graphical Structure

Belief propagation (BP) is a message-passing algorithm that computes the exact marginal distributions at every vertex of a graphical model without cycles. While BP is designed to work correctly on trees, it is routinely applied to general graphical models that may contain cycles, in which case neither convergence, nor correctness in the case of convergence is guaranteed. Nonetheless, BP has gained popularity as it seems to remain effective in many cases of interest, even when the underlying graph is ‘far’ from being a tree. However, the theoretical understanding of BP (and its new relative survey propagation) when applied to CSPs is poor.Contributing to the rigorous understanding of BP, in this paper we relate the convergence of BP to spectral properties of the graph. This encompasses a result for random graphs with a ‘planted’ solution; thus, we obtain the first rigorous result on BP for graph colouring in the case of a complex graphical structure (as opposed to trees). In particular, the analysis shows how belief propagation breaks the symmetry between the 3! possible permutations of the colour classes.

scholarly journals Quantum Machine Learning Tensor Network States

2021 ◽  
Vol 8 ◽  
Author(s):  
Andrey Kardashin ◽  
Alexey Uvarov ◽  
Jacob Biamonte
Keyword(s):  
Machine Learning ◽  
Quantum Algorithm ◽  
Unitary Matrix ◽  
Many Body ◽  
Network Algorithms ◽  
Tensor Networks ◽  
Tensor Network ◽  
Network Methods ◽  
Network States ◽  
Tensor Network States

Tensor network algorithms seek to minimize correlations to compress the classical data representing quantum states. Tensor network algorithms and similar tools—called tensor network methods—form the backbone of modern numerical methods used to simulate many-body physics and have a further range of applications in machine learning. Finding and contracting tensor network states is a computational task, which may be accelerated by quantum computing. We present a quantum algorithm that returns a classical description of a rank-r tensor network state satisfying an area law and approximating an eigenvector given black-box access to a unitary matrix. Our work creates a bridge between several contemporary approaches, including tensor networks, the variational quantum eigensolver (VQE), quantum approximate optimization algorithm (QAOA), and quantum computation.

On Resolving Simultaneous Congruences Using Belief Propagation

2015 ◽  
Vol 27 (3) ◽  
pp. 748-770 ◽  
Author(s):  
Yongseok Yoo ◽  
Sriram Vishwanath
Keyword(s):  
Graphical Models ◽  
Latent Variables ◽  
Graphical Model ◽  
Belief Propagation ◽  
Optimization Problems ◽  
Affinity Propagation ◽  
High Signal ◽  
Ml Estimation ◽  
Combinatorial Optimization Problems ◽  
Simultaneous Congruences

Graphical models and related algorithmic tools such as belief propagation have proven to be useful tools in (approximately) solving combinatorial optimization problems across many application domains. A particularly combinatorially challenging problem is that of determining solutions to a set of simultaneous congruences. Specifically, a continuous source is encoded into multiple residues with respect to distinct moduli, and the goal is to recover the source efficiently from noisy measurements of these residues. This problem is of interest in multiple disciplines, including neural codes, decentralized compression in sensor networks, and distributed consensus in information and social networks. This letter reformulates the recovery problem as an optimization over binary latent variables. Then we present a belief propagation algorithm, a layered variant of affinity propagation, to solve the problem. The underlying encoding structure of multiple congruences naturally results in a layered graphical model for the problem, over which the algorithms are deployed, resulting in a layered affinity propagation (LAP) solution. First, the convergence of LAP to an approximation of the maximum likelihood (ML) estimate is shown. Second, numerical simulations show that LAP converges within a few iterations and that the mean square error of LAP approaches that of the ML estimation at high signal-to-noise ratios.

scholarly journals Block Belief Propagation for Parameter Learning in Markov Random Fields

2019 ◽  
Vol 33 ◽  
pp. 4448-4455
Author(s):  
You Lu ◽  
Zhiyuan Liu ◽  
Bert Huang
Keyword(s):  
Graphical Models ◽  
Random Fields ◽  
Markov Random Fields ◽  
Graphical Model ◽  
Belief Propagation ◽  
Traditional Learning ◽  
Training Methods ◽  
Iteration Complexity ◽  
Markov Random ◽  
Standard Training

Traditional learning methods for training Markov random fields require doing inference over all variables to compute the likelihood gradient. The iteration complexity for those methods therefore scales with the size of the graphical models. In this paper, we propose block belief propagation learning (BBPL), which uses block-coordinate updates of approximate marginals to compute approximate gradients, removing the need to compute inference on the entire graphical model. Thus, the iteration complexity of BBPL does not scale with the size of the graphs. We prove that the method converges to the same solution as that obtained by using full inference per iteration, despite these approximations, and we empirically demonstrate its scalability improvements over standard training methods.

scholarly journals Estimating Differential Latent Variable Graphical Models with Applications to Brain Connectivity

Biometrika ◽  
2020 ◽  
Author(s):  
S Na ◽  
M Kolar ◽  
O Koyejo
Keyword(s):  
Graphical Models ◽  
Latent Variable ◽  
Graphical Model ◽  
Brain Connectivity ◽  
Statistical Error ◽  
Real Data ◽  
Low Rank ◽  
Conditional Dependence ◽  
Gradient Descent Algorithm ◽  
The Difference

Abstract Differential graphical models are designed to represent the difference between the conditional dependence structures of two groups, thus are of particular interest for scientific investigation. Motivated by modern applications, this manuscript considers an extended setting where each group is generated by a latent variable Gaussian graphical model. Due to the existence of latent factors, the differential network is decomposed into sparse and low-rank components, both of which are symmetric indefinite matrices. We estimate these two components simultaneously using a two-stage procedure: (i) an initialization stage, which computes a simple, consistent estimator, and (ii) a convergence stage, implemented using a projected alternating gradient descent algorithm applied to a nonconvex objective, initialized using the output of the first stage. We prove that given the initialization, the estimator converges linearly with a nontrivial, minimax optimal statistical error. Experiments on synthetic and real data illustrate that the proposed nonconvex procedure outperforms existing methods.

scholarly journals Beyond Trees: Analysis and Convergence of Belief Propagation in Graphs with Multiple Cycles

2020 ◽  
Vol 34 (05) ◽  
pp. 7333-7340
Author(s):  
Roie Zivan ◽  
Omer Lev ◽  
Rotem Galiki
Keyword(s):  
Graphical Models ◽  
Belief Propagation ◽  
Optimization Problems ◽  
Optimal Solution ◽  
Damping Factor ◽  
Constraint Optimization ◽  
Single Cycle ◽  
Distributed Constraint Optimization ◽  
Multiple Cycles ◽  
Constraint Optimization Problems

Belief propagation, an algorithm for solving problems represented by graphical models, has long been known to converge to the optimal solution when the graph is a tree. When the graph representing the problem includes a single cycle, the algorithm either converges to the optimal solution or performs periodic oscillations. While the conditions that trigger these two behaviors have been established, the question regarding the convergence and divergence of the algorithm on graphs that include more than one cycle is still open.Focusing on Max-sum, the version of belief propagation for solving distributed constraint optimization problems (DCOPs), we extend the theory on the behavior of belief propagation in general – and Max-sum specifically – when solving problems represented by graphs with multiple cycles. This includes: 1) Generalizing the results obtained for graphs with a single cycle to graphs with multiple cycles, by using backtrack cost trees (BCT). 2) Proving that when the algorithm is applied to adjacent symmetric cycles, the use of a large enough damping factor guarantees convergence to the optimal solution.

scholarly journals Hyper-optimized tensor network contraction

Quantum ◽  
2021 ◽  
Vol 5 ◽  
pp. 410
Author(s):  
Johnnie Gray ◽  
Stefanos Kourtis
Keyword(s):  
Computation Time ◽  
Quantum Circuit ◽  
Optimization Approach ◽  
Many Body ◽  
Tensor Networks ◽  
Randomized Protocols ◽  
Classical Simulation ◽  
Tensor Network ◽  
Speed Up ◽  
Many Body Systems

Tensor networks represent the state-of-the-art in computational methods across many disciplines, including the classical simulation of quantum many-body systems and quantum circuits. Several applications of current interest give rise to tensor networks with irregular geometries. Finding the best possible contraction path for such networks is a central problem, with an exponential effect on computation time and memory footprint. In this work, we implement new randomized protocols that find very high quality contraction paths for arbitrary and large tensor networks. We test our methods on a variety of benchmarks, including the random quantum circuit instances recently implemented on Google quantum chips. We find that the paths obtained can be very close to optimal, and often many orders or magnitude better than the most established approaches. As different underlying geometries suit different methods, we also introduce a hyper-optimization approach, where both the method applied and its algorithmic parameters are tuned during the path finding. The increase in quality of contraction schemes found has significant practical implications for the simulation of quantum many-body systems and particularly for the benchmarking of new quantum chips. Concretely, we estimate a speed-up of over 10,000× compared to the original expectation for the classical simulation of the Sycamore `supremacy' circuits.

scholarly journals Automatic contraction of unstructured tensor networks

2020 ◽  
Vol 8 (1) ◽  
Author(s):  
Adam Jermyn
Keyword(s):  
Partition Function ◽  
Statistical Physics ◽  
Discrete Models ◽  
Correlation Structure ◽  
Central Problem ◽  
Partition Functions ◽  
Lattice Systems ◽  
Tensor Networks ◽  
Tensor Network

The evaluation of partition functions is a central problem in statistical physics. For lattice systems and other discrete models the partition function may be expressed as the contraction of a tensor network. Unfortunately computing such contractions is difficult, and many methods to make this tractable require periodic or otherwise structured networks. Here I present a new algorithm for contracting unstructured tensor networks. This method makes no assumptions about the structure of the network and performs well in both structured and unstructured cases so long as the correlation structure is local.

scholarly journals Disentangling microbial associations from hidden environmental and technical factors via latent graphical models

2019 ◽  
Author(s):  
Zachary D. Kurtz ◽  
Richard Bonneau ◽  
Christian L. Müller
Keyword(s):  
Graphical Models ◽  
Graphical Model ◽  
R Package ◽  
Comprehensive Collection ◽  
Performance Guarantees ◽  
Ecological Association ◽  
Microbial Associations ◽  
Sparse Inverse Covariance Estimation ◽  
Statistical Relationships ◽  
And Function

AbstractDetecting community-wide statistical relationships from targeted amplicon-based and metagenomic profiling of microbes in their natural environment is an important step toward understanding the organization and function of these communities. We present a robust and computationally tractable latent graphical model inference scheme that allows simultaneous identification of parsimonious statistical relationships among microbial species and unobserved factors that influence the prevalence and variability of the abundance measurements. Our method comes with theoretical performance guarantees and is available within the SParse InversE Covariance estimation for Ecological ASsociation Inference (SPIEC-EASI) framework (‘SpiecEasi’ R-package). Using simulations, as well as a comprehensive collection of amplicon-based gut microbiome datasets, we illustrate the method’s ability to jointly identify compositional biases, latent factors that correlate with observed technical covariates, and robust statistical microbial associations that replicate across different gut microbial data sets.

Sign in / Sign up

Export Citation Format

Share Document