scholarly journals Lifted Hinge-Loss Markov Random Fields

2019 ◽  
Vol 33 ◽  
pp. 7975-7983 ◽  
Author(s):  
Sriram Srinivasan ◽  
Behrouz Babaki ◽  
Golnoosh Farnadi ◽  
Lise Getoor
Keyword(s):  
Convex Optimization ◽  
Graphical Models ◽  
Random Fields ◽  
Markov Random Fields ◽  
Optimization Problem ◽  
Convex Optimization Problem ◽  
Map Inference ◽  
Lifted Inference ◽  
Hinge Loss ◽  
Markov Random

Statistical relational learning models are powerful tools that combine ideas from first-order logic with probabilistic graphical models to represent complex dependencies. Despite their success in encoding large problems with a compact set of weighted rules, performing inference over these models is often challenging. In this paper, we show how to effectively combine two powerful ideas for scaling inference for large graphical models. The first idea, lifted inference, is a wellstudied approach to speeding up inference in graphical models by exploiting symmetries in the underlying problem. The second idea is to frame Maximum a posteriori (MAP) inference as a convex optimization problem and use alternating direction method of multipliers (ADMM) to solve the problem in parallel. A well-studied relaxation to the combinatorial optimization problem defined for logical Markov random fields gives rise to a hinge-loss Markov random field (HLMRF) for which MAP inference is a convex optimization problem. We show how the formalism introduced for coloring weighted bipartite graphs using a color refinement algorithm can be integrated with the ADMM optimization technique to take advantage of the sparse dependency structures of HLMRFs. Our proposed approach, lifted hinge-loss Markov random fields (LHL-MRFs), preserves the structure of the original problem after lifting and solves lifted inference as distributed convex optimization with ADMM. In our empirical evaluation on real-world problems, we observe up to a three times speed up in inference over HL-MRFs.

scholarly journals On a Class of Tensor Markov Fields

Entropy ◽  
2020 ◽  
Vol 22 (4) ◽  
pp. 451
Author(s):  
Enrique Hernández-Lemus
Keyword(s):  
Graphical Models ◽  
Random Fields ◽  
Markov Random Fields ◽  
Random Variables ◽  
Probability Measures ◽  
Gibbs Fields ◽  
Markov Random ◽  
Statistical Dependency ◽  
Markov Fields ◽  
Dependency Structures

Here, we introduce a class of Tensor Markov Fields intended as probabilistic graphical models from random variables spanned over multiplexed contexts. These fields are an extension of Markov Random Fields for tensor-valued random variables. By extending the results of Dobruschin, Hammersley and Clifford to such tensor valued fields, we proved that tensor Markov fields are indeed Gibbs fields, whenever strictly positive probability measures are considered. Hence, there is a direct relationship with many results from theoretical statistical mechanics. We showed how this class of Markov fields it can be built based on a statistical dependency structures inferred on information theoretical grounds over empirical data. Thus, aside from purely theoretical interest, the Tensor Markov Fields described here may be useful for mathematical modeling and data analysis due to their intrinsic simplicity and generality.

scholarly journals Learning Interpretable Relational Structures of Hinge-loss Markov Random Fields

2019 ◽  
Author(s):  
Yue Zhang ◽  
Arti Ramesh
Keyword(s):  
Random Fields ◽  
Markov Random Fields ◽  
Learning Algorithm ◽  
Computational Social Science ◽  
Prediction Performance ◽  
Good Prediction ◽  
Relational Structures ◽  
Hinge Loss ◽  
Markov Random ◽  
Combine Uncertainty

Statistical relational models such as Markov logic networks (MLNs) and hinge-loss Markov random fields (HL-MRFs) are specified using templated weighted first-order logic clauses, leading to the creation of complex, yet easy to encode models that effectively combine uncertainty and logic. Learning the structure of these models from data reduces the human effort of identifying the right structures. In this work, we present an asynchronous deep reinforcement learning algorithm to automatically learn HL-MRF clause structures. Our algorithm possesses the ability to learn semantically meaningful structures that appeal to human intuition and understanding, while simultaneously being able to learn structures from data, thus learning structures that have both the desirable qualities of interpretability and good prediction performance. The asynchronous nature of our algorithm further provides the ability to learn diverse structures via exploration, while remaining scalable. We demonstrate the ability of the models to learn semantically meaningful structures that also achieve better prediction performance when compared with a greedy search algorithm, a path-based algorithm, and manually defined clauses on two computational social science applications: i) modeling recovery in alcohol use disorder, and ii) detecting bullying.

Climate field completion via Markov random fields – Application to the HadCRUT4.6 temperature dataset

2021 ◽  
pp. 1-66
Author(s):  
Adam Vaccaro ◽  
Julien Emile-Geay ◽  
Dominque Guillot ◽  
Resherle Verna ◽  
Colin Morice ◽  
...  
Keyword(s):  
Surface Temperature ◽  
Graphical Models ◽  
Random Fields ◽  
Markov Random Fields ◽  
Missing Values ◽  
Interpolation Method ◽  
Synthetic Data ◽  
Warming Trend ◽  
Gaussian Markov Random Fields ◽  
Markov Random

AbstractSurface temperature is a vital metric of Earth’s climate state, but is incompletely observed in both space and time: over half of monthly values are missing from the widely used HadCRUT4.6 global surface temperature dataset. Here we apply GraphEM, a recently developed imputation method, to construct a spatially complete estimate of HadCRUT4.6 temperatures. GraphEM leverages Gaussian Markov random fields (aka Gaussian graphical models) to better estimate covariance relationships within a climate field, detecting anisotropic features such as land/ocean contrasts, orography, ocean currents and wave-propagation pathways. This detection leads to improved estimates of missing values compared to methods (such as kriging) that assume isotropic covariance relationships, as we show with real and synthetic data.This interpolated analysis of HadCRUT4.6 data is available as a 100-member ensemble, propagating information about sampling variability available from the original HadCRUT4.6 dataset. A comparison of NINO3.4 and global mean monthly temperature series with published datasets reveals similarities and differences due in part to the spatial interpolation method. Notably, the GraphEM-completed HadCRUT4.6 global temperature displays a stronger early twenty-first century warming trend than its uninterpolated counterpart, consistent with recent analyses using other datasets. Known events like the 1877/1878 El Niño are recovered with greater fidelity than with kriging, and result in different assessments of changes in ENSO variability through time. Gaussian Markov random fields provide a more geophysically-motivated way to impute missing values in climate fields, and the associated graph provides a powerful tool to analyze the structure of teleconnection patterns. We close with a discussion of wider applications of Markov random fields in climate science.

Interpretable Engagement Models for MOOCs Using Hinge-Loss Markov Random Fields

2020 ◽  
Vol 13 (1) ◽  
pp. 107-122
Author(s):  
Arti Ramesh ◽  
Dan Goldwasser ◽  
Bert Huang ◽  
Hal Daume ◽  
Lise Getoor
Keyword(s):  
Random Fields ◽  
Markov Random Fields ◽  
Hinge Loss ◽  
Markov Random

scholarly journals Domain prediction with probabilistic directional context

2016 ◽  
Author(s):  
Alejandro Ochoa ◽  
Mona Singh
Keyword(s):  
Random Fields ◽  
Markov Random Fields ◽  
Optimization Problem ◽  
Probabilistic Approach ◽  
Combinatorial Optimization Problem ◽  
Protein Domain ◽  
Markov Random ◽  
Extensive Evaluation ◽  
Domain Prediction ◽  
Improved Performance

AbstractMotivationProtein domain prediction is one of the most powerful approaches for sequence-based function prediction. While domain instances are typically predicted independently of each other, newer approaches have demonstrated improved performance by rewarding domain pairs that frequently co-occur within sequences. However, most of these approaches have ignored the order in which domains preferentially co-occur and have also not modeled domain co-occurrence probabilistically.ResultsWe introduce a probabilistic approach for domain prediction that models “directional” domain context. Our method is the first to score all domain pairs within a sequence while taking their order into account, even for non-sequential domains. We show that our approach extends a previous Markov model-based approach to additionally score all pairwise terms, and that it can be interpreted within the context of Markov random fields. We formulate our underlying combinatorial optimization problem as an integer linear program, and demonstrate that it can be solved quickly in practice. Finally, we perform extensive evaluation of domain context methods and demonstrate that incorporating context increases the number of domain predictions by ∼15%, with our approach dPUC2 (Domain Prediction Using Context) outperforming all competing approaches.AvailabilitydPUC2 is available at http://github.com/alexviiia/dpuc2.

scholarly journals Block Belief Propagation for Parameter Learning in Markov Random Fields

2019 ◽  
Vol 33 ◽  
pp. 4448-4455
Author(s):  
You Lu ◽  
Zhiyuan Liu ◽  
Bert Huang
Keyword(s):  
Graphical Models ◽  
Random Fields ◽  
Markov Random Fields ◽  
Graphical Model ◽  
Belief Propagation ◽  
Traditional Learning ◽  
Training Methods ◽  
Iteration Complexity ◽  
Markov Random ◽  
Standard Training

Traditional learning methods for training Markov random fields require doing inference over all variables to compute the likelihood gradient. The iteration complexity for those methods therefore scales with the size of the graphical models. In this paper, we propose block belief propagation learning (BBPL), which uses block-coordinate updates of approximate marginals to compute approximate gradients, removing the need to compute inference on the entire graphical model. Thus, the iteration complexity of BBPL does not scale with the size of the graphs. We prove that the method converges to the same solution as that obtained by using full inference per iteration, despite these approximations, and we empirically demonstrate its scalability improvements over standard training methods.

Sign in / Sign up

Export Citation Format

Share Document