Boosting Variational Inference With Locally Adaptive Step-Sizes

Variational Inference makes a trade-off between the capacity of the variational family and the tractability of finding an approximate posterior distribution. Instead, Boosting Variational Inference allows practitioners to obtain increasingly good posterior approximations by spending more compute. The main obstacle to widespread adoption of Boosting Variational Inference is the amount of resources necessary to improve over a strong Variational Inference baseline. In our work, we trace this limitation back to the global curvature of the KL-divergence. We characterize how the global curvature impacts time and memory consumption, address the problem with the notion of local curvature, and provide a novel approximate backtracking algorithm for estimating local curvature. We give new theoretical convergence rates for our algorithms and provide experimental validation on synthetic and real-world datasets.

Download Full-text

A Bayesian Latent Variable Model of User Preferences with Item Context

Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2018/370 ◽

2018 ◽

Author(s):

Aghiles Salah ◽

Hady W. Lauw

Keyword(s):

Latent Variable ◽

User Preferences ◽

Variational Inference ◽

Latent Variable Model ◽

Personalized Recommendation ◽

Variable Model ◽

Performance Improvements ◽

Collaborative Context ◽

Real World Datasets ◽

Item Item

Personalized recommendation has proven to be very promising in modeling the preference of users over items. However, most existing work in this context focuses primarily on modeling user-item interactions, which tend to be very sparse. We propose to further leverage the item-item relationships that may reflect various aspects of items that guide users' choices. Intuitively, items that occur within the same "context" (e.g., browsed in the same session, purchased in the same basket) are likely related in some latent aspect. Therefore, accounting for the item's context would complement the sparse user-item interactions by extending a user's preference to other items of similar aspects. To realize this intuition, we develop Collaborative Context Poisson Factorization (C2PF), a new Bayesian latent variable model that seamlessly integrates contextual relationships among items into a personalized recommendation approach. We further derive a scalable variational inference algorithm to fit C2PF to preference data. Empirical results on real-world datasets show evident performance improvements over strong factorization models.

Download Full-text

The globalization theorem for the Curvature-Dimension condition

Inventiones mathematicae ◽

10.1007/s00222-021-01040-6 ◽

2021 ◽

Author(s):

Fabio Cavalletti ◽

Emanuel Milman

Keyword(s):

Measure Space ◽

Optimal Transport ◽

A Priori ◽

Metric Measure Space ◽

Local Curvature ◽

Geodesic Space ◽

Third Order ◽

Change Of Variables ◽

Temporal Derivative ◽

Global Curvature

AbstractThe Lott–Sturm–Villani Curvature-Dimension condition provides a synthetic notion for a metric-measure space to have Ricci-curvature bounded from below and dimension bounded from above. We prove that it is enough to verify this condition locally: an essentially non-branching metric-measure space $$(X,\mathsf {d},{\mathfrak {m}})$$ ( X , d , m ) (so that $$(\text {supp}({\mathfrak {m}}),\mathsf {d})$$ ( supp ( m ) , d ) is a length-space and $${\mathfrak {m}}(X) < \infty $$ m ( X ) < ∞ ) verifying the local Curvature-Dimension condition $${\mathsf {CD}}_{loc}(K,N)$$ CD loc ( K , N ) with parameters $$K \in {\mathbb {R}}$$ K ∈ R and $$N \in (1,\infty )$$ N ∈ ( 1 , ∞ ) , also verifies the global Curvature-Dimension condition $${\mathsf {CD}}(K,N)$$ CD ( K , N ) . In other words, the Curvature-Dimension condition enjoys the globalization (or local-to-global) property, answering a question which had remained open since the beginning of the theory. For the proof, we establish an equivalence between $$L^1$$ L 1 - and $$L^2$$ L 2 -optimal-transport–based interpolation. The challenge is not merely a technical one, and several new conceptual ingredients which are of independent interest are developed: an explicit change-of-variables formula for densities of Wasserstein geodesics depending on a second-order temporal derivative of associated Kantorovich potentials; a surprising third-order theory for the latter Kantorovich potentials, which holds in complete generality on any proper geodesic space; and a certain rigidity property of the change-of-variables formula, allowing us to bootstrap the a-priori available regularity. As a consequence, numerous variants of the Curvature-Dimension condition proposed by various authors throughout the years are shown to, in fact, all be equivalent in the above setting, thereby unifying the theory.

Download Full-text