scholarly journals General-Purpose Bayesian Tensor Learning With Automatic Rank Determination and Uncertainty Quantification

2022 ◽  
Vol 4 ◽  
Author(s):  
Kaiqi Zhang ◽  
Cole Hawkins ◽  
Zheng Zhang

A major challenge in many machine learning tasks is that the model expressive power depends on model size. Low-rank tensor methods are an efficient tool for handling the curse of dimensionality in many large-scale machine learning models. The major challenges in training a tensor learning model include how to process the high-volume data, how to determine the tensor rank automatically, and how to estimate the uncertainty of the results. While existing tensor learning focuses on a specific task, this paper proposes a generic Bayesian framework that can be employed to solve a broad class of tensor learning problems such as tensor completion, tensor regression, and tensorized neural networks. We develop a low-rank tensor prior for automatic rank determination in nonlinear problems. Our method is implemented with both stochastic gradient Hamiltonian Monte Carlo (SGHMC) and Stein Variational Gradient Descent (SVGD). We compare the automatic rank determination and uncertainty quantification of these two solvers. We demonstrate that our proposed method can determine the tensor rank automatically and can quantify the uncertainty of the obtained results. We validate our framework on tensor completion tasks and tensorized neural network training tasks.

2018 ◽  
Vol 2018 ◽  
pp. 1-11 ◽  
Author(s):  
Jinzhi Liao ◽  
Jiuyang Tang ◽  
Xiang Zhao ◽  
Haichuan Shang

POI recommendation finds significant importance in various real-life applications, especially when meeting with location-based services, e.g., check-ins social networks. In this paper, we propose to solve POI recommendation through a novel model of dynamic tensor, which is among the first triumphs of its kind. In order to carry out timely recommendation, we predict POI by utilizing a completion algorithm based on fast low-rank tensor. Particularly, the dynamic tensor structure is complemented by the fast low-rank tensor completion algorithm so as to achieve prediction with better performance, where the parameter optimization is achieved by a pigeon-inspired heuristic algorithm. In short, our POI recommendation via the dynamic tensor method can take advantage of the intrinsic characteristics of check-ins data due to the multimode features such as current categories, subsequent categories, and temporal information as well as seasons variations are all integrated into the model. Extensive experiment results not only validate the superiority of our proposed method but also imply the application prospect in large-scale and real-time POI recommendation environment.


2021 ◽  
Author(s):  
Changxiao Cai ◽  
Gen Li ◽  
H. Vincent Poor ◽  
Yuxin Chen

This paper investigates a problem of broad practical interest, namely, the reconstruction of a large-dimensional low-rank tensor from highly incomplete and randomly corrupted observations of its entries. Although a number of papers have been dedicated to this tensor completion problem, prior algorithms either are computationally too expensive for large-scale applications or come with suboptimal statistical performance. Motivated by this, we propose a fast two-stage nonconvex algorithm—a gradient method following a rough initialization—that achieves the best of both worlds: optimal statistical accuracy and computational efficiency. Specifically, the proposed algorithm provably completes the tensor and retrieves all low-rank factors within nearly linear time, while at the same time enjoying near-optimal statistical guarantees (i.e., minimal sample complexity and optimal estimation accuracy). The insights conveyed through our analysis of nonconvex optimization might have implications for a broader family of tensor reconstruction problems beyond tensor completion.


Author(s):  
Tianheng Zhang ◽  
Jianli Zhao ◽  
Qiuxia Sun ◽  
Bin Zhang ◽  
Jianjian Chen ◽  
...  

2019 ◽  
Vol 73 ◽  
pp. 62-69 ◽  
Author(s):  
Wen-Hao Xu ◽  
Xi-Le Zhao ◽  
Teng-Yu Ji ◽  
Jia-Qing Miao ◽  
Tian-Hui Ma ◽  
...  

Author(s):  
Mikhail Krechetov ◽  
Jakub Marecek ◽  
Yury Maximov ◽  
Martin Takac

Low-rank methods for semi-definite programming (SDP) have gained a lot of interest recently, especially in machine learning applications. Their analysis often involves determinant-based or Schatten-norm penalties, which are difficult to implement in practice due to high computational efforts. In this paper, we propose Entropy-Penalized Semi-Definite Programming (EP-SDP), which provides a unified framework for a broad class of penalty functions used in practice to promote a low-rank solution. We show that EP-SDP problems admit an efficient numerical algorithm, having (almost) linear time complexity of the gradient computation; this makes it useful for many machine learning and optimization problems. We illustrate the practical efficiency of our approach on several combinatorial optimization and machine learning problems.


Author(s):  
Jize Xue ◽  
Yongqiang Zhao ◽  
Shaoguang Huang ◽  
Wenzhi Liao ◽  
Jonathan Cheung-Wai Chan ◽  
...  

2020 ◽  
Vol 31 (11) ◽  
pp. 4567-4581 ◽  
Author(s):  
Jize Xue ◽  
Yongqiang Zhao ◽  
Wenzhi Liao ◽  
Jonathan Cheung-Wai Chan ◽  
Seong G. Kong

Sign in / Sign up

Export Citation Format

Share Document