Generalization Error Bounds Using Unlabeled Data

2005 ◽  
pp. 127-142 ◽  
Author(s):  
Matti Kääriäinen
Keyword(s):  
Error Bounds ◽  
Unlabeled Data ◽  
Generalization Error

scholarly journals Wasserstein Soft Label Propagation on Hypergraphs: Algorithm and Generalization Error Bounds

2019 ◽  
Vol 33 ◽  
pp. 3630-3637
Author(s):  
Tingran Gao ◽  
Shahab Asoodeh ◽  
Yi Huang ◽  
James Evans
Keyword(s):  
Error Bounds ◽  
Message Passing ◽  
Learning Algorithm ◽  
Probability Distributions ◽  
Wasserstein Distance ◽  
Semisupervised Learning ◽  
Label Propagation ◽  
Pac Learning ◽  
Generalization Error ◽  
Data Mining Algorithms

Inspired by recent interests of developing machine learning and data mining algorithms on hypergraphs, we investigate in this paper the semi-supervised learning algorithm of propagating ”soft labels” (e.g. probability distributions, class membership scores) over hypergraphs, by means of optimal transportation. Borrowing insights from Wasserstein propagation on graphs [Solomon et al. 2014], we re-formulate the label propagation procedure as a message-passing algorithm, which renders itself naturally to a generalization applicable to hypergraphs through Wasserstein barycenters. Furthermore, in a PAC learning framework, we provide generalization error bounds for propagating one-dimensional distributions on graphs and hypergraphs using 2-Wasserstein distance, by establishing the algorithmic stability of the proposed semisupervised learning algorithm. These theoretical results also shed new lights upon deeper understandings of the Wasserstein propagation on graphs.

scholarly journals Generalization Error Bounds for Kernel Matrix Completion and Extrapolation

2020 ◽  
Vol 27 ◽  
pp. 326-330 ◽  
Author(s):  
Pere Gimenez-Febrer ◽  
Alba Pages-Zamora ◽  
Georgios B. Giannakis
Keyword(s):  
Error Bounds ◽  
Matrix Completion ◽  
Kernel Matrix ◽  
Generalization Error

scholarly journals Generalization Error Bounds of Gradient Descent for Learning Over-Parameterized Deep ReLU Networks

2020 ◽  
Vol 34 (04) ◽  
pp. 3349-3356
Author(s):  
Yuan Cao ◽  
Quanquan Gu
Keyword(s):  
Neural Networks ◽  
Error Bounds ◽  
Gradient Descent ◽  
Deep Neural Networks ◽  
Empirical Studies ◽  
Training Data ◽  
Generalization Error ◽  
Generalization Performance ◽  
Gradient Based ◽  
Random Initialization

Empirical studies show that gradient-based methods can learn deep neural networks (DNNs) with very good generalization performance in the over-parameterization regime, where DNNs can easily fit a random labeling of the training data. Very recently, a line of work explains in theory that with over-parameterization and proper random initialization, gradient-based methods can find the global minima of the training loss for DNNs. However, existing generalization error bounds are unable to explain the good generalization performance of over-parameterized DNNs. The major limitation of most existing generalization bounds is that they are based on uniform convergence and are independent of the training algorithm. In this work, we derive an algorithm-dependent generalization error bound for deep ReLU networks, and show that under certain assumptions on the data distribution, gradient descent (GD) with proper random initialization is able to train a sufficiently over-parameterized DNN to achieve arbitrarily small generalization error. Our work sheds light on explaining the good generalization performance of over-parameterized deep neural networks.

scholarly journals Generalization Error Bounds Via Rényi-, f-Divergences and Maximal Leakage

2021 ◽  
pp. 1-1
Author(s):  
Amedeo Roberto Esposito ◽  
Michael Gastpar ◽  
Ibrahim Issa
Keyword(s):  
Error Bounds ◽  
Generalization Error
Sign in / Sign up

Export Citation Format

Share Document