scholarly journals PAC learning, VC dimension, and the arithmetic hierarchy

2015 ◽  
Vol 54 (7-8) ◽  
pp. 871-883
Author(s):  
Wesley Calvert
Keyword(s):  
Pac Learning ◽  
Vc Dimension ◽  
Arithmetic Hierarchy

scholarly journals On the Complexity of Learning a Class Ratio from Unlabeled Data

2020 ◽  
Vol 69 ◽  
Author(s):  
Benjamin Fish ◽  
Lev Reyzin
Keyword(s):  
Computational Complexity ◽  
Unlabeled Data ◽  
Training Data ◽  
Pac Learning ◽  
Vc Dimension ◽  
Standard Set

In the problem of learning a class ratio from unlabeled data, which we call CR learning, the training data is unlabeled, and only the ratios, or proportions, of examples receiving each label are given. The goal is to learn a hypothesis that predicts the proportions of labels on the distribution underlying the sample. This model of learning is applicable to a wide variety of settings, including predicting the number of votes for candidates in political elections from polls. In this paper, we formally define this class and resolve foundational questions regarding the computational complexity of CR learning and characterize its relationship to PAC learning. Among our results, we show, perhaps surprisingly, that for finite VC classes what can be efficiently CR learned is a strict subset of what can be learned efficiently in PAC, under standard complexity assumptions. We also show that there exist classes of functions whose CR learnability is independent of ZFC, the standard set theoretic axioms. This implies that CR learning cannot be easily characterized (like PAC by VC dimension).

The VC Dimension and Pseudodimension of Two-Layer Neural Networks with Discrete Inputs

1996 ◽  
Vol 8 (3) ◽  
pp. 625-628 ◽  
Author(s):  
Peter L. Bartlett ◽  
Robert C. Williamson
Keyword(s):  
Neural Networks ◽  
Basis Function ◽  
Upper Bounds ◽  
Pac Learning ◽  
Learning Performance ◽  
Sigmoid Function ◽  
Vc Dimension ◽  
Learning Framework ◽  
Probably Approximately Correct ◽  
Training Examples

We give upper bounds on the Vapnik-Chervonenkis dimension and pseudodimension of two-layer neural networks that use the standard sigmoid function or radial basis function and have inputs from {−D, …,D}n. In Valiant's probably approximately correct (pac) learning framework for pattern classification, and in Haussler's generalization of this framework to nonlinear regression, the results imply that the number of training examples necessary for satisfactory learning performance grows no more rapidly than W log (WD), where W is the number of weights. The previous best bound for these networks was O(W4).

scholarly journals On the Complexity of Learning from Label Proportions

2017 ◽  
Author(s):  
Benjamin Fish ◽  
Lev Reyzin
Keyword(s):  
Computational Complexity ◽  
Training Data ◽  
Pac Learning ◽  
Vc Dimension ◽  
Simple Version ◽  
Learning From Label Proportions

In the problem of learning with label proportions (also known as the problem of estimating class ratios), the training data is unlabeled, and only the proportions of examples receiving each label are given. The goal is to learn a hypothesis that predicts the proportions of labels on the distribution underlying the sample. This model of learning is useful in a wide variety of settings, including predicting the number of votes for candidates in political elections from polls. In this paper, we resolve foundational questions regarding the computational complexity of learning in this setting. We formalize a simple version of the setting, and we compare the computational complexity of learning in this model to classical PAC learning. Perhaps surprisingly, we show that what can be learned efficiently in this model is a strict subset of what may be leaned efficiently in PAC, under standard complexity assumptions. We give a characterization in terms of VC dimension, and we show that there are non-trivial problems in this model that can be efficiently learned. We also give an algorithm that demonstrates the feasibility of learning under well-behaved distributions.

scholarly journals The VC Dimension of Metric Balls under Fréchet and Hausdorff Distances

2021 ◽  
Author(s):  
Anne Driemel ◽  
André Nusser ◽  
Jeff M. Phillips ◽  
Ioannis Psarros
Keyword(s):  
Density Estimation ◽  
Lower Bounds ◽  
Upper And Lower Bounds ◽  
Similarity Metrics ◽  
Vc Dimension ◽  
Set Systems ◽  
Polygonal Curves ◽  
The Individual ◽  
Curve Similarity ◽  
Metric Balls

AbstractThe Vapnik–Chervonenkis dimension provides a notion of complexity for systems of sets. If the VC dimension is small, then knowing this can drastically simplify fundamental computational tasks such as classification, range counting, and density estimation through the use of sampling bounds. We analyze set systems where the ground set X is a set of polygonal curves in $$\mathbb {R}^d$$ R d and the sets $$\mathcal {R}$$ R are metric balls defined by curve similarity metrics, such as the Fréchet distance and the Hausdorff distance, as well as their discrete counterparts. We derive upper and lower bounds on the VC dimension that imply useful sampling bounds in the setting that the number of curves is large, but the complexity of the individual curves is small. Our upper and lower bounds are either near-quadratic or near-linear in the complexity of the curves that define the ranges and they are logarithmic in the complexity of the curves that define the ground set.

WHEN CHURCH-ROSSER BECOMES CONTEXT FREE

2007 ◽  
Vol 18 (06) ◽  
pp. 1293-1302 ◽  
Author(s):  
MARTIN KUTRIB ◽  
ANDREAS MALCHER
Keyword(s):  
Linear Time ◽  
Membership Problem ◽  
Closure Properties ◽  
Context Free Language ◽  
Arithmetic Hierarchy ◽  
Context Free ◽  
Free Language

We investigate the intersection of Church-Rosser languages and (strongly) context-free languages. The intersection is still a proper superset of the deterministic context-free languages as well as of their reversals, while its membership problem is solvable in linear time. For the problem whether a given Church-Rosser or context-free language belongs to the intersection we show completeness for the second level of the arithmetic hierarchy. The equivalence of Church-Rosser and context-free languages is Π1-complete. It is proved that all considered intersections are pairwise incomparable. Finally, closure properties under several operations are investigated.

Neural Nets with Superlinear VC-Dimension

ICANN ’94 ◽  
1994 ◽  
pp. 581-584 ◽  
Author(s):  
Wolfgang Maass
Keyword(s):  
Neural Nets ◽  
Vc Dimension
Sign in / Sign up

Export Citation Format

Share Document