Neural Nets with Superlinear VC-Dimension

Neural Computation ◽

10.1162/neco.1994.6.5.877 ◽

1994 ◽

Vol 6 (5) ◽

pp. 877-884 ◽

Cited By ~ 18

Author(s):

Wolfgang Maass

Keyword(s):

Lower Bound ◽

Large Class ◽

Neural Nets ◽

New Method ◽

Neural Net ◽

Vc Dimension ◽

Asymptotically Optimal ◽

Linear Threshold

It has been known for quite a while that the Vapnik-Chervonenkis dimension (VC-dimension) of a feedforward neural net with linear threshold gates is at most O(w · log w), where w is the total number of weights in the neural net. We show in this paper that this bound is in fact asymptotically optimal. More precisely, we exhibit for any depth d ≥ 3 a large class of feedforward neural nets of depth d with w weights that have VC-dimension Ω(w · log w). This lower bound holds even if the inputs are restricted to Boolean values. The proof of this result relies on a new method that allows us to encode more “program-bits” in the weights of a neural net than previously thought possible.

Download Full-text

Vapnik-Chervonenkis Generalization Bounds for Real Valued Neural Networks

Neural Computation ◽

10.1162/neco.1996.8.6.1277 ◽

1996 ◽

Vol 8 (6) ◽

pp. 1277-1299 ◽

Cited By ~ 6

Author(s):

Arne Hole

Keyword(s):

Neural Networks ◽

Uniform Convergence ◽

Lower Bounds ◽

Neural Nets ◽

Vc Dimension ◽

Generalization Ability ◽

Generalization Bounds

We show how lower bounds on the generalization ability of feedforward neural nets with real outputs can be derived within a formalism based directly on the concept of VC dimension and Vapnik's theorem on uniform convergence of estimated probabilities.

Download Full-text

On the Effect of Analog Noise in Discrete-Time Analog Computations

Neural Computation ◽

10.1162/089976698300017359 ◽

1998 ◽

Vol 10 (5) ◽

pp. 1071-1095 ◽

Cited By ~ 53

Author(s):

Wolfgang Maass ◽

Pekka Orponen

Keyword(s):

Discrete Time ◽

Computational Models ◽

Finite Automata ◽

Classical Model ◽

Spiking Neurons ◽

Neural Nets ◽

Analog Computation ◽

Vc Dimension ◽

Digital Computation ◽

New Type

We introduce a model for analog computation with discrete time in the presence of analog noise that is flexible enough to cover the most important concrete cases, such as noisy analog neural nets and networks of spiking neurons. This model subsumes the classical model for digital computation in the presence of noise. We show that the presence of arbitrarily small amounts of analog noise reduces the power of analog computational models to that of finite automata, and we also prove a new type of upper bound for the VC-dimension of computational models with analog noise.

Download Full-text

Advanced Statistical Tools for Improving Yield and Reliability

ISTFA 1999: Conference Proceedings from the 25th International Symposium for Testing and Failure Analysis ◽

10.31399/asm.cp.istfa1999p0233 ◽

1999 ◽

Author(s):

Richard C. Kittler

Keyword(s):

Data Mining ◽

Analysis Data ◽

Neural Nets ◽

Future Trends ◽

Statistical Tools ◽

Mining Technology ◽

Explanatory Variables ◽

Data Coverage ◽

Statistical Approaches ◽

Adequate Data

Abstract Analysis of manufacturing data as a tool for failure analysts often meets with roadblocks due to the complex non-linear behaviors of the relationships between failure rates and explanatory variables drawn from process history. The current work describes how the use of a comprehensive engineering database and data mining technology over-comes some of these difficulties and enables new classes of problems to be solved. The characteristics of the database design necessary for adequate data coverage and unit traceability are discussed. Data mining technology is explained and contrasted with traditional statistical approaches as well as those of expert systems, neural nets, and signature analysis. Data mining is applied to a number of common problem scenarios. Finally, future trends in data mining technology relevant to failure analysis are discussed.

Download Full-text