Almost Linear VC-Dimension Bounds for Piecewise Polynomial Networks

1998 ◽  
Vol 10 (8) ◽  
pp. 2159-2173 ◽  
Author(s):  
Peter L. Bartlett ◽  
Vitaly Maiorov ◽  
Ron Meir
Keyword(s):  
Approximation Error ◽  
Feedforward Neural Networks ◽  
Error Rates ◽  
Upper And Lower Bounds ◽  
Regression Estimation ◽  
Piecewise Polynomial ◽  
Activation Functions ◽  
Vc Dimension ◽  
Number Of Layers ◽  
Polynomial Activation Functions

We compute upper and lower bounds on the VC dimension and pseudodimension of feedforward neural networks composed of piecewise polynomial activation functions. We show that if the number of layers is fixed, then the VC dimension and pseudo-dimension grow as W log W, where W is the number of parameters in the network. This result stands in opposition to the case where the number of layers is unbounded, in which case the VC dimension and pseudo-dimension grow as W2. We combine our results with recently established approximation error rates and determine error bounds for the problem of regression estimation by piecewise polynomial networks with unbounded weights.

Piecewise Polynomial Activation Functions for Feedforward Neural Networks

2019 ◽  
Vol 50 (1) ◽  
pp. 121-147 ◽  
Author(s):  
Ezequiel López-Rubio ◽  
Francisco Ortega-Zamorano ◽  
Enrique Domínguez ◽  
José Muñoz-Pérez
Keyword(s):  
Neural Networks ◽  
Feedforward Neural Networks ◽  
Piecewise Polynomial ◽  
Activation Functions ◽  
Polynomial Activation Functions

scholarly journals The VC Dimension of Metric Balls under Fréchet and Hausdorff Distances

2021 ◽  
Author(s):  
Anne Driemel ◽  
André Nusser ◽  
Jeff M. Phillips ◽  
Ioannis Psarros
Keyword(s):  
Density Estimation ◽  
Lower Bounds ◽  
Upper And Lower Bounds ◽  
Similarity Metrics ◽  
Vc Dimension ◽  
Set Systems ◽  
Polygonal Curves ◽  
The Individual ◽  
Curve Similarity ◽  
Metric Balls

AbstractThe Vapnik–Chervonenkis dimension provides a notion of complexity for systems of sets. If the VC dimension is small, then knowing this can drastically simplify fundamental computational tasks such as classification, range counting, and density estimation through the use of sampling bounds. We analyze set systems where the ground set X is a set of polygonal curves in $$\mathbb {R}^d$$ R d and the sets $$\mathcal {R}$$ R are metric balls defined by curve similarity metrics, such as the Fréchet distance and the Hausdorff distance, as well as their discrete counterparts. We derive upper and lower bounds on the VC dimension that imply useful sampling bounds in the setting that the number of curves is large, but the complexity of the individual curves is small. Our upper and lower bounds are either near-quadratic or near-linear in the complexity of the curves that define the ranges and they are logarithmic in the complexity of the curves that define the ground set.

ACCELERATING TRAINING OF FEEDFORWARD NEURAL NETWORKS

1994 ◽  
Vol 03 (03) ◽  
pp. 339-348
Author(s):  
CARL G. LOONEY
Keyword(s):  
Neural Networks ◽  
Local Minimum ◽  
Random Search ◽  
Feedforward Neural Networks ◽  
Conjugate Gradients ◽  
Activation Functions ◽  
Problematic Behavior ◽  
Methods And Techniques ◽  
Adaptive Step ◽  
Quality Learning

We review methods and techniques for training feedforward neural networks that avoid problematic behavior, accelerate the convergence, and verify the training. Adaptive step gain, bipolar activation functions, and conjugate gradients are powerful stabilizers. Random search techniques circumvent the local minimum trap and avoid specialization due to overtraining. Testing assures quality learning.

scholarly journals Bounds on the rate of convergence for one class of inhomogeneous Markovian queueing models with possible batch arrivals and services

2018 ◽  
Vol 28 (1) ◽  
pp. 141-154 ◽  
Author(s):  
Alexander Zeifman ◽  
Rostislav Razumchik ◽  
Yacov Satin ◽  
Ksenia Kiseleva ◽  
Anna Korotysheva ◽  
...  
Keyword(s):  
Rate Of Convergence ◽  
Queueing Systems ◽  
Approximation Error ◽  
Linear Operators ◽  
Upper And Lower Bounds ◽  
Logarithmic Norm ◽  
Batch Arrivals ◽  
State Dependent ◽  
The Mean ◽  
Number Of Customers

AbstractIn this paper we present a method for the computation of convergence bounds for four classes of multiserver queueing systems, described by inhomogeneous Markov chains. Specifically, we consider an inhomogeneous M/M/S queueing system with possible state-dependent arrival and service intensities, and additionally possible batch arrivals and batch service. A unified approach based on a logarithmic norm of linear operators for obtaining sharp upper and lower bounds on the rate of convergence and corresponding sharp perturbation bounds is described. As a side effect, we show, by virtue of numerical examples, that the approach based on a logarithmic norm can also be used to approximate limiting characteristics (the idle probability and the mean number of customers in the system) of the systems considered with a given approximation error.

scholarly journals Visual Modeling of Data using Convolutional Neural Networks

2019 ◽  
Vol 9 (1) ◽  
pp. 4938-4942
Keyword(s):  
Artificial Intelligence ◽  
Neural Networks ◽  
Computer Vision ◽  
Recurrent Neural Networks ◽  
Feedforward Neural Networks ◽  
Correct Choice ◽  
Activation Functions ◽  
Visual Modeling ◽  
Training Images ◽  
Canadian Institute

Artificial Intelligence has been showing monumental growth in filling the gap between the capabilities of humans and machines. Researchers and scientists work on many aspects to make new things happen. Computer Vision is one of them. To make the system to visualize, neural networks are used. Some of the well-known Neural Networks include CNN, Feedforward Neural Networks (FNN), and Recurrent Neural Networks (RNN) and so on. Among them, CNN is the correct choice for computer vision because they learn relevant features from an image or video similar to the human brain. In this paper, the dataset used is CIFAR-10 (Canadian Institute for Advanced Research) which contains 60,000 images in the size of 32x32. Those images are divided into 10 different classes which contains both training and testing images. The training images are 50,000 and testing images are 10,000. The ten different classes contain airplanes, automobiles, birds, cat, ship, truck, deer, dog, frog and horse images. This paper was mainly concentrated on improving performance using normalization layers and comparing the accuracy achieved using different activation functions like ReLU and Tanh.

Sign in / Sign up

Export Citation Format

Share Document