scholarly journals Learning Rates of lq Coefficient Regularization Learning with Gaussian Kernel

2014 ◽  
Vol 26 (10) ◽  
pp. 2350-2378 ◽  
Author(s):  
Shaobo Lin ◽  
Jinshan Zeng ◽  
Jian Fang ◽  
Zongben Xu
Keyword(s):  
Statistical Learning ◽  
Upper And Lower Bounds ◽  
Gaussian Kernel ◽  
Strong Impact ◽  
Generalization Capability ◽  
Optimal Learning ◽  
Hypothesis Space ◽  
Learning Rates ◽  
Learning Machine ◽  
Powerful Strategy

Regularization is a well-recognized powerful strategy to improve the performance of a learning machine and lq regularization schemes with [Formula: see text] are central in use. It is known that different q leads to different properties of the deduced estimators, say, l2 regularization leads to a smooth estimator, while l1 regularization leads to a sparse estimator. Then how the generalization capability of lq regularization learning varies with q is worthy of investigation. In this letter, we study this problem in the framework of statistical learning theory. Our main results show that implementing lq coefficient regularization schemes in the sample-dependent hypothesis space associated with a gaussian kernel can attain the same almost optimal learning rates for all [Formula: see text]. That is, the upper and lower bounds of learning rates for lq regularization learning are asymptotically identical for all [Formula: see text]. Our finding tentatively reveals that in some modeling contexts, the choice of q might not have a strong impact on the generalization capability. From this perspective, q can be arbitrarily specified, or specified merely by other nongeneralization criteria like smoothness, computational complexity or sparsity.

scholarly journals Learning rates for the risk of kernel-based quantile regression estimators in additive models

2016 ◽  
Vol 14 (03) ◽  
pp. 449-477 ◽  
Author(s):  
Andreas Christmann ◽  
Ding-Xuan Zhou
Keyword(s):  
Hilbert Space ◽  
Quantile Regression ◽  
Reproducing Kernel ◽  
Reproducing Kernel Hilbert Space ◽  
Gaussian Function ◽  
Additive Model ◽  
Additive Models ◽  
Gaussian Kernel ◽  
Optimal Learning ◽  
Learning Rates

Additive models play an important role in semiparametric statistics. This paper gives learning rates for regularized kernel-based methods for additive models. These learning rates compare favorably in particular in high dimensions to recent results on optimal learning rates for purely nonparametric regularized kernel-based quantile regression using the Gaussian radial basis function kernel, provided the assumption of an additive model is valid. Additionally, a concrete example is presented to show that a Gaussian function depending only on one variable lies in a reproducing kernel Hilbert space generated by an additive Gaussian kernel, but does not belong to the reproducing kernel Hilbert space generated by the multivariate Gaussian kernel of the same variance.

scholarly journals Learning Rates for Classification with Gaussian Kernels

2017 ◽  
Vol 29 (12) ◽  
pp. 3353-3380 ◽  
Author(s):  
Shao-Bo Lin ◽  
Jinshan Zeng ◽  
Xiangyu Chang
Keyword(s):  
Binary Classification ◽  
Regression Function ◽  
Learning Rate ◽  
Loss Functions ◽  
Gaussian Kernel ◽  
Support Vector ◽  
Quadratic Loss ◽  
Optimal Learning ◽  
Gaussian Kernels ◽  
Learning Rates

This letter aims at refined error analysis for binary classification using support vector machine (SVM) with gaussian kernel and convex loss. Our first result shows that for some loss functions, such as the truncated quadratic loss and quadratic loss, SVM with gaussian kernel can reach the almost optimal learning rate provided the regression function is smooth. Our second result shows that for a large number of loss functions, under some Tsybakov noise assumption, if the regression function is infinitely smooth, then SVM with gaussian kernel can achieve the learning rate of order [Formula: see text], where [Formula: see text] is the number of samples.

scholarly journals Optimal learning rates for unbiased perception

2010 ◽  
Vol 3 (9) ◽  
pp. 175-175 ◽  
Author(s):  
B. T Backus
Keyword(s):  
Optimal Learning ◽  
Learning Rates

scholarly journals On the High Dimentional Information Processing in Quaternionic Domain and its Applications

2018 ◽  
Vol 7 (2) ◽  
pp. 177
Author(s):  
Sushil Kumar ◽  
Bipin Kumar Tripathi
Keyword(s):  
Wide Spectrum ◽  
Neural System ◽  
High Dimensional ◽  
Generalization Capability ◽  
Phase Information ◽  
High Dimension Data ◽  
Higher Dimensional ◽  
Dimensional Parameters ◽  
Learning Machine ◽  
Communication Control

<p>There are various high dimensional engineering and scientific applications in communication, control, robotics, computer vision, biometrics, etc.; where researchers are facing problem to design an intelligent and robust neural system which can process higher dimensional information efficiently. The conventional real-valued neural networks are tried to solve the problem associated with high dimensional parameters, but the required network structure possesses high complexity and are very time consuming and weak to noise. These networks are also not able to learn magnitude and phase values simultaneously in space.<strong> </strong> The quaternion is the number, which possesses the magnitude in all four directions and phase information is embedded within it. This paper presents a well generalized learning machine with a quaternionic domain neural network that can finely process magnitude and phase information of high dimension data without any hassle. The learning and generalization capability of the proposed learning machine is presented through a wide spectrum of simulations which demonstrate the significance of the work.</p>

scholarly journals Least Square Regularized Regression for Multitask Learning

2013 ◽  
Vol 2013 ◽  
pp. 1-7
Author(s):  
Yong-Li Xu ◽  
Di-Rong Chen ◽  
Han-Xiong Li
Keyword(s):  
Hilbert Space ◽  
Learning Algorithms ◽  
Multitask Learning ◽  
Least Square ◽  
Regularized Regression ◽  
Task Learning ◽  
Hypothesis Space ◽  
Learning Rates ◽  
Prediction Function ◽  
Optimal Function

The study of multitask learning algorithms is one of very important issues. This paper proposes a least-square regularized regression algorithm for multi-task learning with hypothesis space being the union of a sequence of Hilbert spaces. The algorithm consists of two steps of selecting the optimal Hilbert space and searching for the optimal function. We assume that the distributions of different tasks are related to a set of transformations under which any Hilbert space in the hypothesis space is norm invariant. We prove that under the above assumption the optimal prediction function of every task is in the same Hilbert space. Based on this result, a pivotal error decomposition is founded, which can use samples of related tasks to bound excess error of the target task. We obtain an upper bound for the sample error of related tasks, and based on this bound, potential faster learning rates are obtained compared to single-task learning algorithms.

A Study on the Optimal Double Parameters for Steepest Descent with Momentum

2015 ◽  
Vol 27 (4) ◽  
pp. 982-1004 ◽  
Author(s):  
Naimin Zhang
Keyword(s):  
Stability Analysis ◽  
Steepest Descent ◽  
Quadratic Functions ◽  
Optimal Parameters ◽  
Optimal Learning ◽  
Descent Algorithms ◽  
Learning Rates ◽  
Global Optimal ◽  
The Stability

This letter presents the stability analysis for two steepest descent algorithms with momentum for quadratic functions. The corresponding local optimal parameters in Torii and Hagan ( 2002 ) and Zhang ( 2013 ) are extended to the global optimal parameters, that is, both the optimal learning rates and the optimal momentum factors are obtained simultaneously which make for the fastest convergence.

Sign in / Sign up

Export Citation Format

Share Document