scholarly journals Learning Rates of Least-Square Regularized Regression

2005 ◽  
Vol 6 (2) ◽  
pp. 171-192 ◽  
Author(s):  
Qiang Wu ◽  
Yiming Ying ◽  
Ding-Xuan Zhou
Keyword(s):  
Least Square ◽  
Regularized Regression ◽  
Learning Rates

scholarly journals Least Square Regularized Regression for Multitask Learning

2013 ◽  
Vol 2013 ◽  
pp. 1-7
Author(s):  
Yong-Li Xu ◽  
Di-Rong Chen ◽  
Han-Xiong Li
Keyword(s):  
Hilbert Space ◽  
Learning Algorithms ◽  
Multitask Learning ◽  
Least Square ◽  
Regularized Regression ◽  
Task Learning ◽  
Hypothesis Space ◽  
Learning Rates ◽  
Prediction Function ◽  
Optimal Function

The study of multitask learning algorithms is one of very important issues. This paper proposes a least-square regularized regression algorithm for multi-task learning with hypothesis space being the union of a sequence of Hilbert spaces. The algorithm consists of two steps of selecting the optimal Hilbert space and searching for the optimal function. We assume that the distributions of different tasks are related to a set of transformations under which any Hilbert space in the hypothesis space is norm invariant. We prove that under the above assumption the optimal prediction function of every task is in the same Hilbert space. Based on this result, a pivotal error decomposition is founded, which can use samples of related tasks to bound excess error of the target task. We obtain an upper bound for the sample error of related tasks, and based on this bound, potential faster learning rates are obtained compared to single-task learning algorithms.

LEARNING RATES OF REGULARIZED REGRESSION FOR FUNCTIONAL DATA

2009 ◽  
Vol 07 (06) ◽  
pp. 839-850 ◽  
Author(s):  
YONG-LI XU ◽  
DI-RONG CHEN
Keyword(s):  
Functional Data Analysis ◽  
Functional Data ◽  
Input Data ◽  
Reproducing Kernel ◽  
Reproducing Kernel Hilbert Space ◽  
Natural Extension ◽  
Least Square ◽  
Regularized Regression ◽  
Fast Learning ◽  
Learning Rates

The study of regularized learning algorithms is a very important issue and functional data analysis extends classical methods. We establish the learning rates of the least square regularized regression algorithm in reproducing kernel Hilbert space for functional data. With the iteration method, we obtain fast learning rate for functional data. Our result is a natural extension for least square regularized regression algorithm when the dimension of input data is finite.

LEAST SQUARE REGRESSION WITH COEFFICIENT REGULARIZATION BY GRADIENT DESCENT

2012 ◽  
Vol 10 (01) ◽  
pp. 1250005 ◽  
Author(s):  
JUAN HUANG ◽  
HONG CHEN ◽  
LUOQING LI
Keyword(s):  
Integral Operator ◽  
Explicit Expression ◽  
Gradient Descent ◽  
Stochastic Gradient ◽  
Least Square ◽  
Stochastic Gradient Descent ◽  
Least Square Regression ◽  
Learning Rates ◽  
Gradient Descent Algorithm ◽  
Regularization Parameters

We propose a stochastic gradient descent algorithm for the least square regression with coefficient regularization. An explicit expression of the solution via sampling operator and empirical integral operator is derived. Learning rates are given in terms of the suitable choices of the step sizes and regularization parameters.

REGULARIZED LEAST SQUARE REGRESSION WITH SPHERICAL POLYNOMIAL KERNELS

2009 ◽  
Vol 07 (06) ◽  
pp. 781-801 ◽  
Author(s):  
LUOQING LI
Keyword(s):  
Theoretical Analysis ◽  
Integral Operators ◽  
Excess Risk ◽  
Least Square ◽  
Least Square Regression ◽  
Learning Rates ◽  
Explicit Bounds ◽  
Spherical Polynomial ◽  
Polynomial Kernels

This article considers regularized least square regression on the sphere. It develops a theoretical analysis of the generalization performances of regularized least square regression algorithm with spherical polynomial kernels. The explicit bounds are derived for the excess risk error. The learning rates depend on the eigenvalues of spherical polynomial integral operators and on the dimension of spherical polynomial spaces.

Sign in / Sign up

Export Citation Format

Share Document