Coefficient-Based Regression with Non-Identical Unbounded Sampling

We investigate a coefficient-based least squares regression problem with indefinite kernels from non-identical unbounded sampling processes. Here non-identical unbounded sampling means the samples are drawn independently but not identically from unbounded sampling processes. The kernel is not necessarily symmetric or positive semi-definite. This leads to additional difficulty in the error analysis. By introducing a suitable reproducing kernel Hilbert space (RKHS) and a suitable intermediate integral operator, elaborate analysis is presented by means of a novel technique for the sample error. This leads to satisfactory results.

Download Full-text

Kernel PLS Regression II: Kernel Partial Least Squares Regression by Projecting Both Independent and Dependent Variables into Reproducing Kernel Hilbert Space

2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC) ◽

10.1109/smc.2018.00350 ◽

2018 ◽

Author(s):

Yan Pei

Keyword(s):

Hilbert Space ◽

Least Squares ◽

Partial Least Squares ◽

Partial Least Squares Regression ◽

Reproducing Kernel ◽

Reproducing Kernel Hilbert Space ◽

Pls Regression ◽

Least Squares Regression ◽

Kernel Partial Least Squares ◽

Dependent Variables

Download Full-text

Orthogonal least squares for radial basis function network in reproducing Kernel Hilbert space

IFAC Proceedings Volumes ◽

10.1016/s1474-6670(17)31576-8 ◽

2004 ◽

Vol 37 (12) ◽

pp. 847-852

Author(s):

Chooleewan Dachapak ◽

Shunshoku Kanae ◽

Zi-Jiang Yang ◽

Kiyoshi Wada

Keyword(s):

Hilbert Space ◽

Radial Basis Function ◽

Least Squares ◽

Basis Function ◽

Reproducing Kernel ◽

Reproducing Kernel Hilbert Space ◽

Radial Basis Function Network ◽

Orthogonal Least Squares ◽

Radial Basis ◽

Function Network

Download Full-text

Online regularized pairwise learning with least squares loss

Analysis and Applications ◽

10.1142/s0219530519410070 ◽

2019 ◽

Vol 18 (01) ◽

pp. 49-78 ◽

Cited By ~ 1

Author(s):

Cheng Wang ◽

Ting Hu

Keyword(s):

Hilbert Space ◽

Least Squares ◽

Online Algorithm ◽

Reproducing Kernel ◽

Sharp Estimate ◽

Reproducing Kernel Hilbert Space ◽

Learning Sequence ◽

Pairwise Learning ◽

Regularization Parameters

In this paper, we study online algorithm for pairwise problems generated from the Tikhonov regularization scheme associated with the least squares loss function and a reproducing kernel Hilbert space (RKHS). This work establishes the convergence for the last iterate of the online pairwise algorithm with the polynomially decaying step sizes and varying regularization parameters. We show that the obtained error rate in [Formula: see text]-norm can be nearly optimal in the minimax sense under some mild conditions. Our analysis is achieved by a sharp estimate for the norms of the learning sequence and the characterization of RKHS using its associated integral operators and probability inequalities for random variables with values in a Hilbert space.

Download Full-text

Error analysis of reproducing kernel Hilbert space method for solving functional integral equations

Journal of Computational and Applied Mathematics ◽

10.1016/j.cam.2016.01.008 ◽

2016 ◽

Vol 300 ◽

pp. 300-311 ◽

Cited By ~ 12

Author(s):

Esmail Babolian ◽

Shahnam Javadi ◽

Eslam Moradi

Keyword(s):

Hilbert Space ◽

Error Analysis ◽

Integral Equations ◽

Reproducing Kernel ◽

Reproducing Kernel Hilbert Space ◽

Functional Integral ◽

Functional Integral Equations ◽

Space Method

Download Full-text

Stability and error analysis of the reproducing kernel Hilbert space method for the solution of weakly singular Volterra integral equation on graded mesh

Applied Numerical Mathematics ◽

10.1016/j.apnum.2017.05.010 ◽

2017 ◽

Vol 120 ◽

pp. 197-214 ◽

Cited By ~ 7

Author(s):

Hossein Beyrami ◽

Taher Lotfi ◽

Katayoun Mahdiani

Keyword(s):

Hilbert Space ◽

Integral Equation ◽

Error Analysis ◽

Volterra Integral Equation ◽

Reproducing Kernel ◽

Reproducing Kernel Hilbert Space ◽

Graded Mesh ◽

Weakly Singular ◽

Stability And Error Analysis ◽

Space Method

Download Full-text

Constructive Analysis for Least Squares Regression with GeneralizedK-Norm Regularization

Abstract and Applied Analysis ◽

10.1155/2014/458459 ◽

2014 ◽

Vol 2014 ◽

pp. 1-7 ◽

Cited By ~ 1

Author(s):

Cheng Wang ◽

Weilin Nie

Keyword(s):

Integral Operator ◽

Least Squares ◽

Learning Rate ◽

Spectral Theorem ◽

Least Squares Regression ◽

Constructive Approach ◽

Projection Technique ◽

Sample Error ◽

Regularization Error ◽

Error Decomposition

We introduce a constructive approach for the least squares algorithms with generalizedK-norm regularization. Different from the previous studies, a stepping-stone function is constructed with some adjustable parameters in error decomposition. It makes the analysis flexible and may be extended to other algorithms. Based on projection technique for sample error and spectral theorem for integral operator in regularization error, we finally derive a learning rate.

Download Full-text

Convergence rates of Kernel Conjugate Gradient for random design regression

Analysis and Applications ◽

10.1142/s0219530516400017 ◽

2016 ◽

Vol 14 (06) ◽

pp. 763-794 ◽

Cited By ~ 7

Author(s):

Gilles Blanchard ◽

Nicole Krämer

Keyword(s):

Least Squares ◽

Conjugate Gradient ◽

Reproducing Kernel ◽

Convergence Rates ◽

Reproducing Kernel Hilbert Space ◽

Regression Function ◽

Rates Of Convergence ◽

Least Squares Regression ◽

Hilbert Norm ◽

Effective Dimensionality

We prove statistical rates of convergence for kernel-based least squares regression from i.i.d. data using a conjugate gradient (CG) algorithm, where regularization against overfitting is obtained by early stopping. This method is related to Kernel Partial Least Squares, a regression method that combines supervised dimensionality reduction with least squares projection. Following the setting introduced in earlier related literature, we study so-called “fast convergence rates” depending on the regularity of the target regression function (measured by a source condition in terms of the kernel integral operator) and on the effective dimensionality of the data mapped into the kernel space. We obtain upper bounds, essentially matching known minimax lower bounds, for the ℒ2 (prediction) norm as well as for the stronger Hilbert norm, if the true regression function belongs to the reproducing kernel Hilbert space. If the latter assumption is not fulfilled, we obtain similar convergence rates for appropriate norms, provided additional unlabeled data are available.

Download Full-text

Distributed least squares prediction for functional linear regression

Inverse Problems ◽

10.1088/1361-6420/ac4153 ◽

2021 ◽

Author(s):

Hongzhi Tong

Keyword(s):

Least Squares ◽

Reproducing Kernel ◽

Learning Strategy ◽

Reproducing Kernel Hilbert Space ◽

Target Function ◽

Regularity Conditions ◽

Least Squares Regression ◽

Functional Linear Model ◽

Data Set ◽

Functional Linear Regression

Abstract To cope with the challenges of memory bottleneck and algorithmic scalability when massive data sets are involved, we propose a distributed least squares procedure in the framework of functional linear model and reproducing kernel Hilbert space. This approach divides the big data set into multiple subsets, applies regularized least squares regression on each of them, and then averages the individual outputs as a ﬁnal prediction. We establish the non-asymptotic prediction error bounds for the proposed learning strategy under some regularity conditions. When the target function only has weak regularity, we also introduce some unlabelled data to construct a semi-supervised approach to enlarge the number of the partitioned subsets. Results in present paper provide a theoretical guarantee that the distributed algorithm can achieve the optimal rate of convergence while allowing the whole data set to be partitioned into a large number of subsets for parallel processing.

Download Full-text