On Learning Vector-Valued Functions

2005 ◽  
Vol 17 (1) ◽  
pp. 177-204 ◽  
Author(s):  
Charles A. Micchelli ◽  
Massimiliano Pontil
Keyword(s):  
Hilbert Space ◽  
Learning Theory ◽  
Reproducing Kernel ◽  
Reproducing Kernel Hilbert Space ◽  
Support Vector ◽  
Translation Invariant ◽  
Minimal Norm ◽  
Finite Set ◽  
Vector Valued Functions ◽  
Vector Valued

In this letter, we provide a study of learning in a Hilbert space of vector-valued functions. We motivate the need for extending learning theory of scalar-valued functions by practical considerations and establish some basic results for learning vector-valued functions that should prove useful in applications. Specifically, we allow an output space Y to be a Hilbert space, and we consider a reproducing kernel Hilbert space of functions whose values lie in Y. In this setting, we derive the form of the minimal norm interpolant to a finite set of data and apply it to study some regularization functionals that are important in learning theory. We consider specific examples of such functionals corresponding to multiple-output regularization networks and support vector machines, for both regression and classification. Finally, we provide classes of operator-valued kernels of the dot product and translation-invariant type.

scholarly journals Combining Dissimilarities in a Hyper Reproducing Kernel Hilbert Space for Complex Human Cancer Prediction

2009 ◽  
Vol 2009 ◽  
pp. 1-9 ◽  
Author(s):  
Manuel Martín-Merino ◽  
Ángela Blanco ◽  
Javier De Las Rivas
Keyword(s):  
Hilbert Space ◽  
Dna Microarrays ◽  
Reproducing Kernel ◽  
Human Cancer ◽  
Reproducing Kernel Hilbert Space ◽  
Programming Algorithm ◽  
Support Vector ◽  
Cancer Prediction ◽  
Additional Information ◽  
Misclassification Errors

DNA microarrays provide rich profiles that are used in cancer prediction considering the gene expression levels across a collection of related samples. Support Vector Machines (SVM) have been applied to the classification of cancer samples with encouraging results. However, they rely on Euclidean distances that fail to reflect accurately the proximities among sample profiles. Then, non-Euclidean dissimilarities provide additional information that should be considered to reduce the misclassification errors. In this paper, we incorporate in the -SVM algorithm a linear combination of non-Euclidean dissimilarities. The weights of the combination are learnt in a (Hyper Reproducing Kernel Hilbert Space) HRKHS using a Semidefinite Programming algorithm. This approach allows us to incorporate a smoothing term that penalizes the complexity of the family of distances and avoids overfitting. The experimental results suggest that the method proposed helps to reduce the misclassification errors in several human cancer problems.

scholarly journals Generalized coherent states, reproducing kernels, and quantum support vector machines

2017 ◽  
Vol 17 (15&16) ◽  
pp. 1292-1306 ◽  
Author(s):  
Rupak Chatterjee ◽  
Ting Yu
Keyword(s):  
Hilbert Space ◽  
Coherent States ◽  
Reproducing Kernel ◽  
Reproducing Kernel Hilbert Space ◽  
Feature Space ◽  
Reproducing Kernels ◽  
Kernel Functions ◽  
Support Vector ◽  
Machine Learning Classification ◽  
Generalized Coherent States

The support vector machine (SVM) is a popular machine learning classification method which produces a nonlinear decision boundary in a feature space by constructing linear boundaries in a transformed Hilbert space. It is well known that these algorithms when executed on a classical computer do not scale well with the size of the feature space both in terms of data points and dimensionality. One of the most significant limitations of classical algorithms using non-linear kernels is that the kernel function has to be evaluated for all pairs of input feature vectors which themselves may be of substantially high dimension. This can lead to computationally excessive times during training and during the prediction process for a new data point. Here, we propose using both canonical and generalized coherent states to calculate specific nonlinear kernel functions. The key link will be the reproducing kernel Hilbert space (RKHS) property for SVMs that naturally arise from canonical and generalized coherent states. Specifically, we discuss the evaluation of radial kernels through a positive operator valued measure (POVM) on a quantum optical system based on canonical coherent states. A similar procedure may also lead to calculations of kernels not usually used in classical algorithms such as those arising from generalized coherent states.

scholarly journals Generalized linear models for geometrical current predictors: An application to predict garment fit

2019 ◽  
Vol 20 (6) ◽  
pp. 562-591
Author(s):  
Sonia Barahona ◽  
Pablo Centella ◽  
Ximo Gual-Arnau ◽  
M. Victoria Ibáñez ◽  
Amelia Simó
Keyword(s):  
Functional Data ◽  
Orthonormal Basis ◽  
Reproducing Kernel ◽  
Linear Models ◽  
Reproducing Kernel Hilbert Space ◽  
Alternative Methods ◽  
Covariance Operator ◽  
Support Vector ◽  
Ordinal Response ◽  
Vector Valued

The aim of this article is to model an ordinal response variable in terms of vector-valued functional data included on a vector-valued reproducing kernel Hilbert space (RKHS). In particular, we focus on the vector-valued RKHS obtained when a geometrical object (body) is characterized by a current and on the ordinal regression model. A common way to solve this problem in functional data analysis is to express the data in the orthonormal basis given by decomposition of the covariance operator. But our data present very important differences with respect to the usual functional data setting. On the one hand, they are vector-valued functions, and on the other, they are functions in an RKHS with a previously defined norm. We propose to use three different bases: the orthonormal basis given by the kernel that defines the RKHS, a basis obtained from decomposition of the integral operator defined using the covariance function and a third basis that combines the previous two. The three approaches are compared and applied to an interesting problem: building a model to predict the fit of children's garment sizes, based on a 3D database of the Spanish child population. Our proposal has been compared with alternative methods that explore the performance of other classifiers (Support Vector Machine and [Formula: see text]-NN), and with the result of applying the classification method proposed in this work, from different characterizations of the objects (landmarks and multivariate anthropometric measurements instead of currents), obtaining in all these cases worst results.

ESTIMATING THE APPROXIMATION ERROR IN LEARNING THEORY

2003 ◽  
Vol 01 (01) ◽  
pp. 17-41 ◽  
Author(s):  
STEVE SMALE ◽  
DING-XUAN ZHOU
Keyword(s):  
Hilbert Space ◽  
Learning Theory ◽  
Hilbert Spaces ◽  
Reproducing Kernel ◽  
Interpolation Space ◽  
Reproducing Kernel Hilbert Space ◽  
Approximation Error ◽  
Approximation Problem ◽  
Linear Operators ◽  
Kernel Hilbert Spaces

Let B be a Banach space and (ℋ,‖·‖ℋ) be a dense, imbedded subspace. For a ∈ B, its distance to the ball of ℋ with radius R (denoted as I(a, R)) tends to zero when R tends to infinity. We are interested in the rate of this convergence. This approximation problem arose from the study of learning theory, where B is the L2 space and ℋ is a reproducing kernel Hilbert space. The class of elements having I(a, R) = O(R-r) with r > 0 is an interpolation space of the couple (B, ℋ). The rate of convergence can often be realized by linear operators. In particular, this is the case when ℋ is the range of a compact, symmetric, and strictly positive definite linear operator on a separable Hilbert space B. For the kernel approximation studied in Learning Theory, the rate depends on the regularity of the kernel function. This yields error estimates for the approximation by reproducing kernel Hilbert spaces. When the kernel is smooth, the convergence is slow and a logarithmic convergence rate is presented for analytic kernels in this paper. The purpose of our results is to provide some theoretical estimates, including the constants, for the approximation error required for the learning theory.

A NOTE ON STABILITY OF ERROR BOUNDS IN STATISTICAL LEARNING THEORY

2011 ◽  
Vol 09 (04) ◽  
pp. 369-382
Author(s):  
MING LI ◽  
ANDREA CAPONNETTO
Keyword(s):  
Hilbert Space ◽  
Statistical Learning ◽  
Learning Theory ◽  
Error Bounds ◽  
Wide Class ◽  
Reproducing Kernel ◽  
Reproducing Kernel Hilbert Space ◽  
Statistical Learning Theory ◽  
Regression Function ◽  
Functional Space

We consider a wide class of error bounds developed in the context of statistical learning theory which are expressed in terms of functionals of the regression function, for instance, its norm in a reproducing kernel Hilbert space or other functional space. These bounds are unstable in the sense that a small perturbation of the regression function can induce an arbitrary large increase of the relevant functional and make the error bound useless. Using a known result involving Fano inequality, we show how stability can be recovered.

scholarly journals Lebesgue Decomposition of Non-Commutative Measures

2020 ◽  
Author(s):  
Michael T Jury ◽  
Robert T W Martin
Keyword(s):  
Hilbert Space ◽  
Reproducing Kernel ◽  
Fock Space ◽  
Reproducing Kernel Hilbert Space ◽  
Semigroup Algebra ◽  
Space Theory ◽  
Lebesgue Decomposition ◽  
Sesquilinear Forms ◽  
Positive Linear Functionals ◽  
Algebra Theory

Abstract We extend the Lebesgue decomposition of positive measures with respect to Lebesgue measure on the complex unit circle to the non-commutative (NC) multi-variable setting of (positive) NC measures. These are positive linear functionals on a certain self-adjoint subspace of the Cuntz–Toeplitz $C^{\ast }-$algebra, the $C^{\ast }-$algebra of the left creation operators on the full Fock space. This theory is fundamentally connected to the representation theory of the Cuntz and Cuntz–Toeplitz $C^{\ast }-$algebras; any *−representation of the Cuntz–Toeplitz $C^{\ast }-$algebra is obtained (up to unitary equivalence), by applying a Gelfand–Naimark–Segal construction to a positive NC measure. Our approach combines the theory of Lebesgue decomposition of sesquilinear forms in Hilbert space, Lebesgue decomposition of row isometries, free semigroup algebra theory, NC reproducing kernel Hilbert space theory, and NC Hardy space theory.

scholarly journals Multi-omics-based prediction of hybrid performance in canola

2021 ◽  
Author(s):  
Dominic Knoch ◽  
Christian R. Werner ◽  
Rhonda C. Meyer ◽  
David Riewe ◽  
Amine Abbadi ◽  
...  
Keyword(s):  
Hilbert Space ◽  
Genetic Markers ◽  
Reproducing Kernel ◽  
Prediction Models ◽  
Agronomic Traits ◽  
Reproducing Kernel Hilbert Space ◽  
Hybrid Performance ◽  
Hybrid Prediction ◽  
Gaussian Kernels ◽  
Breeding Programmes

Abstract Key message Complementing or replacing genetic markers with transcriptomic data and use of reproducing kernel Hilbert space regression based on Gaussian kernels increases hybrid prediction accuracies for complex agronomic traits in canola. In plant breeding, hybrids gained particular importance due to heterosis, the superior performance of offspring compared to their inbred parents. Since the development of new top performing hybrids requires labour-intensive and costly breeding programmes, including testing of large numbers of experimental hybrids, the prediction of hybrid performance is of utmost interest to plant breeders. In this study, we tested the effectiveness of hybrid prediction models in spring-type oilseed rape (Brassica napus L./canola) employing different omics profiles, individually and in combination. To this end, a population of 950 F1 hybrids was evaluated for seed yield and six other agronomically relevant traits in commercial field trials at several locations throughout Europe. A subset of these hybrids was also evaluated in a climatized glasshouse regarding early biomass production. For each of the 477 parental rapeseed lines, 13,201 single nucleotide polymorphisms (SNPs), 154 primary metabolites, and 19,479 transcripts were determined and used as predictive variables. Both, SNP markers and transcripts, effectively predict hybrid performance using (genomic) best linear unbiased prediction models (gBLUP). Compared to models using pure genetic markers, models incorporating transcriptome data resulted in significantly higher prediction accuracies for five out of seven agronomic traits, indicating that transcripts carry important information beyond genomic data. Notably, reproducing kernel Hilbert space regression based on Gaussian kernels significantly exceeded the predictive abilities of gBLUP models for six of the seven agronomic traits, demonstrating its potential for implementation in future canola breeding programmes.

Sign in / Sign up

Export Citation Format

Share Document