Finding Small Sets of Random Fourier Features for Shift-Invariant Kernel Approximation

2016 ◽  
pp. 42-54 ◽  
Author(s):  
Frank-M. Schleif ◽  
Ata Kaban ◽  
Peter Tino
Keyword(s):  
Kernel Approximation ◽  
Invariant Kernel ◽  
Random Fourier Features

scholarly journals Random Fourier Features via Fast Surrogate Leverage Weighted Sampling

2020 ◽  
Vol 34 (04) ◽  
pp. 4844-4851
Author(s):  
Fanghui Liu ◽  
Xiaolin Huang ◽  
Yudong Chen ◽  
Jie Yang ◽  
Johan Suykens
Keyword(s):  
State Of The Art ◽  
Sampling Strategy ◽  
Prediction Performance ◽  
Current State ◽  
Kernel Approximation ◽  
Sampling Process ◽  
New Strategy ◽  
The Matrix ◽  
Benchmark Datasets ◽  
Random Fourier Features

In this paper, we propose a fast surrogate leverage weighted sampling strategy to generate refined random Fourier features for kernel approximation. Compared to the current state-of-the-art method that uses the leverage weighted scheme (Li et al. 2019), our new strategy is simpler and more effective. It uses kernel alignment to guide the sampling process and it can avoid the matrix inversion operator when we compute the leverage function. Given n observations and s random features, our strategy can reduce the time complexity for sampling from O(ns2+s3) to O(ns2), while achieving comparable (or even slightly better) prediction performance when applied to kernel ridge regression (KRR). In addition, we provide theoretical guarantees on the generalization performance of our approach, and in particular characterize the number of random features required to achieve statistical guarantees in KRR. Experiments on several benchmark datasets demonstrate that our algorithm achieves comparable prediction performance and takes less time cost when compared to (Li et al. 2019).

scholarly journals Data-driven Random Fourier Features using Stein Effect

2017 ◽  
Author(s):  
Wei-Cheng Chang ◽  
Chun-Liang Li ◽  
Yiming Yang ◽  
Barnabás Póczos
Keyword(s):  
Monte Carlo ◽  
Large Scale ◽  
Randomized Algorithm ◽  
Data Driven ◽  
Equal Weight ◽  
Quasi Monte Carlo ◽  
Empirical Risk ◽  
Kernel Approximation ◽  
Random Fourier Features ◽  
Stein Effect

Large-scale kernel approximation is an important problem in machine learning research. Approaches using random Fourier features have become increasingly popular \cite{Rahimi_NIPS_07}, where kernel approximation is treated as empirical mean estimation via Monte Carlo (MC) or Quasi-Monte Carlo (QMC) integration \cite{Yang_ICML_14}. A limitation of the current approaches is that all the features receive an equal weight summing to 1. In this paper, we propose a novel shrinkage estimator from "Stein effect", which provides a data-driven weighting strategy for random features and enjoys theoretical justifications in terms of lowering the empirical risk. We further present an efficient randomized algorithm for large-scale applications of the proposed method. Our empirical results on six benchmark data sets demonstrate the advantageous performance of this approach over representative baselines in both kernel approximation and supervised learning tasks.

scholarly journals Learning Adaptive Random Features

2019 ◽  
Vol 33 ◽  
pp. 4229-4236
Author(s):  
Yanjun Li ◽  
Kai Zhang ◽  
Jun Wang ◽  
Sanjiv Kumar
Keyword(s):  
Supervised Learning ◽  
Sampling Rate ◽  
Scaling Up ◽  
Monte Carlo Integration ◽  
Sampling Schemes ◽  
Generalization Bounds ◽  
Learning Framework ◽  
Kernel Approximation ◽  
Random Fourier Features ◽  
Spectral Sampling

Random Fourier features are a powerful framework to approximate shift invariant kernels with Monte Carlo integration, which has drawn considerable interest in scaling up kernel-based learning, dimensionality reduction, and information retrieval. In the literature, many sampling schemes have been proposed to improve the approximation performance. However, an interesting theoretic and algorithmic challenge still remains, i.e., how to optimize the design of random Fourier features to achieve good kernel approximation on any input data using a low spectral sampling rate? In this paper, we propose to compute more adaptive random Fourier features with optimized spectral samples (wj’s) and feature weights (pj’s). The learning scheme not only significantly reduces the spectral sampling rate needed for accurate kernel approximation, but also allows joint optimization with any supervised learning framework. We establish generalization bounds using Rademacher complexity, and demonstrate advantages over previous methods. Moreover, our experiments show that the empirical kernel approximation provides effective regularization for supervised learning.

Graph Diffusion Kernel LMS using Random Fourier Features

2020 ◽  
Author(s):  
Vinay Chakravarthi Gogineni ◽  
Vitor R. M. Elias ◽  
Wallace A. Martins ◽  
Stefan Werner
Keyword(s):  
Diffusion Kernel ◽  
Random Fourier Features

A nonlinear anisotropic diffusion equation for image restoration with forward-backward diffusivities

2021 ◽  
Vol 14 ◽  
Author(s):  
Santosh Kumar ◽  
Nitendra Kumar ◽  
Khursheed Alam
Keyword(s):  
Image Processing ◽  
Anisotropic Diffusion ◽  
Diffusion Models ◽  
Smoothing Kernel ◽  
Proposed Model ◽  
Invariant Kernel ◽  
Nonlinear Anisotropic Diffusion ◽  
Anisotropic Diffusion Equation ◽  
Frequent Problem ◽  
Deblurring And Denoising

Background: In the image processing area, deblurring and denoising are the most challenging hurdles. The deblurring image by a spatially invariant kernel is a frequent problem in the field of image processing. Methods: For deblurring and denoising, the total variation (TV norm) and nonlinear anisotropic diffusion models are powerful tools. In this paper, nonlinear anisotropic diffusion models for image denoising and deblurring are proposed. The models are developed in the following manner: first multiplying the magnitude of the gradient in the anisotropic diffusion model, and then apply priori smoothness on the solution image by Gaussian smoothing kernel. Results: The finite difference method is used to discretize anisotropic diffusion models with forward-backward diffusivities. Conclusion: The results of the proposed model are given in terms of the improvement.

Sign in / Sign up

Export Citation Format

Share Document