Wide-sense Stationary Stochastic Processes

2020 ◽  
pp. 467-491
Author(s):  
Pierre Brémaud
Keyword(s):  
Stochastic Processes ◽  
Wide Sense ◽  
Stationary Stochastic Processes

scholarly journals Learning Theory Estimates with Observations from General Stationary Stochastic Processes

2016 ◽  
Vol 28 (12) ◽  
pp. 2853-2889 ◽  
Author(s):  
Hanyuan Hang ◽  
Yunlong Feng ◽  
Ingo Steinwart ◽  
Johan A. K. Suykens
Keyword(s):  
Stochastic Processes ◽  
Type Inequality ◽  
Empirical Risk Minimization ◽  
Risk Minimization ◽  
Mixing Processes ◽  
Gaussian Kernels ◽  
Empirical Risk ◽  
Learning Rates ◽  
Stationary Stochastic Processes ◽  
Learning Schemes

This letter investigates the supervised learning problem with observations drawn from certain general stationary stochastic processes. Here by general, we mean that many stationary stochastic processes can be included. We show that when the stochastic processes satisfy a generalized Bernstein-type inequality, a unified treatment on analyzing the learning schemes with various mixing processes can be conducted and a sharp oracle inequality for generic regularized empirical risk minimization schemes can be established. The obtained oracle inequality is then applied to derive convergence rates for several learning schemes such as empirical risk minimization (ERM), least squares support vector machines (LS-SVMs) using given generic kernels, and SVMs using gaussian kernels for both least squares and quantile regression. It turns out that for independent and identically distributed (i.i.d.) processes, our learning rates for ERM recover the optimal rates. For non-i.i.d. processes, including geometrically [Formula: see text]-mixing Markov processes, geometrically [Formula: see text]-mixing processes with restricted decay, [Formula: see text]-mixing processes, and (time-reversed) geometrically [Formula: see text]-mixing processes, our learning rates for SVMs with gaussian kernels match, up to some arbitrarily small extra term in the exponent, the optimal rates. For the remaining cases, our rates are at least close to the optimal rates. As a by-product, the assumed generalized Bernstein-type inequality also provides an interpretation of the so-called effective number of observations for various mixing processes.

Sign in / Sign up

Export Citation Format

Share Document