Optimal minimax rates against non-smooth alternatives

This study investigates optimal minimax rates for specification testing when the alternative hypothesis is built on a set of non-smooth functions. The set consists of bounded functions that are not necessarily differentiable with no smoothness constraints imposed on their derivatives. In the instrumental variable regression set up with an unknown error variance structure, we find that the optimal minimax rate is n−1/4, where n is the sample size. The rate is achieved by a simple test based on the difference between non-parametric and parametric variance estimators. Simulation studies illustrate that the test has reasonable power against various non-smooth alternatives. The empirical application to Engel curves specification emphasizes the good applicability of the test.

Minimax Rates of ℓp-Losses for High-Dimensional Linear Errors-in-Variables Models over ℓq-Balls

In this paper, the high-dimensional linear regression model is considered, where the covariates are measured with additive noise. Different from most of the other methods, which are based on the assumption that the true covariates are fully obtained, results in this paper only require that the corrupted covariate matrix is observed. Then, by the application of information theory, the minimax rates of convergence for estimation are investigated in terms of the ℓp(1≤p<∞)-losses under the general sparsity assumption on the underlying regression parameter and some regularity conditions on the observed covariate matrix. The established lower and upper bounds on minimax risks agree up to constant factors when p=2, which together provide the information-theoretic limits of estimating a sparse vector in the high-dimensional linear errors-in-variables model. An estimator for the underlying parameter is also proposed and shown to be minimax optimal in the ℓ2-loss.

Online Learning over a Finite Action Set with Limited Switching

This paper studies the value of switching actions in the Prediction From Experts problem (PFE) and Adversarial Multiarmed Bandits problem (MAB). First, we revisit the well-studied and practically motivated setting of PFE with switching costs. Many algorithms achieve the minimax optimal order for both regret and switches in expectation; however, high probability guarantees are an open problem. We present the first algorithms that achieve this optimal order for both quantities with high probability. This also implies the first high probability guarantees for several other problems, and, in particular, is efficiently adaptable to online combinatorial optimization with limited switching. Next, to investigate the value of switching actions more granularly, we introduce the switching budget setting, which limits algorithms to a fixed number of (costless) switches. Using this result and several reductions, we unify previous work and completely characterize the complexity of this switching budget setting up to small polylogarithmic factors: for both PFE and MAB, for all switching budgets, and for both expectation and high probability guarantees. Interestingly, as the switching budget decreases, the minimax regret rate admits a phase transition for PFE but not for MAB. These results recover and generalize the known minimax rates for the (arbitrary) switching cost setting.

Super-resolution of near-colliding point sources

We consider the problem of stable recovery of sparse signals of the form $$\begin{equation*}F(x)=\sum_{j=1}^d a_j\delta(x-x_j),\quad x_j\in\mathbb{R},\;a_j\in\mathbb{C}, \end{equation*}$$from their spectral measurements, known in a bandwidth $\varOmega $ with absolute error not exceeding $\epsilon&gt;0$. We consider the case when at most $p\leqslant d$ nodes $\{x_j\}$ of $F$ form a cluster whose extent is smaller than the Rayleigh limit ${1\over \varOmega }$, while the rest of the nodes is well separated. Provided that $\epsilon \lessapprox \operatorname{SRF}^{-2p+1}$, where $\operatorname{SRF}=(\varOmega \varDelta )^{-1}$ and $\varDelta $ is the minimal separation between the nodes, we show that the minimax error rate for reconstruction of the cluster nodes is of order ${1\over \varOmega }\operatorname{SRF}^{2p-1}\epsilon $, while for recovering the corresponding amplitudes $\{a_j\}$ the rate is of the order $\operatorname{SRF}^{2p-1}\epsilon $. Moreover, the corresponding minimax rates for the recovery of the non-clustered nodes and amplitudes are ${\epsilon \over \varOmega }$ and $\epsilon $, respectively. These results suggest that stable super-resolution is possible in much more general situations than previously thought. Our numerical experiments show that the well-known matrix pencil method achieves the above accuracy bounds.

Kernel regression, minimax rates and effective dimensionality: Beyond the regular case

We investigate if kernel regularization methods can achieve minimax convergence rates over a source condition regularity assumption for the target function. These questions have been considered in past literature, but only under specific assumptions about the decay, typically polynomial, of the spectrum of the the kernel mapping covariance operator. In the perspective of distribution-free results, we investigate this issue under much weaker assumption on the eigenvalue decay, allowing for more complex behavior that can reflect different structure of the data at different scales.