An Exponential Tail Bound for the Deleted Estimate

There is an accumulating evidence in the literature that stability of learning algorithms is a key characteristic that permits a learning algorithm to generalize. Despite various insightful results in this direction, there seems to be an overlooked dichotomy in the type of stability-based generalization bounds we have in the literature. On one hand, the literature seems to suggest that exponential generalization bounds for the estimated risk, which are optimal, can be only obtained through stringent, distribution independent and computationally intractable notions of stability such as uniform stability. On the other hand, it seems that weaker notions of stability such as hypothesis stability, although it is distribution dependent and more amenable to computation, can only yield polynomial generalization bounds for the estimated risk, which are suboptimal. In this paper, we address the gap between these two regimes of results. In particular, the main question we address here is whether it is possible to derive exponential generalization bounds for the estimated risk using a notion of stability that is computationally tractable and distribution dependent, but weaker than uniform stability. Using recent advances in concentration inequalities, and using a notion of stability that is weaker than uniform stability but distribution dependent and amenable to computation, we derive an exponential tail bound for the concentration of the estimated risk of a hypothesis returned by a general learning rule, where the estimated risk is expressed in terms of the deleted estimate. Interestingly, we note that our final bound has similarities to previous exponential generalization bounds for the deleted estimate, in particular, the result of Bousquet and Elisseeff (2002) for the regression case.

Refinements of Gaussian Tail Inequality

In this paper, we first prove a theorem which gives considerably better bound for 0 ≤ t ≤ 1/2 than Gaussian tail inequality (or tail bound for normal density) and thus is a refinement of Gaussian tail inequality in this case. Next we present an interesting result which provides a refinement of Gaussian tail inequality for t > √ 3. Besides, we also prove an improvement of Gaussian tail inequality for 0 < t ≤ 1/2. Finally, we present a more general result which includes a variety of interesting results as special cases.

On Proportions of Fit Individuals in Population of Mutation-Based Evolutionary Algorithm with Tournament Selection

In this article, we consider a fitness-level model of a non-elitist mutation-only evolutionary algorithm (EA) with tournament selection. The model provides upper and lower bounds for the expected proportion of the individuals with fitness above given thresholds. In the case of so-called monotone mutation, the obtained bounds imply that increasing the tournament size improves the EA performance. As corollaries, we obtain an exponentially vanishing tail bound for the Randomized Local Search on unimodal functions and polynomial upper bounds on the runtime of EAs on the 2-SAT problem and on a family of Set Cover problems proposed by E. Balas.

Complex random matrices have no real eigenvalues

Let [Formula: see text] where [Formula: see text] are iid copies of a mean zero, variance one, subgaussian random variable. Let [Formula: see text] be an [Formula: see text] random matrix with entries that are iid copies of [Formula: see text]. We prove that there exists a [Formula: see text] such that the probability that [Formula: see text] has any real eigenvalues is less than [Formula: see text] where [Formula: see text] only depends on the subgaussian moment of [Formula: see text]. The bound is optimal up to the value of the constant [Formula: see text]. The principal component of the proof is an optimal tail bound on the least singular value of matrices of the form [Formula: see text] where [Formula: see text] is a deterministic complex matrix with the condition that [Formula: see text] for some constant [Formula: see text] depending on the subgaussian moment of [Formula: see text]. For this class of random variables, this result improves on the results of Pan–Zhou [Circular law, extreme singular values and potential theory, J. Multivariate Anal. 101(3) (2010) 645–656] and Rudelson–Vershynin [The Littlewood–Offord problem and invertibility of random matrices, Adv. Math. 218(2) (2008) 600–633]. In the proof of the tail bound, we develop an optimal small-ball probability bound for complex random variables that generalizes the Littlewood–Offord theory developed by Tao–Vu [From the Littlewood–Offord problem to the circular law: Universality of the spectral distribution of random matrices, Bull. Amer. Math. Soc.[Formula: see text]N.S.[Formula: see text] 46(3) (2009) 377–396; Inverse Littlewood–Offord theorems and the condition number of random discrete matrices, Ann. of Math.[Formula: see text] 169(2) (2009) 595–632] and Rudelson–Vershynin [The Littlewood–Offord problem and invertibility of random matrices, Adv. Math. 218(2) (2008) 600–633; Smallest singular value of a random rectangular matrix, Comm. Pure Appl. Math. 62(12) (2009) 1707–1739].

On Tail Bounds for Random Recursive Trees

We consider a multivariate distributional recursion of sum type, as arises in the probabilistic analysis of algorithms and random trees. We prove an upper tail bound for the solution using Chernoff's bounding technique by estimating the Laplace transform. The problem is traced back to the corresponding problem for binary search trees by stochastic domination. The result obtained is applied to the internal path length and Wiener index of random b-ary recursive trees with weighted edges and random linear recursive trees. Finally, lower tail bounds for the Wiener index of these trees are given.

On Tail Bounds for Random Recursive Trees

We consider a multivariate distributional recursion of sum type, as arises in the probabilistic analysis of algorithms and random trees. We prove an upper tail bound for the solution using Chernoff's bounding technique by estimating the Laplace transform. The problem is traced back to the corresponding problem for binary search trees by stochastic domination. The result obtained is applied to the internal path length and Wiener index of randomb-ary recursive trees with weighted edges and random linear recursive trees. Finally, lower tail bounds for the Wiener index of these trees are given.