scholarly journals Statistical Query Algorithms for Mean Vector Estimation and Stochastic Convex Optimization

2021 ◽  
Author(s):  
Vitaly Feldman ◽  
Cristóbal Guzmán ◽  
Santosh Vempala
Keyword(s):  
Machine Learning ◽  
Convex Optimization ◽  
Lower Bounds ◽  
Differential Privacy ◽  
Linear Optimization ◽  
General Setting ◽  
Upper And Lower Bounds ◽  
Statistical Queries ◽  
The Mean ◽  
Mean Vector

Stochastic convex optimization, by which the objective is the expectation of a random convex function, is an important and widely used method with numerous applications in machine learning, statistics, operations research, and other areas. We study the complexity of stochastic convex optimization given only statistical query (SQ) access to the objective function. We show that well-known and popular first-order iterative methods can be implemented using only statistical queries. For many cases of interest, we derive nearly matching upper and lower bounds on the estimation (sample) complexity, including linear optimization in the most general setting. We then present several consequences for machine learning, differential privacy, and proving concrete lower bounds on the power of convex optimization–based methods. The key ingredient of our work is SQ algorithms and lower bounds for estimating the mean vector of a distribution over vectors supported on a convex body in Rd. This natural problem has not been previously studied, and we show that our solutions can be used to get substantially improved SQ versions of Perceptron and other online algorithms for learning halfspaces.

scholarly journals On the mean and dispersion of the Moore-Penrose generalized inverse of a Wishart matrix

2020 ◽  
Vol 36 (36) ◽  
pp. 124-133
Author(s):  
Shinpei Imori ◽  
Dietrich Von Rosen
Keyword(s):  
Lower Bounds ◽  
Upper Bound ◽  
Degrees Of Freedom ◽  
Generalized Inverse ◽  
Upper And Lower Bounds ◽  
Exact Results ◽  
Identity Matrix ◽  
Wishart Matrix ◽  
The Mean ◽  
Scale Matrix

The Moore-Penrose inverse of a singular Wishart matrix is studied. When the scale matrix equals the identity matrix the mean and dispersion matrices of the Moore-Penrose inverse are known. When the scale matrix has an arbitrary structure no exact results are available. The article complements the existing literature by deriving upper and lower bounds for the expectation and an upper bound for the dispersion of the Moore-Penrose inverse. The results show that the bounds become large when the number of rows (columns) of the Wishart matrix are close to the degrees of freedom of the distribution.

scholarly journals The Mean Width of Circumscribed Random Polytopes

2010 ◽  
Vol 53 (4) ◽  
pp. 614-628 ◽  
Author(s):  
Károly J. Böröczky ◽  
Rolf Schneider
Keyword(s):  
Convex Body ◽  
Asymptotic Formula ◽  
Uniform Distribution ◽  
Lower Bounds ◽  
Upper And Lower Bounds ◽  
Simplicial Polytope ◽  
Mean Width ◽  
The Mean ◽  
The Difference ◽  
Precise Asymptotic

AbstractFor a given convex body K in ℝd, a random polytope K(n) is defined (essentially) as the intersection of n independent closed halfspaces containing K and having an isotropic and (in a specified sense) uniform distribution. We prove upper and lower bounds of optimal orders for the difference of the mean widths of K(n) and K as n tends to infinity. For a simplicial polytope P, a precise asymptotic formula for the difference of the mean widths of P(n) and P is obtained.

Estimates of the mean size of the subset image under composition of random mappings

2018 ◽  
Vol 28 (5) ◽  
pp. 331-338 ◽  
Author(s):  
Andrey M. Zubkov ◽  
Aleksandr A. Serov
Keyword(s):  
Lower Bounds ◽  
Upper And Lower Bounds ◽  
Random Mappings ◽  
Mean Size ◽  
The Mean ◽  
The Difference ◽  
Time Memory Tradeoff

Abstract Let XN be a set of N elements and F1, F2,… be a sequence of random independent equiprobable mappings XN → N. For a subset S0 ⊂ XN, |S0|=m, we consider a sequence of its images St=Ft(…F2(F1(S0))…), t=1,2… An approach to the exact recurrent computation of distribution of |St| is described. Two-sided inequalities forM{|St|||S0|=m} such that the difference between the upper and lower bounds is o(m)for m, t, N → ∞, mt=o(N) are derived. The results are of interest for the analysis of time-memory tradeoff algorithms.

scholarly journals Make Up Your Mind: The Price of Online Queries in Differential Privacy

2019 ◽  
Vol 9 (1) ◽  
Author(s):  
Mark Bun ◽  
Thomas Steinke ◽  
Jonathan Ullman
Keyword(s):  
Lower Bounds ◽  
Real Line ◽  
Differential Privacy ◽  
Adaptive Model ◽  
Search Queries ◽  
The Real ◽  
Statistical Queries ◽  
Online Queries ◽  
Single Batch

We consider the problem of answering queries about a sensitive dataset subject to differential privacy. The queries may be chosen adversarially from a larger set $Q$ of allowable queries in one of three ways, which we list in order from easiest to hardest to answer:Offline: The queries are chosen all at once and the differentially private mechanism answers the queries in a single batch. Online: The queries are chosen all at once, but the mechanism only receives the queries in a streaming fashion and must answer each query before seeing the next query. Adaptive: The queries are chosen one at a time and the mechanism must answer each query before the next query is chosen. In particular, each query may depend on the answers given to previous queries.Many differentially private mechanisms are just as efficient in the adaptive model as they are in the offline model. Meanwhile, most lower bounds for differential privacy hold in the offline setting. This suggests that the three models may be equivalent. We prove that these models are all, in fact, distinct. Specifically, we show that there is a family of statistical queries such that exponentially more queries from this family can be answered in the offline model than in the online model. We also exhibit a family of search queries such that exponentially more queries from this family can be answered in the online model than in the adaptive model. We also investigate whether such separations might hold for simple queries like threshold queries over the real line.

scholarly journals Minimax Estimation of a Mean Vector for Distributions on a Compact Set

1990 ◽  
Vol 20 (2) ◽  
pp. 173-179
Author(s):  
Richard Dykstra
Keyword(s):  
Dirichlet Process ◽  
General Setting ◽  
Minimax Estimation ◽  
Compact Set ◽  
Dirichlet Process Prior ◽  
Minimax Estimator ◽  
Error Type ◽  
Squared Error ◽  
The Mean ◽  
Mean Vector

AbstractMinimax estimation procedures for the mean vector of a distribution on a compact set under squared error type loss functions are considered. In particular, a Dirichlet process prior is used to show that a linear function of is a minimax estimator in the class of all measurable estimators and all possible distributions. This effort extends some earlier work of Bühlmann to a more general setting.

Another look at the moment bounds on reliability

1994 ◽  
Vol 31 (03) ◽  
pp. 777-787 ◽  
Author(s):  
Debasis Sengupta
Keyword(s):  
Lower Bounds ◽  
Reliability Function ◽  
Upper And Lower Bounds ◽  
Finite Moment ◽  
Complicated Form ◽  
The Mean ◽  
The Moment ◽  
Moment Bounds

In this paper a unified derivation of the upper and lower bounds (in terms of the mean) of an IFR, DFR, IFRA, DFRA, NBU or NWU reliability function is presented. The method of proof provides a simpler alternative to the various proofs known in the literature. Moreover, this method can be used to generalize the existing results in two ways, as demonstrated here. First, the bounds for the reliability function are obtained in terms of any finite moment in all these cases. Subsequently we provide a moment bound on a reliability function which ages faster or slower than a known one in some sense. The existing literature does not offer any of these generalizations, except for a few results which are available in an unnecessarily complicated form.

scholarly journals Sharp Upper and Lower Bounds of VDB Topological Indices of Digraphs

Symmetry ◽  
2021 ◽  
Vol 13 (10) ◽  
pp. 1903
Author(s):  
Juan Monsalve ◽  
Juan Rada
Keyword(s):  
Lower Bounds ◽  
Topological Index ◽  
General Setting ◽  
Upper Bounds ◽  
Topological Indices ◽  
Vertex Degree ◽  
Upper And Lower Bounds ◽  
Lower And Upper Bounds ◽  
Extremal Values

A vertex-degree-based (VDB, for short) topological index φ induced by the numbers φij was recently defined for a digraph D, as φD=12∑uvφdu+dv−, where du+ denotes the out-degree of the vertex u,dv− denotes the in-degree of the vertex v, and the sum runs over the set of arcs uv of D. This definition generalizes the concept of a VDB topological index of a graph. In a general setting, we find sharp lower and upper bounds of a symmetric VDB topological index over Dn, the set of all digraphs with n non-isolated vertices. Applications to well-known topological indices are deduced. We also determine extremal values of symmetric VDB topological indices over OTn and OG, the set of oriented trees with n vertices, and the set of all orientations of a fixed graph G, respectively.

Another look at the moment bounds on reliability

1994 ◽  
Vol 31 (3) ◽  
pp. 777-787 ◽  
Author(s):  
Debasis Sengupta
Keyword(s):  
Lower Bounds ◽  
Reliability Function ◽  
Upper And Lower Bounds ◽  
Finite Moment ◽  
Complicated Form ◽  
The Mean ◽  
The Moment ◽  
Moment Bounds

In this paper a unified derivation of the upper and lower bounds (in terms of the mean) of an IFR, DFR, IFRA, DFRA, NBU or NWU reliability function is presented. The method of proof provides a simpler alternative to the various proofs known in the literature. Moreover, this method can be used to generalize the existing results in two ways, as demonstrated here. First, the bounds for the reliability function are obtained in terms of any finite moment in all these cases. Subsequently we provide a moment bound on a reliability function which ages faster or slower than a known one in some sense. The existing literature does not offer any of these generalizations, except for a few results which are available in an unnecessarily complicated form.

Sign in / Sign up

Export Citation Format

Share Document