scholarly journals Differentially Private Inference for Binomial Data

2020 ◽  
Vol 10 (1) ◽  
Author(s):  
Jordan Alexander Awan ◽  
Aleksandra Slavkovic
Keyword(s):  
Confidence Intervals ◽  
Differential Privacy ◽  
Empirical Distribution ◽  
Random Variable ◽  
Linear Constraints ◽  
Type I ◽  
Finite Sample ◽  
Hypothesis Tests ◽  
Binomial Data ◽  
Uniformly Most Powerful

We derive uniformly most powerful (UMP) tests for simple and one-sided hypotheses for a population proportion within the framework of Differential Privacy (DP), optimizing finite sample performance. We show that in general, DP hypothesis tests can be written in terms of linear constraints, and for exchangeable data can always be expressed as a function of the empirical distribution. Using this structure, we prove a `Neyman-Pearson lemma' for binomial data under DP, where the DP-UMP only depends on the sample sum. Our tests can also be stated as a post-processing of a random variable, whose distribution we coin ``Truncated-Uniform-Laplace'' (Tulap), a generalization of the Staircase and discrete Laplace distributions. Furthermore, we obtain exact p-values, which are easily computed in terms of the Tulap random variable. Using the above techniques, we show that our tests can be applied to give uniformly most accurate one-sided confidence intervals and optimal confidence distributions. We also derive uniformly most powerful unbiased (UMPU) two-sided tests, which lead to uniformly most accurate unbiased (UMAU) two-sided confidence intervals. We show that our results can be applied to distribution-free hypothesis tests for continuous data. Our simulation results demonstrate that all our tests have exact type I error, and are more powerful than current techniques.

scholarly journals Statistical Approximating Distributions Under Differential Privacy

2018 ◽  
Vol 8 (1) ◽  
Author(s):  
Yue Wang ◽  
Daniel Kifer ◽  
Jaewoo Lee ◽  
Vishesh Karwa
Keyword(s):  
Central Limit Theorem ◽  
Limit Theorem ◽  
Confidence Intervals ◽  
Central Limit ◽  
Differential Privacy ◽  
Prior Work ◽  
Finite Sample ◽  
The Central Limit Theorem ◽  
Approximating Distributions

Statistics computed from data are viewed as random variables. When they are used for tasks like hypothesis testing and confidence intervals, their true finite sample distributions are often replaced by approximating distributions that are easier to work with (for example, the Gaussian, which results from using approximations justified by the Central Limit Theorem). When data are perturbed by differential privacy, the approximating distributions also need to be modified. Prior work provided various competing methods for creating such approximating distributions with little formal justification beyond the fact that they worked well empirically. In this paper, we study the question of how to generate statistical approximating distributions for differentially private statistics, provide finite sample guarantees for the quality of the approximations.

scholarly journals Analyzing Nested Experimental Designs - A User-Friendly Resampling Method to Determine Experimental Significance

2021 ◽  
Author(s):  
Rishikesh U Kulkarni ◽  
Catherine L Wang ◽  
Carolyn R Bertozzi
Keyword(s):  
Confidence Intervals ◽  
Biomedical Research ◽  
Test Construction ◽  
Error Rates ◽  
Experimental Designs ◽  
Statistical Hypothesis ◽  
P Value ◽  
Type I ◽  
Hypothesis Tests ◽  
Python Package

While hierarchical experimental designs are near-ubiquitous in neuroscience and biomedical research, researchers often do not take the structure of their datasets into account while performing statistical hypothesis tests. Resampling-based methods are a flexible strategy for performing these analyses but are difficult due to the lack of open-source software to automate test construction and execution. To address this, we report Hierarch, a Python package to perform hypothesis tests and compute confidence intervals on hierarchical experimental designs. Using a combination of permutation resampling and bootstrap aggregation, Hierarch can be used to perform hypothesis tests that maintain nominal Type I error rates and generate confidence intervals that maintain the nominal coverage probability without making distributional assumptions about the dataset of interest. Hierarch makes use of the Numba JIT compiler to reduce p-value computation times to under one second for typical datasets in biomedical research. Hierarch also enables researchers to construct user-defined resampling plans that take advantage of Hierarch's Numba-accelerated functions. Hierarch is freely available as a Python package at https://github.com/rishi-kulkarni/hierarch.

Distributionally Robust Conditional Quantile Prediction with Fixed Design

2021 ◽  
Author(s):  
Meng Qi ◽  
Ying Cao ◽  
Zuo-Jun (Max) Shen
Keyword(s):  
Empirical Distribution ◽  
Random Variable ◽  
Practical Method ◽  
Data Driven ◽  
Finite Sample ◽  
Response Variable ◽  
Conditional Quantile ◽  
Measure Concentration ◽  
Fixed Design ◽  
Distributionally Robust

Conditional quantile prediction involves estimating/predicting the quantile of a response random variable conditioned on observed covariates. The existing literature assumes the availability of independent and identically distributed (i.i.d.) samples of both the covariates and the response variable. However, such an assumption often becomes restrictive in many real-world applications. By contrast, we consider a fixed-design setting of the covariates, under which neither the response variable nor the covariates have i.i.d. samples. The present study provides a new data-driven distributionally robust framework under a fixed-design setting. We propose a regress-then-robustify method by constructing a surrogate empirical distribution of the noise. The solution of our framework coincides with a simple yet practical method that involves only regression and sorting, therefore providing an explanation for its empirical success. Measure concentration results are obtained for the surrogate empirical distribution, which further lead to finite-sample performance guarantees and asymptotic consistency. Numerical experiments are conducted to demonstrate the advantages of our approach. This paper was accepted by Hamid Nazerzadeh, Special Issue on Data-Driven Prescriptive Analytics.

scholarly journals p < 0,05, ¿Criterio mágico para resolver cualquier problema o leyenda urbana?

2012 ◽  
Vol 17 (2) ◽  
pp. 203
Author(s):  
Pedro Monterrey Gutiérrez Monterrey Gutiérrez
Keyword(s):  
Data Analysis ◽  
Type I Error ◽  
Random Variable ◽  
Statistical Hypothesis ◽  
P Value ◽  
Type I ◽  
Statistical Methodology ◽  
Hypothesis Tests ◽  
Data Analysis Procedure ◽  
Scientific Papers

Hypothesis testing is a well-known procedure for data analysis widely used in scientific papers but, at the same time, strongly criticized and its use questioned and restricted in some cases due to inconsistencies observed from their application. This issue is analyzed in this paper on the basis of the fundamentals of the statistical methodology and the different approaches that have been historically developed to solve the problem of statistical hypothesis analysis highlighting a not well known point: the P value is a random variable. The fundamentals of Fisher´s, Neyman-Pearson´s and Bayesian´s solutions are analyzed and based on them, the inconsistency of the commonly used procedure of determining a p value, compare it to a type I error value (usually 0.05) and get a conclusion is discussed and, on their basis, inconsistencies of the data analysis procedure are identified, procedure consisting in the identification of a P value, the comparison of the P-value with a type-I error value –which<br />is usually considered to be 0.05– and upon this the decision on the conclusions of the analysis. Additionally, recommendations on the<br />best way to proceed when solving a problem are presented, as well as the methodological and teaching challenges to be faced when analyzing correctly the data and determining the validity of the hypotheses.<br /><strong>Key words</strong>: Neyman-Pearson’s hypothesis tests, Fisher’s significance tests, Bayesian hypothesis tests, Vancouver norms, P-value, null-hypothesis.

One Size Fits All?

Methodology ◽  
2012 ◽  
Vol 8 (1) ◽  
pp. 23-38 ◽  
Author(s):  
Manuel C. Voelkle ◽  
Patrick E. McKnight
Keyword(s):  
Repeated Measures ◽  
Structural Equation ◽  
Type I Error ◽  
Equation Modeling ◽  
Type I ◽  
Finite Sample ◽  
Traditional Methods ◽  
Repeated Measures Anova ◽  
Special Cases ◽  
Latent Curve

The use of latent curve models (LCMs) has increased almost exponentially during the last decade. Oftentimes, researchers regard LCM as a “new” method to analyze change with little attention paid to the fact that the technique was originally introduced as an “alternative to standard repeated measures ANOVA and first-order auto-regressive methods” (Meredith & Tisak, 1990, p. 107). In the first part of the paper, this close relationship is reviewed, and it is demonstrated how “traditional” methods, such as the repeated measures ANOVA, and MANOVA, can be formulated as LCMs. Given that latent curve modeling is essentially a large-sample technique, compared to “traditional” finite-sample approaches, the second part of the paper addresses the question to what degree the more flexible LCMs can actually replace some of the older tests by means of a Monte-Carlo simulation. In addition, a structural equation modeling alternative to Mauchly’s (1940) test of sphericity is explored. Although “traditional” methods may be expressed as special cases of more general LCMs, we found the equivalence holds only asymptotically. For practical purposes, however, no approach always outperformed the other alternatives in terms of power and type I error, so the best method to be used depends on the situation. We provide detailed recommendations of when to use which method.

Two-Condition Within-Participant Statistical Mediation Analysis: A Path-Analytic Framework

2019 ◽  
Author(s):  
Amanda Kay Montoya ◽  
Andrew F. Hayes
Keyword(s):  
Confidence Intervals ◽  
Indirect Effect ◽  
Mediation Analysis ◽  
Analytic Approach ◽  
Analytic Framework ◽  
Hypothesis Tests ◽  
Bootstrap Confidence Intervals ◽  
Complex Models ◽  
Path Analytic ◽  
Statistical Mediation Analysis

Researchers interested in testing mediation often use designs where participants are measured on a dependent variable Y and a mediator M in both of two different circumstances. The dominant approach to assessing mediation in such a design, proposed by Judd, Kenny, and McClelland (2001), relies on a series of hypothesis tests about components of the mediation model and is not based on an estimate of or formal inference about the indirect effect. In this paper we recast Judd et al.’s approach in the path-analytic framework that is now commonly used in between-participant mediation analysis. By so doing, it is apparent how to estimate the indirect effect of a within-participant manipulation on some outcome through a mediator as the product of paths of influence. This path analytic approach eliminates the need for discrete hypothesis tests about components of the model to support a claim of mediation, as Judd et al’s method requires, because it relies only on an inference about the product of paths— the indirect effect. We generalize methods of inference for the indirect effect widely used in between-participant designs to this within-participant version of mediation analysis, including bootstrap confidence intervals and Monte Carlo confidence intervals. Using this path analytic approach, we extend the method to models with multiple mediators operating in parallel and serially and discuss the comparison of indirect effects in these more complex models. We offer macros and code for SPSS, SAS, and Mplus that conduct these analyses.

xtgeebcv: A command for bias-corrected sandwich variance estimation for GEE analyses of cluster randomized trials

2020 ◽  
Vol 20 (2) ◽  
pp. 363-381 ◽  
Author(s):  
John A. Gallis ◽  
Fan Li ◽  
Elizabeth L. Turner
Keyword(s):  
Randomized Trials ◽  
Variance Estimation ◽  
Public Health Education ◽  
Standard Errors ◽  
Future Research ◽  
Type I ◽  
Cluster Randomized Trials ◽  
Finite Sample ◽  
Individual Level ◽  
Cluster Randomized

Cluster randomized trials, where clusters (for example, schools or clinics) are randomized to comparison arms but measurements are taken on individuals, are commonly used to evaluate interventions in public health, education, and the social sciences. Analysis is often conducted on individual-level outcomes, and such analysis methods must consider that outcomes for members of the same cluster tend to be more similar than outcomes for members of other clusters. A popular individual-level analysis technique is generalized estimating equations (GEE). However, it is common to randomize a small number of clusters (for example, 30 or fewer), and in this case, the GEE standard errors obtained from the sandwich variance estimator will be biased, leading to inflated type I errors. Some bias-corrected standard errors have been proposed and studied to account for this finite-sample bias, but none has yet been implemented in Stata. In this article, we describe several popular bias corrections to the robust sandwich variance. We then introduce our newly created command, xtgeebcv, which will allow Stata users to easily apply finite-sample corrections to standard errors obtained from GEE models. We then provide examples to demonstrate the use of xtgeebcv. Finally, we discuss suggestions about which finite-sample corrections to use in which situations and consider areas of future research that may improve xtgeebcv.

scholarly journals Nonparametric bootstrap inference for the targeted highly adaptive least absolute shrinkage and selection operator (LASSO) estimator

2020 ◽  
Vol 0 (0) ◽  
Author(s):  
Weixin Cai ◽  
Mark van der Laan
Keyword(s):  
Confidence Intervals ◽  
Nonparametric Estimation ◽  
Cross Validation ◽  
Data Distribution ◽  
Average Treatment Effect ◽  
Finite Sample ◽  
Nonparametric Bootstrap ◽  
A Value ◽  
Variation Norm ◽  
Selection Operator

AbstractThe Highly-Adaptive least absolute shrinkage and selection operator (LASSO) Targeted Minimum Loss Estimator (HAL-TMLE) is an efficient plug-in estimator of a pathwise differentiable parameter in a statistical model that at minimal (and possibly only) assumes that the sectional variation norm of the true nuisance functions (i.e., relevant part of data distribution) are finite. It relies on an initial estimator (HAL-MLE) of the nuisance functions by minimizing the empirical risk over the parameter space under the constraint that the sectional variation norm of the candidate functions are bounded by a constant, where this constant can be selected with cross-validation. In this article we establish that the nonparametric bootstrap for the HAL-TMLE, fixing the value of the sectional variation norm at a value larger or equal than the cross-validation selector, provides a consistent method for estimating the normal limit distribution of the HAL-TMLE. In order to optimize the finite sample coverage of the nonparametric bootstrap confidence intervals, we propose a selection method for this sectional variation norm that is based on running the nonparametric bootstrap for all values of the sectional variation norm larger than the one selected by cross-validation, and subsequently determining a value at which the width of the resulting confidence intervals reaches a plateau. We demonstrate our method for 1) nonparametric estimation of the average treatment effect when observing a covariate vector, binary treatment, and outcome, and for 2) nonparametric estimation of the integral of the square of the multivariate density of the data distribution. In addition, we also present simulation results for these two examples demonstrating the excellent finite sample coverage of bootstrap-based confidence intervals.

ASYMPTOTICALLY UMP PANEL UNIT ROOT TESTS—THE EFFECT OF HETEROGENEITY IN THE ALTERNATIVES

2014 ◽  
Vol 31 (3) ◽  
pp. 539-559 ◽  
Author(s):  
I. Gaia Becheri ◽  
Feike C. Drost ◽  
Ramon van den Akker
Keyword(s):  
Unit Root ◽  
Generalized Least Squares ◽  
Applied Economics ◽  
Unit Root Tests ◽  
Finite Sample ◽  
Asymptotic Power ◽  
Optimal Tests ◽  
Independent Panel ◽  
Local Asymptotic Power ◽  
Uniformly Most Powerful

In a Gaussian, heterogeneous, cross-sectionally independent panel with incidental intercepts, Moon, Perron, and Phillips (2007, Journal of Econometrics 141, 416–459) present an asymptotic power envelope yielding an upper bound to the local asymptotic power of unit root tests. In case of homogeneous alternatives this envelope is known to be sharp, but this paper shows that it is not attainable for heterogeneous alternatives. Using limit experiment theory we derive a sharp power envelope. We also demonstrate that, among others, one of the likelihood ratio based tests in Moon et al. (2007, Journal of Econometrics 141, 416–459), a pooled generalized least squares (GLS) based test using the Breitung and Meyer (1994, Applied Economics 25, 353–361) device, and a new test based on the asymptotic structure of the model are all asymptotically UMP (Uniformly Most Powerful). Thus, perhaps somewhat surprisingly, pooled regression-based tests may yield optimal tests in case of heterogeneous alternatives. Although finite-sample powers are comparable, the new test is easy to implement and has superior size properties.

scholarly journals A SMOOTH TEST FOR THE EQUALITY OF DISTRIBUTIONS

2012 ◽  
Vol 29 (2) ◽  
pp. 419-446 ◽  
Author(s):  
Anil K. Bera ◽  
Aurobindo Ghosh ◽  
Zhijie Xiao
Keyword(s):  
New York ◽  
Goodness Of Fit ◽  
Age Distribution ◽  
Distribution Functions ◽  
Type I ◽  
Finite Sample ◽  
Community Rating ◽  
Smooth Test ◽  
Two Samples ◽  
Group Insurance

The two-sample version of the celebrated Pearson goodness-of-fit problem has been a topic of extensive research, and several tests like the Kolmogorov-Smirnov and Cramér-von Mises have been suggested. Although these tests perform fairly well as omnibus tests for comparing two probability density functions (PDFs), they may have poor power against specific departures such as in location, scale, skewness, and kurtosis. We propose a new test for the equality of two PDFs based on a modified version of the Neyman smooth test using empirical distribution functions minimizing size distortion in finite samples. The suggested test can detect the specific directions of departure from the null hypothesis. Specifically, it can identify deviations in the directions of mean, variance, skewness, or tail behavior. In a finite sample, the actual probability of type-I error depends on the relative sizes of the two samples. We propose two different approaches to deal with this problem and show that, under appropriate conditions, the proposed tests are asymptotically distributed as chi-squared. We also study the finite sample size and power properties of our proposed test. As an application of our procedure, we compare the age distributions of employees with small employers in New York and Pennsylvania with group insurance before and after the enactment of the “community rating” legislation in New York. It has been conventional wisdom that if community rating is enforced (where the group health insurance premium does not depend on age or any other physical characteristics of the insured), then the insurance market will collapse, since only older or less healthy patients would prefer group insurance. We find that there are significant changes in the age distribution in the population in New York owing mainly to a shift in location and scale.

Sign in / Sign up

Export Citation Format

Share Document