The conditionality principle in high-dimensional regression

Summary Consider a high-dimensional linear regression problem, where the number of covariates is larger than the number of observations and the interest is in estimating the conditional variance of the response variable given the covariates. A conditional and an unconditional framework are considered, where conditioning is with respect to the covariates, which are ancillary to the parameter of interest. In recent papers, a consistent estimator was developed in the unconditional framework when the marginal distribution of the covariates is normal with known mean and variance. In the present work, a certain Bayesian hypothesis test is formulated under the conditional framework, and it is shown that the Bayes risk is a constant. This implies that no consistent estimator exists in the conditional framework. However, when the marginal distribution of the covariates is normal, the conditional error of the above consistent estimator converges to zero, with probability converging to one. It follows that even in the conditional setting, information about the marginal distribution of an ancillary statistic may have a significant impact on statistical inference. The practical implication in the context of high-dimensional regression models is that additional observations where only the covariates are given are potentially very useful and should not be ignored. This finding is most relevant to semi-supervised learning problems where covariate information is easy to obtain.

MULTIMODALITY p**-FORMULA AND CONFIDENCE REGIONS

Barndorff-Nielsen’s celebrated p*-formula and variations thereof have amongst their various attractions the ability to approximate bimodal distributions. In this paper we show that in general this requires a crucial adjustment to the basic formula. The adjustment is based on a simple idea and straightforward to implement, yet delivers important improvements. It is based on recognizing that certain outcomes are theoretically impossible and the density of the MLE should then equal zero, rather than the positive density that a straight application of p* would suggest. This has implications for inference and we show how to use the new p**-formula to construct improved confidence regions. These can be disjoint as a consequence of the bimodality. The degree of bimodality depends heavily on the value of an approximate ancillary statistic and conditioning on the observed value of this statistic is therefore desirable. The p**-formula naturally delivers the relevant conditional distribution. We illustrate these results in small and large samples using a simple nonlinear regression model and errors in variables model where the measurement errors in dependent and explanatory variables are correlated and allow for weak proxies.

Unconditional Optimality of Conditional Tests in the Presence of Nuisance Parameters

The results of Chatterjee and Chanopadhyay (1990) are extended here to the case where nuisance parameters are present. The extension involves the inclusion of the nuisance parameter alongwith the partial ancillary statistic in the loss functions. It is shown that the usual best conditional test is actually unconditionally optimum irrespective of any knowledge about the nuisance parameter. Some standard examples are considered where loss functions can be so chosen as to realise certain intuitively reasonable properties.

Ancillarity and the Limited Information Maximum-Likelihood Estimation of a Structural Equation in a Simultaneous Equation System

The concepts of the curved exponential family of distributions and ancillarity are applied to estimation problems of a single structural equation in a simultaneous equation model, and the effect of conditioning on ancillary statistics on the limited information maximum-likelihood (LIML) estimator is investigated. The asymptotic conditional covariance matrix of the LIML estimator conditioned on the second-order asymptotic maximal ancillary statistic is shown to be efficiently estimated by Liu and Breen's formula. The effect of conditioning on a second-order asymptotic ancillary statistic, i.e., the smallest characteristic root associated with the LIML estimation, is analyzed by means of an asymptotic expansion of the distribution as well as the exact distribution. The smallest root helps to give an intuitively appealing measure of precision of the LIML estimator.

Geometrical theory of higher-order asymptotics of test, interval estimator and conditional inference

Higher order asymptotic powers of one-sided and two-sided tests, both conditional and unconditional, are evaluated for a one-parameter curved exponential family of distributions by using differential-geometrical notions. Higher order asymptotic powers and the expected size of one-sided and two-sided interval estimators, both conditional and unconditional, are also obtained. The tests and interval estimators, which are third-order most powerful at any arbitrary specified one point are explicitly designed with the help of the approximate ancillary statistic. The characteristics of the conditional inference are elucidated, by proving the equivalence up to the third order of the conditional inference and likelihood ratio inference. Geometrical notions such as curvatures and angles play a fundamental role in the present theory.