Assessing the Global and Local Uncertainty of Scientific Evidence in the Presence of Model Misspecification

Scientists need to compare the support for models based on observed phenomena. The main goal of the evidential paradigm is to quantify the strength of evidence in the data for a reference model relative to an alternative model. This is done via an evidence function, such as ΔSIC, an estimator of the sample size scaled difference of divergences between the generating mechanism and the competing models. To use evidence, either for decision making or as a guide to the accumulation of knowledge, an understanding of the uncertainty in the evidence is needed. This uncertainty is well characterized by the standard statistical theory of estimation. Unfortunately, the standard theory breaks down if the models are misspecified, as is commonly the case in scientific studies. We develop non-parametric bootstrap methodologies for estimating the sampling distribution of the evidence estimator under model misspecification. This sampling distribution allows us to determine how secure we are in our evidential statement. We characterize this uncertainty in the strength of evidence with two different types of confidence intervals, which we term “global” and “local.” We discuss how evidence uncertainty can be used to improve scientific inference and illustrate this with a reanalysis of the model identification problem in a prominent landscape ecology study using structural equations.

Convergence Analysis of the Hessian Estimation Evolution Strategy

The class of algorithms called Hessian Estimation Evolution Strategies (HE-ESs) update the covariance matrix of their sampling distribution by directly estimating the curvature of the objective function. The approach is practically efficient, as attested by respectable performance on the BBOB testbed, even on rather irregular functions. In this paper we formally prove two strong guarantees for the (1+4)-HE-ES, a minimal elitist member of the family: stability of the covariance matrix update, and as a consequence, linear convergence on all convex quadratic problems at a rate that is independent of the problem instance.

WAYS IN WHICH HIGH-SCHOOL STUDENTS UNDERSTAND THE SAMPLING DISTRIBUTION FOR PROPORTIONS

In Spain, curricular guidelines as well as the university-entrance tests for social-science high-school students (17–18 years old) include sampling distributions. To analyse the understanding of this concept we investigated a sample of 234 students. We administered a questionnaire to them and ask half for justifications of their answers. The questionnaire consisted of four sampling tasks with two sample sizes (n = 100 and 10) and population proportions (equal or different to 0.5)systematically varied. The experiment gathered twofold data from the students simultaneously, namely about their perception of the mean and about their understanding of variation of the sampling distribution. The analysis of students’ responses indicates a good understanding of the relationship between the theoretical proportion in the population and the sample proportion. Sampling variability, however, was overestimated in bigger samples. We also observed various types of biased thinking in the students: the equiprobability and recency biases, as well as deterministic pre-conceptions. The effect of the task variables on the students’ responses is also discussed here. First published December 2020 at Statistics Education Research Journal: Archives

Estimating the fundamental niche: accounting for the uneven availability of existing climates

In the last years, studies that question important conceptual and methodological aspects in the field of ecological niche modeling (and species distribution modeling) have cast doubts on the validity of the existing methodologies. Particularly, it has been broadly discussed whether it is possible to estimate the fundamental niche of a species using presence data. Although it has being identified that the main limitation is that presence data come from the realized niche, which is a subset of the fundamental niche, most of the existing methods lack the ability to overcome it, and then, they fit objects that are more similar to the realized niche. To overcome this limitation, we propose to use the region that is accessible to the species (based on its dispersal abilities) to determine a sampling distribution in environmental space that allow us to quantify the likelihood of observing a particular environmental combination in a sample of presence points. We incorporate this sampling distribution into a multivariate normal model (Mahalanobis model) by creating a weight function that modifies the probabilities of observing an environmental combination in a sample of presences as a way to account for the uneven availability of environmental conditions. We show that the parameters of the modified, weighted-normal model can be approximated by a maximum likelihood estimation approach, and used to draw ellipsoids (confidence regions) that represent the shape of the fundamental niche of the species. We illustrate the application of our model with two worked examples: (i) using presence points of an invasive species and an accessible area that includes only its native range, to evaluate whether the fitted model predicts confirmed establishments of the species outside its native range, and (ii) using presence data of closely related species with known accessible areas to exhibit how the different dispersal abilities of the species constraint a classic Mahalanobis model. Taking into account the distribution of environmental conditions that are accessible to the species indeed affected the estimation of the ellipsoids used to model their fundamental niches.