Sample size determination for Bayesian hierarchical models commonly used in psycholinguistics

Although Bayesian data analysis has the great advantage that one need not specify the sample size in advance of running an experiment, there are nevertheless situations where it becomes necessary to have at least an initial ballpark estimate for a target sample size. An example where this becomes necessary is grant applications. In this paper, we adapt a simulation-based method proposed by Wang and Gelfand, 2002 (A simulation-based approach to Bayesian sample size determination for performance under a given model and for separating models. Statistical Science, 193-208) for a Bayes-factor based design analysis. We demonstrate how relatively complex hierarchical models (which are commonly used in psycholinguistics) can be used to determine approximate sample sizes for planning experiments. The code is available for researchers to adapt for their own purposes and applications at https://osf.io/hjgrm/.

Bayesian sample size re-estimation using power priors

The sample size of a randomized controlled trial is typically chosen in order for frequentist operational characteristics to be retained. For normally distributed outcomes, an assumption for the variance needs to be made which is usually based on limited prior information. Especially in the case of small populations, the prior information might consist of only one small pilot study. A Bayesian approach formalizes the aggregation of prior information on the variance with newly collected data. The uncertainty surrounding prior estimates can be appropriately modelled by means of prior distributions. Furthermore, within the Bayesian paradigm, quantities such as the probability of a conclusive trial are directly calculated. However, if the postulated prior is not in accordance with the true variance, such calculations are not trustworthy. In this work we adapt previously suggested methodology to facilitate sample size re-estimation. In addition, we suggest the employment of power priors in order for operational characteristics to be controlled.