Sample Size and Accuracy of Estimates in Multilevel Models

Methodology ◽  
2011 ◽  
Vol 7 (3) ◽  
pp. 111-120 ◽  
Author(s):  
Omar Paccagnella
Keyword(s):  
Variance Components ◽  
Multilevel Models ◽  
Intraclass Correlation ◽  
Correlation Coefficients ◽  
Estimation Procedure ◽  
Small Sample ◽  
Standard Errors ◽  
Sample Sizes ◽  
Monte Carlo Simulation Study ◽  
Small Sample Sizes

In a multilevel framework several researches have investigated the behavior of estimates in finite samples, particularly for continuous dependent variables. Some findings show poor precise estimates for the variance components. On the other hand, discrete response multilevel models have been investigated less widely. In this paper we analyze the influence of different factors on the accuracy of estimates and standard errors of estimates in a binary response 2-level model, through a Monte Carlo simulation study. We investigate the hypothesis of: (a) small sample sizes; (b) different intraclass correlation coefficients; (c) different numbers of quadrature points in the estimation procedure. Standard errors of estimates are studied through a noncoverage indicator. In all instances we have considered, the point estimates are unbiased (even with very small sample sizes), while the variance components are underestimated. The accuracy of the standard errors of variance estimates needs a very large number of groups.

scholarly journals Adequate Sample Sizes for a Three-Level Growth Model

2021 ◽  
Vol 12 ◽  
Author(s):  
Eunsoo Lee ◽  
Sehee Hong
Keyword(s):  
Sample Size ◽  
Growth Model ◽  
Multilevel Models ◽  
Growth Models ◽  
Intraclass Correlation ◽  
Small Sample ◽  
Sample Sizes ◽  
Level 3 ◽  
Unbiased Estimates ◽  
Small Sample Sizes

Multilevel models have been developed for addressing data that come from a hierarchical structure. In particular, due to the increase of longitudinal studies, a three-level growth model is frequently used to measure the change of individuals who are nested in groups. In multilevel modeling, sufficient sample sizes are needed to obtain unbiased estimates and enough power to detect individual or group effects. However, there are few sample size guidelines for three-level growth models. Therefore, it is important that researchers recognize the possibility of unreliable results when sample sizes are small. The purpose of this study is to find adequate sample sizes for a three-level growth model under realistic conditions. A Monte Carlo simulation was performed under 12 conditions: (1) level-2 sample size (10, 30), (2) level-3 sample size (30, 50, 100) (3) intraclass correlation at level-3 (0.05, 0.15). The study examined the following outcomes: convergence rate, relative parameter bias, mean square error (MSE), 95% coverage rate and power. The results indicate that estimates of the regression coefficients are unbiased, but the variance component tends to be inaccurate with small sample sizes.

scholarly journals vICC: Varying Intraclass Correlation Coefficients in R

2020 ◽  
Author(s):  
Donald Ray Williams
Keyword(s):  
Variance Components ◽  
Multilevel Models ◽  
Wide Spectrum ◽  
Intraclass Correlation ◽  
Correlation Coefficients ◽  
R Package ◽  
Group Level ◽  
Future Directions ◽  
Underlying Assumption ◽  
Intraclass Correlation Coefficients

In mixed-effects (a.k.a, hierarchical or multilevel) models, intraclass correlation coefficients (ICC) are commonly computed, with applications spanning from characterizing group-level homogeneity to measurement reliability. While there are a wide spectrum of applications, an underlying assumption of each is that the variance components used in their computation are fixed and non-varying. The methodology in the R package \textbf{vICC} was specifically designedto quantify variation in the ICC. This can be used to identify groups that are more (or less) homogeneous, as well as which groups are adequately described by the non-varying ICC. I end with future directions.

No Evidence that Experiencing Physical Warmth Promotes Interpersonal Warmth: Two Failures to Replicate Williams and Bargh (2008)

2018 ◽  
Author(s):  
Christopher Chabris ◽  
Patrick Ryan Heck ◽  
Jaclyn Mandart ◽  
Daniel Jacob Benjamin ◽  
Daniel J. Simons
Keyword(s):  
Null Hypothesis ◽  
Small Sample ◽  
Sample Sizes ◽  
Double Blind ◽  
Bayesian Analyses ◽  
Physical Warmth ◽  
Small Sample Sizes ◽  
Interpersonal Warmth

Williams and Bargh (2008) reported that holding a hot cup of coffee caused participants to judge a person’s personality as warmer, and that holding a therapeutic heat pad caused participants to choose rewards for other people rather than for themselves. These experiments featured large effects (r = .28 and .31), small sample sizes (41 and 53 participants), and barely statistically significant results. We attempted to replicate both experiments in field settings with more than triple the sample sizes (128 and 177) and double-blind procedures, but found near-zero effects (r = –.03 and .02). In both cases, Bayesian analyses suggest there is substantially more evidence for the null hypothesis of no effect than for the original physical warmth priming hypothesis.

scholarly journals G-computation and machine learning for estimating the causal effects of binary exposure statuses on binary outcomes

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Florent Le Borgne ◽  
Arthur Chatton ◽  
Maxime Léger ◽  
Rémi Lenain ◽  
Yohann Foucher
Keyword(s):  
Machine Learning ◽  
Support Vector Machine ◽  
Statistical Power ◽  
Small Sample ◽  
Causal Effects ◽  
Small Samples ◽  
Support Vector ◽  
Sample Sizes ◽  
Super Learner ◽  
Small Sample Sizes

AbstractIn clinical research, there is a growing interest in the use of propensity score-based methods to estimate causal effects. G-computation is an alternative because of its high statistical power. Machine learning is also increasingly used because of its possible robustness to model misspecification. In this paper, we aimed to propose an approach that combines machine learning and G-computation when both the outcome and the exposure status are binary and is able to deal with small samples. We evaluated the performances of several methods, including penalized logistic regressions, a neural network, a support vector machine, boosted classification and regression trees, and a super learner through simulations. We proposed six different scenarios characterised by various sample sizes, numbers of covariates and relationships between covariates, exposure statuses, and outcomes. We have also illustrated the application of these methods, in which they were used to estimate the efficacy of barbiturates prescribed during the first 24 h of an episode of intracranial hypertension. In the context of GC, for estimating the individual outcome probabilities in two counterfactual worlds, we reported that the super learner tended to outperform the other approaches in terms of both bias and variance, especially for small sample sizes. The support vector machine performed well, but its mean bias was slightly higher than that of the super learner. In the investigated scenarios, G-computation associated with the super learner was a performant method for drawing causal inferences, even from small sample sizes.

scholarly journals What can we Learn from Studies Based on Small Sample Sizes? Comment on Regan, Lakhanpal, and Anguiano (2012)

2013 ◽  
Vol 113 (1) ◽  
pp. 221-224 ◽  
Author(s):  
David R. Johnson ◽  
Lauren K. Bachan
Keyword(s):  
Sample Size ◽  
Recent Article ◽  
Small Sample ◽  
Sample Sizes ◽  
Relationship Outcomes ◽  
Small Sample Sizes

In a recent article, Regan, Lakhanpal, and Anguiano (2012) highlighted the lack of evidence for different relationship outcomes between arranged and love-based marriages. Yet the sample size ( n = 58) used in the study is insufficient for making such inferences. This reply discusses and demonstrates how small sample sizes reduce the utility of this research.

Sign in / Sign up

Export Citation Format

Share Document