scholarly journals Sample Size Determination and Optimal Design of Simple Pretest-Posttest Experimental Designs: Introduction, Software, and Illustrations

2021 ◽  
Author(s):  
Metin Bulus
Keyword(s):  
Optimal Design ◽  
Sample Size ◽  
Small Sample ◽  
Experimental Designs ◽  
Sample Size Determination ◽  
Size Determination ◽  
Sample Sizes ◽  
Control Groups ◽  
Small Sample Sizes ◽  
And Control

A recent systematic review of experimental studies conducted in Turkey between 2010 and 2020 reported that small sample sizes had been a significant drawback (Bulus and Koyuncu, 2021). A small chunk of the studies were small-scale true experiments (subjects randomized into the treatment and control groups). The remaining studies consisted of quasi-experiments (subjects in treatment and control groups were matched on pretest or other covariates) and weak experiments (neither randomized nor matched but had the control group). They had an average sample size below 70 for different domains and outcomes. These small sample sizes imply a strong (and perhaps erroneous) assumption about the minimum relevant effect size (MRES) of intervention before an experiment is conducted; that is, a standardized intervention effect of Cohen’s d < 0.50 is not relevant to education policy or practice. Thus, an introduction to sample size determination for pretest-posttest simple experimental designs is warranted. This study describes nuts and bolts of sample size determination, derives expressions for optimal design under differential cost per treatment and control units, provide convenient tables to guide sample size decisions for MRES values between 0.20 ≤ Cohen’s d ≤ 0.50, and describe the relevant software along with illustrations.

scholarly journals What can we Learn from Studies Based on Small Sample Sizes? Comment on Regan, Lakhanpal, and Anguiano (2012)

2013 ◽  
Vol 113 (1) ◽  
pp. 221-224 ◽  
Author(s):  
David R. Johnson ◽  
Lauren K. Bachan
Keyword(s):  
Sample Size ◽  
Recent Article ◽  
Small Sample ◽  
Sample Sizes ◽  
Relationship Outcomes ◽  
Small Sample Sizes

In a recent article, Regan, Lakhanpal, and Anguiano (2012) highlighted the lack of evidence for different relationship outcomes between arranged and love-based marriages. Yet the sample size ( n = 58) used in the study is insufficient for making such inferences. This reply discusses and demonstrates how small sample sizes reduce the utility of this research.

scholarly journals Efficient and flexible simulation-based sample size determination for clinical trials with multiple design parameters

2020 ◽  
pp. 096228022097579
Author(s):  
Duncan T Wilson ◽  
Richard Hooper ◽  
Julia Brown ◽  
Amanda J Farrin ◽  
Rebecca EA Walwyn
Keyword(s):  
Sample Size ◽  
Computer Experiments ◽  
Small Sample ◽  
Sample Size Determination ◽  
Design Parameters ◽  
Size Determination ◽  
Parametric Regression ◽  
Software Packages ◽  
Primary Endpoints ◽  
Simulation Based

Simulation offers a simple and flexible way to estimate the power of a clinical trial when analytic formulae are not available. The computational burden of using simulation has, however, restricted its application to only the simplest of sample size determination problems, often minimising a single parameter (the overall sample size) subject to power being above a target level. We describe a general framework for solving simulation-based sample size determination problems with several design parameters over which to optimise and several conflicting criteria to be minimised. The method is based on an established global optimisation algorithm widely used in the design and analysis of computer experiments, using a non-parametric regression model as an approximation of the true underlying power function. The method is flexible, can be used for almost any problem for which power can be estimated using simulation, and can be implemented using existing statistical software packages. We illustrate its application to a sample size determination problem involving complex clustering structures, two primary endpoints and small sample considerations.

The Relationship Between Sample Sizes and Effect Sizes in Systematic Reviews in Education

2009 ◽  
Vol 31 (4) ◽  
pp. 500-506 ◽  
Author(s):  
Robert Slavin ◽  
Dewi Smith
Keyword(s):  
Sample Size ◽  
Effect Size ◽  
Secondary Mathematics ◽  
Significant Negative Correlation ◽  
Small Sample ◽  
Effect Sizes ◽  
Sample Sizes ◽  
Large Samples ◽  
Small Sample Sizes ◽  
The Relationship

Research in fields other than education has found that studies with small sample sizes tend to have larger effect sizes than those with large samples. This article examines the relationship between sample size and effect size in education. It analyzes data from 185 studies of elementary and secondary mathematics programs that met the standards of the Best Evidence Encyclopedia. As predicted, there was a significant negative correlation between sample size and effect size. The differences in effect sizes between small and large experiments were much greater than those between randomized and matched experiments. Explanations for the effects of sample size on effect size are discussed.

scholarly journals Adequate Sample Sizes for a Three-Level Growth Model

2021 ◽  
Vol 12 ◽  
Author(s):  
Eunsoo Lee ◽  
Sehee Hong
Keyword(s):  
Sample Size ◽  
Growth Model ◽  
Multilevel Models ◽  
Growth Models ◽  
Intraclass Correlation ◽  
Small Sample ◽  
Sample Sizes ◽  
Level 3 ◽  
Unbiased Estimates ◽  
Small Sample Sizes

Multilevel models have been developed for addressing data that come from a hierarchical structure. In particular, due to the increase of longitudinal studies, a three-level growth model is frequently used to measure the change of individuals who are nested in groups. In multilevel modeling, sufficient sample sizes are needed to obtain unbiased estimates and enough power to detect individual or group effects. However, there are few sample size guidelines for three-level growth models. Therefore, it is important that researchers recognize the possibility of unreliable results when sample sizes are small. The purpose of this study is to find adequate sample sizes for a three-level growth model under realistic conditions. A Monte Carlo simulation was performed under 12 conditions: (1) level-2 sample size (10, 30), (2) level-3 sample size (30, 50, 100) (3) intraclass correlation at level-3 (0.05, 0.15). The study examined the following outcomes: convergence rate, relative parameter bias, mean square error (MSE), 95% coverage rate and power. The results indicate that estimates of the regression coefficients are unbiased, but the variance component tends to be inaccurate with small sample sizes.

scholarly journals A Note on Inferences About the Probability of Success

2020 ◽  
Vol 18 (1) ◽  
Author(s):  
Rand Wilcox
Keyword(s):  
Confidence Interval ◽  
Sample Size ◽  
Small Sample ◽  
Extensive Literature ◽  
Sample Sizes ◽  
Probability Of Success ◽  
Small Sample Sizes ◽  
A Minor ◽  
Sample Case

There is an extensive literature dealing with inferences about the probability of success. A minor goal in this note is to point out when certain recommended methods can be unsatisfactory when the sample size is small. The main goal is to report results on the two-sample case. Extant results suggest using one of four methods. The results indicate when computing a 0.95 confidence interval, two of these methods can be more satisfactory when dealing with small sample sizes.

A quantum leap in the reproducibility, precision, and sensitivity of gene expression profile analysis even when sample size is extremely small

2015 ◽  
Vol 13 (04) ◽  
pp. 1550018 ◽  
Author(s):  
Kevin Lim ◽  
Zhenhua Li ◽  
Kwok Pui Choi ◽  
Limsoon Wong
Keyword(s):  
Gene Expression ◽  
Sample Size ◽  
Small Sample Size ◽  
Statistical Tests ◽  
Transcript Level ◽  
Small Sample ◽  
Sample Sizes ◽  
Quantum Leap ◽  
Two Samples ◽  
Small Sample Sizes

Transcript-level quantification is often measured across two groups of patients to aid the discovery of biomarkers and detection of biological mechanisms involving these biomarkers. Statistical tests lack power and false discovery rate is high when sample size is small. Yet, many experiments have very few samples (≤ 5). This creates the impetus for a method to discover biomarkers and mechanisms under very small sample sizes. We present a powerful method, ESSNet, that is able to identify subnetworks consistently across independent datasets of the same disease phenotypes even under very small sample sizes. The key idea of ESSNet is to fragment large pathways into smaller subnetworks and compute a statistic that discriminates the subnetworks in two phenotypes. We do not greedily select genes to be included based on differential expression but rely on gene-expression-level ranking within a phenotype, which is shown to be stable even under extremely small sample sizes. We test our subnetworks on null distributions obtained by array rotation; this preserves the gene–gene correlation structure and is suitable for datasets with small sample size allowing us to consistently predict relevant subnetworks even when sample size is small. For most other methods, this consistency drops to less than 10% when we test them on datasets with only two samples from each phenotype, whereas ESSNet is able to achieve an average consistency of 58% (72% when we consider genes within the subnetworks) and continues to be superior when sample size is large. We further show that the subnetworks identified by ESSNet are highly correlated to many references in the biological literature. ESSNet and supplementary material are available at: http://compbio.ddns.comp.nus.edu.sg:8080/essnet .

scholarly journals Efficacy of St. John's Wort for Treating Mild to Moderate Depression

2007 ◽  
Vol 12 (3) ◽  
pp. 184-195 ◽  
Author(s):  
Tera Gahlsdorf ◽  
Robert Krause ◽  
Margaret W. Beal
Keyword(s):  
Small Sample ◽  
Sample Sizes ◽  
Control Groups ◽  
St John’S Wort ◽  
St John's Wort ◽  
Moderate Depression ◽  
Pharmaceutical Group ◽  
Conflicting Evidence ◽  
Small Sample Sizes ◽  
Larger Sample

Current studies regarding the efficacy of the herb St. John's wort (SJW) in treating mild to moderate cases of depression show conflicting evidence. In this article, we review the literature and consider similarities and differences between studies showing some efficacy and those showing none. Twelve published reports were reviewed. The majority of studies indicated the efficacy of SJW in the treatment of mild to moderate cases of depression. Most trials have had small sample sizes and either a placebo group or a standard pharmaceutical group. Two studies (both pediatric) were uncontrolled. Studies generally reported outcomes that had positive implications for their financial supporters and/or those with whom the primary investigators had acknowledged financial affiliations. More studies that have larger sample sizes and include placebo and pharmaceutical control groups are needed.

Sign in / Sign up

Export Citation Format

Share Document