scholarly journals Using and Understanding Power in Psychological Research: A Survey Study

2021 ◽  
Vol 7 (1) ◽  
Author(s):  
Elizabeth Collins ◽  
Roger Watt
Keyword(s):  
Effect Size ◽  
Power Analysis ◽  
A Priori ◽  
Small Sample ◽  
Survey Study ◽  
Estimation Methods ◽  
P Value ◽  
Size Estimation ◽  
Sample Sizes ◽  
Effect Size Estimation

Statistical power is key to planning studies if understood and used correctly. Power is the probability of obtaining a statistically significant p-value, given a set alpha, sample size, and population effect size. The literature suggests that psychology studies are underpowered due to small sample sizes, and that researchers do not hold accurate intuitions about sensible sample sizes and associated levels of power. In this study, we surveyed 214 psychological researchers, and asked them about their experiences of using a priori power analysis, effect size estimation methods, post hoc power, and their understanding of what the term “power” actually means. Power analysis use was high, although participants reported difficulties with complex research designs, and effect size estimation. Participants also typically could not accurately define power. If psychological researchers are expected to compute a priori power analyses to plan their research, clearer educational material and guidelines should be made available.

scholarly journals Improving effect size estimation and statistical power with multi-echo fMRI and its impact on understanding the neural systems supporting mentalizing

2015 ◽  
Author(s):  
Michael V. Lombardo ◽  
Bonnie Auyeung ◽  
Rosemary J. Holt ◽  
Jack Waldman ◽  
Amber N. V. Ruigrok ◽  
...  
Keyword(s):  
Effect Size ◽  
Statistical Power ◽  
Small Sample ◽  
Cognitive Domain ◽  
Group Level ◽  
Size Estimation ◽  
Sample Sizes ◽  
Brain Organization ◽  
Effect Size Estimation ◽  
Size Estimates

AbstractFunctional magnetic resonance imaging (fMRI) research is routinely criticized for being statistically underpowered due to characteristically small sample sizes and much larger sample sizes are being increasingly recommended. Additionally, various sources of artifact inherent in fMRI data can have detrimental impact on effect size estimates and statistical power. Here we show how specific removal of non-BOLD artifacts can improve effect size estimation and statistical power in task-fMRI contexts, with particular application to the social-cognitive domain of mentalizing/theory of mind. Non-BOLD variability identification and removal is achieved in a biophysical and statistically principled manner by combining multi-echo fMRI acquisition and independent components analysis (ME-ICA). Group-level effect size estimates on two different mentalizing tasks were enhanced by ME-ICA at a median rate of 24% in regions canonically associated with mentalizing, while much more substantial boosts (40-149%) were observed in non-canonical cerebellar areas. This effect size boosting is primarily a consequence of reduction of non-BOLD noise at the subject-level, which then translates into consequent reductions in between-subject variance at the group-level. Power simulations demonstrate that enhanced effect size enables highly-powered studies at traditional sample sizes. Cerebellar effects observed after applying ME-ICA may be unobservable with conventional imaging at traditional sample sizes. Thus, ME-ICA allows for principled design-agnostic non-BOLD artifact removal that can substantially improve effect size estimates and statistical power in task-fMRI contexts. ME-ICA could help issues regarding statistical power and non-BOLD noise and enable potential for novel discovery of aspects of brain organization that are currently under-appreciated and not well understood.

Effect size estimation: Methods and examples

2012 ◽  
Vol 49 (8) ◽  
pp. 1039-1047 ◽  
Author(s):  
Lut Berben ◽  
Susan M. Sereika ◽  
Sandra Engberg
Keyword(s):  
Effect Size ◽  
Estimation Methods ◽  
Size Estimation ◽  
Effect Size Estimation

scholarly journals Contrasts and Correlations in Effect-Size Estimation

2000 ◽  
Vol 11 (6) ◽  
pp. 446-453 ◽  
Author(s):  
Ralph L. Rosnow ◽  
Robert Rosenthal ◽  
Donald B. Rubin
Keyword(s):  
Effect Size ◽  
Real Life ◽  
Size Estimation ◽  
Sample Sizes ◽  
Standardized Measures ◽  
The Real ◽  
Two Samples ◽  
Correlational Approach ◽  
Effect Size Estimation ◽  
Size Display

This article describes procedures for presenting standardized measures of effect size when contrasts are used to ask focused questions of data. The simplest contrasts consist of comparisons of two samples (e.g., based on the independent t statistic). Useful effect-size indices in this situation are members of the g family (e.g., Hedges's g and Cohen's d) and the Pearson r. We review expressions for calculating these measures and for transforming them back and forth, and describe how to adjust formulas for obtaining g or d from t, or r from g, when the sample sizes are unequal. The real-life implications of d or g calculated from t become problematic when there are more than two groups, but the correlational approach is adaptable and interpretable, although more complex than in the case of two groups. We describe a family of four conceptually related correlation indices: the alerting correlation, the contrast correlation, the effect-size correlation, and the BESD (binomial effect-size display) correlation. These last three correlations are identical in the simple setting of only two groups, but differ when there are more than two groups.

scholarly journals A Multi-faceted Mess: A Review of Statistical Power Analysis in Psychology Journal Articles

2019 ◽  
Author(s):  
Rob Cribbie ◽  
Nataly Beribisky ◽  
Udi Alter
Keyword(s):  
Sample Size ◽  
Effect Size ◽  
Power Analysis ◽  
Statistical Power ◽  
Type I Error ◽  
A Priori ◽  
Type I ◽  
Specific Level ◽  
Maximum Sample Size ◽  
Power Analyses

Many bodies recommend that a sample planning procedure, such as traditional NHST a priori power analysis, is conducted during the planning stages of a study. Power analysis allows the researcher to estimate how many participants are required in order to detect a minimally meaningful effect size at a specific level of power and Type I error rate. However, there are several drawbacks to the procedure that render it “a mess.” Specifically, the identification of the minimally meaningful effect size is often difficult but unavoidable for conducting the procedure properly, the procedure is not precision oriented, and does not guide the researcher to collect as many participants as feasibly possible. In this study, we explore how these three theoretical issues are reflected in applied psychological research in order to better understand whether these issues are concerns in practice. To investigate how power analysis is currently used, this study reviewed the reporting of 443 power analyses in high impact psychology journals in 2016 and 2017. It was found that researchers rarely use the minimally meaningful effect size as a rationale for the chosen effect in a power analysis. Further, precision-based approaches and collecting the maximum sample size feasible are almost never used in tandem with power analyses. In light of these findings, we offer that researchers should focus on tools beyond traditional power analysis when sample planning, such as collecting the maximum sample size feasible.

scholarly journals Statistical agnostic mapping: a framework in neuroimaging based on concentration inequalities

2019 ◽  
Author(s):  
J.M. Gorriz ◽  
◽  
◽  
Keyword(s):  
Large Error ◽  
Small Sample ◽  
Concentration Inequalities ◽  
P Value ◽  
Sample Sizes ◽  
Limited Sample ◽  
Pattern Recognition Problem ◽  
Actual Risk ◽  
Out Of Sample

ABSTRACTIn the 70s a novel branch of statistics emerged focusing its effort in selecting a function in the pattern recognition problem, which fulfils a definite relationship between the quality of the approximation and its complexity. These data-driven approaches are mainly devoted to problems of estimating dependencies with limited sample sizes and comprise all the empirical out-of sample generalization approaches, e.g. cross validation (CV) approaches. Although the latter are not designed for testing competing hypothesis or comparing different models in neuroimaging, there are a number of theoretical developments within this theory which could be employed to derive a Statistical Agnostic (non-parametric) Mapping (SAM) at voxel or multi-voxel level. Moreover, SAMs could relieve i) the problem of instability in limited sample sizes when estimating the actual risk via the CV approaches, e.g. large error bars, and provide ii) an alternative way of Family-wise-error (FWE) corrected p-value maps in inferential statistics for hypothesis testing. In this sense, we propose a novel framework in neuroimaging based on concentration inequalities, which results in (i) a rigorous development for model validation with a small sample/dimension ratio, and (ii) a less-conservative procedure than FWE p-value correction, to determine the brain significance maps from the inferences made using small upper bounds of the actual risk.

On determining sample size in experiments involving laboratory animals

2018 ◽  
Vol 52 (4) ◽  
pp. 341-350 ◽  
Author(s):  
Michael FW Festing
Keyword(s):  
Sample Size ◽  
Common Sense ◽  
Effect Size ◽  
Power Analysis ◽  
Laboratory Animals ◽  
Sample Sizes

Scientists using laboratory animals are under increasing pressure to justify their sample sizes using a “power analysis”. In this paper I review the three methods currently used to determine sample size: “tradition” or “common sense”, the “resource equation” and the “power analysis”. I explain how, using the “KISS” approach, scientists can make a provisional choice of sample size using any method, and then easily estimate the effect size likely to be detectable according to a power analysis. Should they want to be able to detect a smaller effect they can increase their provisional sample size and recalculate the effect size. This is simple, does not need any software and provides justification for the sample size in the terms used in a power analysis.

Sign in / Sign up

Export Citation Format

Share Document