On the Difference between a-priori and observed statistical power — A comment on “statistical power and sample size calculations: A primer for pediatric surgeons”

2020 ◽  
Vol 55 (1) ◽  
pp. 203-205
Author(s):  
Arne Schröder ◽  
Christina Oetzmann von Sochaczewski
Keyword(s):  
Sample Size ◽  
Statistical Power ◽  
A Priori ◽  
Sample Size Calculations ◽  
The Difference

scholarly journals Statistical Primer for Athletic Trainers: Understanding the Role of Statistical Power in Comparative Athletic Training Research

2018 ◽  
Vol 53 (7) ◽  
pp. 716-719
Author(s):  
Monica R. Lininger ◽  
Bryan L. Riemann
Keyword(s):  
Sample Size ◽  
Athletic Training ◽  
Treatment Effect ◽  
Statistical Power ◽  
Athletic Trainers ◽  
A Priori ◽  
Significant Treatment ◽  
Training Research ◽  
Α Level

Objective: To describe the concept of statistical power as related to comparative interventions and how various factors, including sample size, affect statistical power.Background: Having a sufficiently sized sample for a study is necessary for an investigation to demonstrate that an effective treatment is statistically superior. Many researchers fail to conduct and report a priori sample-size estimates, which then makes it difficult to interpret nonsignificant results and causes the clinician to question the planning of the research design.Description: Statistical power is the probability of statistically detecting a treatment effect when one truly exists. The α level, a measure of differences between groups, the variability of the data, and the sample size all affect statistical power.Recommendations: Authors should conduct and provide the results of a priori sample-size estimations in the literature. This will assist clinicians in determining whether the lack of a statistically significant treatment effect is due to an underpowered study or to a treatment's actually having no effect.

scholarly journals A Multi-faceted Mess: A Review of Statistical Power Analysis in Psychology Journal Articles

2019 ◽  
Author(s):  
Rob Cribbie ◽  
Nataly Beribisky ◽  
Udi Alter
Keyword(s):  
Sample Size ◽  
Effect Size ◽  
Power Analysis ◽  
Statistical Power ◽  
Type I Error ◽  
A Priori ◽  
Type I ◽  
Specific Level ◽  
Maximum Sample Size ◽  
Power Analyses

Many bodies recommend that a sample planning procedure, such as traditional NHST a priori power analysis, is conducted during the planning stages of a study. Power analysis allows the researcher to estimate how many participants are required in order to detect a minimally meaningful effect size at a specific level of power and Type I error rate. However, there are several drawbacks to the procedure that render it “a mess.” Specifically, the identification of the minimally meaningful effect size is often difficult but unavoidable for conducting the procedure properly, the procedure is not precision oriented, and does not guide the researcher to collect as many participants as feasibly possible. In this study, we explore how these three theoretical issues are reflected in applied psychological research in order to better understand whether these issues are concerns in practice. To investigate how power analysis is currently used, this study reviewed the reporting of 443 power analyses in high impact psychology journals in 2016 and 2017. It was found that researchers rarely use the minimally meaningful effect size as a rationale for the chosen effect in a power analysis. Further, precision-based approaches and collecting the maximum sample size feasible are almost never used in tandem with power analyses. In light of these findings, we offer that researchers should focus on tools beyond traditional power analysis when sample planning, such as collecting the maximum sample size feasible.

scholarly journals Causality in Statistical Power: Isomorphic Properties of Measurement, Research Design, Effect Size, and Sample Size

Scientifica ◽  
2016 ◽  
Vol 2016 ◽  
pp. 1-5 ◽  
Author(s):  
R. Eric Heidel
Keyword(s):  
Sample Size ◽  
Cognitive Dissonance ◽  
Research Design ◽  
Effect Size ◽  
Statistical Power ◽  
Research Study ◽  
A Priori ◽  
Sample Size Calculation ◽  
Design Effect

Statistical power is the ability to detect a significant effect, given that the effect actually exists in a population. Like most statistical concepts, statistical power tends to induce cognitive dissonance in hepatology researchers. However, planning for statistical power by ana priorisample size calculation is of paramount importance when designing a research study. There are five specific empirical components that make up ana priorisample size calculation: the scale of measurement of the outcome, the research design, the magnitude of the effect size, the variance of the effect size, and the sample size. A framework grounded in the phenomenon of isomorphism, or interdependencies amongst different constructs with similar forms, will be presented to understand the isomorphic effects of decisions made on each of the five aforementioned components of statistical power.

scholarly journals Not just methods: User expertise explains the variability of outcomes of genome-wide studies

2016 ◽  
Author(s):  
Katie E. Lotterhos ◽  
Olivier François ◽  
Michael G.B. Blum
Keyword(s):  
Summer School ◽  
Statistical Power ◽  
A Priori ◽  
Simulated Data ◽  
Genome Scan ◽  
Genome Wide ◽  
The Difference ◽  
Difficult Challenge ◽  
Genomic Regions ◽  
Candidate Loci

AbstractGenome scan approaches promise to map genomic regions involved in adaptation of individuals to their environment. Outcomes of genome scans have been shown to depend on several factors including the underlying demography, the adaptive scenario, and the software or method used. We took advantage of a pedagogical experiment carried out during a summer school to explore the effect of an unexplored source of variability, which is the degree of user expertise.Participants were asked to analyze three simulated data challenges with methods presented during the summer school. In addition to submitting lists, participants evaluated a priori their level of expertise. We measured the quality of each genome scan analysis by computing a score that depends on false discovery rate and statistical power. In an easy and a difficult challenge, less advanced participants obtained similar scores compared to advanced ones, demonstrating that participants with little background in genome scan methods were able to learn how to use complex software after short introductory tutorials. However, in a challenge ofintermediate difficulty, advanced participants obtained better scores. To explain the difference, we introduce a probabilistic model that shows that a larger variation in scores is expected for SNPs of intermediate difficulty of detection. We conclude that practitioners should develop their statistical and computational expertise to follow the development of complex methods. To encourage training, we release the website of the summer school where users can submit lists of candidate loci, which will be scored and compared to the scores obtained by previous users.

A priori power considerations in orthodontic research: a 3 year meta-epidemiologic study

2019 ◽  
Vol 42 (4) ◽  
pp. 454-459
Author(s):  
Sophia Gratsia ◽  
Despina Koletsi ◽  
Padhraig S Fleming ◽  
Nikolaos Pandis
Keyword(s):  
Sample Size ◽  
Study Design ◽  
Strong Predictor ◽  
Statistical Significance ◽  
A Priori ◽  
Epidemiologic Study ◽  
Observational Research ◽  
Power Calculations ◽  
Sample Size Calculations

Summary Aim To assess the prevalence of a priori power calculations in orthodontic literature and to identify potential associations with a number of study characteristics, including journal, year of publication and statistical significance of the outcome. Materials and methods The electronic archives of four leading orthodontic journals with the highest impact factor (American Journal of Orthodontics and Dentofacial Orthopedics, AJODO; European Journal of Orthodontics, EJO; Angle Orthodontist, ANGLE; Orthodontics and Craniofacial Research, OCR) were assessed over a 3 year period until December 2018. The proportion of articles reporting a priori power calculations were recorded, and the association with journal, year of publication, study design, continent of authorship, number of centres and researchers, statistical significance of results and reporting of confidence intervals (CIs) was assessed. Univariable and multivariable regression were used to identify significant predictors. Results Overall, 654 eligible articles were retrieved, with the majority published in the AJODO (n = 246, 37.6%), followed by ANGLE (n = 222, 33.9%) and EJO (n = 139, 21.3%). A total of 233 studies (35.6%) presented power considerations a priori along with sample size calculations. Study design was a very strong predictor with interventional design presenting 3.02 times higher odds for a priori power assumptions compared to observational research [odds ratio (OR): 3.02; 95% CIs: 2.06, 4.42; P < 0.001]. Conclusions Presentation of a priori power considerations for sample size calculations was not universal in contemporary orthodontic literature, while specific study designs such as observational or animal and in vitro studies were less likely to report such considerations.

Quantifying annual patterns in the frequency of mammalian births: do goodness-of-fit tests provide adequate inferences?

2012 ◽  
Vol 60 (6) ◽  
pp. 381 ◽  
Author(s):  
Evan Watkins ◽  
Julian Di Stefano
Keyword(s):  
Sample Size ◽  
Effect Size ◽  
Statistical Power ◽  
Goodness Of Fit ◽  
A Priori ◽  
Simulated Data ◽  
Maximum Effect ◽  
Chi Square ◽  
Goodness Of Fit Tests ◽  
Power Studies

Hypotheses relating to the annual frequency distribution of mammalian births are commonly tested using a goodness-of-fit procedure. Several interacting factors influence the statistical power of these tests, but no power studies have been conducted using scenarios derived from biological hypotheses. Corresponding to theories relating reproductive output to seasonal resource fluctuation, we simulated data reflecting a winter reduction in birth frequency to test the effect of four factors (sample size, maximum effect size, the temporal pattern of response and the number of categories used for analysis) on the power of three goodness-of-fit procedures – the G and Chi-square tests and Watson’s U2 test. Analyses resulting in high power all had a large maximum effect size (60%) and were associated with a sample size of 200 on most occasions. The G-test was the most powerful when data were analysed using two temporal categories (winter and other) while Watson’s U2 test achieved the highest power when 12 monthly categories were used. Overall, the power of most modelled scenarios was low. Consequently, we recommend using power analysis as a research planning tool, and have provided a spreadsheet enabling a priori power calculations for the three tests considered.

scholarly journals Power, positive predictive value, and sample size calculations for random field theory-based fMRI inference

2019 ◽  
Author(s):  
Dirk Ostwald ◽  
Sebastian Schneider ◽  
Rasmus Bruckner ◽  
Lilla Horvath
Keyword(s):  
Field Theory ◽  
Positive Predictive Value ◽  
Random Field ◽  
Sample Size ◽  
Predictive Value ◽  
Multiple Testing ◽  
Statistical Power ◽  
Random Field Theory ◽  
Sample Size Calculations ◽  
Cluster Level

AbstractRecent discussions on the reproducibility of task-related functional magnetic resonance imaging (fMRI) studies have emphasized the importance of power and sample size calculations in fMRI study planning. In general, statistical power and sample size calculations are dependent on the statistical inference framework that is used to test hypotheses. Bibliometric analyses suggest that random field theory (RFT)-based voxel- and cluster-level fMRI inference are the most commonly used approaches for the statistical evaluation of task-related fMRI data. However, general power and sample size calculations for these inference approaches remain elusive. Based on the mathematical theory of RFT-based inference, we here develop power and positive predictive value (PPV) functions for voxel- and cluster-level inference in both uncorrected single test and corrected multiple testing scenarios. Moreover, we apply the theoretical results to evaluate the sample size necessary to achieve desired power and PPV levels based on an fMRI pilot study.

Sign in / Sign up

Export Citation Format

Share Document