scholarly journals Sample size, sample size planning, and the impact of study context: systematic review and recommendations by the example of psychological depression treatment

2021 ◽  
pp. 1-7
Author(s):  
Raphael Schuster ◽  
Tim Kaiser ◽  
Yannik Terhorst ◽  
Eva Maria Messner ◽  
Lucia-Maria Strohmeier ◽  
...  
Keyword(s):  
Sample Size ◽  
Study Design ◽  
Statistical Power ◽  
Face To Face ◽  
Internet Interventions ◽  
Study Planning ◽  
Sample Size Planning ◽  
Highly Correlated ◽  
Study Context ◽  
The Impact

Abstract Background Sample size planning (SSP) is vital for efficient studies that yield reliable outcomes. Hence, guidelines, emphasize the importance of SSP. The present study investigates the practice of SSP in current trials for depression. Methods Seventy-eight randomized controlled trials published between 2013 and 2017 were examined. Impact of study design (e.g. number of randomized conditions) and study context (e.g. funding) on sample size was analyzed using multiple regression. Results Overall, sample size during pre-registration, during SSP, and in published articles was highly correlated (r's ≥ 0.887). Simultaneously, only 7–18% of explained variance related to study design (p = 0.055–0.155). This proportion increased to 30–42% by adding study context (p = 0.002–0.005). The median sample size was N = 106, with higher numbers for internet interventions (N = 181; p = 0.021) compared to face-to-face therapy. In total, 59% of studies included SSP, with 28% providing basic determinants and 8–10% providing information for comprehensible SSP. Expected effect sizes exhibited a sharp peak at d = 0.5. Depending on the definition, 10.2–20.4% implemented intense assessment to improve statistical power. Conclusions Findings suggest that investigators achieve their determined sample size and pre-registration rates are increasing. During study planning, however, study context appears more important than study design. Study context, therefore, needs to be emphasized in the present discussion, as it can help understand the relatively stable trial numbers of the past decades. Acknowledging this situation, indications exist that digital psychiatry (e.g. Internet interventions or intense assessment) can help to mitigate the challenge of underpowered studies. The article includes a short guide for efficient study planning.

Exploring power in response inhibition tasks using the bootstrap: The impact of number of participants, number of trials, effect magnitude, and study design

2019 ◽  
Author(s):  
Curtis David Von Gunten ◽  
Bruce D Bartholow
Keyword(s):  
Study Design ◽  
Effect Size ◽  
Statistical Power ◽  
Bootstrap Sampling ◽  
Reliable Measure ◽  
Internal Reliability ◽  
The Impact ◽  
Power Analyses ◽  
Effect Magnitude

A primary psychometric concern with laboratory-based inhibition tasks has been their reliability. However, a reliable measure may not be necessary or sufficient for reliably detecting effects (statistical power). The current study used a bootstrap sampling approach to systematically examine how the number of participants, the number of trials, the magnitude of an effect, and study design (between- vs. within-subject) jointly contribute to power in five commonly used inhibition tasks. The results demonstrate the shortcomings of relying solely on measurement reliability when determining the number of trials to use in an inhibition task: high internal reliability can be accompanied with low power and low reliability can be accompanied with high power. For instance, adding additional trials once sufficient reliability has been reached can result in large gains in power. The dissociation between reliability and power was particularly apparent in between-subject designs where the number of participants contributed greatly to power but little to reliability, and where the number of trials contributed greatly to reliability but only modestly (depending on the task) to power. For between-subject designs, the probability of detecting small-to-medium-sized effects with 150 participants (total) was generally less than 55%. However, effect size was positively associated with number of trials. Thus, researchers have some control over effect size and this needs to be considered when conducting power analyses using analytic methods that take such effect sizes as an argument. Results are discussed in the context of recent claims regarding the role of inhibition tasks in experimental and individual difference designs.

scholarly journals Strength of functional signature correlates with effect size in autism

2016 ◽  
Author(s):  
Sara Ballouz ◽  
Jesse Gillis
Keyword(s):  
Study Design ◽  
Effect Size ◽  
Statistical Power ◽  
Fragile X ◽  
Meta Analysis ◽  
Autism Spectrum ◽  
Protein Interaction Data ◽  
Functional Convergence ◽  
Study Designs ◽  
The Impact

AbstractBackgroundDisagreements over genetic signatures associated with disease have been particularly prominent in the field of psychiatric genetics, creating a sharp divide between disease burdens attributed to common and rare variation, with study designs independently targeting each. Meta-analysis within each of these study designs is routine, whether using raw data or summary statistics, but combining results across study designs is atypical. However, tests of functional convergence are used across all study designs, where candidate gene sets are assessed for overlaps with previously known properties. This suggests one possible avenue for combining not study data, but the functional conclusions that they reach.MethodIn this work, we test for functional convergence in autism spectrum disorder (ASD) across different study types, and specifically whether the degree to which a gene is implicated in autism is correlated with the degree to which it drives functional convergence. Because different study designs are distinguishable by their differences in effect size, this also provides a unified means of incorporating the impact of study design into the analysis of convergence.ResultsWe detected remarkably significant positive trends in aggregate (p < 2.2e-16) with 14 individually significant properties (FDR<0.01), many in areas researchers have targeted based on different reasoning, such as the fragile X mental retardation protein (FMRP) interactor enrichment (FDR 0.003). We are also able to detect novel technical effects and we see that network enrichment from protein-protein interaction data is heavily confounded with study design, arising readily in control data.ConclusionsWe see a convergent functional signal for a subset of known and novel functions in ASD from all sources of genetic variation. Meta-analytic approaches explicitly accounting for different study designs can be adapted to other diseases to discover novel functional associations and increase statistical power.

scholarly journals Considerations of sample size and power calculations given a range of analytical scenarios

2021 ◽  
Author(s):  
Alice Carter ◽  
Kate Tilling ◽  
Marcus Robert Munafo
Keyword(s):  
Sample Size ◽  
Statistical Power ◽  
Poor Quality ◽  
Research Quality ◽  
Design Stage ◽  
Design Consideration ◽  
Power Calculations ◽  
Critical Design ◽  
Study Planning ◽  
Research Questions

The sample size of a study is a key design and planning consideration. However, sample size and power calculations are often either poorly reported or not reported at all, which suggests they may not form a routine part of study planning. Inadequate understanding of sample size and statistical power can result in poor quality studies. Journals increasingly require a justification of sample size, for example through the use of reporting checklists. However, for meaningful improvements in research quality to be made, researchers need to consider sample size and power at the design stage of a study, rather than at the publication stage. Here we briefly illustrate sample size and statistical power in the context of different research questions and how they should be viewed as a critical design consideration.

SAMPLE SIZE PLANNING IN QUANTITATIVE L2 RESEARCH

2020 ◽  
Vol 42 (4) ◽  
pp. 849-870
Author(s):  
Reza Norouzian
Keyword(s):  
Sample Size ◽  
Statistical Power ◽  
Open Science ◽  
Effect Sizes ◽  
Size Estimation ◽  
Alternative Approach ◽  
Sample Size Planning ◽  
Effect Size Estimation ◽  
Definition Of ◽  
Actual Size

AbstractResearchers are traditionally advised to plan for their required sample size such that achieving a sufficient level of statistical power is ensured (Cohen, 1988). While this method helps distinguishing statistically significant effects from the nonsignificant ones, it does not help achieving the higher goal of accurately estimating the actual size of those effects in an intended study. Adopting an open-science approach, this article presents an alternative approach, accuracy in effect size estimation (AESE), to sample size planning that ensures that researchers obtain adequately narrow confidence intervals (CI) for their effect sizes of interest thereby ensuring accuracy in estimating the actual size of those effects. Specifically, I (a) compare the underpinnings of power-analytic and AESE methods, (b) provide a practical definition of narrow CIs, (c) apply the AESE method to various research studies from L2 literature, and (d) offer several flexible R programs to implement the methods discussed in this article.

Citation Analysis of Articles about Hand Surgery Published in Orthopaedic and Hand Surgery Journals

2019 ◽  
Vol 24 (01) ◽  
pp. 36-44 ◽  
Author(s):  
Yuki Fujihara ◽  
Nasa Fujihara ◽  
Michiro Yamamoto ◽  
Hitoshi Hirata
Keyword(s):  
Clinical Trial ◽  
Sample Size ◽  
Randomized Clinical Trial ◽  
Study Design ◽  
Comparative Studies ◽  
Clinical Studies ◽  
Hand Surgery ◽  
The Impact ◽  
Larger Sample ◽  
Surgery Journals

Background: To date, little is known about the characteristics of highly cited studies in hand surgery compared with other orthopaedic subspecialties. We aimed to assess the position of hand surgery within the orthopedic surgery literature. Methods: We conducted a bibliographic analysis using the Web of Science database to review 1,568 articles published between January 2012 and December 2012 in 4 relevant general orthopedic and 2 hand surgery journals. We used the number of citations within 3 years of publication to measure the impact of each paper. To analyze prognostic factors using logistic regression analysis, we extracted data on orthopedic subspecialty, published journal, location of authorship, and type of study for all articles. For clinical studies, we also recorded details on study design and sample size. Results: Of eligible hand surgery articles (n = 307), the majority (62%) were case reports/series. Only 19% were comparative studies, comprising a significantly smaller proportion of comparative studies from other subspecialties in general orthopedic journals. Systematic reviews/meta-analyses generated a significantly higher number of average citations, whereas educational reviews were consistently cited less frequently than other study types (14.9 and 6.1 average citations, respectively). Being published in the Journal of Bone and Joint Surgery, American volume, having authorship in North America or Europe and Australia, focusing on subspecialties like hip & knee, sports, or shoulder, utilizing a comparative or randomized clinical trial study design, and having a larger sample size increased the odds of receiving more citations. Conclusions: Clinical studies related to hand surgery published in general orthopedic journals are most often of lower quality study design. Having a larger sample size or using a comparative study or randomized clinical trial design can improve the quality of study and may ultimately increase the impact factor of hand surgery journals.

scholarly journals Population Pharmacokinetic Assessment of the Effect of Food on Piperaquine Bioavailability in Patients with Uncomplicated Malaria

2014 ◽  
Vol 58 (4) ◽  
pp. 2052-2058 ◽  
Author(s):  
Joel Tarning ◽  
Niklas Lindegardh ◽  
Khin Maung Lwin ◽  
Anna Annerberg ◽  
Lily Kiricharoen ◽  
...  
Keyword(s):  
Monte Carlo ◽  
Study Design ◽  
Statistical Power ◽  
Mixed Effects ◽  
Distribution Model ◽  
Population Pharmacokinetic ◽  
Final Model ◽  
Content Type ◽  
Mixed Effects Modeling ◽  
The Impact

ABSTRACTPreviously published literature reports various impacts of food on the oral bioavailability of piperaquine. The aim of this study was to use a population modeling approach to investigate the impact of concomitant intake of a small amount of food on piperaquine pharmacokinetics. This was an open, randomized comparison of piperaquine pharmacokinetics when administered as a fixed oral formulation once daily for 3 days with (n= 15) and without (n= 15) concomitant food to patients with uncomplicatedPlasmodium falciparummalaria in Thailand. Nonlinear mixed-effects modeling was used to characterize the pharmacokinetics of piperaquine and the influence of concomitant food intake. A modified Monte Carlo mapped power approach was applied to evaluate the relationship between statistical power and various degrees of covariate effect sizes of the given study design. Piperaquine population pharmacokinetics were described well in fasting and fed patients by a three-compartment distribution model with flexible absorption. The final model showed a 25% increase in relative bioavailability per dose occasion during recovery from malaria but demonstrated no clinical impact of concomitant intake of a low-fat meal. Body weight and age were both significant covariates in the final model. The novel power approach concluded that the study was adequately powered to detect a food effect of at least 35%. This modified Monte Carlo mapped power approach may be a useful tool for evaluating the power to detect true covariate effects in mixed-effects modeling and a given study design. A small amount of food does not affect piperaquine absorption significantly in acute malaria.

Sample-Size Planning for More Accurate Statistical Power: A Method Adjusting Sample Effect Sizes for Publication Bias and Uncertainty

2017 ◽  
Vol 28 (11) ◽  
pp. 1547-1562 ◽  
Author(s):  
Samantha F. Anderson ◽  
Ken Kelley ◽  
Scott E. Maxwell
Keyword(s):  
Sample Size ◽  
Publication Bias ◽  
Effect Size ◽  
Statistical Power ◽  
Web Applications ◽  
R Package ◽  
Effect Sizes ◽  
Alternative Approach ◽  
Sample Size Planning ◽  
User Friendly

The sample size necessary to obtain a desired level of statistical power depends in part on the population value of the effect size, which is, by definition, unknown. A common approach to sample-size planning uses the sample effect size from a prior study as an estimate of the population value of the effect to be detected in the future study. Although this strategy is intuitively appealing, effect-size estimates, taken at face value, are typically not accurate estimates of the population effect size because of publication bias and uncertainty. We show that the use of this approach often results in underpowered studies, sometimes to an alarming degree. We present an alternative approach that adjusts sample effect sizes for bias and uncertainty, and we demonstrate its effectiveness for several experimental designs. Furthermore, we discuss an open-source R package, BUCSS, and user-friendly Web applications that we have made available to researchers so that they can easily implement our suggested methods.

scholarly journals Statistical power: implications for planning MEG studies

2019 ◽  
Author(s):  
Maximilien Chaumon ◽  
Aina Puce ◽  
Nathalie George
Keyword(s):  
Sample Size ◽  
Statistical Power ◽  
Single Subject ◽  
List Type ◽  
Group Level ◽  
Spatial Properties ◽  
Human Connectome Project ◽  
Neurophysiological Studies ◽  
Subject Variability ◽  
The Impact

AbstractStatistical power is key for robust, replicable science. Here, we systematically explored how numbers of trials and subjects affect statistical power in MEG sensor-level data. More specifically, we simulated “experiments” using the MEG resting-state dataset of the Human Connectome Project (HCP). We divided the data in two conditions, injected a dipolar source at a known anatomical location in the “signal condition”, but not in the “noise condition”, and detected significant differences at sensor level with classical paired t-tests across subjects. Group-level detectability of these simulated effects varied drastically with anatomical origin. We thus examined in detail which spatial properties of the sources affected detectability, looking specifically at the distance from closest sensor and orientation of the source, and at the variability of these parameters across subjects. In line with previous single-subject studies, we found that the most detectable effects originate from source locations that are closest to the sensors and oriented tangentially with respect to the head surface. In addition, cross-subject variability in orientation also affected group-level detectability, boosting detection in regions where this variability was small and hindering detection in regions where it was large. Incidentally, we observed a considerable covariation of source position, orientation, and their cross-subject variability in individual brain anatomical space, making it difficult to assess the impact of each of these variables independently of one another. We thus also performed simulations where we controlled spatial properties independently of individual anatomy. These additional simulations confirmed the strong impact of distance and orientation and further showed that orientation variability across subjects affects detectability, whereas position variability does not.Importantly, our study indicates that strict unequivocal recommendations as to the ideal number of trials and subjects for any experiment cannot be realistically provided for neurophysiological studies. Rather, it highlights the importance of considering the spatial constraints underlying expected sources of activity while designing experiments.HighlightsAdequate sample size (number of subjects and trials) is key to robust neuroscienceWe simulated evoked MEG experiments and examined sensor-level detectabilityStatistical power varied by source distance, orientation & between-subject variabilityConsider source detectability at sensor-level when designing MEG studiesSample size for MEG studies? Consider source with lowest expected statistical power

Sign in / Sign up

Export Citation Format

Share Document