Sample Size Justifications in Gait & Posture

Background: Context regarding how researchers determine the sample size of their experiments is important for interpreting the results and determining their value and meaning. Between 2018 and 2019, the journal Gait & Posture introduced a requirement for sample size justification in their author guidelines.Research Question: How frequently and in what ways are sample sizes justified in Gait & Posture research articles and was the inclusion of a guideline requiring sample size justification associated with a change in practice?Methods: The guideline was not in place prior to May 2018 and was in place from 25th July 2019. All articles in the three most recent volumes of the journal (84-86) and the three most recent, pre-guideline volumes (60-62) at time of preregistration were included in this analysis. This provided an initial sample of 324 articles (176 pre-guideline and 148 post-guideline). Articles were screened by two authors to extract author data, article metadata and sample size justification data. Specifically, screeners identified if (yes or no) and how sample sizes were justified. Six potential justification types (Measure Entire Population, Resource Constraints, Accuracy, A priori Power Analysis, Heuristics, No Justification) and an additional option of Other/Unsure/Unclear were used.Results: In most cases, authors of Gait & Posture articles did not provide a justification for their study’s sample size. The inclusion of the guideline was associated with a modest increase in the percentage of articles providing a justification (16.6% to 28.1%). A priori power calculations were the dominant type of justification, but many were not reported in enough detail to allow replication.Significance: Gait & Posture researchers should be more transparent in how they determine their sample sizes and carefully consider if they are suitable. Editors and journals may consider adding a similar guideline as a low-resource way to improve sample size justification reporting.

Sample Size Justification

An important step when designing a study is to justify the sample size that will be collected. The key aim of a sample size justification is to explain how the collected data is expected to provide valuable information given the inferential goals of the researcher. In this overview article six approaches are discussed to justify the sample size in a quantitative empirical study: 1) collecting data from (an)almost) the entire population, 2) choosing a sample size based on resource constraints, 3) performing an a-priori power analysis, 4) planning for a desired accuracy, 5) using heuristics, or 6) explicitly acknowledging the absence of a justification. An important question to consider when justifying sample sizes is which effect sizes are deemed interesting, and the extent to which the data that is collected informs inferences about these effect sizes. Depending on the sample size justification chosen, researchers could consider 1) what the smallest effect size of interest is, 2) which minimal effect size will be statistically significant, 3) which effect sizes they expect (and what they base these expectations on), 4) which effect sizes would be rejected based on a confidence interval around the effect size, 5) which ranges of effects a study has sufficient power to detect based on a sensitivity power analysis, and 6) which effect sizes are plausible in a specific research area. Researchers can use the guidelines presented in this article to improve their sample size justification, and hopefully, align the informational value of a study with their inferential goals.

Simulation-Based Power Analysis for Factorial Analysis of Variance Designs

Researchers often rely on analysis of variance (ANOVA) when they report results of experiments. To ensure that a study is adequately powered to yield informative results with an ANOVA, researchers can perform an a priori power analysis. However, power analysis for factorial ANOVA designs is often a challenge. Current software solutions do not allow power analyses for complex designs with several within-participants factors. Moreover, power analyses often need [Formula: see text] or Cohen’s f as input, but these effect sizes are not intuitive and do not generalize to different experimental designs. We have created the R package Superpower and online Shiny apps to enable researchers without extensive programming experience to perform simulation-based power analysis for ANOVA designs of up to three within- or between-participants factors. Predicted effects are entered by specifying means, standard deviations, and, for within-participants factors, the correlations. The simulation provides the statistical power for all ANOVA main effects, interactions, and individual comparisons. The software can plot power across a range of sample sizes, can control for multiple comparisons, and can compute power when the homogeneity or sphericity assumption is violated. This Tutorial demonstrates how to perform a priori power analysis to design informative studies for main effects, interactions, and individual comparisons and highlights important factors that determine the statistical power for factorial ANOVA designs.

The quality of research on mental health effects of the COVID-19 pandemic: A note of caution after a systematic review

Background: SARS-CoV-2 pandemic has spurred scientific production in diverse fieldsof knowledge, including mental health. Yet, the quality of current research may bechallenged by the urgent need to provide immediate results to understand and alleviatethe consequences of the pandemic. This systematic review aims to examine compliancewith basic methodological quality criteria and open science practices on the mental healtheffects of the COVID-19 pandemic.Methods: A systematic search was performed using PubMed and Scopus databases onthe 13th of May. Empirical studies, published in peer-reviewed journals in English,between February and May 2020, were included. The dependent variable(s) required tobe quantitative and related to mental health. Exclusion criteria included clinicalpharmacological trials, and studies using psychophysiological or biological recordings.The study protocol was previously pre-registered in https://osf.io/bk3gw/.Findings: Twenty-eight studies were identified. More than 75% met the requirementsrelated to reporting key methodological and statistical information. However, 89.3% usedconvenience samples and 92.86% lacked of a priori power analysis. There was lowcompliance with open science recommendations, such as pre-registration of studies (0%)and availability of databases (3.57%), which raise concerns about the validity,generalisability, and reproducibility of the findings.Interpretation: While the importance of offering rapid evidence-based responses tomitigate mental health problems stemming from the COVID-19 pandemic is undeniable,it should not be done at the expense of sacrificing scientific rigor. The results of this studymay stimulate researchers and funding agencies to try to orchestrate efforts and resourcesand follow standard codes of good science.

Decreased Hip Labral Width Measured via Preoperative MRI is Associated with Inferior Outcomes for Arthroscopic Labral Repair

Objectives: There are few pre-operative prognostic factors for hip labral repair outcomes. The objective of this study was to determine if hip labrum width measured on MRI was predictive of outcomes Methods: A retrospective review of prospectively gathered hip arthroscopy patients from 2010 to 2017 was performed. Inclusion criteria was defined as: patients >18 years old with radiographic evidence of femoroacetabular impingement who underwent a primary labral repair with >2 years of follow-up. Exclusion criteria was defined as: inadequate imaging, prior hip surgery, Tonnis grade ≥2 or lateral central edge angle <25 degrees. An a-priori power analysis was performed. MRI measurements of labral width were conducted by two blinded musculoskeletal fellowship-trained radiologists at standardized “clockface” locations using a previously validated technique. Outcomes were assessed using the Harris Hip Score (HHS), Modified HHS (mHSS), and NonArthritic Hip Score (NAHS). For mHHS, a minimal clinically important difference (MCID) and Patient Acceptable Symptomatic State (PASS) of 8 and 74 were used, respectively. Patients were divided into groups by labral width of ≤4mm and >4mm. Statistical analysis was performed using: linear and polynomial regression, Mann-Whitney U, Fischer exact, and interclass-correlation coefficients (ICC) testing Results: One hundred and three patients (107 hips) met criteria (mean age 39.4years+/-17, BMI 25.0+/-4, 51%right-sided, 68%female). Mean labrum measurements and number of patients with ≤4mm labrums at the 12:00 (indirect rectus), 3:00 (Psoas U), and 1:30 (point ½ between) positions were 7.1mm+/-2.2; 15 labrums≤4mm, 7.0 mm+/-2.0;13 labrums≤4mm, and 5.5+/-1.9; 27 labrums≤4mm, respectively. ICC agreements were good to excellent between readers at all positions (0.83-0.91,p<0.001). Pre-operative HHS, mHHS, and NAHS were not statistically different (p>0.05). Sex, laterality, and BMI had no significant effect on outcomes (p>0.05).HHS, mHHS, and NAHS scores were found to be significantly lower in the ≤4mm group at each location tested (p<0.001); including mHHS at the 12:00 (67vs87), 3:00, (69vs87) and 1:30 (74vs88) positions. The proportion of ≤4mm patients that reached MCID was significantly lower(p<0.001) at the 12:00 (47%vs91%), 3:00 (54%vs89%) and 1:30 (63%vs93%) positions. The proportion of ≤4mm patients above PASS was significantly lower (p<0.001) at the 12:00 (40%vs84%), 3:00 (31%vs84%) and 1:30 (52%vs86%) positions.Linear regression modelling was not significant at any position (p>0.05). Polynomial regression was significant at the 12:00 (R2=0.23,p<0.001), 3:00 (R2=0.17,p<0.001), and 1:30 (R2=0.26,p<0.001). Conclusion: A non-linear relationship may exist between labral width and patient outcomes following labral repair. Labrum width of ≤4mm measured via MRI may negatively impact labral repair outcomes. Future research may determine if torn labrums ≤4mm should be reconstructed instead of repaired.

Simulation-Based Power-Analysis for Factorial ANOVA Designs

Researchers often rely on analysis of variance (ANOVA) when they report results of experiments. To ensure a study is adequately powered to yield informative results when performing an ANOVA, researchers can perform an a-priori power analysis. However, power analysis for factorial ANOVA designs is often a challenge. Current software solutions do not allow power analyses for complex designs with several within-subject factors. Moreover, power analyses often need partial eta-squared or Cohen's $f$ as input, but these effect sizes are not intuitive and do not generalize to different experimental designs. We have created the R package Superpower and online Shiny apps to enable researchers without extensive programming experience to perform simulation-based power analysis for ANOVA designs of up to three within- or between-subject factors. Predicted effects are entered by specifying means, standard deviations, and for within-subject factors the correlations. The simulation provides the statistical power for all ANOVA main effects, interactions, and individual comparisons, and allows researchers to correct for multiple comparisons. The software can plot power across a range of sample sizes, can control error rates for multiple comparisons, and can compute power when the homogeneity or sphericity assumptions are violated. This tutorial will demonstrate how to perform a-priori power analysis to design informative studies for main effects, interactions, and individual comparisons, and highlights important factors that determine the statistical power for factorial ANOVA designs.

Fragility Index, power, strength and robustness of findings in sports medicine and arthroscopic surgery: a secondary analysis of data from a study on use of the Fragility Index in sports surgery

Background A recent study concluded that most findings reported as significant in sports medicine and arthroscopic surgery are not “robust” when evaluated with the Fragility Index (FI). A secondary analysis of data from a previous study was performed to investigate (1) the correctness of the findings, (2) the association between FI, p-value and post hoc power, (3) median power to detect a medium effect size, and (4) the implementation of sample size analysis in these randomized controlled trials (RCTs). Methods In addition to the 48 studies listed in the appendix accompanying the original study by Khan et al. (2017) we did a follow-up literature search and 18 additional studies were found. In total 66 studies were included in the analysis. We calculated post hoc power, p-values and confidence intervals associated with the main outcome variable. Use of a priori power analysis was recorded. The median power to detect small (h > 0.2), medium (h > 0.5), or large effect (h > 0.8) with a baseline proportion of events of 10% and 30% in each study included was calculated. Three simulation data sets were used to validate our findings. Results Inconsistencies were found in eight studies. A priori power analysis was missing in one-fourth of studies (16/66). The median power to detect a medium effect size with a baseline proportion of events of 10% and 30% was 42% and 43%, respectively. The FI was inherently associated with the achieved p-value and post hoc power. Discussion A relatively high proportion of studies had inconsistencies. The FI is a surrogate measure for p-value and post hoc power. Based on these studies, the median power in this field of research is suboptimal. There is an urgent need to investigate how well research claims in orthopedics hold in a replicated setting and the validity of research findings.

Can the Kerlan-Jobe Orthopaedic Clinic Shoulder and Elbow Score Be Reliably Administered Over the Phone? A Randomized Study

Background: The Kerlan-Jobe Orthopaedic Clinic (KJOC) shoulder and elbow outcome score is a functional assessment tool for the upper extremity of the overhead athlete, which is currently validated for administration in person. Purpose/Hypothesis: The purpose of this study was to validate the KJOC score for administration over the phone. The hypothesis was that no difference will exist in KJOC scores for the same patient between administration in person versus over the phone. Study Design: Cohort study (diagnosis); Level of evidence, 2. Methods: Fifty patients were randomized to fill out the KJOC questionnaire either over the phone first (25 patients) or in person first (25 patients) based on an a priori power analysis. One week after the patients completed the initial KJOC on the phone or in person, they then filled out the score via the opposite method. Results were compared per question and for overall score. Results: There was a mean ± SD of 8 ± 5 days between when patients completed the first and second questionnaires. There were no significant differences in the overall KJOC score between the phone and paper groups ( P = .139). The intraclass correlation coefficient comparing paper and phone scores was 0.802 (95% CI, 0.767-0.883; P < .001), with a Cronbach alpha of 0.89. On comparison of individual questions, there were significant differences for questions 1, 3, and 8 ( P = .013, .023, and .042, respectively). Conclusion: The KJOC questionnaire can be administered over the phone with no significant difference in overall score as compared with that from in-person administration.