Developing a core outcome set for traumatic brachial plexus injuries: a systematic review of outcomes

ObjectiveTo identify what outcomes have been assessed in traumatic brachial plexus injury (TBPI) research to inform the development of a core outcome set for TBPI.DesignSystematic review.MethodMedline (OVID), EMBASE, CINAHL and AMED were systematically searched for studies evaluating the clinical effectiveness of interventions in adult TBPIs from January 2013 to September 2018 updated in May 2021. Two authors independently screened papers. Outcome reporting bias was assessed. All outcomes were extracted verbatim from studies. Patient-reported outcomes or performance outcome measures were extracted directly from the instrument. Variation in outcome reporting was determined by assessing the number of unique outcomes reported across all included studies. Outcomes were categorised into domains using a prespecified taxonomy.ResultsVerbatim outcomes (n=1491) were extracted from 138 studies including 32 questionnaires. Unique outcomes (n=157) were structured into 4 core areas and 11 domains. Outcomes within the musculoskeletal domain were measured in 86% of studies, physical functioning in 25%, emotional functioning in 25% and adverse events in 33%. We identified 63 different methods for measuring muscle strength, 16 studies for range of movement and 63 studies did not define how they measured movement. More than two-thirds of the outcomes were incompletely reported in prospective studies.ConclusionThis review of outcome reporting in TBPI research demonstrated an impairment focus and heterogeneity. A core outcome set would ensure standardised and relevant outcomes are reported to facilitate future systematic review and meta-analysis.PROSPERO registration numberCRD42018109843.

Selective Reporting of Outcomes in Tinnitus Trials: Comparison of Trial Registries With Corresponding Publications

Objectives: We aimed to study the prevalence of selective reporting of primary and secondary outcomes in tinnitus trials and to examine if selective reporting of outcome measures is influenced by the nature and direction of its results.Background: Selective reporting of outcome measures has been reported in several biomedical fields and can influence the clinical usefulness and implementation of outcomes of clinical trials. It is reported as one of the obstacles in finding an effective intervention for tinnitus.Methods: ClinicalTrials.gov (CT.gov) was used to identify all registered interventional tinnitus trials up to December 2015. A standardized search was used to find corresponding publications up to March 2018. The prespecified outcomes in CT.gov were compared with the outcomes reported in corresponding publication(s). The effects of the (lack of) statistical significance of trial results and the effects of funding source on record adherence were evaluated. Changes in registration elements were assessed with the Archive site of CT.gov.Results: We found corresponding publications for 60 (64.5%) of 93 eligible tinnitus trials registered in CT.gov. Of all the publications, five (7.5%) fully reported outcome measures entirely in line with the prespecified outcome measures. Discrepancies between the prespecified and reported outcomes were found in a total of 51 (76.1%) of the studies for primary outcomes, whereas 62 (92.5%) of the studies had discrepancies in secondary outcomes. In secondary outcomes, statistical significance of trial results influenced CT.gov record adherence. In addition, there was a statistically significant difference in the rate of discrepancy in industry-funded [n = 98 (87.5%) discrepant outcomes] and non-industry funded trials [n = 172 (74.5%) discrepant outcomes] (p = 0.01). Finally, 15 (25.9%) trialists made modifications in registered outcome measures during or after the trial period.Conclusion: Tinnitus trials suffer from substantial outcome reporting bias. Awareness of its presence must be raised to limit the obstacles of finding an effective intervention for tinnitus.

Misreporting of Results of Research in Psychiatry

Few studies address publication and outcome reporting biases of randomized controlled trials (RCTs) in psychiatry. The objective of this study was to determine publication and outcome reporting bias in RCTs funded by the Stanley Medical Research Institute (SMRI), a U.S. based, non-profit organization funding RCTs in schizophrenia and bipolar disorder. We identified all RCTs (n = 280) funded by SMRI between 2000 and 2011, and using non-public, final study reports and published manuscripts, we classified the results as positive or negative in terms of the drug compared to placebo. Design, outcome measures and statistical methods specified in the original protocol were compared to the published manuscript. Of 280 RCTs funded by SMRI between 2000 and 2011, at the time of this writing, three RCTs were ongoing and 39 were not performed. Among the 238 completed RCTs, 86 (36.1%) reported positive and 152 (63.9%) reported negative results: 86% (74/86) of those with positive findings were published in contrast to 53% (80/152) of those with negative findings (P < .001). In 70% of the manuscripts published, there were major discrepancies between the published manuscript and the original RCT protocol (change in the primary outcome measure or statistics, change in a number of patient groups, 25% or more reduction in sample size). We conclude that publication bias and outcome reporting bias is common in papers reporting RCTs in schizophrenia and bipolar disorder. These data have major implications regarding the validity of the reports of clinical trials published in the literature.

Correcting for outcome reporting bias in a meta-analysis: A meta-regression approach

Outcome reporting bias (ORB) refers to the biasing effect caused by researchers selectively reporting outcomes based on their statistical significance. ORB leads to inflated average effect size estimates in a meta-analysis if only the outcome with the largest effect size is reported due to ORB. We propose a new method (CORB) to correct for ORB that includes an estimate of the variability of the outcomes' effect size as a moderator in a meta-regression model. An estimate of the variability of the outcomes' effect size can be computed by assuming a correlation among the outcomes. Results of a Monte-Carlo simulation study showed that effect size in meta-analyses may be severely overestimated without any correction for ORB. The CORB method accurately estimates effect size when overestimation caused by ORB is the largest. Applying the new method to a meta-analysis on the effect of playing violent video games on aggressive cognition showed that the average effect size estimate decreased when correcting for ORB. We recommend to routinely apply methods to correct for ORB in any meta-analysis. We provide annotated R code and functions to facilitate researchers to apply the CORB method.

Outcome Reporting bias in Exercise Oncology trials (OREO): a cross-sectional study

Background: Despite evidence of selective outcome reporting across multiple disciplines, this has not yet been assessed in trials studying the effects of exercise in people with cancer. Therefore, the purpose of our study was to explore prospectively registered randomised controlled trials (RCTs) in exercise oncology for evidence of selective outcome reporting. Methods: Eligible trials were RCTs that 1) investigated the effects of at least partially supervised exercise interventions in people with cancer; 2) were preregistered (i.e. registered before the first patient was recruited) on a clinical trials registry; and 3) reported results in a peer-reviewed published manuscript. We searched the PubMed database from the year of inception to September 2020 to identify eligible exercise oncology RCTs clinical trial registries. Eligible trial registrations and linked published manuscripts were compared to identify the proportion of sufficiently preregistered outcomes reported correctly in the manuscripts, and cases of outcome omission, switching, and silently introduction of non- novel outcomes. Results: We identified 31 eligible RCTs and 46 that were ineligible due to retrospective registration. Of the 405 total prespecified outcomes across the 31 eligible trials, only 6.2% were preregistered complete methodological detail. Only 16% (n=148/929) of outcomes reported in published results manuscripts were linked with sufficiently preregistered outcomes without outcome switching. We found 85 total cases of outcome switching. A high proportion (41%) of preregistered outcomes were omitted from the published results manuscripts, and many published outcomes (n=394; 42.4%) were novel outcomes that had been silently introduced (median, min-max=10, 0-50 per trial). We found no examples of preregistered efficacy outcomes that were measured, assessed, and analysed as planned. Conclusions: We found evidence suggestive of widespread selective outcome reporting and non-reporting bias (omitted preregistered outcomes, outcome switching, and silently introduced novel outcomes). The existence of such reporting discrepancies has implications for the integrity and credibility of RCTs in exercise oncology.

Outcome Reporting Bias in COVID-19 mRNA Vaccine Clinical Trials

Relative risk reduction and absolute risk reduction measures in the evaluation of clinical trial data are poorly understood by health professionals and the public. The absence of reported absolute risk reduction in COVID-19 vaccine clinical trials can lead to outcome reporting bias that affects the interpretation of vaccine efficacy. The present article uses clinical epidemiologic tools to critically appraise reports of efficacy in Pfzier/BioNTech and Moderna COVID-19 mRNA vaccine clinical trials. Based on data reported by the manufacturer for Pfzier/BioNTech vaccine BNT162b2, this critical appraisal shows: relative risk reduction, 95.1%; 95% CI, 90.0% to 97.6%; p = 0.016; absolute risk reduction, 0.7%; 95% CI, 0.59% to 0.83%; p < 0.000. For the Moderna vaccine mRNA-1273, the appraisal shows: relative risk reduction, 94.1%; 95% CI, 89.1% to 96.8%; p = 0.004; absolute risk reduction, 1.1%; 95% CI, 0.97% to 1.32%; p < 0.000. Unreported absolute risk reduction measures of 0.7% and 1.1% for the Pfzier/BioNTech and Moderna vaccines, respectively, are very much lower than the reported relative risk reduction measures. Reporting absolute risk reduction measures is essential to prevent outcome reporting bias in evaluation of COVID-19 vaccine efficacy.

A randomised controlled trial comparing palate surgery at 6 months versus 12 months of age (the TOPS trial): a statistical analysis plan

Background Cleft palate is among the most common birth abnormalities. The success of primary surgery in the early months of life is crucial for successful feeding, hearing, dental development, and facial growth. Over recent decades, age at palatal surgery in infancy has reduced. The Timing Of Primary Surgery for cleft palate (TOPS) trial aims to determine whether, in infants with cleft palate, it is better to perform primary surgery at age 6 or 12 months (corrected for gestational age). Methods/design The TOPS trial is an international, two-arm, parallel group, randomised controlled trial. The primary outcome is insufficient velopharyngeal function at 5 years of age. Secondary outcomes, measured at 12 months, 3 years, and 5 years of age, include measures of speech development, safety of the procedure, hearing level, middle ear function, dentofacial development, and growth. The analysis approaches for primary and secondary outcomes are described here, as are the descriptive statistics which will be reported. The TOPS protocol has been published previously. Discussion This paper provides details of the planned statistical analyses for the TOPS trial and will reduce the risk of outcome reporting bias and data-driven results. Trial registration ClinicalTrials.gov NCT00993551. Registered on 9 October 2009.

Visual representations of meta-analyses of multiple outcomes: Extensions to forest plots, funnel plots, and caterpillar plots

Meta-analytic datasets can be large, especially when in primary studies multiple effect sizes are reported. The visualization of meta-analytic data is therefore useful to summarize data and understand information reported in primary studies. The gold standard figures in meta-analysis are forest and funnel plots. However, none of these plots can yet account for the existence of multiple effect sizes within primary studies. This manuscript describes extensions to the funnel plot, forest plot and caterpillar plot to adapt them to three-level meta-analyses. For forest plots, we propose to plot the study-specific effects and their precision, and to add additional confidence intervals that reflect the sampling variance of individual effect sizes. For caterpillar plots and funnel plots, we recommend to plot individual effect sizes and averaged study-effect sizes in two separate graphs. For the funnel plot, plotting separate graphs might improve the detection of both publication bias and/or selective outcome reporting bias.

Use of wearable biometric monitoring devices to measure outcomes in randomized clinical trials: a methodological systematic review

Background Wearable biometric monitoring devices (BMDs) have the potential to transform the conduct of randomized controlled trials (RCTs) by shifting the collection of outcome data from single measurements at predefined time points to dense continuous measurements. Methods Methodological systematic review to understand how recent RCTs used BMDs to measure outcomes and to describe the reporting of these RCTs. Electronic search was performed in the Cochrane Central Register of Controlled Trials, PubMed, and EMBASE and completed a page-by-page hand search in five leading medical journals between January 1, 2018, and December 31, 2018. Three reviewers independently extracted all primary and secondary outcomes collected using BMDs, and assessed (1) the definitions used to summarize BMD outcome data; (2) whether the validity, reliability, and responsiveness of sensors was reported; (3) the discrepancy with outcomes prespecified in public clinical trial registries; and (4) the methods used to manage missing and incomplete BMD outcome data. Results Of the 4562 records screened, 75 RCTs were eligible. Among them, 24% tested a pharmacological intervention and 57% used an inertial measurement sensor to measure physical activity. Included trials involved 464 outcomes (average of 6 [SD = 8] outcomes per trial). In total, 35 trials used a BMD to measure a primary outcome. Several issues affected the value and transparency of trials using BMDs to measure outcomes. First, the definition of outcomes used in the trials was highly heterogeneous (e.g., 21 diabetes trials had 266 outcomes and 153 had different unique definitions to measure diabetes control), which limited the combination and comparison of results. Second, information on the validity, reliability, and responsiveness of sensors used was lacking in 74% of trials. Third, half (53%) of the outcomes measured with BMDs had not been prespecified, with a high risk of outcome reporting bias. Finally, reporting on the management of incomplete outcome data (e.g., due to suboptimal compliance with the BMD) was absent in 68% of RCTs. Conclusions Use of BMDs to measure outcomes is becoming the norm rather than the exception in many fields. Yet, trialists need to account for several methodological issues when specifying and conducting RCTs using these novel tools.