Selection Protocol for Replication in Sports and Exercise Science

Introduction: To improve the rigor of science, experimental evidence for scientific claims ideally needs to be replicated repeatedly with sufficiently similar procedures to increase the collective confidence in the veracity of those claims. Large replication projects in psychology, cancer biology and social science have evaluated the replicability of their fields but no collaborative effort has been undertaken in sports and exercise science. We propose to undertake such an effort here. As this is the first large replication project in this field, there is no agreed-upon protocol for selecting studies to replicate. Criticism of the previous selection protocols include claims they were non-randomized and non-representative, and alleged to be biased. Any selection protocol in sports and exercise science must be unbiased and representative to provide an accurate estimate of replicability of the field. The aim of this document is to produce a protocol for selecting studies to replicate for inclusion in a large replication project in sports and exercise science. Methods: The proposed selection protocol uses multiple inclusion and exclusion criteria for replication study selection, including: the year of publication and citation rankings, research disciplines, study types, the research question and key dependent variable, study methods and feasibility. Studies selected for replication will be stratified into pools based on instrumentation and expertise required and will then be allocated to volunteer laboratories for replication. Replication outcomes will be assessed using a multiple inferential strategy and descriptive information will be reported regarding the final number of included and excluded studies, and original author contact.

The quality of reporting RCT abstracts in four major orthodontics journals for the period 2012–2017

Introduction: Clear reporting of the abstracts of randomised controlled trials (RCTs) facilitates the assessment and identification of such trials. Aim: To assess whether authors in the orthodontic field of research currently report RCT abstracts adequately, as defined by the Consolidated Standards of Reporting Trials (CONSORT) statement. Design: An observational retrospective study. Methods: Electronic searches with supplementary hand searching were undertaken to identify RCTs published in (1) American Journal of Orthodontics and Dentofacial Orthopedics ( AJO-DO), (2) Angle Orthodontist ( AO), (3) European Journal of Orthodontics ( EJO) and (4) Journal of Orthodontics ( JO) for the period from January 2012 to December 2017. The completeness of the abstract reporting was evaluated using a modified CONSORT for abstract statement checklist. Results: A total of 3678 articles were retrieved, but only 224 RCT abstracts were identified and assessed. A high volume of RCTs were published with either the AO (39%) or AJO-DO (32%); the majority of the RCT abstracts (93.6%) were structured. The mean overall abstract reporting quality score was 69.1% (95% confidence interval = 67.5–70.7). In relation to individual quality items, the majority of the RCT abstracts (range = 96–100%) demonstrated clear reporting of the author/contact details, trial design, participants, interventions, objectives, outcomes, number of participants randomised to each group, recruitment, results and conclusions. However, reporting of the title, trial registration, funding and number of analysed participants were only moderately adequate and reporting of the assessment of blinding and adverse events were the least-reported items in the identified abstracts. Conclusions: As several CONSORT reporting items were poorly reported, it is the responsibility of authors, referees and editors alike to ensure that the CONSORT guidelines are followed.

PP29 Evaluating Supplementary Search Methods: Outcomes To Measure And Why

Introduction:In a recently published review of supplementary search methods, we proposed that researchers could usefully record time taken to search and present outcome values in similar way to existing studies, to facilitate generalisability of outcomes, where appropriate. We also discuss the idea of linking literature search effectiveness to study value. In this vignette, we discuss which outcomes we believe are important to measure and why. We discuss this in the context of the review of supplementary search methods and using a recently submitted evaluation of contacting study authors for context.Methods:In a recently completed systematic review, we contacted eighty-two study authors to ask three questions. We aimed to measure the following outcomes when contacting study authors: Effectiveness - determined as number of contacts compared to number of replies; Efficiency - i) time to make contact and ii) time between contact and reply. We determined this in hours, minutes and seconds, in line with other studies; Cost - determined by comparing the efficiency of contacting authors with the effectiveness; and Value - determined by reading and comparing the published studies with the replies received to see if any unique data were identified.Results:Effectiveness: thirty-eight answers were received from eighty-two possible contacts. Efficiency: In total, author contact took six hours, fifty-four minutes and twenty-five seconds across thirty-nine weeks. Replies were received across zero to thirty-nine days (median fourteen days). Cost: Cost for staff time was GBP 80.33 (EUR 91.20) or GBP 2.11 (EUR 2.40) per e-mail reply received. Value: We were able to identify value in author replies for each of the questions asked.Conclusions:In a recently published review of supplementary search methods, and a linked evaluation of the effectiveness of contacting study authors, we suggest outcomes that should be measured to determine effectiveness of literature search methods. We conclude that measuring these outcomes demonstrate both effectiveness and value.