Risk of bias versus quality assessment of randomised controlled trials: cross sectional study

ObjectivesTo assess the value of pilot and feasibility studies to randomised controlled trials (RCTs) funded by the National Institute for Health Research (NIHR) Health Technology Assessment (HTA) programme. To explore the methodological components of pilot/feasibility studies and how they inform full RCTs.Study designCross-sectional study.SettingBoth groups included NIHR HTA programme funded studies in the period 1 January 2010–31 December 2014 (decision date). Group 1: stand-alone pilot/feasibility studies published in the HTA Journal or accepted for publication. Group 2: all funded RCT applications funded by the HTA programme, including reference to an internal and/or external pilot/feasibility study. The methodological components were assessed using an adapted framework from a previous study.Main outcome measuresThe proportion of stand-alone pilot and feasibility studies which recommended proceeding to full trial and what study elements were assessed. The proportion of ‘HTA funded’ trials which used internal and external pilot and feasibility studies to inform the design of the trial.ResultsGroup 1 identified 15 stand-alone pilot/feasibility studies. Study elements most commonly assessed weretesting recruitment(100% in both groups),feasibility(83%, 100%) andsuggestions for further study/investigation(83%, 100%). Group 2 identified 161 ‘HTA funded’ applications: 59 cited an external pilot/feasibility study wheretesting recruitment(50%, 73%) andfeasibility(42%, 73%) were the most commonly reported study elements: 92 reported an internal pilot/feasibility study wheretesting recruitment(93%, 100%) andfeasibility(44%, 92%) were the most common study elements reported.Conclusions‘HTA funded’ research which includes pilot and feasibility studies assesses a variety of study elements. Pilot and feasibility studies serve an important role when determining the most appropriate trial design. However, how they are reported and in what context requires caution when interpreting the findings and delivering a definitive trial.

Download Full-text

Analysis of conference abstracts of prosthodontic randomised-controlled trials presented at IADR general sessions (2002–2015): a cross-sectional study of the relationship between demographic characteristics, reporting quality and final publication

BMJ Open ◽

10.1136/bmjopen-2019-034635 ◽

2020 ◽

Vol 10 (2) ◽

pp. e034635

Author(s):

Junsheng Chen ◽

Yubin Cao ◽

Meijie Wang ◽

Xueqi Gan ◽

Chunjie Li ◽

...

Keyword(s):

Randomised Controlled Trials ◽

Reporting Quality ◽

Cross Sectional Study ◽

Publication Rate ◽

Demographic Characteristics ◽

Sectional Study ◽

Cross Sectional ◽

Randomised Controlled ◽

The Relationship

ObjectivesTo analyse the relationship between demographic characteristics, reporting quality and final publication rate of conference abstracts of prosthodontic randomised-controlled trials (RCTs) presented at International Association for Dental Research (IADR) general sessions (2002–2015).DesignA cross-sectional study on conference abstracts.MethodsConference abstracts of prosthodontic RCTs presented at IADR general sessions (2002–2015) were obtained. Literature search was performed in multiple databases to confirm the final publication status of conference abstracts. Two investigators independently extracted the data including conference date, origin, presentation type, exact p value, number of centres, institution type, overall conclusion, subspecialty, publication time and journal. The reporting quality of abstracts was assessed by two investigators according to the Consolidated Standards of Reporting Trials statement. The relationship between demographic characteristics, reporting quality and final publication was analysed by χ2test.Setting, participants and interventionsNot applicable.Primary and secondary outcome measuresFinal publication rate, demographic characteristics and reporting quality of conference abstracts of prosthodontic RCTs presented at IADR general sessions (2002–2015).ResultsOf the 340 prosthodontic RCT abstracts, 43.24% were published. The mean time to final publication was 22.86 months. Europe contributed the most number of abstracts but Asia and Australia had the highest publication rate. Oral presentation, multicentre trial and complete denture and overdenture subspecialty were associated with a higher publication rate. Reporting quality of eligibility criteria of participants, random assignment and primary outcome results for each group correlated with a higher final publication rate.ConclusionsOver half of conference abstracts of prosthodontic RCTs presented at IADR general sessions (2002–2015) were unpublished. Oral presentation and multiple centres were associated with higher publication rates. Abstracts’ reporting quality addressing participant recruitment, assignment and primary results correlated with trials’ validity and applicability. Conference attendees may refer to this research to identify valid and applicable prosthodontic trials but should treat and apply results cautiously.

Download Full-text

A protocol of a cross-sectional study evaluating an online tool for early career peer reviewers assessing reports of randomised controlled trials

BMJ Open ◽

10.1136/bmjopen-2017-017462 ◽

2017 ◽

Vol 7 (9) ◽

pp. e017462 ◽

Cited By ~ 5

Author(s):

Anthony Chauvin ◽

David Moher ◽

Doug Altman ◽

David L Schriger ◽

Sabina Alam ◽

...

Keyword(s):

Randomised Controlled Trials ◽

Cross Sectional Study ◽

Early Career ◽

Sectional Study ◽

Cross Sectional ◽

Online Tool ◽

Peer Reviewers ◽

The Mean ◽

Randomised Controlled ◽

Completeness Of Reporting

IntroductionSystematic reviews evaluating the impact of interventions to improve the quality of peer review for biomedical publications highlighted that interventions were limited and have little impact. This study aims to compare the accuracy of early career peer reviewers who use an innovative online tool to the usual peer reviewer process in evaluating the completeness of reporting and switched primary outcomes in completed reports.Methods and analysisThis is a cross-sectional study of individual two-arm parallel-group randomised controlled trials (RCTs) published in the BioMed Central series medical journals,BMJ,BMJ OpenandAnnals of Emergency Medicineand indexed with the publication type ‘Randomised Controlled Trial’. First, we will develop an online tool and training module based (a) on the Consolidated Standards of Reporting Trials (CONSORT) 2010 checklist and the Explanation and Elaboration document that would be dedicated to junior peer reviewers for assessing the completeness of reporting of key items and (b) the Centre for Evidence-Based Medicine Outcome Monitoring Project process used to identify switched outcomes in completed reports of the primary results of RCTs when initially submitted. Then, we will compare the performance of early career peer reviewers who use the online tool to the usual peer review process in identifying inadequate reporting and switched outcomes in completed reports of RCTs at initial journal submission. The primary outcome will be the mean number of items accurately classified per manuscript. The secondary outcomes will be the mean number of items accurately classified per manuscript for the CONSORT items and the sensitivity, specificity and likelihood ratio to detect the item as adequately reported and to identify a switch in outcomes. We aim to include 120 RCTs and 120 early career peer reviewers.Ethics and disseminationThe research protocol was approved by the ethics committee of the INSERM Institutional Review Board (21 January 2016). The study is based on voluntary participation and informed written consent.Trial registration numberNCT03119376.

Download Full-text

Risk of bias and methodological issues in randomised controlled trials of acupuncture for knee osteoarthritis: a cross-sectional study

BMJ Open ◽

10.1136/bmjopen-2017-019847 ◽

2018 ◽

Vol 8 (3) ◽

pp. e019847 ◽

Cited By ~ 5

Author(s):

Pengli Jia ◽

Li Tang ◽

Jiajie Yu ◽

Andy H Lee ◽

Xu Zhou ◽

...

Keyword(s):

Knee Osteoarthritis ◽

Randomised Controlled Trials ◽

Primary Outcome ◽

Sample Size Calculation ◽

Risk Of Bias ◽

Controlled Trials ◽

Cochrane Central Register ◽

Methodological Issues ◽

Cross Sectional ◽

Randomised Controlled

ObjectiveTo assess risk of bias and to investigate methodological issues concerning the design, conduct and analysis of randomised controlled trials (RCTs) testing acupuncture for knee osteoarthritis (KOA).MethodsPubMed, EMBASE, Cochrane Central Register of Controlled Trials and four major Chinese databases were searched for RCTs that investigated the effect of acupuncture for KOA. The Cochrane tool was used to examine the risk of bias of eligible RCTs. Their methodological details were examined using a standardised and pilot-tested questionnaire of 48 items, together with the association between four predefined factors and important methodological quality indicators.ResultsA total of 248 RCTs were eligible, of which 39 (15.7%) used computer-generated randomisation sequence. Of the 31 (12.5%) trials that stated the allocation concealment, only one used central randomisation. Twenty-five (10.1%) trials mentioned that their acupuncture procedures were standardised, but only 18 (7.3%) specified how the standardisation was achieved. The great majority of trials (n=233, 94%) stated that blinding was in place, but 204 (87.6%) did not clarify who was blinded. Only 27 (10.9%) trials specified the primary outcome, for which 7 used intention-to-treat analysis. Only 17 (6.9%) trials included details on sample size calculation; none preplanned an interim analysis and associated stopping rule. In total, 46 (18.5%) trials explicitly stated that loss to follow-up occurred, but only 6 (2.4%) provided some information to deal with the issue. No trials prespecified, conducted or reported any subgroup or adjusted analysis for the primary outcome.ConclusionThe overall risk of bias was high among published RCTs testing acupuncture for KOA. Methodological limitations were present in many important aspects of design, conduct and analyses. These findings inform the development of evidence-based methodological guidance for future trials assessing the effect of acupuncture for KOA.

Download Full-text

A Cross-sectional literature survey showed the reporting quality of multicentre randomised controlled trials should be improved

Journal of Clinical Epidemiology ◽

10.1016/j.jclinepi.2021.05.008 ◽

2021 ◽

Author(s):

Xuan Zhang ◽

Wai Ching Lam ◽

Fan Liu ◽

Mengdan Li ◽

Lin Zhang ◽

...

Keyword(s):

Randomised Controlled Trials ◽

Reporting Quality ◽

Literature Survey ◽

Controlled Trials ◽

Cross Sectional ◽

Randomised Controlled

Download Full-text

Comparative efficacy and safety of interventions for treating head lice: a protocol for systematic review and network meta-analysis

BMJ Paediatrics Open ◽

10.1136/bmjpo-2021-001129 ◽

2021 ◽

Vol 5 (1) ◽

pp. e001129

Author(s):

Bill Stevenson ◽

Wubshet Tesfaye ◽

Julia Christenson ◽

Cynthia Mathew ◽

Solomon Abrha ◽

...

Keyword(s):

Systematic Review ◽

Randomised Controlled Trials ◽

Meta Analysis ◽

Grey Literature ◽

Risk Of Bias ◽

Controlled Trials ◽

Head Lice ◽

Efficacy And Safety ◽

Randomised Controlled ◽

Meta Analyses

BackgroundHead lice infestation is a major public health problem around the globe. Its treatment is challenging due to product failures resulting from rapidly emerging resistance to existing treatments, incorrect treatment applications and misdiagnosis. Various head lice treatments with different mechanism of action have been developed and explored over the years, with limited report on systematic assessments of their efficacy and safety. This work aims to present a robust evidence summarising the interventions used in head lice.MethodThis is a systematic review and network meta-analysis which will be reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-analyses statement for network meta-analyses. Selected databases, including PubMed, Embase, MEDLINE, Web of Science, CINAHL and Cochrane Central Register of Controlled Trials will be systematically searched for randomised controlled trials exploring head lice treatments. Searches will be limited to trials published in English from database inception till 2021. Grey literature will be identified through Open Grey, AHRQ, Grey Literature Report, Grey Matters, ClinicalTrials.gov, WHO International Clinical Trials Registry and International Standard Randomised Controlled Trials Number registry. Additional studies will be sought from reference lists of included studies. Study screening, selection, data extraction and assessment of methodological quality will be undertaken by two independent reviewers, with disagreements resolved via a third reviewer. The primary outcome measure is the relative risk of cure at 7 and 14 days postinitial treatment. Secondary outcome measures may include adverse drug events, ovicidal activity, treatment compliance and acceptability, and reinfestation. Information from direct and indirect evidence will be used to generate the effect sizes (relative risk) to compare the efficacy and safety of individual head lice treatments against a common comparator (placebo and/or permethrin). Risk of bias assessment will be undertaken by two independent reviewers using the Cochrane Risk of Bias tool and the certainty of evidence assessed using the Grading of Recommendations, Assessment, Development and Evaluations guideline for network meta-analysis. All quantitative analyses will be conducted using STATA V.16.DiscussionThe evidence generated from this systematic review and meta-analysis is intended for use in evidence-driven treatment of head lice infestations and will be instrumental in informing health professionals, public health practitioners and policy-makers.PROSPERO registration numberCRD42017073375.

Download Full-text

A Systematic Review of Cost-Effectiveness Analyses Alongside Randomised Controlled Trials of Acupuncture

Acupuncture in Medicine ◽

10.1136/acupmed-2012-010178 ◽

2012 ◽

Vol 30 (4) ◽

pp. 273-285 ◽

Cited By ~ 38

Author(s):

Song-Yi Kim ◽

Hyangsook Lee ◽

Younbyoung Chae ◽

Hi-Joon Park ◽

Hyejung Lee

Keyword(s):

Cost Effectiveness ◽

Usual Care ◽

Randomised Controlled Trials ◽

Risk Of Bias ◽

Economic Evaluations ◽

Controlled Trials ◽

German Studies ◽

Life Years ◽

Randomised Controlled ◽

The Cost

Objective To summarise the evidence on the cost-effectiveness of acupuncture. Methods We identified full economic evaluations such as cost-effectiveness analysis (CEA), cost-utility analysis (CUA) and cost-benefit analysis (CBA) alongside randomised controlled trials (RCTs) that assessed the consequences and costs of acupuncture for any medical condition. Eleven electronic databases were searched up to March 2011 without language restrictions. Eligible RCTs were assessed using the Cochrane criteria for risk of bias and a modified version of the checklist for economic evaluation. The general characteristics and the results of each economic analysis such as incremental cost-effectiveness ratios (ICERs) were extracted. Results Of 17 included studies, nine were CUAs that measured quality-adjusted life years (QALYs) and eight were CEAs that assessed effectiveness of acupuncture based on improvements in clinical symptoms. All CUAs showed that acupuncture with or without usual care was cost-effective compared with waiting list control or usual care alone, with ICERs ranging from ¢3011/QALY (dysmenorrhoea) to ¢22 298/QALY (allergic rhinitis) in German studies, and from £3855/QALY (osteoarthritis) to £9951/QALY (headache) in UK studies. In the CEAs, acupuncture was beneficial at a relatively low cost in six European and Asian studies. All CUAs were well-designed with a low risk of bias, but this was not the case for CEAs. Conclusions Overall, this review demonstrates the cost-effectiveness of acupuncture. Despite such promising results, any generalisation of these results needs to be made with caution given the diversity of diseases and the different status of acupuncture in the various countries.

Download Full-text

Methodological quality of systematic reviews on treatments for depression: a cross-sectional study

Epidemiology and Psychiatric Sciences ◽

10.1017/s2045796017000208 ◽

2017 ◽

Vol 27 (6) ◽

pp. 619-627 ◽

Cited By ~ 6

Author(s):

V. C. H. Chung ◽

X. Y. Wu ◽

Y. Feng ◽

R. S. T. Ho ◽

S. Y. S. Wong ◽

...

Keyword(s):

Systematic Reviews ◽

Methodological Quality ◽

A Priori ◽

Cross Sectional Study ◽

Risk Of Bias ◽

Sectional Study ◽

Pharmacological Treatments ◽

Cross Sectional ◽

Depression Treatments

Aims.Depression is one of the most common mental disorders and identifying effective treatment strategies is crucial for the control of depression. Well-conducted systematic reviews (SRs) and meta-analyses can provide the best evidence for supporting treatment decision-making. Nevertheless, the trustworthiness of conclusions can be limited by lack of methodological rigour. This study aims to assess the methodological quality of a representative sample of SRs on depression treatments.Methods.A cross-sectional study on the bibliographical and methodological characteristics of SRs published on depression treatments trials was conducted. Two electronic databases (the Cochrane Database of Systematic Reviews and the Database of Abstracts of Reviews of Effects) were searched for potential SRs. SRs with at least one meta-analysis on the effects of depression treatments were considered eligible. The methodological quality of included SRs was assessed using the validated AMSTAR (Assessing the Methodological Quality of Systematic Reviews) tool. The associations between bibliographical characteristics and scoring on AMSTAR items were analysed using logistic regression analysis.Results.A total of 358 SRs were included and appraised. Over half of included SRs (n = 195) focused on non-pharmacological treatments and harms were reported in 45.5% (n = 163) of all studies. Studies varied in methods and reporting practices: only 112 (31.3%) took the risk of bias among primary studies into account when formulating conclusions; 245 (68.4%) did not fully declare conflict of interests; 93 (26.0%) reported an ‘a priori’ design and 104 (29.1%) provided lists of both included and excluded studies. Results from regression analyses showed: more recent publications were more likely to report ‘a priori’ designs [adjusted odds ratio (AOR) 1.31, 95% confidence interval (CI) 1.09–1.57], to describe study characteristics fully (AOR 1.16, 95% CI 1.06–1.28), and to assess presence of publication bias (AOR 1.13, 95% CI 1.06–1.19), but were less likely to list both included and excluded studies (AOR 0.86, 95% CI 0.81–0.92). SRs published in journals with higher impact factor (AOR 1.14, 95% CI 1.04–1.25), completed by more review authors (AOR 1.12, 95% CI 1.01–1.24) and SRs on non-pharmacological treatments (AOR 1.62, 95% CI 1.01–2.59) were associated with better performance in publication bias assessment.Conclusion.The methodological quality of included SRs is disappointing. Future SRs should strive to improve rigour by considering of risk of bias when formulating conclusions, reporting conflict of interests and authors should explicitly describe harms. SR authors should also use appropriate methods to combine the results, prevent language and publication biases, and ensure timely updates.

Download Full-text