scholarly journals Tolerating bad health research: the continuing scandal

2021 ◽  
Author(s):  
Stefania Pirosca ◽  
Frances Shiely ◽  
Mike Clarke ◽  
Shaun Treweek
Keyword(s):  
High Risk ◽  
Cochrane Review ◽  
Research Community ◽  
Risk Of Bias ◽  
Randomised Trials ◽  
Main Text ◽  
Ethical Approval ◽  
The Cost ◽  
Trial Participants ◽  
Trial Cost

Abstract BackgroundAt the 2015 REWARD/EQUATOR conference on research waste, the late Doug Altman revealed that his only regret about his 1994 BMJ paper ‘The scandal of poor medical research’ was that he used the word ‘poor’ rather than ‘bad’. But how much research is bad? And what would improve things?Main textWe focus on randomised trials and look at scale, participants and cost. We randomly selected up to two quantitative intervention reviews published by all clinical Cochrane Review Groups between May 2020 and April 2021. Data including risk of bias, number of participants, intervention type and country were extracted for all trials included in selected reviews. High risk of bias trials were classed as bad. The cost of high risk of bias trials was estimated using published estimates of trial cost per-participant.We identified 96 reviews authored by 546 reviewers from 49 clinical Cochrane Review Groups that included 1,659 trials done in 84 countries. Of the 1,640 trials providing risk of bias information, 1,013 (62%) were high risk of bias (bad), 494 (30%) unclear and 133 (8%) low risk of bias. Bad trials were spread across all clinical areas and all countries. Well over 220,000 participants (or 56% of all participants) were in bad trials. The low estimate of the cost of bad trials was £726 million; our high estimate was over £8 billion. We have five recommendations: trials should be neither funded (1) or given ethical approval (2) unless they have a statistician and methodologist; trialists should use a risk of bias tool at design (3); more statisticians and methodologists should be trained and supported (4); there should be more funding into applied methodology research and infrastructure (5). ConclusionsMost randomised trials are bad and most trial participants will be in one. The research community has tolerated this for decades. This has to stop: we need to put rigour and methodology where it belongs– at the centre of our science.

scholarly journals Quality assessment of studies included in Cochrane oral health systematic reviews

2020 ◽  
Author(s):  
Ahmad Sofi-Mahmudi ◽  
Pouria Iranparvar ◽  
Maryam Shakiba ◽  
Erfan Shamsoddin ◽  
Hossein Mohammad-Rahimi ◽  
...  
Keyword(s):  
Clinical Trials ◽  
Oral Health ◽  
High Risk ◽  
Systematic Reviews ◽  
Randomized Clinical Trials ◽  
Cochrane Review ◽  
Risk Of Bias ◽  
Double Blind ◽  
Cochrane Reviews ◽  
Performance Bias

AbstractObjectivesThe Risk of Bias (RoB) and other characteristics of randomized clinical trials included in Cochrane oral health systematic reviews were assessed.Study Design and SettingsAll the trials included in Cochrane oral health systematic reviews were examined. The RoB was evaluated for all the included clinical trials according to the Cochrane review standards. The Overall Risk of Bias (ORoB) was defined in this study based on the criteria for determining the overall bias in Cochrane’s RoB tool-v2. Descriptive analyses were carried out to determine the frequency of each intended variable.ResultsA total of 2565 studies were included in our analysis. The majority of the studies (n=1600) had sample sizes of 50 or higher. As for blinding, 907 studies were labelled as double-blind. Performance bias showed the highest rate of high risk (31.4%). Almost half of the studies had a high ORoB compared to 11.1% with low ORoB. The studies that used placebos had higher low ORoB (14.8% vs. 10.7%). The double-blind studies had the highest low ORoB (23.6%). The studies with a cross-over design had the highest low ORoB (28.8%).ConclusionOverall, the RoB for the studies on dentistry and oral health in Cochrane reviews was deemed high.

scholarly journals Cluster randomised trials of individual-level interventions were at high risk of bias

2021 ◽  
Author(s):  
Christina Easter ◽  
Jennifer A. Thompson ◽  
Sandra Eldridge ◽  
Monica Taljaard ◽  
Karla Hemming
Keyword(s):  
High Risk ◽  
Risk Of Bias ◽  
Randomised Trials ◽  
Individual Level ◽  
Cluster Randomised Trials ◽  
Cluster Randomised

scholarly journals Dialysis and plasmapheresis for schizophrenia: a systematic review

2020 ◽  
Vol 50 (8) ◽  
pp. 1233-1240 ◽  
Author(s):  
Emily R. Cox ◽  
Katie F. M. Marwick ◽  
Robert W. Hunter ◽  
Josef Priller ◽  
Stephen M. Lawrie
Keyword(s):  
Systematic Review ◽  
Clinical Trials ◽  
High Risk ◽  
Detrimental Effect ◽  
Immune Dysfunction ◽  
Risk Of Bias ◽  
Randomised Trials ◽  
Inclusion Criteria ◽  
High Quality ◽  
Double Blinded

AbstractIncreasing evidence suggests that circulating factors and immune dysfunction may contribute to the pathogenesis of schizophrenia. In particular, proinflammatory cytokines, complement and autoantibodies against CNS epitopes have recently been associated with psychosis. Related concepts in previous decades led to several clinical trials of dialysis and plasmapheresis as treatments for schizophrenia. These trials may have relevance for the current understanding of schizophrenia. We aimed to identify whether dialysis or plasmapheresis are beneficial interventions in schizophrenia. We conducted a systematic search in major electronic databases for high-quality studies (double-blinded randomised trials with sham controls) applying either haemodialysis or plasmapheresis as an intervention in patients with schizophrenia, published in English from the start of records until September 2018. We found nine studies meeting inclusion criteria, reporting on 105 patients in total who received either sham or active intervention. One out of eight studies reported a beneficial effect of haemodialysis on schizophrenia, one a detrimental effect and six no effect. The sole trial of plasmapheresis found it to be ineffective. Adverse events were reported in 23% of patients. Studies were at unclear or high risk of bias. It is unlikely that haemodialysis is a beneficial treatment in schizophrenia, although the studies were of small size and could not consider potential subgroups. Plasmapheresis was only addressed by one study and warrants further exploration as a treatment modality in schizophrenia.

scholarly journals Quality Assessment of Clinical Trials Included in Cochrane Oral Health Systematic Reviews

2021 ◽  
Author(s):  
Ahmad Sofi-Mahmudi ◽  
Pouria Iranparvar ◽  
Maryam Shakiba ◽  
Erfan Shamsoddin ◽  
Hossein Mohammad-Rahimi ◽  
...  
Keyword(s):  
Clinical Trials ◽  
Oral Health ◽  
High Risk ◽  
Systematic Reviews ◽  
Cochrane Review ◽  
Risk Of Bias ◽  
Double Blind ◽  
Cochrane Reviews ◽  
Health Studies ◽  
Performance Bias

Abstract Background: Risk of Bias (RoB) and other characteristics of randomised clinical trials included in Cochrane oral health systematic reviews were assessed.Methods: All the trials included in Cochrane oral health systematic reviews were identified and examined. The RoB was evaluated for all the included clinical trials according to the Cochrane review standards. The Overall Risk of Bias (ORoB) was defined in this study based on the criteria in Cochrane’s RoB tool-v2. Descriptive analyses were carried out to determine the frequency of each intended variable.Results: In a total of 2565 included studies, the majority (n=1600) had 50 or higher sample sizes. As for blinding, 907 studies were labelled as double-blind. Performance bias showed the highest rate of high risk (31.4%). Almost half of the studies had a high ORoB compared to 11.1% with low ORoB. The studies that used placebos had higher low ORoB (14.8% vs 10.7%). The double-blind studies had the highest low ORoB (23.6%). The studies with a cross-over design had the highest low ORoB (28.8%).Conclusion: The RoB of oral health studies in Cochrane reviews was deemed high. Special efforts may be required to improve conducting and reporting of trials in this area.

scholarly journals Trial registration and selective outcome reporting in 585 clinical trials investigating drugs for prevention of postoperative nausea and vomiting

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Manuel Riemer ◽  
Peter Kranke ◽  
Antonia Helf ◽  
Debora Mayer ◽  
Maria Popp ◽  
...  
Keyword(s):  
Clinical Trials ◽  
High Risk ◽  
Postoperative Nausea ◽  
Cochrane Review ◽  
Postoperative Nausea And Vomiting ◽  
Risk Of Bias ◽  
The Body ◽  
Trial Registration ◽  
Selective Outcome Reporting ◽  
Outcome Reporting

Abstract Background Selective outcome reporting in clinical trials introduces bias in the body of evidence distorting clinical decision making. Trial registration aims to prevent this bias and is suggested by the International Committee of Medical Journal Editors (ICMJE) since 2004. Methods The 585 randomized controlled trials (RCTs) published between 1965 and 2017 that were included in a recently published Cochrane review on antiemetic drugs for prevention of postoperative nausea and vomiting were selected. In a retrospective study, we assessed trial registration and selective outcome reporting by comparing study publications with their registered protocols according to the ‘Cochrane Risk of bias’ assessment tool 1.0. Results In the Cochrane review, the first study which referred to a registered trial protocol was published in 2004. Of all 585 trials included in the Cochrane review, 334 RCTs were published in 2004 or later, of which only 22% (75/334) were registered. Among the registered trials, 36% (27/75) were pro- and 64% (48/75) were retrospectively registered. 41% (11/27) of the prospectively registered trials were free of selective outcome reporting bias, 22% (6/27) were incompletely registered and assessed as unclear risk, and 37% (10/27) were assessed as high risk. Major outcome discrepancies between registered and published high risk trials were a change from the registered primary to a published secondary outcome (32%), a new primary outcome (26%), and different outcome assessment times (26%). Among trials with high risk of selective outcome reporting 80% favoured at least one statistically significant result. Registered trials were assessed more often as ‘overall low risk of bias’ compared to non-registered trials (64% vs 28%). Conclusions In 2017, 13 years after the ICMJE declared prospective protocol registration a necessity for reliable clinical studies, the frequency and quality of trial registration in the field of PONV is very poor. Selective outcome reporting reduces trustworthiness in findings of clinical trials. Investigators and clinicians should be aware that only following a properly registered protocol and transparently reporting of predefined outcomes, regardless of the direction and significance of the result, will ultimately strengthen the body of evidence in the field of PONV research in the future.

A Critical Overview of Systematic Reviews of Chemotherapy for Advanced and Locally Advanced Pancreatic Cancer using both AMSTAR2 and ROBIS as Quality Assessment Tools

2020 ◽  
Vol 15 ◽  
Author(s):  
Amit Dang ◽  
Surendar Chidirala ◽  
Prashanth Veeranki ◽  
BN Vallish
Keyword(s):  
Pancreatic Cancer ◽  
High Risk ◽  
Systematic Reviews ◽  
Locally Advanced ◽  
Advanced Pancreatic Cancer ◽  
Risk Of Bias ◽  
Low Risk ◽  
Locally Advanced Pancreatic Cancer ◽  
Grade 3 ◽  
Critical Overview

Background: We performed a critical overview of published systematic reviews (SRs) of chemotherapy for advanced and locally advanced pancreatic cancer, and evaluated their quality using AMSTAR2 and ROBIS tools. Materials and Methods: PubMed and Cochrane Central Library were searched for SRs on 13th June 2020. SRs with metaanalysis which included only randomized controlled trials and that had assessed chemotherapy as one of the treatment arms were included. The outcome measures, which were looked into, were progression-free survival (PFS), overall survival (OS), and adverse events (AEs) of grade 3 or above. Two reviewers independently assessed all the SRs with both ROBIS and AMSTAR2. Results: Out of the 1,879 identified records, 26 SRs were included for the overview. Most SRs had concluded that gemcitabine-based combination regimes, prolonged OS and PFS, but increased the incidence of grade 3-4 toxicities, when compared to gemcitabine monotherapy, but survival benefits were not consistent when gemcitabine was combined with molecular targeted agents. As per ROBIS, 24/26 SRs had high risk of bias, with only 1/26 SR having low risk of bias. As per AMSTAR2, 25/26 SRs had critically low, and 1/26 SR had low, confidence in the results. The study which scored ‘low’ risk of bias in ROBIS scored ‘low confidence in results’ in AMSTAR2. The inter-rater reliability for scoring the overall confidence in the SRs with AMSTAR2 and the overall domain in ROBIS was substantial; ROBIS: kappa=0.785, SEM=0.207, p<0.001; AMSTAR2: kappa=0.649, SEM=0.323, p<0.001. Conclusion: Gemcitabine-based combination regimens can prolong OS and PFS but also worsen AEs when compared to gemcitabine monotherapy. The included SRs have an overall low methodological quality and high risk of bias as per AMSTAR2 and ROBIS respectively.

scholarly journals Efficacy and moderators of efficacy of trauma-focused cognitive behavioural therapies in children and adolescents: protocol for an individual participant data meta-analysis from randomised trials

BMJ Open ◽  
2021 ◽  
Vol 11 (2) ◽  
pp. e047212
Author(s):  
Anke de Haan ◽  
Caitlin Hitchcock ◽  
Richard Meiser-Stedman ◽  
Markus A Landolt ◽  
Isla Kuhn ◽  
...  
Keyword(s):  
Children And Adolescents ◽  
Meta Analysis ◽  
Behavioural Therapy ◽  
Risk Of Bias ◽  
Randomised Trials ◽  
First Line ◽  
Cognitive Behavioural ◽  
Individual Participant ◽  
Behavioural Therapies ◽  
First Line Treatment

IntroductionTrauma-focused cognitive behavioural therapies are the first-line treatment for posttraumatic stress disorder (PTSD) in children and adolescents. Nevertheless, open questions remain with respect to efficacy: why does this first-line treatment not work for everyone? For whom does it work best? Individual clinical trials often do not provide sufficient statistical power to examine and substantiate moderating factors. To overcome the issue of limited power, an individual participant data meta-analysis of randomised trials evaluating forms of trauma-focused cognitive behavioural therapy in children and adolescents aged 6–18 years will be conducted.Methods and analysisWe will update the National Institute for Health and Care Excellence guideline literature search from 2018 with an electronic search in the databases PsycINFO, MEDLINE, Embase, Cochrane Central Register of Controlled Trials and CINAHL with the terms (trauma* OR stress*) AND (cognitive therap* OR psychotherap*) AND (trial* OR review*). Electronic searches will be supplemented by a comprehensive grey literature search in archives and trial registries. Only randomised trials that used any manualised psychological treatment—that is a trauma-focused cognitive behavioural therapy for children and adolescents—will be included. The primary outcome variable will be child-reported posttraumatic stress symptoms (PTSS) post-treatment. Proxy-reports (teacher, parent and caregiver) will be analysed separately. Secondary outcomes will include follow-up assessments of PTSS, PTSD diagnosis and symptoms of comorbid disorders such as depression, anxiety-related and externalising problems. Random-effects models applying restricted maximum likelihood estimation will be used for all analyses. We will use the Revised Cochrane Risk of Bias tool to measure risk of bias.Ethics and disseminationContributing study authors need to have permission to share anonymised data. Contributing studies will be required to remove patient identifiers before providing their data. Results will be published in a peer-reviewed journal and presented at international conferences.PROSPERO registration numberCRD42019151954.

scholarly journals A systematic review of pharmacist-led audit and feedback interventions to influence prescribing behaviour in general practice settings

2021 ◽  
Vol 29 (Supplement_1) ◽  
pp. i34-i35
Author(s):  
M Carter ◽  
N Abutheraa ◽  
N Ivers ◽  
J Grimshaw ◽  
S Chapman ◽  
...  
Keyword(s):  
General Practice ◽  
Systematic Reviews ◽  
Health Professionals ◽  
Meta Analysis ◽  
Cochrane Review ◽  
Audit And Feedback ◽  
Cochrane Library ◽  
Randomised Trials ◽  
Prescribing Behaviour ◽  
Meta Analyses

Abstract Introduction Audit and Feedback (A&F) involves measuring data about practice, comparing it with clinical guidelines, professional standards or peer performance, and then feeding back the data to individuals/groups of health professionals to encourage change in practice (if required). A 2012 Cochrane review (1) found A&F was effective in changing health professionals’ behaviour and suggested that the person who delivers the A&F intervention influences its effect. Increasingly, pharmacists work in general practice and often have responsibility for medication review and repeat prescriptions. The effectiveness of pharmacist-led A&F in influencing prescribing behaviour is uncertain. Aim This secondary analysis from an ongoing update of the original Cochrane review aims to identify and describe pharmacist-led A&F interventions and evaluate their impact on prescribing behaviour in general practice compared with no intervention. Methods This sub-review is registered with PROSPERO: CRD42020194355 and complies with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (2). For the updated Cochrane review, the Cochrane Effective Practice and Organization of Care Group searched MEDLINE (1946 to present), EMBASE, CINAHL and Cochrane Library (March 2019) to identify randomised trials featuring A&F interventions. For this sub-review, authors screened titles and abstracts (May 2020) to identify trials involving pharmacist-led A&F interventions in primary care, extracted data, and assessed risk of bias (RoB) in eligible studies. Review results are summarised descriptively. Heterogeneity will be assessed and a random-effects meta-analysis is planned. Publication bias for selected outcomes and the certainty of the body of evidence will be evaluated and presented. Sub-group analyses will be conducted. Results Titles and abstracts of 295 studies identified for inclusion in the Cochrane A&F review update were screened. Eleven studies (all cluster-randomised trials) conducted in 9 countries (Denmark, Italy, Netherlands, Norway, Republic of Ireland, UK, Australia, Malaysia, USA) were identified for inclusion (Figure 1). Six studies had low RoB, two had high risk due to dissimilarities between trial arms at baseline and/or insufficient detail about randomisation, and three studies had unclear RoB. Studies examined the effect of A&F on prescribing for specific conditions (e.g. hypertension), medications (e.g. antibiotics), populations (e.g. patients &gt;70), and prescribing errors (e.g. inappropriate dose). The pharmacist delivering A&F was a colleague of intervention participants in five studies. Pharmacists’ levels of skill and experience varied; seven studies reported details of pharmacist training undertaken for trial purposes. A&F interventions in nine studies demonstrated changes in prescribing, including reductions in errors or inappropriate prescribing according to the study aims and smaller increases in unwanted prescribing compared with the control group. Data analyses are ongoing (results will be available for the conference). Conclusion The preliminary results demonstrate the effectiveness of pharmacist-led A&F interventions in different countries and health systems with influencing prescribing practice to align more closely with guidance. Studies measured different prescribing behaviours; meta-analysis is unlikely to include all 11 studies. Further detailed analysis including feedback format/content/frequency and pharmacist skill level/experience, work-base (external/internal to recipients), will examine the impact of specific features on intervention effectiveness. References 1. Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012(6):CD000259. 2. Moher D, Liberati A, Tetzlaff J, Altman DG, Group P. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097.

scholarly journals Benefits and safety of transdermal glyceryl trinitrate in acute stroke: a systematic review and meta-analysis of randomised trials (protocol)

BMJ Open ◽  
2021 ◽  
Vol 11 (1) ◽  
pp. e043591
Author(s):  
Beng Leong Lim ◽  
Wei Feng Lee ◽  
Wei Ming Ng ◽  
Wei Ling Tay ◽  
Wui Ling Chan
Keyword(s):  
Systematic Review ◽  
Ischaemic Stroke ◽  
Glyceryl Trinitrate ◽  
Acute Stroke ◽  
Meta Analysis ◽  
Cochrane Review ◽  
Cochrane Library ◽  
Placebo Control ◽  
Randomised Trials ◽  
Patient Subgroup

IntroductionHigh blood pressure (BP) in acute stroke has adverse outcomes. Transdermal glyceryl trinitrate (GTN) has beneficial properties in controlling BP. The 2016 meta-analysis and 2017 Cochrane review showed that transdermal GTN was beneficial in a small patient subgroup with stroke onset ≤6 hours. Larger studies focusing on this patient subgroup have since been conducted. We report the protocol for an updated systematic review and meta-analysis on the safety and benefits of transdermal GTN in acute stroke.Methods and analysisWe will search Medline, Pubmed, Embase, CINAHL and Cochrane Library from inception until June 2020 for randomised trials that report the efficacy and safety of transdermal GTN versus placebo/control therapy among adult patients with acute stroke. Primary outcomes include in-hospital mortality, BP lowering and late functional status. Secondary outcomes include early, late, resource utilisation and surrogate outcomes. Safety outcomes include reported adverse events. Reviewers will first screen titles and abstracts, and then full texts, to identify eligible studies. Independently and in duplicate, they will extract data, assess risk of bias (RoB) using a modified Cochrane RoB tool and quality of evidence using Grading of Recommendations, Assessment, Development and Evaluation. Disagreement will be resolved by discussion and consultation with an external reviewer if necessary. Using a random-effects model, we will report effect sizes using relative risks and 95% CIs. We will perform predefined subgroup analyses: intracerebral haemorrhage versus ischaemic stroke; minor (NIHSS (National Institutes of Health Stroke Scale) ≤five) versus major (NIHSS >five) ischaemic stroke; ischaemic stroke with versus without thrombolysis; prehospital versus non-prehospital settings; time from stroke to randomisation ≤6 versus >6 hours and high versus low overall RoB studies. We will also perform trial sequential analysis for the primary outcomes.Ethics and disseminationEthics board approval is unnecessary. PROSPERO registration has been obtained. The results will be disseminated through publication in a peer-reviewed journal.PROSPERO registration numberCRD42020173093.

Sign in / Sign up

Export Citation Format

Share Document