scholarly journals Reporting and interpretation of results from clinical trials that did not claim a treatment difference: survey of four general medical journals

BMJ Open ◽  
2019 ◽  
Vol 9 (9) ◽  
pp. e024785 ◽  
Author(s):  
Simon Gates ◽  
Elizabeth Ealing
Keyword(s):  
Primary Outcome ◽  
Significant Treatment Effect ◽  
Treatment Benefit ◽  
Significant Treatment ◽  
Medical Journals ◽  
P Values ◽  
Significant Difference ◽  
General Medical Journals ◽  
Treatment Comparisons ◽  
Randomised Controlled

ObjectivesTo describe and summarise how the results of randomised controlled trials (RCTs) that did not find a significant treatment effect are reported, and to estimate how commonly trial reports make unwarranted claims.DesignWe performed a retrospective survey of published RCTs, published in four high impact factor general medical journals between June 2016 and June 2017.SettingTrials conducted in all settings were included.Participants94 reports of RCTs that did not find a difference in their main comparison or comparisons were included.InterventionsAll interventions.Primary and secondary outcomesWe recorded the way the results of each trial for its primary outcome or outcomes were described in Results and Conclusions sections of the Abstract, using a 10-category classification. Other outcomes were whether confidence intervals (CIs) and p values were presented for the main treatment comparisons, and whether the results and conclusions referred to measures of uncertainty. We estimated the proportion of papers that made claims that were not justified by the results, or were open to multiple interpretations.Results94 trial reports (120 treatment comparisons) were included. In Results sections, for 58/120 comparisons (48.3%) the results of the study were re-stated, without interpretation, and 38/120 (31.7%) stated that there was no statistically significant difference. In Conclusions, 65/120 treatment comparisons (54.2%) stated that there was no treatment benefit, 14/120 (11.7%) that there was no significant benefit and 16/120 (13.3%) that there was no significant difference. CIs and p values were both presented by 84% of studies (79/94), but only 3/94 studies referred to uncertainty when drawing conclusions.ConclusionsThe majority of trials (54.2%) inappropriately interpreted a result that was not statistically significant as indicating no treatment benefit. Very few studies interpreted the result as indicating a lack of evidence against the null hypothesis of zero difference between the trial arms.

scholarly journals Reporting and interpretation of results from clinical trials that did not claim a treatment difference

2018 ◽  
Author(s):  
Simon Gates
Keyword(s):  
Primary Outcome ◽  
High Impact ◽  
Secondary Outcome ◽  
Controlled Trials ◽  
Treatment Benefit ◽  
Treatment Difference ◽  
Randomized Controlled ◽  
Significant Difference ◽  
Treatment Comparisons ◽  
Randomised Controlled

Objectives: To describe and summarise the reporting of “non-significant” results in randomized controlled trials (RCTs), and to estimate how commonly trial reports make erroneous claims of no treatment difference based on a non-statistically significant result.Design: Retrospective survey of published RCTs.Setting: Four high impact factor general medical journals, published between June 2016 and June 2017.Participants: Reports of randomised controlled trials that did not find a difference between the interventions they compared.Interventions: Not an interventional study.Primary and secondary outcome measures: We recorded the way each trial’s results for its primary outcome or outcomes were described in the Results and Conclusions sections of the Abstract, using a 10-category classification. We estimated the proportion of papers that made claims that were not justified by the results, or were open to multiple interpretations.Results: Eighty-five trial reports were included, reporting 111 treatment comparisons. The majority of papers made unjustified or confusing statements. In the Results section of abstracts, for 55/111 comparisons (49.5%) the study’s results were re-stated, without interpretation, and 34/111 (30.6%) stated that there was not a statistically significant difference. In the conclusions, 61/111 treatment comparisons (55%) stated that there was no treatment benefit, 14/111 (12.6%) that there was no significant benefit, and 13/111 (11.7%) that there was no significant difference. Conclusions: Despite decades of warnings, the error of concluding a lack of treatment benefit from a non-statistically significant result remains common.

scholarly journals Unpublished rating scales: A major source of bias in randomised controlled trials of treatments for schizophrenia

2000 ◽  
Vol 176 (3) ◽  
pp. 249-252 ◽  
Author(s):  
Max Marshall ◽  
Austin Lockwood ◽  
Caroline Bradley ◽  
Clive Adams ◽  
Claire Joy ◽  
...  
Keyword(s):  
Gold Standard ◽  
Randomised Controlled Trials ◽  
Rating Scales ◽  
Significant Treatment Effect ◽  
Control Groups ◽  
Significant Treatment ◽  
Treatment Groups ◽  
Definition Of ◽  
Randomised Controlled ◽  
And Control

BackgroundA recent review suggested an association between using unpublished scales in clinical trials and finding significant results.AimsTo determine whether such an association existed in schizophrenia trials.MethodThree hundred trials were randomly selected from the Cochrane Schizophrenia Group's Register. All comparisons between treatment groups and control groups using rating scales were identified. The publication status of each scale was determined and claims of a significant treatment effect were recorded.ResultsTrials were more likely to report that a treatment was superior to control when an unpublished scale was used to make the comparison (relative risk 1.37 (95% C11.12–1.68)). This effect increased when a ‘gold-standard’ definition of treatment superiority was applied (RR 1.94 (95% C11.35–2.79)). In non-pharmacological trials, one-third of ‘gold-standard’ claims of treatment superiority would not have been made if published scales had been used.ConclusionsUnpublished scales are a source of bias in schizophrenia trials.

scholarly journals Efficiency and safety of ondansetron in preventing postanaesthesia shivering

2016 ◽  
Vol 98 (6) ◽  
pp. 358-366 ◽  
Author(s):  
k He ◽  
H Zhao ◽  
HC Zhou
Keyword(s):  
Lower Risk ◽  
Randomised Controlled Trials ◽  
Primary Outcome ◽  
Outcome Measure ◽  
Meta Analysis ◽  
Primary Outcome Measure ◽  
Cochrane Library ◽  
Significant Difference ◽  
Secondary Outcomes ◽  
Randomised Controlled

Introduction Shivering is one of the most frequent complications of operation during the postanaesthesia period. Ondansetron has been proved to be valid in preventing postanaesthesia shivering (PAS) in several studies. However, its efficiency and safety are still disputable. We therefore performed an updated meta-analysis of randomised controlled trials (RCTs) for evaluation and to clarify this issue. Methods A literature search using the PubMed, Embase™ and Cochrane Library databases was performed (from inception to January 2015). RCTs that evaluated the efficiency and safety of ondansetron in the prevention of PAS were included in the meta-analysis. The primary outcome measure was incidence of PAS, and secondary outcomes included subgroup analysis and the side effects of ondansetron. Results A total of 8 RCTs containing 905 subjects were identified as suitable for this review. Compared with placebo, ondansetron was associated with a significant reduction of PAS (relative risk [RR]: 0.33, 95% confidence interval [CI]: 0.19–0.58, p=0.0001) while no difference was detected between ondansetron and pethidine (RR: 0.89, 95% CI: 0.41–1.94, p=0.78). There was no significant difference between ondansetron and placebo or pethidine in terms of risk of bradycardia but ondansetron was associated with a lower risk of hypotension (RR: 0.26, 95% CI: 0.08–0.79, p=0.020) than placebo. There was no difference in hypotension when ondansetron was compared with pethidine. Conclusions Ondansetron can prevent PAS effectively and reduce the risk of hypotension.

scholarly journals O04 Clinical and cost-effectiveness of ultrasound-guided intra-articular corticosteroid and local anaesthetic injection for hip OA: a randomised controlled trial (HIT)

Rheumatology ◽  
2020 ◽  
Vol 59 (Supplement_2) ◽  
Author(s):  
Zoe Paskins ◽  
Kieran Bromley ◽  
Martyn Lewis ◽  
Gemma Hughes ◽  
Emily Hughes ◽  
...  
Keyword(s):  
Cost Effectiveness ◽  
Randomised Controlled Trial ◽  
Pain Intensity ◽  
Primary Outcome ◽  
Controlled Trial ◽  
Hip Pain ◽  
Ultrasound Guided ◽  
Life Years ◽  
Significant Difference ◽  
Randomised Controlled

Abstract Background Evidence of the effectiveness of intra-articular corticosteroid injection for hip osteoarthritis (OA) is limited. The HIT trial compared the clinical and cost-effectiveness of an ultrasound-guided intra-articular hip injection (USGI) of 40mg triamcinolone acetonide and 4ml 1% lidocaine hydrochloride combined with best current treatment (BCT) with (i) BCT alone (primary objective) and (ii) an USGI of 5ml 1% lidocaine only combined with BCT (EudraCT:2014-003412-37). Methods This was a pragmatic, three-parallel arm, single-blind, randomised controlled trial in adults with moderate-severe painful hip OA recruited from community musculoskeletal services and primary care. Participants were randomised equally to: (1) BCT alone, (2) BCT plus USGI triamcinolone/lidocaine, or (3) BCT plus USGI lidocaine only. Outcomes were collected postally at 2 weeks, 2, 4 and 6 months. The primary outcome was self-reported current hip pain intensity (0-10 numeric rating scale (NRS)) over 6 months (repeated measures analysis). Secondary outcomes included function (WOMAC), and, for cost-utility analysis, general health (EQ-5D-5L) and healthcare utilisation. 204 participants were required to detect a minimum difference of 1 point in mean pain NRS score between arms (1) and (2) with 80% power (5% two-tailed significance level, 15% loss to follow-up). Analysis was by intention-to-treat. Results 199 participants were recruited (43% male, mean age 63 years), 67 to arm (1) and 66 each to arms (2) and (3). Primary outcome completion rates were 95% at 2 weeks, 94% at 2 months, 90% at 4 months, and 89% at 6 months. Greater mean improvement in hip pain intensity (0-10 NRS) over 6 months was seen with BCT plus USGI triamcinolone/lidocaine compared with BCT alone: -1.43 (95%CI -2.15,-0.72). Greater mean improvement in pain intensity was seen at 2 weeks (-3.17; -4.06,-2.28) and 2 months (-1.81;-2.71,-0.92), but not at 4 (-0.86;-1.78,0.05) or 6 months (0.12; -0.80,1.04). Participants treated with BCT plus USGI triamcinolone/lidocaine compared with BCT alone had greater mean improvement in function (WOMAC-F -5.47;(-9.41,-1.53)) over 6 months. There was no statistically significant difference in hip pain intensity over 6 months between BCT plus USGI triamcinolone/lidocaine compared with BCT plus USGI lidocaine (-0.52;-1.21,0.18). There was one possible treatment-related serious adverse event: a participant with no signs of infection at randomisation died from endocarditis four months after USGI triamcinolone/lidocaine. BCT plus USGI triamcinolone/lidocaine was less costly (mean cost difference per participant £-161.59) and associated with significantly higher quality-adjusted life-years (QALYs) than BCT only over 6 months (mean difference 0.0477 (0.0257,0.0699). Conclusion USGI triamcinolone/lidocaine plus BCT leads to greater improvements in pain and function over 6 months in adults with hip OA than BCT alone, and was highly cost-effective. There was no significant difference in hip pain intensity between the groups receiving USGI triamcinolone/lidocaine and USGI lidocaine only, raising the possibility of a degree of placebo effect. Disclosures Z. Paskins None. K. Bromley None. M. Lewis None. G. Hughes None. E. Hughes None. A. Cherrington None. A. Hall None. M. Holden None. R. Oppong None. J. Kigozi None. K. Stevenson None. A. Menon None. P. Roberts None. G. Peat None. C. Jinks None. N.E. Foster None. C.D. Mallen None. E. Roddy None.

scholarly journals Efficacy of nature-based therapy for individuals with stress-related illnesses: randomised controlled trial

2018 ◽  
Vol 213 (1) ◽  
pp. 404-411 ◽  
Author(s):  
Ulrika Karlsson Stigsdotter ◽  
Sus Sola Corazon ◽  
Ulrik Sidenius ◽  
Patrik Karlsson Nyed ◽  
Helmer Bøving Larsen ◽  
...  
Keyword(s):  
Randomised Controlled Trial ◽  
Primary Outcome ◽  
Controlled Trial ◽  
Well Being ◽  
Behavioural Therapy ◽  
Primary Outcome Measure ◽  
Secondary Outcome ◽  
Significant Difference ◽  
Increasing Demand ◽  
Randomised Controlled

BackgroundStress-related illnesses are a major threat to public health, and there is increasing demand for validated treatments.AimsTo test the efficacy of nature-based therapy (NBT) for patients with stress-related illnesses.MethodRandomised controlled trial (ClinicalTrials.gov ID NCT01849718) comparing Nacadia® NBT (NNBT) with the cognitive–behavioural therapy known as Specialised Treatment for Severe Bodily Distress Syndromes (STreSS). In total, 84 participants were randomly allocated to one of the two treatments. The primary outcome measure was the mean aggregate score on the Psychological General Well-Being Index (PGWBI).ResultsBoth treatments resulted in a significant increase in the PGWBI (primary outcome) and a decrease in burnout (the Shirom–Melamed Burnout Questionnaire, secondary outcome), which were both sustained 12 months later. No significant difference in efficacy was found between NNBT and STreSS for primary outcome and secondary outcomes.ConclusionsThe study showed no statistical evidence of a difference between NNBT and STreSS for treating patients with stress-related illnesses.Declaration of interestNone.

scholarly journals Influence of overstated abstract conclusions on clinicians: a web-based randomised controlled trial

BMJ Open ◽  
2017 ◽  
Vol 7 (12) ◽  
pp. e018355 ◽  
Author(s):  
Kiyomi Shinohara ◽  
Takuya Aoki ◽  
Ryuhei So ◽  
Yasushi Tsujimoto ◽  
Aya M Suganuma ◽  
...  
Keyword(s):  
Primary Care ◽  
Primary Care Physicians ◽  
Primary Outcome ◽  
Controlled Trial ◽  
Mean Difference ◽  
Secondary Outcome ◽  
Outcome Information ◽  
Significant Difference ◽  
Registration Number ◽  
Randomised Controlled

ObjectivesTo investigate whether overstatements in abstract conclusions influence primary care physicians’ evaluations when they read reports of randomised controlled trials (RCTs)DesignRCT setting: This study was a parallel-group randomised controlled survey, conducted online while masking the study hypothesis.ParticipantsVolunteers were recruited from members of the Japan Primary Care Association in January 2017. We sent email invitations to 7040 primary care physicians. Among the 787 individuals who accessed the website, 622 were eligible and automatically randomised into ‘without overstatement’ (n=307) and ‘with overstatement’ (n=315) groups.InterventionsWe selected five abstracts from published RCTs with at least one non-significant primary outcome and overstatement in the abstract conclusion. To construct a version without overstatement, we rewrote the conclusion sections. The methods and results sections were standardised to provide the necessary information of primary outcome information when it was missing in the original abstract. Participants were randomly assigned to read an abstract either with or without overstatements and asked to evaluate the benefit of the intervention.Outcome measuresThe primary outcome was the participants’ evaluation of the benefit of the intervention discussed in the abstract, on a scale from 0 to 10. A secondary outcome was the validity of the conclusion.ResultsThere was no significant difference between the groups with respect to their evaluation of the benefit of the intervention (mean difference: 0.07, 95% CI −0.28 to 0.42, p=0.69). Participants in the ‘without’ group considered the study conclusion to be more valid than those in the ‘with’ group (mean difference: 0.97, 95% CI 0.59 to 1.36, P<0.001).ConclusionThe overstatements in abstract conclusions did not significantly influence the primary care physicians’ evaluations of the intervention effect when necessary information about the primary outcomes was distinctly reported.Trial registration numberUMIN000025317; Pre-results.

scholarly journals Droperidolv. haloperidol for sedation of aggressive behaviour in acute mental health: Randomised controlled trial

2015 ◽  
Vol 206 (3) ◽  
pp. 223-228 ◽  
Author(s):  
Leonie Calver ◽  
Vincent Drinkwater ◽  
Rahul Gupta ◽  
Colin B. Page ◽  
Geoffrey K. Isbister
Keyword(s):  
Mental Health ◽  
Randomised Controlled Trial ◽  
Primary Outcome ◽  
Controlled Trial ◽  
Behavioural Disturbance ◽  
Significant Difference ◽  
Psychiatric Units ◽  
Secondary Outcomes ◽  
Randomised Controlled ◽  
Effective Sedation

BackgroundAgitation and aggression are significant problems in acute psychiatric units. There is little consensus on which drug is most effective and safest for sedation of these patients.AimsTo compare the effectiveness and safety of haloperidolv. droperidol for patients with agitation and aggression.MethodIn a masked, randomised controlled trial (ACTRN12611000565943) intramuscular droperidol (10 mg) was compared with intramuscular haloperidol (10 mg) for adult patients with acute behavioural disturbance in a psychiatric intensive care unit. The primary outcome was time to sedation within 120 min. Secondary outcomes were use of additional sedation, adverse events and staff injuries.ResultsFrom 584 patients, 110 were randomised to haloperidol and 118 to droperidol. Effective sedation occurred in 210 (92%) patients within 120 min. There was no significant difference in median time to sedation: 20 min (interquartile range 15–30, range 10–75) for haloperidolv. 25 min (IQR 15–30, range 10–115) for droperidol (P= 0.89). Additional sedation was used more often with haloperidol (13%v. 5%,P= 0.06), but adverse effects were less common with haloperidol (1%v. 5%,P= 0.12). There were 8 staff injuries.ConclusionsBoth haloperidol and droperidol were effective for sedation of patients with acute behavioural disturbance.

scholarly journals Time to intubation with McGrath™ videolaryngoscope versus direct laryngoscope in powered air-purifying respirator: a randomised controlled trial

2021 ◽  
Author(s):  
QY Goh ◽  
SA Lie ◽  
Z Tan ◽  
PYB Tan ◽  
SY Ng ◽  
...  
Keyword(s):  
Tracheal Intubation ◽  
Primary Outcome ◽  
Controlled Trial ◽  
Surgical Patients ◽  
Intubation Difficulty ◽  
Direct Laryngoscope ◽  
Significant Difference ◽  
Scale Scores ◽  
Difficulty Scale ◽  
Randomised Controlled

Introduction: During the COVID-19 pandemic, multiple guidelines have recommended the videolaryngoscope for tracheal intubation. However, there is no evidence that videolaryngoscope reduces time to tracheal intubation, which is important for COVID-19 patients with respiratory failure. Methods: To simulate intubation of COVID-19 patients, we randomised 28 elective surgical patients to be intubated with either the McGrath™ MAC videolaryngoscope or the direct laryngoscope by specialist anaesthetists donning 3M™ Jupiter™ powered air-purifying respirators (PAPR) and N95 masks. Primary outcome was time to intubation. Results: The median (IQR) times to intubation were 61s (37–63 s) and 41.5s (37–56 s) in the videolaryngoscope and direct laryngoscope groups respectively (p = 0.35). The closest mean (SD) distances between the anaesthetist and the patient during intubation were 21.6 cm (4.8 cm) and 17.6 cm (5.3 cm) in the videolaryngoscope and direct laryngoscope groups, respectively (p = 0.045). There were no significant differences in the median intubation difficulty scale scores, proportion of successful intubation at first laryngoscopic attempt and proportion of intubations requiring adjuncts. Intubations for all the patients were successful with no adverse event. Conclusion: There was no significant difference in the time to intubation by specialist anaesthetists who were donned in PAPR and N95 masks on elective surgical patients with either the McGrath™ videolaryngoscope or direct laryngoscope. The distance between the anaesthetist and patient was significantly further with the videolaryngoscope. The direct laryngoscope could be an equal alternative to videolaryngoscope for specialist anaesthetists when resources are limited or disrupted due to the pandemic.

Sign in / Sign up

Export Citation Format

Share Document