Reporting and interpretation of results from clinical trials that did not claim a treatment difference

Objectives: To describe and summarise the reporting of “non-significant” results in randomized controlled trials (RCTs), and to estimate how commonly trial reports make erroneous claims of no treatment difference based on a non-statistically significant result.Design: Retrospective survey of published RCTs.Setting: Four high impact factor general medical journals, published between June 2016 and June 2017.Participants: Reports of randomised controlled trials that did not find a difference between the interventions they compared.Interventions: Not an interventional study.Primary and secondary outcome measures: We recorded the way each trial’s results for its primary outcome or outcomes were described in the Results and Conclusions sections of the Abstract, using a 10-category classification. We estimated the proportion of papers that made claims that were not justified by the results, or were open to multiple interpretations.Results: Eighty-five trial reports were included, reporting 111 treatment comparisons. The majority of papers made unjustified or confusing statements. In the Results section of abstracts, for 55/111 comparisons (49.5%) the study’s results were re-stated, without interpretation, and 34/111 (30.6%) stated that there was not a statistically significant difference. In the conclusions, 61/111 treatment comparisons (55%) stated that there was no treatment benefit, 14/111 (12.6%) that there was no significant benefit, and 13/111 (11.7%) that there was no significant difference. Conclusions: Despite decades of warnings, the error of concluding a lack of treatment benefit from a non-statistically significant result remains common.

Reporting and interpretation of results from clinical trials that did not claim a treatment difference: survey of four general medical journals

BMJ Open ◽

10.1136/bmjopen-2018-024785 ◽

2019 ◽

Vol 9 (9) ◽

pp. e024785 ◽

Cited By ~ 6

Author(s):

Simon Gates ◽

Elizabeth Ealing

Keyword(s):

Primary Outcome ◽

Significant Treatment Effect ◽

Treatment Benefit ◽

Significant Treatment ◽

Medical Journals ◽

P Values ◽

Significant Difference ◽

General Medical Journals ◽

Treatment Comparisons ◽

ObjectivesTo describe and summarise how the results of randomised controlled trials (RCTs) that did not find a significant treatment effect are reported, and to estimate how commonly trial reports make unwarranted claims.DesignWe performed a retrospective survey of published RCTs, published in four high impact factor general medical journals between June 2016 and June 2017.SettingTrials conducted in all settings were included.Participants94 reports of RCTs that did not find a difference in their main comparison or comparisons were included.InterventionsAll interventions.Primary and secondary outcomesWe recorded the way the results of each trial for its primary outcome or outcomes were described in Results and Conclusions sections of the Abstract, using a 10-category classification. Other outcomes were whether confidence intervals (CIs) and p values were presented for the main treatment comparisons, and whether the results and conclusions referred to measures of uncertainty. We estimated the proportion of papers that made claims that were not justified by the results, or were open to multiple interpretations.Results94 trial reports (120 treatment comparisons) were included. In Results sections, for 58/120 comparisons (48.3%) the results of the study were re-stated, without interpretation, and 38/120 (31.7%) stated that there was no statistically significant difference. In Conclusions, 65/120 treatment comparisons (54.2%) stated that there was no treatment benefit, 14/120 (11.7%) that there was no significant benefit and 16/120 (13.3%) that there was no significant difference. CIs and p values were both presented by 84% of studies (79/94), but only 3/94 studies referred to uncertainty when drawing conclusions.ConclusionsThe majority of trials (54.2%) inappropriately interpreted a result that was not statistically significant as indicating no treatment benefit. Very few studies interpreted the result as indicating a lack of evidence against the null hypothesis of zero difference between the trial arms.

Preventing dementia

Advances in Psychiatric Treatment ◽

10.1192/apt.11.3.176 ◽

2005 ◽

Vol 11 (3) ◽

pp. 176-183 ◽

Cited By ~ 18

Author(s):

Nitin Purandare ◽

Clive Ballard ◽

Alistair Burns

Keyword(s):

Risk Factors ◽

Randomised Controlled Trials ◽

Primary Outcome ◽

Secondary Outcome ◽

Controlled Trials ◽

Multiple Risk Factors ◽

Incident Dementia ◽

Treatment Of Hypertension ◽

Intervention Trials ◽

Epidemiological evidence has identified key strategies that may be used in the prevention of both Alzheimer's disease and vascular dementia. These strategies may be loosely divided into three areas: treatment of vascular risk factors, neuroprotection and increasing neuronal reserves. Evidence from randomised controlled trials already exists for treatment of hypertension but results for statins have been disappointing. Most of the intervention trials currently in progress focus on one or two risk factors and include cognition or dementia only as a secondary outcome. The potential of intervention strategies awaits confirmation by randomised controlled trials that target multiple risk factors in at-risk people with mild cognitive impairment, with incident dementia as the primary outcome.

Side Effects of Intravenous Patient-Controlled Analgesia with Remifentanil Compared with Intermittent Epidural Bolus for Labour Analgesia – A Randomized Controlled Trial

PRILOZI ◽

10.2478/prilozi-2020-0009 ◽

2019 ◽

Vol 40 (3) ◽

pp. 99-108

Author(s):

Dafina Karadjova ◽

Mirjana Shosholcheva ◽

Еmilija Ivanov ◽

Аtanas Sivevski ◽

Ivo Kjaev ◽

...

Keyword(s):

Side Effects ◽

Epidural Analgesia ◽

Primary Outcome ◽

Controlled Trial ◽

Secondary Outcome ◽

Patient Controlled Analgesia ◽

Obstetric Anaesthesia ◽

Randomized Controlled ◽

Significant Difference ◽

Remifentanil Group

Abstract Introduction: Epidural analgesia is considered a gold standard in obstetric anaesthesia and analgesia. However, in situation when it is contraindicated, unwanted by the patient or simply unavailable, remifentanil can be an excellent alternative. The goal of our study is to analyse the side effects of intravenous patient-controlled analgesia (IV PCA) with remifentanil compared with epidural analgesia during delivery. Material and methods: This study included 155 pregnant women in term for birth, divided into 2 groups: a remifentanil group (RG), and an epidural group (EG). Patients in the RG received intravenous PCA with remifentanil, while patients in the ЕG received epidural analgesia with programmed intermittent bolus dosing. Our primary outcome was maternal safety; the secondary outcome was neonatal safety. Results: The results present a significantly lower SaO2 value of the parturients in the RG (96.95 ± 1.4 vs 98.22 ± 0.6), and a significantly higher respiratory rate per minute in the EG at all time points after the onset of analgesia (20.85 ± 1.4 vs 18.67 ± 0.9). There was more frequent sedation, nausea and vomiting in the RG, while in the EG there was a more elevated temperature, itching and irregularities in the CTG record. Regarding the newborn, there was no significant difference between the two groups in the Apgar scores, pH, pCO2, pO2, and bicarbonate, while there was a significantly lower value of the base excess in the RG group. Conclusion: PCA with remifentanil is safe for the mother, foetus and the newborn, with minimal side effects. Continuous respiratory monitoring, oxygen supply and following of all consensus recommendations are mandatory.

Efficacy of nature-based therapy for individuals with stress-related illnesses: randomised controlled trial

The British Journal of Psychiatry ◽

10.1192/bjp.2018.2 ◽

2018 ◽

Vol 213 (1) ◽

pp. 404-411 ◽

Cited By ~ 12

Author(s):

Ulrika Karlsson Stigsdotter ◽

Sus Sola Corazon ◽

Ulrik Sidenius ◽

Patrik Karlsson Nyed ◽

Helmer Bøving Larsen ◽

...

Keyword(s):

Randomised Controlled Trial ◽

Primary Outcome ◽

Controlled Trial ◽

Well Being ◽

Behavioural Therapy ◽

Primary Outcome Measure ◽

Secondary Outcome ◽

Significant Difference ◽

Increasing Demand ◽

BackgroundStress-related illnesses are a major threat to public health, and there is increasing demand for validated treatments.AimsTo test the efficacy of nature-based therapy (NBT) for patients with stress-related illnesses.MethodRandomised controlled trial (ClinicalTrials.gov ID NCT01849718) comparing Nacadia® NBT (NNBT) with the cognitive–behavioural therapy known as Specialised Treatment for Severe Bodily Distress Syndromes (STreSS). In total, 84 participants were randomly allocated to one of the two treatments. The primary outcome measure was the mean aggregate score on the Psychological General Well-Being Index (PGWBI).ResultsBoth treatments resulted in a significant increase in the PGWBI (primary outcome) and a decrease in burnout (the Shirom–Melamed Burnout Questionnaire, secondary outcome), which were both sustained 12 months later. No significant difference in efficacy was found between NNBT and STreSS for primary outcome and secondary outcomes.ConclusionsThe study showed no statistical evidence of a difference between NNBT and STreSS for treating patients with stress-related illnesses.Declaration of interestNone.

Influence of overstated abstract conclusions on clinicians: a web-based randomised controlled trial

BMJ Open ◽

10.1136/bmjopen-2017-018355 ◽

2017 ◽

Vol 7 (12) ◽

pp. e018355 ◽

Cited By ~ 4

Author(s):

Kiyomi Shinohara ◽

Takuya Aoki ◽

Ryuhei So ◽

Yasushi Tsujimoto ◽

Aya M Suganuma ◽

...

Keyword(s):

Primary Care ◽

Primary Care Physicians ◽

Primary Outcome ◽

Controlled Trial ◽

Mean Difference ◽

Secondary Outcome ◽

Outcome Information ◽

Significant Difference ◽

Registration Number ◽

ObjectivesTo investigate whether overstatements in abstract conclusions influence primary care physicians’ evaluations when they read reports of randomised controlled trials (RCTs)DesignRCT setting: This study was a parallel-group randomised controlled survey, conducted online while masking the study hypothesis.ParticipantsVolunteers were recruited from members of the Japan Primary Care Association in January 2017. We sent email invitations to 7040 primary care physicians. Among the 787 individuals who accessed the website, 622 were eligible and automatically randomised into ‘without overstatement’ (n=307) and ‘with overstatement’ (n=315) groups.InterventionsWe selected five abstracts from published RCTs with at least one non-significant primary outcome and overstatement in the abstract conclusion. To construct a version without overstatement, we rewrote the conclusion sections. The methods and results sections were standardised to provide the necessary information of primary outcome information when it was missing in the original abstract. Participants were randomly assigned to read an abstract either with or without overstatements and asked to evaluate the benefit of the intervention.Outcome measuresThe primary outcome was the participants’ evaluation of the benefit of the intervention discussed in the abstract, on a scale from 0 to 10. A secondary outcome was the validity of the conclusion.ResultsThere was no significant difference between the groups with respect to their evaluation of the benefit of the intervention (mean difference: 0.07, 95% CI −0.28 to 0.42, p=0.69). Participants in the ‘without’ group considered the study conclusion to be more valid than those in the ‘with’ group (mean difference: 0.97, 95% CI 0.59 to 1.36, P<0.001).ConclusionThe overstatements in abstract conclusions did not significantly influence the primary care physicians’ evaluations of the intervention effect when necessary information about the primary outcomes was distinctly reported.Trial registration numberUMIN000025317; Pre-results.

Mycophenolate Mofetil for Induction Treatment of Lupus Nephritis: A Systematic Review and Metaanalysis

The Journal of Rheumatology ◽

10.3899/jrheum.100130 ◽

2010 ◽

Vol 38 (1) ◽

pp. 69-78 ◽

Cited By ~ 77

Author(s):

ZAHI TOUMA ◽

DAFNA D. GLADMAN ◽

MURRAY B. UROWITZ ◽

JOSEPH BEYENE ◽

ELIZABETH M. ULERYK ◽

...

Keyword(s):

Mycophenolate Mofetil ◽

Lupus Nephritis ◽

Primary Outcome ◽

Gastrointestinal Symptoms ◽

Induction Treatment ◽

Controlled Trials ◽

Efficacy And Safety ◽

Endstage Renal Disease ◽

Randomized Controlled ◽

Significant Difference

Objective.To systematically review the efficacy and safety of mycophenolic acid and mycophenolate mofetil (MMF) compared to cyclophosphamide (CYC) for the induction treatment of lupus nephritis (LN).Methods.Medline, Embase, the Cochrane Center Register of Controlled Trials, and abstracts presented in major international conferences were searched for randomized controlled trials. The primary outcome was renal remission (complete, partial, and overall) and secondary outcomes were adverse events during study period and longterm followup data. Data were compared between groups and relative risk (RR) and 95% CI were calculated.Results.Four trials of a total of 618 patients were included. MMF was not superior to CYC for renal remission (partial RR 0.94, 95% CI 0.80 to 1.12; complete RR 0.67, 95% CI 0.35 to 1.28, and overall RR 0.89, 95% CI 0.71 to 1.10). There was a significant reduction in alopecia (RR 5.77, 95% CI 1.56 to 21.38) and amenorrhea (RR 6.64, 95% CI 2.00 to 22.07) with the use of MMF compared to CYC. These results should be interpreted with caution given the width of the CI. There was no significant difference for infections, leukopenia, gastrointestinal symptoms, herpes zoster, endstage renal disease, and death among groups during study period and longterm followup data.Conclusion.We could not show that MMF is superior to CYC for the induction treatment of LN. Patients treated with MMF showed reduced risk of certain side effects. MMF can be used as an alternative to CYC for the induction treatment of LN.

Comparison of postoperative analgesic effects in response to either dexamethasone or dexmedetomidine as local anesthetic adjuvants: a systematic review and meta-analysis of randomized controlled trials

Journal of Anesthesia ◽

10.1007/s00540-021-02895-y ◽

2021 ◽

Author(s):

Zhen-Guo Song ◽

Shen-Yue Pang ◽

Gui-Yue Wang ◽

Zhao Zhang

Keyword(s):

Nerve Block ◽

Meta Analysis ◽

Secondary Outcome ◽

Controlled Trials ◽

Type I ◽

Analgesic Effects ◽

Randomized Controlled ◽

Upper Limb Surgery ◽

Significant Difference

AbstractThis review compares the effects of peripheral dexamethasone and dexmedetomidine on postoperative analgesia. We included six randomized controlled trials (354 patients) through a systematic literature search. We found that analgesia duration was comparable between dexamethasone and dexmedetomidine (58.59 min, 95% CI (confidence interval), − 66.13, 183.31 min) with extreme heterogeneity. Secondary outcome was also compared and no significant difference was observed in sensory block onset and duration and motor block duration and also for postoperative nausea and vomiting. It is noteworthy that dexamethasone reduced analgesic consumption (fentanyl) by 29.12 mcg compared with dexmedetomidine. We performed subgroup analyses and found no significant difference between the following: (1) lidocaine vs ropivacaine (P = 0.28), (2) nerve block vs nerve block + general anesthesia (P = 0.47), and (3) upper limb surgery vs thoracoscopic pneumonectomy (P = 0.27). We applied trial sequential analysis to assess the risks of type I and II errors and concluded that the meta-analysis was insufficiently powered to answer the clinical question, and further analysis is needed to establish which adjuvant is better. In conclusion, we believe that existing research indicates that dexamethasone and dexmedetomidine have equivalent analgesic effects in peripheral nerve blocks.

3,4-diaminopyridine Treatment for Lambert-Eaton Myasthenic Syndrome in Adults: A Meta-analysis of Randomized Controlled Trials

10.21203/rs.3.rs-34187/v1 ◽

2020 ◽

Author(s):

Na Zhang ◽

Daojun Hong ◽

Taohui Ouyang ◽

Wei Meng ◽

Jingwei Huang ◽

...

Keyword(s):

Primary Outcome ◽

Meta Analysis ◽

Autoimmune Disorder ◽

Placebo Treatment ◽

Secondary Outcome ◽

Controlled Trials ◽

Randomized Controlled ◽

Compound Muscle Action Potentials ◽

Myasthenic Syndrome

Abstract Background: Lambert-Eaton myasthenic syndrome (LEMS) is a rare autoimmune disorder of neuromuscular transmission. The objective was to examine the efficacy and safetyof 3,4-diaminopyridine (3,4-DAP) in LEMS. Methods: We searched several databases to identify relevant studies, including PubMed, EMBASE, Cochrane Neuromuscular Disease Group Specialized Register and the Cochrane Controlled Trials Register.The primary outcome Quantitative Myasthenia Gravis (QMG) muscle score and secondary outcome compound muscle action potentials (CMAP) amplitude were pooled by meta-analysis. Results: Six randomised controlled trials (RCT) involving 115 patients with LEMS were included. A meta-analysis of the primary outcome showed a significant improvement of 2.33 (95% CI, -2.81 to -1.85, p<0.001) in QMG muscle score after treatment with 3,4-DAP. Moreover, a meta-analysis of the secondary outcome showed that the overall mean CMAP amplitude improved significantly in LEMS patients with 3, 4-DAP treatment, compared with placebo treatment(mean difference 1.63 mV, 95% CI, 0.85 to 2.41, p<0.001). The overall assessment of all included trials showed low risk of bias and low heterogeneity. Conclusioins: The pooled results demonsrated that 3,4-DAP had a significant effect on LEMS treatment, with improvements in muscle strength score and CMAP amplitude, which are moderate to high evidence from randomized controlled trials.

Three-Hourly versus Two-Hourly Feeding Interval in Stable Preterm Infants: A Systematic Review and Meta-Analysis of Randomized Controlled Trials

10.21203/rs.3.rs-888420/v1 ◽

2021 ◽

Author(s):

Jogender Kumar ◽

Jitendra Meena ◽

Pradeep Debata ◽

Jeeva Mary Sankar ◽

Praveen Kumar ◽

...

Keyword(s):

Preterm Infants ◽

Primary Outcome ◽

Meta Analysis ◽

Stage Ii ◽

Controlled Trials ◽

Randomized Controlled ◽

Significant Difference ◽

Registration Number ◽

Group 1

Abstract Evidence from randomized controlled trials (RCTs) suggests that three-hourly feeding is safe and might help achieve full feeds earlier in preterm infants. We systematically compared the benefits and harms of three-hourly and two-hourly feeding schedules in preterm infants. We searched electronic databases (MEDLINE, CINAHL, Embase, Web of Science, and Scopus) and trial registries until 30 July 2021 for RCTs comparing the two feeding schedules. We did random-effects meta-analysis using RevMan 5.4 software. The primary outcome was the incidence of stage II or III necrotizing enterocolitis (NEC). Other outcomes were the incidence of any stage NEC, sepsis, mortality, time to full enteral feeds, and hospital stay. Six trials (872 participants) are included. There was no significant difference in the incidence of stage II/III NEC (3 trials; 530 participants; RR 1.39; 95% CI: 0.53,3.65; I2 -0%, low certainty), and any stage NEC (5 studies; 767 participants; RR 0.98; 95% CI: 0.53,1.82; I2 0%, very-low certainty) between three and two-hourly feeding groups. There was no difference in the time to achieve full feeds (5 trials; 755 participants; MD: -0.0 days; 95% CI: -0.32, 0.31, low certainty) or other outcomes. On subgroup analysis, neonates with birthweight <1000 grams and in the three-hourly feeding regime achieved full enteral feeds slower than those in the two-hourly feeding group (1 trial; 84 participants; MD: 2.9 days, 95% CI: 1.16, 4.64, low certainty).ConclusionIn stable preterm infants (1000-1500 grams), three-hourly feeding can be followed safely. In infants <1000 grams, two-hourly feeding should be continued till further evidence.Prospero Registration Number: CRD42021246568

3,4-diaminopyridine Treatment for Lambert-Eaton Myasthenic Syndrome in Adults: A Meta-analysis of Randomized Controlled Trials

10.21203/rs.3.rs-34187/v2 ◽

2020 ◽

Author(s):

Na Zhang ◽

Daojun Hong ◽

Taohui Ouyang ◽

Wei Meng ◽

Jingwei Huang ◽

...

Keyword(s):

Meta Analysis ◽

Autoimmune Disorder ◽

Placebo Treatment ◽

Secondary Outcome ◽

Controlled Trials ◽

Randomized Controlled ◽

Compound Muscle Action Potentials ◽

Myasthenic Syndrome ◽

Abstract Background: Lambert-Eaton myasthenic syndrome (LEMS) is a rare autoimmune disorder of neuromuscular transmission. The objective was to examine the efficacy and safety of 3,4-diaminopyridine (3,4-DAP) in patients with LEMS. Methods: We searched several databases to identify relevant studies, including PubMed, EMBASE, Cochrane Neuromuscular Disease Group Specialized Register and the Cochrane Controlled Trials Register.The primary outcome, quantitative myasthenia gravis (QMG) score and the secondary outcome, compound muscle action potentials (CMAP) amplitude were pooled by meta-analysis. Results: Six randomised controlled trials (RCT) involving 115 patients with LEMS were included. QMG score showed a significant decrease (improvement) of 2.33 points (95% CI, -2.81 to -1.85, p<0.001) after treatment with 3,4-DAP. Moreover, the overall mean CMAP amplitude improved significantly in LEMS patients with 3, 4-DAP treatment, compared with placebo treatment (mean difference 1.63 mV, 95% CI, 0.85 to 2.41, p<0.001). The overall assessment of all included trials showed a low risk of bias and low heterogeneity. Conclusions: The pooled results of randomized controlled trials demonsrated with moderate to high evidence that 3,4-DAP has a significant effect on LEMS treatment, with improvements in muscle strength score and CMAP amplitude.