Comparison of treatment effect sizes associated with surrogate and final patient relevant outcomes in randomised controlled trials: meta-epidemiological study

AbstractWe aimed to establish the treatment effect of physical activity for depression in young people through meta-analysis. Four databases were searched to September 2016 for randomised controlled trials of physical activity interventions for adolescents and young adults, 12–25 years, experiencing a diagnosis or threshold symptoms of depression. Random-effects meta-analysis was used to estimate the standardised mean difference (SMD) between physical activity and control conditions. Subgroup analysis and meta-regression investigated potential treatment effect modifiers. Acceptability was estimated using dropout. Trials were assessed against risk of bias domains and overall quality of evidence was assessed using GRADE criteria. Seventeen trials were eligible and 16 provided data from 771 participants showing a large effect of physical activity on depression symptoms compared to controls (SMD = −0.82, 95% CI = −1.02 to −0.61, p < 0.05, I2 = 38%). The effect remained robust in trials with clinical samples (k = 5, SMD = −0.72, 95% CI = −1.15 to −0.30), and in trials using attention/activity placebo controls (k = 7, SMD = −0.82, 95% CI = −1.05 to −0.59). Dropout was 11% across physical activity arms and equivalent in controls (k = 12, RD = −0.01, 95% CI = −0.04 to 0.03, p = 0.70). However, the quality of RCT-level evidence contributing to the primary analysis was downgraded two levels to LOW (trial-level risk of bias, suspected publication bias), suggesting uncertainty in the size of effect and caution in its interpretation. While physical activity appears to be a promising and acceptable intervention for adolescents and young adults experiencing depression, robust clinical effectiveness trials that minimise risk of bias are required to increase confidence in the current finding. The specific intervention characteristics required to improve depression remain unclear, however best candidates given current evidence may include, but are not limited to, supervised, aerobic-based activity of moderate-to-vigorous intensity, engaged in multiple times per week over eight or more weeks. Further research is needed. (Registration: PROSPERO-CRD 42015024388).

Download Full-text

Treatment effect bias in randomised controlled trials using surrogate outcomes: a preliminary cohort study analysis

Trials ◽

10.1186/1745-6215-12-s1-a73 ◽

2011 ◽

Vol 12 (S1) ◽

Author(s):

Oriana Ciani ◽

Rod S Taylor

Keyword(s):

Cohort Study ◽

Treatment Effect ◽

Randomised Controlled Trials ◽

Controlled Trials ◽

Study Analysis ◽

Surrogate Outcomes ◽

Randomised Controlled

Download Full-text

Who Benefits from Electroconvulsive Therapy?

The British Journal of Psychiatry ◽

10.1192/bjp.160.3.355 ◽

1992 ◽

Vol 160 (3) ◽

pp. 355-359 ◽

Cited By ~ 55

Author(s):

Heather Buchan ◽

Eve Johnstone ◽

K. McPherson ◽

R. L. Palmer ◽

T. J. Crow ◽

...

Keyword(s):

Electroconvulsive Therapy ◽

Treatment Effect ◽

Randomised Controlled Trials ◽

Controlled Trials ◽

Combining Data ◽

Randomised Controlled ◽

Improved Outcome

This paper describes the results obtained by combining data from the Northwick Park and Leicester randomised controlled trials of ECT. Patients who suffered from depression in which retardation and delusions were features and who received real ECT had a significantly improved outcome at the end of four weeks of treatment (as measured by improvement in the HRSD) in comparison with those who received simulated ECT. However, this treatment effect was not detectable at six-month follow-up. Patients who were neither retarded nor deluded did not benefit significantly from real as opposed to simulated ECT.

Download Full-text

Effect of interventions for the well-being, satisfaction and flourishing of general practitioners—a systematic review

BMJ Open ◽

10.1136/bmjopen-2020-046599 ◽

2021 ◽

Vol 11 (8) ◽

pp. e046599

Author(s):

Diana Naehrig ◽

Aaron Schokman ◽

Jessica Kate Hughes ◽

Ronald Epstein ◽

Ian B Hickie ◽

...

Keyword(s):

Systematic Review ◽

Randomised Controlled Trials ◽

Intervention Studies ◽

Well Being ◽

Risk Of Bias ◽

Effect Sizes ◽

Controlled Trials ◽

Positive Outcomes ◽

Mindfulness Interventions ◽

Randomised Controlled

ObjectivesClinician well-being has been recognised as an important pillar of healthcare. However, research mainly addresses mitigating the negative aspects of stress or burnout, rather than enabling positive aspects. With the added strain of a pandemic, identifying how best to maintain and support the well-being, satisfaction and flourishing of general practitioners (GPs) is now more important than ever.DesignSystematic review.Data sourcesWe searched MEDLINE, PsycINFO, Embase, CINAHL and Scopus from 2000 to 2020.Study selectionIntervention studies with more than 50% GPs in the sample evaluating self-reported well-being, satisfaction and related positive outcomes were included. The Cochrane Risk of Bias 2 tool was applied.ResultsWe retrieved 14 792 records, 94 studies underwent full-text review. We included 19 studies in total. Six randomised controlled trials, three non-randomised, controlled trials, eight non-controlled studies of individual or organisational interventions with a total of 1141 participants. There were two quasi-experimental articles evaluating health system policy change. Quantitative and qualitative positive outcomes were extracted and analysed. Individual mindfulness interventions were the most common (k=9) with medium to large within-group (0.37–1.05) and between-group (0.5–1.5) effect sizes for mindfulness outcomes, and small-to-medium effect sizes for other positive outcomes including resilience, compassion and empathy. Studies assessing other intervention foci or other positive outcomes (including well-being, satisfaction) were of limited size and quality.ConclusionsThere is remarkably little evidence on how to improve GPs well-being beyond using mindfulness interventions, particularly for interventions addressing organisational or system factors. This was further undermined by inconsistent reporting, and overall high risk of bias. We need to conduct research in this space with the same rigour with which we approach clinical intervention studies in patients.PROSPERO registration numberCRD42020164699.Funding sourceDr Diana Naehrig is funded through the Raymond Seidler PhD scholarship.

Download Full-text

Machine learning analysis plans for randomised controlled trials: detecting treatment effect heterogeneity with strict control of type I error

Trials ◽

10.1186/s13063-020-4076-y ◽

2020 ◽

Vol 21 (1) ◽

Cited By ~ 1

Author(s):

James A. Watson ◽

Chris C. Holmes

Keyword(s):

Machine Learning ◽

Treatment Effect ◽

Randomised Controlled Trials ◽

Type I Error ◽

Controlled Trials ◽

Type I ◽

Treatment Effect Heterogeneity ◽

Randomised Controlled ◽

Effect Heterogeneity ◽

Learning Analysis

Download Full-text

Recommended motor assessments based on psychometric properties in individuals with dementia: a systematic review

European Review of Aging and Physical Activity ◽

10.1186/s11556-019-0228-z ◽

2019 ◽

Vol 16 (1) ◽

Cited By ~ 2

Author(s):

Sandra Trautwein ◽

Philipp Maurus ◽

Bettina Barisch-Fritz ◽

Anela Hadzic ◽

Alexander Woll

Keyword(s):

Physical Activity ◽

Systematic Review ◽

Psychometric Properties ◽

Randomised Controlled Trials ◽

Effect Sizes ◽

Controlled Trials ◽

Retest Reliability ◽

Frequency Of Use ◽

Randomised Controlled ◽

Test Retest Reliability

Abstract Background Motor assessments are important to determine effectiveness of physical activity in individuals with dementia (IWD). However, inappropriate and non-standardised assessments without sound psychometric properties have been used. This systematic review aims to examine psychometric properties of motor assessments in IWD combined with frequency of use and effect sizes and to provide recommendations based on observed findings. We performed a two-stage systematic literature search using Pubmed, Web of Science, Cochrane Library, ALOIS, and Scopus (inception - July/September 2018, English and German). The first search purposed to identify motor assessments used in randomised controlled trials assessing effectiveness of physical activity in IWD and to display their frequency of use and effect sizes. The second search focused on psychometric properties considering influence of severity and aetiology of dementia and cueing on test-retest reliability. Two reviewers independently extracted and analysed findings of eligible studies in a narrative synthesis. Results Literature searches identified 46 randomised controlled trials and 21 psychometric property studies. While insufficient information was available for validity, we observed sufficient inter-rater and relative test-retest reliability but unacceptable absolute test-retest reliability for most assessments. Combining these findings with frequency of use and effect sizes, we recommend Functional Reach Test, Groningen Meander Walking Test (time), Berg Balance Scale, Performance Oriented Mobility Assessment, Timed Up & Go Test, instrumented gait analysis (spatiotemporal parameters), Sit-to-Stand assessments (repetitions> 1), and 6-min walk test. It is important to consider that severity and aetiology of dementia and cueing influenced test-retest reliability of some assessments. Conclusion This review establishes an important foundation for future investigations. Sufficient relative reliability supports the conclusiveness of recommended assessments at group level, while unacceptable absolute reliability advices caution in assessing intra-individual changes. Moreover, influences on test-retest reliability suggest tailoring assessments and instructions to IWD and applying cueing only where it is inevitable. Considering heterogeneity of included studies and insufficient examination in various areas, these recommendations are not comprehensive. Further research, especially on validity and influences on test-retest reliability, as well as standardisation and development of tailored assessments for IWD is crucial. This systematic review was registered in PROSPERO (CRD42018105399).

Download Full-text

A study of target effect sizes in randomised controlled trials published in the Health Technology Assessment journal

Trials ◽

10.1186/s13063-018-2886-y ◽

2018 ◽

Vol 19 (1) ◽

Cited By ~ 7

Author(s):

Joanne C. Rothwell ◽

Steven A. Julious ◽

Cindy L. Cooper

Keyword(s):

Health Technology Assessment ◽

Technology Assessment ◽

Randomised Controlled Trials ◽

Health Technology ◽

Effect Sizes ◽

Controlled Trials ◽

Target Effect ◽

Randomised Controlled

Download Full-text

Adjusting for treatment switching in randomised controlled trials – A simulation study and a simplified two-stage method

Statistical Methods in Medical Research ◽

10.1177/0962280214557578 ◽

2014 ◽

Vol 26 (2) ◽

pp. 724-751 ◽

Cited By ~ 23

Author(s):

Nicholas R Latimer ◽

KR Abrams ◽

PC Lambert ◽

MJ Crowther ◽

AJ Wailoo ◽

...

Keyword(s):

Treatment Effect ◽

Survival Benefit ◽

Randomised Controlled Trials ◽

Health Technology ◽

Control Group ◽

Controlled Trials ◽

Inverse Probability ◽

Treatment Switching ◽

Overall Survival Benefit ◽

Randomised Controlled

Estimates of the overall survival benefit of new cancer treatments are often confounded by treatment switching in randomised controlled trials (RCTs) – whereby patients randomised to the control group are permitted to switch onto the experimental treatment upon disease progression. In health technology assessment, estimates of the unconfounded overall survival benefit associated with the new treatment are needed. Several switching adjustment methods have been advocated in the literature, some of which have been used in health technology assessment. However, it is unclear which methods are likely to produce least bias in realistic RCT-based scenarios. We simulated RCTs in which switching, associated with patient prognosis, was permitted. Treatment effect size and time dependency, switching proportions and disease severity were varied across scenarios. We assessed the performance of alternative adjustment methods based upon bias, coverage and mean squared error, related to the estimation of true restricted mean survival in the absence of switching in the control group. We found that when the treatment effect was not time-dependent, rank preserving structural failure time models (RPSFTM) and iterative parameter estimation methods produced low levels of bias. However, in the presence of a time-dependent treatment effect, these methods produced higher levels of bias, similar to those produced by an inverse probability of censoring weights method. The inverse probability of censoring weights and structural nested models produced high levels of bias when switching proportions exceeded 85%. A simplified two-stage Weibull method produced low bias across all scenarios and provided the treatment switching mechanism is suitable, represents an appropriate adjustment method.

Download Full-text

Examination of overall treatment effect and the proportion attributable to contextual effect in osteoarthritis: meta-analysis of randomised controlled trials

Annals of the Rheumatic Diseases ◽

10.1136/annrheumdis-2015-208387 ◽

2016 ◽

Vol 75 (11) ◽

pp. 1964-1970 ◽

Cited By ~ 59

Author(s):

Kun Zou ◽

Jean Wong ◽

Natasya Abdullah ◽

Xi Chen ◽

Toby Smith ◽

...

Keyword(s):

Treatment Effect ◽

Randomised Controlled Trials ◽

Specific Effect ◽

Pain Reduction ◽

Contextual Effect ◽

Controlled Trials ◽

Large Sample Size ◽

Duration Of Treatment ◽

Manual Search ◽

Randomised Controlled

ObjectiveTo examine the overall treatment effect and the proportion attributable to contextual effect (PCE) in randomised controlled trials (RCTs) of diverse treatments for osteoarthritis (OA).MethodsWe searched Medline, Embase, Central, Science Citation Index, AMED and CINAHL through October 2014, supplemented with manual search of reference lists, published meta-analyses and systematic reviews. Included were RCTs in OA comparing placebo with representative complementary, pharmacological, non-pharmacological and surgical treatments. The primary outcome was pain. Secondary outcomes were function and stiffness. The effect size (ES) of overall treatment effect and the PCE were pooled using random-effects model. Subgroup analyses and meta-regression were conducted to examine determinants of the PCE.ResultsIn total, 215 trials (41 392 participants) were included. The overall treatment effect for pain ranged from the smallest with lavage (ES=0.46, 95% CI 0.24 to 0.68) to the largest with topical non-steroidal anti-inflammatory drugs (ES=1.37, 95% CI 1.19 to 1.55). On average, 75% (PCE=0.75, 95% CI 0.72 to 0.79) of pain reduction was attributable to contextual effect. It varied by treatment from 47% (PCE=0.47, 95% CI 0.32 to 0.70) for intra-articular corticosteroid to 91% (PCE=0.91, 95% CI 0.60 to 1.37) for joint lavage. Similar results were observed for function and stiffness. Treatment delivered by needle/injection and other means than oral medication, longer duration of treatment, large sample size (≥100 per arm) and public funding source were associated with increased PCE for pain reduction.ConclusionsThe majority (75%) of the overall treatment effect in OA RCTs is attributable to contextual effects rather than the specific effect of treatments. Reporting overall treatment effect and PCE, in addition to traditional ES, permits a more balanced, clinically meaningful interpretation of RCT results. This would help dispel the frequent discordance between conclusions from RCT evidence and clinical experience—the ‘efficacy paradox’.

Download Full-text