Potential importance of residual confounding in systematic reviews of observational studies: Answer to Mathur and VanderTweele

IntroductionSometimes, observational studies may provide important evidence that allow inferences of causality between exposure and outcome (although on most occasions only low certainty evidence). Authors, frequently and perhaps usually at the behest of the journals to which they are submitting, avoid using causal language when addressing evidence from observational studies. This is true even when the issue of interest is the causal effect of an intervention or exposure. Clarity of thinking and appropriateness of inferences may be enhanced through the use of language that reflects the issue under consideration. The objectives of this study are to systematically evaluate the extent and nature of causal language use in systematic reviews of observational studies and to relate that to the actual intent of the investigation.Methods and analysisWe will conduct a systematic survey of systematic reviews of observational studies addressing modifiable exposures and their possible impact on patient-important outcomes. We will randomly select 200 reviews published in 2019, stratified in a 1:1 ratio by use and non-use of the Grading of Recommendations Assessment, Development, and Evaluation (GRADE). Teams of two reviewers will independently assess study eligibility and extract data using a standardised data extraction forms, with resolution of disagreement by discussion and, if necessary, by third party adjudication. Through examining the inferences, they make in their papers’ discussion, we will evaluate whether the authors’ intent was to address causation or association. We will summarise the use of causal language in the study title, abstract, study question and results using descriptive statistics. Finally, we will assess whether the language used is consistent with the intention of the authors. We will determine whether results in reviews that did or did not use GRADE differ.Ethics and disseminationEthics approval for this study is not required. We will disseminate the results through publication in a peer-reviewed journals.RegistrationOpen Science Framework (osf.io/vh8yx).

Download Full-text

Lenvatinib and sorafenib for differentiated thyroid cancer after radioactive iodine: a systematic review and economic evaluation

Health Technology Assessment ◽

10.3310/hta24020 ◽

2020 ◽

Vol 24 (2) ◽

pp. 1-180 ◽

Cited By ~ 3

Author(s):

Nigel Fleeman ◽

Rachel Houten ◽

Adrian Bagust ◽

Marty Richardson ◽

Sophie Beale ◽

...

Keyword(s):

Thyroid Cancer ◽

Cost Effectiveness ◽

Systematic Reviews ◽

Observational Studies ◽

Differentiated Thyroid Cancer ◽

Radioactive Iodine ◽

Clinical Effectiveness ◽

Economic Evaluations ◽

Base Case ◽

The Cost

Background Thyroid cancer is a rare cancer, accounting for only 1% of all malignancies in England and Wales. Differentiated thyroid cancer (DTC) accounts for ≈94% of all thyroid cancers. Patients with DTC often require treatment with radioactive iodine. Treatment for DTC that is refractory to radioactive iodine [radioactive iodine-refractory DTC (RR-DTC)] is often limited to best supportive care (BSC). Objectives We aimed to assess the clinical effectiveness and cost-effectiveness of lenvatinib (Lenvima®; Eisai Ltd, Hertfordshire, UK) and sorafenib (Nexar®; Bayer HealthCare, Leverkusen, Germany) for the treatment of patients with RR-DTC. Data sources EMBASE, MEDLINE, PubMed, The Cochrane Library and EconLit were searched (date range 1999 to 10 January 2017; searched on 10 January 2017). The bibliographies of retrieved citations were also examined. Review methods We searched for randomised controlled trials (RCTs), systematic reviews, prospective observational studies and economic evaluations of lenvatinib or sorafenib. In the absence of relevant economic evaluations, we constructed a de novo economic model to compare the cost-effectiveness of lenvatinib and sorafenib with that of BSC. Results Two RCTs were identified: SELECT (Study of [E7080] LEnvatinib in 131I-refractory differentiated Cancer of the Thyroid) and DECISION (StuDy of sorafEnib in loCally advanced or metastatIc patientS with radioactive Iodine-refractory thyrOid caNcer). Lenvatinib and sorafenib were both reported to improve median progression-free survival (PFS) compared with placebo: 18.3 months (lenvatinib) vs. 3.6 months (placebo) and 10.8 months (sorafenib) vs. 5.8 months (placebo). Patient crossover was high (≥ 75%) in both trials, confounding estimates of overall survival (OS). Using OS data adjusted for crossover, trial authors reported a statistically significant improvement in OS for patients treated with lenvatinib compared with those given placebo (SELECT) but not for patients treated with sorafenib compared with those given placebo (DECISION). Both lenvatinib and sorafenib increased the incidence of adverse events (AEs), and dose reductions were required (for > 60% of patients). The results from nine prospective observational studies and 13 systematic reviews of lenvatinib or sorafenib were broadly comparable to those from the RCTs. Health-related quality-of-life (HRQoL) data were collected only in DECISION. We considered the feasibility of comparing lenvatinib with sorafenib via an indirect comparison but concluded that this would not be appropriate because of differences in trial and participant characteristics, risk profiles of the participants in the placebo arms and because the proportional hazard assumption was violated for five of the six survival outcomes available from the trials. In the base-case economic analysis, using list prices only, the cost-effectiveness comparison of lenvatinib versus BSC yields an incremental cost-effectiveness ratio (ICER) per quality-adjusted life-year (QALY) gained of £65,872, and the comparison of sorafenib versus BSC yields an ICER of £85,644 per QALY gained. The deterministic sensitivity analyses show that none of the variations lowered the base-case ICERs to < £50,000 per QALY gained. Limitations We consider that it is not possible to compare the clinical effectiveness or cost-effectiveness of lenvatinib and sorafenib. Conclusions Compared with placebo/BSC, treatment with lenvatinib or sorafenib results in an improvement in PFS, objective tumour response rate and possibly OS, but dose modifications were required to treat AEs. Both treatments exhibit estimated ICERs of > £50,000 per QALY gained. Further research should include examination of the effects of lenvatinib, sorafenib and BSC (including HRQoL) for both symptomatic and asymptomatic patients, and the positioning of treatments in the treatment pathway. Study registration This study is registered as PROSPERO CRD42017055516. Funding The National Institute for Health Research Health Technology Assessment programme.

Download Full-text

Translation and validation of PubMed and Embase search filters for identification of systematic reviews, intervention studies, and observational studies in the field of first aid

Journal of the Medical Library Association JMLA ◽

10.5195/jmla.2021.1219 ◽

2021 ◽

Vol 109 (4) ◽

Author(s):

Bert Avau ◽

Hans Van Remoortel ◽

Emmy De Buck

Keyword(s):

Systematic Reviews ◽

Observational Studies ◽

Intervention Studies ◽

First Aid ◽

Search Filter ◽

Filter Performance ◽

Ovid Medline ◽

Search Filters ◽

Study Type ◽

Evidence Summaries

Objective: The aim of this project was to validate search filters for systematic reviews, intervention studies, and observational studies translated from Ovid MEDLINE and Embase syntax and used for searches in PubMed and Embase.com during the development of evidence summaries supporting first aid guidelines. We aimed to achieve a balance among recall, specificity, precision, and number needed to read (NNR).Methods: Reference gold standards were constructed per study type derived from existing evidence summaries. Search filter performance was assessed through retrospective searches and measurement of relative recall, specificity, precision, and NNR when using the translated search filters. Where necessary, search filters were optimized. Adapted filters were validated in separate validation gold standards.Results: Search filters for systematic reviews and observational studies reached recall of ≥85% in both PubMed and Embase. Corresponding specificities for systematic review filters were ≥96% in both databases, with a precision of 9.7% (NNR 10) in PubMed and 5.4% (NNR 19) in Embase. For observational study filters, specificity, precision, and NNR were 68%, 2%, and 51 in PubMed and 47%, 0.8%, and 123 in Embase, respectively. These filters were considered sufficiently effective. Search filters for intervention studies reached a recall of 85% and 83% in PubMed and Embase, respectively. Optimization led to recall of ≥95% with specificity, precision, and NNR of 49%, 1.3%, and 79 in PubMed and 56%, 0.74%, and 136 in Embase, respectively.Conclusions: We report validated filters to search for systematic reviews, observational studies, and intervention studies in guideline projects in PubMed and Embase.com.

Download Full-text

A nephrology guide to reading and using systematic reviews of observational studies

Nephrology Dialysis Transplantation ◽

10.1093/ndt/gfv257 ◽

2015 ◽

Vol 30 (10) ◽

pp. 1615-1621 ◽

Cited By ~ 1

Author(s):

Pietro Ravani ◽

Paul E. Ronksley ◽

Matthew T. James ◽

Giovanni F. Strippoli

Keyword(s):

Systematic Reviews ◽

Observational Studies

Download Full-text

A Mixed-Methods Approach to Synthesizing Evidence on Mediators of Intervention Effects

Western Journal of Nursing Research ◽

10.1177/0193945911402365 ◽

2011 ◽

Vol 33 (7) ◽

pp. 870-900 ◽

Cited By ~ 5

Author(s):

Jennifer Leeman ◽

YunKyung Chang ◽

Corrine I. Voils ◽

Jamie L. Crandell ◽

Margarete Sandelowski

Keyword(s):

Mixed Methods ◽

Systematic Reviews ◽

Behavioral Change ◽

Observational Studies ◽

Intervention Studies ◽

Meta Analysis ◽

Intervention Effects ◽

Mediation Effects ◽

Mixed Methods Approach ◽

Using Data

Greater understanding of the mechanisms (mediators) by which behavioral-change interventions work is critical to developing theory and refining interventions. Although systematic reviews have been advocated as a method for exploring mediators, this is rarely done. One challenge is that intervention researchers typically test only two paths of the mediational model: the effect of the intervention on mediators and on outcomes. The authors addressed this challenge by drawing information not only from intervention studies but also from observational studies that provide data on associations between potential mediators and outcomes. They also reviewed qualitative studies of participants’ perceptions of why and how interventions worked. Using data from intervention ( n = 37) and quantitative observational studies ( n = 55), the authors conducted a meta-analysis of the mediation effects of eight variables. Qualitative findings ( n = 6) contributed to more in-depth explanations for findings. The methods used have potential to contribute to understanding of core mechanisms of behavioral-change interventions.

Download Full-text

Induction of labour before 41 weeks or expectant management until 42 weeks: data from observational studies, randomized trials, systematic reviews and meta-analysis

Obstetrics and gynecology: News, Opinions, Training ◽

10.33029/2303-9698-2021-9-2-19-33 ◽

2021 ◽

Vol 9 (2) ◽

pp. 19-33

Author(s):

O.R. Baev ◽

◽

I.I. Baranov ◽

◽

Keyword(s):

Systematic Reviews ◽

Observational Studies ◽

Randomized Trials ◽

Meta Analysis ◽

Expectant Management ◽

Induction Of Labour

Download Full-text

A New Method for Partial Correction of Residual Confounding in Time-Series and Other Observational Studies

American Journal of Epidemiology ◽

10.1093/aje/kwx013 ◽

2017 ◽

Vol 185 (10) ◽

pp. 941-949 ◽

Cited By ~ 8

Author(s):

W. Dana Flanders ◽

Matthew J. Strickland ◽

Mitchel Klein

Keyword(s):

Time Series ◽

Observational Studies ◽

New Method ◽

Residual Confounding ◽

Partial Correction

Download Full-text

Use of E-values for addressing confounding in observational studies—an empirical assessment of the literature

International Journal of Epidemiology ◽

10.1093/ije/dyz261 ◽

2020 ◽

Vol 49 (5) ◽

pp. 1482-1494 ◽

Cited By ~ 10

Author(s):

Manuel R Blum ◽

Yuan Jin Tan ◽

John P A Ioannidis

Keyword(s):

Observational Studies ◽

Literature Search ◽

Original Data ◽

Case Control ◽

Careful Consideration ◽

Effect Sizes ◽

Systematic Literature Search ◽

Similar Magnitude ◽

Unmeasured Confounding ◽

Residual Confounding

Abstract Background E-values are a recently introduced approach to evaluate confounding in observational studies. We aimed to empirically assess the current use of E-values in published literature. Methods We conducted a systematic literature search for all publications, published up till the end of 2018, which cited at least one of two inceptive E-value papers and presented E-values for original data. For these case publications we identified control publications, matched by journal and issue, where the authors had not calculated E-values. Results In total, 87 papers presented 516 E-values. Of the 87 papers, 14 concluded that residual confounding likely threatens at least some of the main conclusions. Seven of these 14 named potential uncontrolled confounders. 19 of 87 papers related E-value magnitudes to expected strengths of field-specific confounders. The median E-value was 1.88, 1.82, and 2.02 for the 43, 348, and 125 E-values where confounding was felt likely to affect the results, unlikely to affect the results, or not commented upon, respectively. The 69 case-control publication pairs dealt with effect sizes of similar magnitude. Of 69 control publications, 52 did not comment on unmeasured confounding and 44/69 case publications concluded that confounding was unlikely to affect study conclusions. Conclusions Few papers using E-values conclude that confounding threatens their results, and their E-values overlap in magnitude with those of papers acknowledging susceptibility to confounding. Facile automation in calculating E-values may compound the already poor handling of confounding. E-values should not be a substitute for careful consideration of potential sources of unmeasured confounding. If used, they should be interpreted in the context of expected confounding in specific fields.

Download Full-text