scholarly journals Risk of bias in observational studies using routinely collected data of comparative effectiveness research: a meta-research study

BMC Medicine ◽  
2021 ◽  
Vol 19 (1) ◽  
Author(s):  
Van Thu Nguyen ◽  
Mishelle Engleton ◽  
Mauricia Davison ◽  
Philippe Ravaud ◽  
Raphael Porcher ◽  
...  

Abstract Background To assess the completeness of reporting, research transparency practices, and risk of selection and immortal bias in observational studies using routinely collected data for comparative effectiveness research. Method We performed a meta-research study by searching PubMed for comparative effectiveness observational studies evaluating therapeutic interventions using routinely collected data published in high impact factor journals from 01/06/2018 to 30/06/2020. We assessed the reporting of the study design (i.e., eligibility, treatment assignment, and the start of follow-up). The risk of selection bias and immortal time bias was determined by assessing if the time of eligibility, the treatment assignment, and the start of follow-up were synchronized to mimic the randomization following the target trial emulation framework. Result Seventy-seven articles were identified. Most studies evaluated pharmacological treatments (69%) with a median sample size of 24,000 individuals. In total, 20% of articles inadequately reported essential information of the study design. One-third of the articles (n = 25, 33%) raised some concerns because of unclear reporting (n = 6, 8%) or were at high risk of selection bias and/or immortal time bias (n = 19, 25%). Only five articles (25%) described a solution to mitigate these biases. Six articles (31%) discussed these biases in the limitations section. Conclusion Reporting of essential information of study design in observational studies remained suboptimal. Selection bias and immortal time bias were common methodological issues that researchers and physicians should be aware of when interpreting the results of observational studies using routinely collected data.

2012 ◽  
Vol 30 (34) ◽  
pp. 4208-4214 ◽  
Author(s):  
Katrina Armstrong

Comparative effectiveness research (CER) seeks to assist consumers, clinicians, purchasers, and policy makers to make informed decisions to improve health care at both the individual and population levels. CER includes evidence generation and evidence synthesis. Randomized controlled trials are central to CER because of the lack of selection bias, with the recent development of adaptive and pragmatic trials increasing their relevance to real-world decision making. Observational studies comprise a growing proportion of CER because of their efficiency, generalizability to clinical practice, and ability to examine differences in effectiveness across patient subgroups. Concerns about selection bias in observational studies can be mitigated by measuring potential confounders and analytic approaches, including multivariable regression, propensity score analysis, and instrumental variable analysis. Evidence synthesis methods include systematic reviews and decision models. Systematic reviews are a major component of evidence-based medicine and can be adapted to CER by broadening the types of studies included and examining the full range of benefits and harms of alternative interventions. Decision models are particularly suited to CER, because they make quantitative estimates of expected outcomes based on data from a range of sources. These estimates can be tailored to patient characteristics and can include economic outcomes to assess cost effectiveness. The choice of method for CER is driven by the relative weight placed on concerns about selection bias and generalizability, as well as pragmatic concerns related to data availability and timing. Value of information methods can identify priority areas for investigation and inform research methods.


Author(s):  
Sharon H. Giordano

Observational studies are increasingly being used for comparative effectiveness research. These studies can have the greatest impact when randomized trials are not feasible or when randomized studies have not included the population or outcomes of interest. However, careful attention must be paid to study design to minimize the likelihood of selection biases. Analytic techniques, such as multivariable regression modeling, propensity score analysis, and instrumental variable analysis, also can also be used to help address confounding. Oncology has many existing large and clinically rich observational databases that can be used for comparative effectiveness research. With careful study design, observational studies can produce valid results to assess the benefits and harms of a treatment or intervention in representative real-world populations.


2018 ◽  
Vol 7 (1) ◽  
pp. 57-66 ◽  
Author(s):  
Alexis A Krumme ◽  
Ajinkya Pawar ◽  
Sebastian Schneeweiss ◽  
Robert J Glynn ◽  
Niteesh K Choudhry ◽  
...  

2019 ◽  
Vol 34 (11) ◽  
pp. 2193-2200 ◽  
Author(s):  
K R Heineman ◽  
D B Kuiper ◽  
Sla Bastide-van Gemert ◽  
M J Heineman ◽  
M Hadders-Algra

Abstract STUDY QUESTION Do ovarian stimulation (OS) and the in vitro laboratory procedures affect offsprings’ cognitive and behavioural outcome at 9 years? SUMMARY ANSWER OS and the in vitro laboratory procedures or the combination of both were not associated with cognitive and behavioural outcome at age 9 years. WHAT IS KNOWN ALREADY ART is not associated with an adverse short-term developmental outcome of the offspring, but limited knowledge is available on the offspring’s long-term neurodevelopmental condition. STUDY DESIGN, SIZE, DURATION A 9-year longitudinal, assessor-blinded, prospective follow-up study of 169 out of 215 singletons (79%) born between March 2005 and December 2006 was performed. PARTICIPANTS/MATERIALS, SETTING, METHODS Singletons born following IVF or ICSI with OS (n = 57), born after modified natural cycle IVF/ICSI (MNC-IVF/ICSI; n = 46) and born after natural conception to subfertile couples (Sub-NC; n = 66), were assessed at 9 years. This study design, with two ART groups and a subfertile reference group, allows for disentangling the effects of OS and ART procedures on developmental outcome. Cognitive outcome was evaluated with the Wechsler abbreviated scale of intelligence and the NEPSY-II. Behaviour was assessed with the child behaviour checklist (CBCL) and teacher report form (TRF). Univariable analyses and multiple linear regression models were used. MAIN RESULTS AND THE ROLE OF CHANCE There was no significant difference in intelligence quotient (IQ) scores between ART groups (mean IQ (95% CI): OS 114.8 (83.2–142.6); MNC 114.0 (90.2–140.8); Sub-NC 115.4 (87.9–141.2), P = 0.746). Multivariable analyses did not reveal a statistically significant association between ART group and total, verbal and performance IQ. CBCL and TRF scores did not differ significantly between ART groups (P = 0.090 and 0.507, respectively). Multivariable analyses did not demonstrate a statistically significant association between ART group and CBCL and TRF total, or internalising and externalising T-scores. No significant correlations between time to pregnancy (TTP)—a proxy for the severity of parental subfertility—and outcome measures were found (Spearman rho between −0.050 and 0.049, NS), which was confirmed with multivariable analyses. LIMITATIONS, REASONS FOR CAUTION The attrition rate of 21% may be considered as a limitation of the study; however, after a follow-up period of 9 years, this rate is generally considered acceptable, and there were no significant differences in background characteristics between children with and without follow-up, making an attrition-related selection bias less likely. Another limitation of the study is the relatively small sample size, which could contribute to selection bias, hamper generalizability to the ART population and lead to false negative findings as a result of underpowering. An a priori power analysis on total IQ indicated that the OS-IVF/ICSI and Sub-NC groups should contain 64 children, confirming that our study including 57 and 66 children, respectively, was slightly underpowered. WIDER IMPLICATIONS OF THE FINDINGS Our study indicated that OS and the in vitro laboratory procedures or the combination of both and TTP were not associated with cognitive and behavioural outcome at 9 years. These are reassuring results for both parents and clinicians involved in ART. STUDY FUNDING/COMPETING INTEREST(S) The study was financially supported by the University Medical Center Groningen (UMCG), two graduate schools of the UMCG (BCN and SHARE) and the Cornelia Stichting. The sponsors of the study had no role in study design, data collection, data analysis, data interpretation or writing of the report. The authors have no conflicts of interest to declare.


Sign in / Sign up

Export Citation Format

Share Document