OP208 Did Health Technology Assessments Make the Wrong Call? Quantitative Bias Analysis: Alectinib versus Ceritinib in Non-Small Cell Lung Cancer

IntroductionThe German health technology assessment (HTA) rejected additional benefit of alectinib for second line (2L) ALK+ NSCLC, citing possible biases from missing ECOG performance status data and unmeasured confounding in real-world evidence (RWE) for 2L ceritinib that was submitted as a comparator to the single arm alectinib trial. Alectinib was approved in the US and therefore US post-launch RWE can be used to evaluate this HTA decision.MethodsWe compared the real-world effectiveness of alectinib with ceritinib in 2L post-crizotinib ALK+ NSCLC using the nationwide Flatiron Health electronic health record (EHR)-derived de-identified database. Using quantitative bias analysis (QBA), we estimated the strength of (i) unmeasured confounding and (ii) deviation from missing-at-random (MAR) assumptions needed to nullify any overall survival (OS) benefit.ResultsAlectinib had significantly longer median OS than ceritinib in complete case analysis. The estimated effect size (Hazard Ratio: 0.55) was robust to risk ratios of unmeasured confounder-outcome and confounder-exposure associations of <2.4.Based on tipping point analysis, missing baseline ECOG performance status for ceritinib-treated patients (49% missing) would need to be more than 3.4-times worse than expected under MAR to nullify the OS benefit observed for alectinib.ConclusionsOnly implausible levels of bias reversed our conclusions. These methods could provide a framework to explore uncertainty and aid decision-making for HTAs to enable patient access to innovative therapies.

Intermittent opioid use and ischemic placental disease: Study quantifies risks, raises questions

Discomfort and, to a lesser extent, pain are common complaints during pregnancy, and some patients may turn to opioids for pain relief. Esposito and colleagues (Am J Epidemiol. 2021, in press) report associations between intermittent exposure to opioids during pregnancy and the risk of ischemic placental disease (IPD) – a syndrome that includes preeclampsia, placental abruption, small for gestational age (SGA) births, and preterm delivery. They found that early opioid exposure in pregnancy was associated with a modestly increased risk for abruption, SGA births, and preterm delivery, and both early and late exposure was associated with the greatest risk for these outcomes. Surprisingly, preeclampsia was not associated with opioid use. Through quantitative bias analysis, the authors cleverly tackle a number of biases to assess their roles in explaining the associations, including unmeasured confounding, outcome misclassification, and residual confounding; none exerted strong influences on the associations. Although the findings appear fairly robust on the surface, the lack of association between intermittent opioid use and preeclampsia, and important differences in characteristics of patients in the opioid exposed group compared to the unexposed group, suggest that further study is needed to clarify the relationship between intermittent opioid use, lifestyle factors, and IPD risk.

The validity of the claims-based definition of rheumatoid arthritis evaluated in 64 hospitals in Japan

Background An administrative database covering a whole population such as the national database in Japan may be used to estimate the nationwide prevalence of diseases including rheumatoid arthritis (RA) when a well-validated definition of the disease is available. In Japan, the record linkage between the administrative database and medical charts in hospitals is strictly prohibited. A “hospital-based” validation study is one of few possible validation studies where claims kept inside the study hospital are rearranged into the database structure. Methods We selected random samples of 19,734 patients from approximately 1.6 million patients who received medical care between February 2018 and January 2019 in one of the 64 hospitals of the Tokushukai Medical Group. We excluded patients whose observation period was less than 365 days and identified 334 patients who met the definition of “possible cases of RA” whose medical charts were then independently evaluated by two rheumatologists. In a sensitivity analysis, we assessed bias due to misclassifying some patients with RA who did not meet the definition of “possible cases of RA” as a patient with no RA. Results The kappa coefficient between the two rheumatologists was 0.80. The prevalence of RA in the study population was estimated to be 0.56%. We found that [condition code of RA] and ([any disease-modifying antirheumatic drug] or [oral corticosteroid with no systemic autoimmune diseases (other than RA) and no polymyalgia rheumatica]) had a relatively high sensitivity (approximately 73%) and a high positive predictive value (approximately 80%). In a sensitivity analysis, we found that when some patients with RA who did not meet the definition of “possible cases of RA” were misclassified as a patient with no RA, then this would lead to underestimation of the prevalence of the definition-positive patients and the adjusted prevalence. Conclusions We recommend using the claims-based definition of RA (found in the current validation study) to estimate the prevalence of RA in Japan. We also suggest estimating the adjusted prevalence using the quantitative bias analysis method, since the prevalence of the disease in the “hospital-based” validation study is different from that in the administrative database. Trial registration The current study is not a clinical trial and hence not subject to trial registration.

A systematic review of quantitative bias analysis applied to epidemiological research

Background Quantitative bias analysis (QBA) measures study errors in terms of direction, magnitude and uncertainty. This systematic review aimed to describe how QBA has been applied in epidemiological research in 2006–19. Methods We searched PubMed for English peer-reviewed studies applying QBA to real-data applications. We also included studies citing selected sources or which were identified in a previous QBA review in pharmacoepidemiology. For each study, we extracted the rationale, methodology, bias-adjusted results and interpretation and assessed factors associated with reproducibility. Results Of the 238 studies, the majority were embedded within papers whose main inferences were drawn from conventional approaches as secondary (sensitivity) analyses to quantity-specific biases (52%) or to assess the extent of bias required to shift the point estimate to the null (25%); 10% were standalone papers. The most common approach was probabilistic (57%). Misclassification was modelled in 57%, uncontrolled confounder(s) in 40% and selection bias in 17%. Most did not consider multiple biases or correlations between errors. When specified, bias parameters came from the literature (48%) more often than internal validation studies (29%). The majority (60%) of analyses resulted in &gt;10% change from the conventional point estimate; however, most investigators (63%) did not alter their original interpretation. Degree of reproducibility related to inclusion of code, formulas, sensitivity analyses and supplementary materials, as well as the QBA rationale. Conclusions QBA applications were rare though increased over time. Future investigators should reference good practices and include details to promote transparency and to serve as a reference for other researchers.

Invited Commentary: Quantitative Bias Analysis can see the Forest for the Trees

The article by Jiang et al (Am J. Epidemiol.) extends quantitative bias analysis from the realm of statistical models to the realm of machine learning algorithms. Given the rooting of statistical models in the spirit of explanation and the rooting of machine learning algorithms in the spirt of prediction, this extension is thought provoking indeed. Some such thoughts are expounded here.