Population Adjustment Methods for Indirect Comparisons: A Review of National Institute for Health and Care Excellence Technology Appraisals

AbstractObjectivesIndirect comparisons via a common comparator (anchored comparisons) are commonly used in health technology assessment. However, common comparators may not be available, or the comparison may be biased due to differences in effect modifiers between the included studies. Recently proposed population adjustment methods aim to adjust for differences between study populations in the situation where individual patient data are available from at least one study, but not all studies. They can also be used when there is no common comparator or for single-arm studies (unanchored comparisons). We aim to characterise the use of population adjustment methods in technology appraisals (TAs) submitted to the United Kingdom National Institute for Health and Care Excellence (NICE).MethodsWe reviewed NICE TAs published between 01/01/2010 and 20/04/2018.ResultsPopulation adjustment methods were used in 7 percent (18/268) of TAs. Most applications used unanchored comparisons (89 percent, 16/18), and were in oncology (83 percent, 15/18). Methods used included matching-adjusted indirect comparisons (89 percent, 16/18) and simulated treatment comparisons (17 percent, 3/18). Covariates were included based on: availability, expert opinion, effective sample size, statistical significance, or cross-validation. Larger treatment networks were commonplace (56 percent, 10/18), but current methods cannot account for this. Appraisal committees received results of population-adjusted analyses with caution and typically looked for greater cost effectiveness to minimise decision risk.ConclusionsPopulation adjustment methods are becoming increasingly common in NICE TAs, although their impact on decisions has been limited to date. Further research is needed to improve upon current methods, and to investigate their properties in simulation studies.

Download Full-text

Target estimands for efficient decision making: Response to comments on “Assessing the performance of population adjustment methods for anchored indirect comparisons: A simulation study”

Statistics in Medicine ◽

10.1002/sim.8965 ◽

2021 ◽

Vol 40 (11) ◽

pp. 2759-2763

Author(s):

David M. Phillippo ◽

Sofia Dias ◽

Anthony E. Ades ◽

Nicky J. Welton

Keyword(s):

Decision Making ◽

Simulation Study ◽

Indirect Comparisons ◽

Population Adjustment

Download Full-text

A simple solution to the inadequacy of asymptotic likelihood-based inference for response-adaptive clinical trials

Statistical Papers ◽

10.1007/s00362-021-01234-3 ◽

2021 ◽

Author(s):

Alessandro Baldi Antognini ◽

Marco Novelli ◽

Maroussa Zagoraiou

Keyword(s):

Clinical Trials ◽

Primary Outcome ◽

Wald Test ◽

Simple Solution ◽

Nuisance Parameters ◽

Simulation Studies ◽

Fundamental Properties ◽

Adaptive Clinical Trials ◽

Treatment Comparisons ◽

Practical Recommendations

AbstractThe present paper discusses drawbacks and limitations of likelihood-based inference in sequential clinical trials for treatment comparisons managed via Response-Adaptive Randomization. Taking into account the most common statistical models for the primary outcome—namely binary, Poisson, exponential and normal data—we derive the conditions under which (i) the classical confidence intervals degenerate and (ii) the Wald test becomes inconsistent and strongly affected by the nuisance parameters, also displaying a non monotonic power. To overcome these drawbacks, we provide a very simple solution that could preserve the fundamental properties of likelihood-based inference. Several illustrative examples and simulation studies are presented in order to confirm the relevance of our results and provide some practical recommendations.

Download Full-text

When does the use of individual patient data in network meta-analysis make a difference? A simulation study

BMC Medical Research Methodology ◽

10.1186/s12874-020-01198-2 ◽

2021 ◽

Vol 21 (1) ◽

Author(s):

Steve Kanters ◽

Mohammad Ehsanul Karim ◽

Kristian Thorlund ◽

Aslam Anis ◽

Nick Bansback

Keyword(s):

Simulation Study ◽

Treatment Effect ◽

Mean Squared Error ◽

Individual Patient Data ◽

Meta Analysis ◽

Model Performance ◽

Patient Data ◽

Meta Regression ◽

Treatment Comparisons ◽

The Impact

Abstract Background The use of individual patient data (IPD) in network meta-analyses (NMA) is rapidly growing. This study aimed to determine, through simulations, the impact of select factors on the validity and precision of NMA estimates when combining IPD and aggregate data (AgD) relative to using AgD only. Methods Three analysis strategies were compared via simulations: 1) AgD NMA without adjustments (AgD-NMA); 2) AgD NMA with meta-regression (AgD-NMA-MR); and 3) IPD-AgD NMA with meta-regression (IPD-NMA). We compared 108 parameter permutations: number of network nodes (3, 5 or 10); proportion of treatment comparisons informed by IPD (low, medium or high); equal size trials (2-armed with 200 patients per arm) or larger IPD trials (500 patients per arm); sparse or well-populated networks; and type of effect-modification (none, constant across treatment comparisons, or exchangeable). Data were generated over 200 simulations for each combination of parameters, each using linear regression with Normal distributions. To assess model performance and estimate validity, the mean squared error (MSE) and bias of treatment-effect and covariate estimates were collected. Standard errors (SE) and percentiles were used to compare estimate precision. Results Overall, IPD-NMA performed best in terms of validity and precision. The median MSE was lower in the IPD-NMA in 88 of 108 scenarios (similar results otherwise). On average, the IPD-NMA median MSE was 0.54 times the median using AgD-NMA-MR. Similarly, the SEs of the IPD-NMA treatment-effect estimates were 1/5 the size of AgD-NMA-MR SEs. The magnitude of superior validity and precision of using IPD-NMA varied across scenarios and was associated with the amount of IPD. Using IPD in small or sparse networks consistently led to improved validity and precision; however, in large/dense networks IPD tended to have negligible impact if too few IPD were included. Similar results also apply to the meta-regression coefficient estimates. Conclusions Our simulation study suggests that the use of IPD in NMA will considerably improve the validity and precision of estimates of treatment effect and regression coefficients in the most NMA IPD data-scenarios. However, IPD may not add meaningful validity and precision to NMAs of large and dense treatment networks when negligible IPD are used.

Download Full-text

Health technology assessment in the United Kingdom

International Journal of Technology Assessment in Health Care ◽

10.1017/s0266462309090618 ◽

2009 ◽

Vol 25 (S1) ◽

pp. 178-181 ◽

Cited By ~ 10

Author(s):

Michael Drummond ◽

David Banta

Keyword(s):

Health Care ◽

United Kingdom ◽

Cost Effectiveness ◽

Health Technology Assessment ◽

Technology Assessment ◽

Clinical Excellence ◽

Present Situation ◽

Health Technology ◽

The United Kingdom ◽

Short Run

Objectives: The aim of this study was to describe generally the development and present situation with health technology assessment (HTA) in the United Kingdom.Methods: The methods used are a review of important materials that have described the development process and present situation, supplemented by some personal experiences.Results: The United Kingdom has been characterized historically as a country with a strong interest in evidence in health care, both clinical trials for efficacy and cost-effectiveness analyses. However, this evidence was not well-linked to the needs of the National Health Services (NHS) before formation of the NHS R&D Programme in 1991, The R&D Programme brought substantial resources into HTA and related activities, with the central aim of improving health care in Britain and increasing value for money. However, policy makers as well as staff of the R&D Programme were dissatisfied with the use of the HTA results in clinical and administrative practice. Therefore, the National Institute of Clinical Excellence (NICE) was formed in 1999. NICE issues guidance intended to influence practical decision making in health care at the national and local levels, based on efficacy information and, in some cases, economic analyses. NICE is now also seeking ways to maximize impacts on practice.Conclusions: The UK experience shows that information on clinical and cost-effectiveness may not be enough to change practice, at least in the short-run. Still, one may conclude that the United Kingdom now has one of the few most important and influential HTA programs in the world.

Download Full-text

Assumptions of Mixed Treatment Comparisons in Health Technology Assessments - Challenges and Possible Steps for Practical Application

PLoS ONE ◽

10.1371/journal.pone.0160712 ◽

2016 ◽

Vol 11 (8) ◽

pp. e0160712 ◽

Cited By ~ 3

Author(s):

Stefanie Reken ◽

Sibylle Sturtz ◽

Corinna Kiefer ◽

Yvonne-Beatrice Böhler ◽

Beate Wieseler

Keyword(s):

Health Technology ◽

Practical Application ◽

Mixed Treatment ◽

Treatment Comparisons ◽

Mixed Treatment Comparisons

Download Full-text

PMC34 BIBLIOGRAPHIC REVIEW OF DISCRETE EVENTS SIMULATION STUDIES IN HEALTH TECHNOLOGY EVALUATION

Value in Health ◽

10.1016/s1098-3015(10)65588-x ◽

2007 ◽

Vol 10 (6) ◽

pp. A461

Author(s):

T Monleon ◽

JM Rodriguez-Barrios ◽

D Serrano

Keyword(s):

Health Technology ◽

Simulation Studies ◽

Technology Evaluation ◽

Discrete Events ◽

Health Technology Evaluation

Download Full-text

PP206 Health Technology Assessment Guidance In The United Kingdom: Addressing Issues Specific To Medical Devices

International Journal of Technology Assessment in Health Care ◽

10.1017/s0266462321001318 ◽

2021 ◽

Vol 37 (S1) ◽

pp. 26-26

Author(s):

Scott Gibson ◽

Sita Saunders ◽

Amanda Hansson Hedblom ◽

Maximilian Blüher ◽

Rafael Torrejon Torres ◽

...

Keyword(s):

United Kingdom ◽

Health Technology Assessment ◽

Medical Devices ◽

Technology Assessment ◽

Healthcare Providers ◽

Economic Assessment ◽

Health Technology ◽

Training Costs ◽

Medical Technologies ◽

The United Kingdom

IntroductionThe United Kingdom spends approximately GBP4.2 billion (USD5.6 billion; EUR4.7 billion) each year on medical devices, but healthcare providers receive little health technology assessment (HTA) guidance on cost-effective device procurement. Our objective was to assess the availability of HTA guidance for medical technologies and to identify key challenges related to the economic assessment of these technologies.MethodsNational Institute for Health and Care Excellence technology appraisal (TA) and Medical Technologies Evaluation Programme (MTEP) appraisals published online between November 2009 and October 2020 were identified. The “case for adoption” recommendation, type of devices, and critiques of economic analyses for each MTEP appraisal were extracted and categorized.ResultsIn comparison to 415 publicly available TAs for pharmaceuticals, only 45 medical technologies have been appraised through the MTEP. MTEP-submitted technologies can be categorized into diagnostic (7), monitoring (3), prophylaxis (5), therapeutic (28), and other (2). Furthermore, 11 were implants, seven were used by patients, and 27 had provider interaction. Major points of MTEP criticism were a failure to model cost consequences, training costs, and organizational impact. There was also the barrier of transferring costs across budgeting divisions.ConclusionsIn comparison to HTA guidance for pharmaceuticals, there is a dearth of medical device guidance. Therapeutic and implantable devices appear to be disproportionately overrepresented in the MTEP process. This may be because their appraisal is most akin to pharmaceuticals, for which HTA processes are well established. To encourage more HTAs of medical devices, HTA guidance should elaborate on issues specifically related to medical devices.

Download Full-text