OP123 The Use Of Surrogate Outcomes In National Institute For Health And Care Excellence (NICE) Highly Specialised Technology Evaluations: A Review Of Published Guidance

IntroductionThe use of surrogate outcomes in health technology assessment (HTA) is increasing and methods for validating surrogate relationships have been published. However, these may not be fully applicable to ultra-rare diseases due to challenges such as scarcity of evidence and heterogenous populations. This study reviews and summarizes the use of surrogate outcomes and committee's considerations in the evaluations within the National Institute for Health and Care Excellence's (NICE) Highly Specialised Technology (HST) programme, which was established in 2013 in response to the challenges associated with the assessment of ultra-rare diseases.MethodsAll HST evaluation documents published before November 2020 were reviewed. Data extracted included surrogate outcomes used, rationales, the committee's considerations on the validity and generalizability of the surrogate relationships, related uncertainties, and other factors considered in decision-making.ResultsSeven out of the eighteen published HST topics used surrogate outcomes. The rationale for most of the surrogate relationships focused on biological plausibility. Common concerns raised by the committee included the generalizability of the surrogate relationship to the condition of interest, the lack of validation, and inability to prove or quantify the magnitude of benefits associated with the surrogate relationships. In some topics, other aspects of the evidence and clinical/patient expert's opinions were also considered by the committee.ConclusionsThe use of surrogate outcomes is common in NICE HST evaluations and the challenges in supporting surrogate relationships with more than biological plausibility are recognized. However, our review indicates that, the committee considers more than just biological plausibility and will take into account other related factors.

Patient-important outcomes reported in randomized controlled trials of pharmacologic treatments for COVID-19: a protocol of a META-epidemiological study

Background The coronavirus disease 19 (covid-19) pandemic has underscored the need to expedite clinical research, which may lead investigators to shift away from measuring patient-important outcomes (PIO), limiting research applicability. We aim to investigate if randomized controlled trials (RCTs) of covid-19 pharmacological therapies include PIOs. Methods We will perform a meta-epidemiological study of RCTs that included people at risk for, or with suspected, probable, or confirmed covid-19, examining any pharmacological treatment or blood product aimed at prophylaxis or treatment. We will obtain data from all RCTs identified in a living network metanalysis (NMA). The main data sources are the living WHO covid-19 database up to 1 March 2021 and six additional Chinese databases up to 20 February 2021. Two reviewers independently will review each citation, full-text article, and abstract data. To categorize the outcomes according to their importance to patients, we will adapt a previously defined hierarchy: a) mortality, b) quality of life/ functional status/symptoms, c) morbidity, and d) surrogate outcomes. Outcomes within the category a) and b) will be considered critically important to patients, and outcomes within the category c) will be regarded as important. We will use descriptive statistics to assess the proportion of studies that report each category of outcomes. We will perform univariable and multivariable analysis to explore associations between trial characteristics and the likelihood of reporting PIOs. Discussion The findings from this meta-epidemiological study will help health care professionals and researchers understand if the current covid-19 trials are effectively assessing and reporting the outcomes that are important to patients. If a deficiency in capturing PIOs is identified, this information may help inform the development of future RCTs in covid-19. Systematic review registrations Open Science Framework registration: osf.io/6xgjz.

A comparison of the Food and Drug Administration’s and Health Canada’s regulatory decisions about failed confirmatory trials for oncology drugs: an observational study

Background Oncology drugs are frequently approved on the basis of surrogate outcomes that require further trials to confirm the benefits, but at times these trials fail and regulators need to decide whether to withdraw approval for the indication and/or to remove the drug from the market. This study compares decisions by the Food and Drug Administration and Health Canada about oncology drugs that were approved using either Accelerated Approval (FDA) or Notice of Compliance with conditions (NOC/c, Health Canada) and that failed confirmatory trials. Methods Drug/indications approved by the FDA through its Accelerated Approval Pathway and that later failed confirmatory studies were identified from a published study and additional information on these drugs was collected from Drugs@FDA. Health Canada websites were searched on September 11, 2021 for the same group of drugs to determine if they were approved in Canada under the NOC/c pathway for the same indication as in the US. Information from both the FDA and Health Canada about these products was entered into an Excel spreadsheet. Decisions about whether to withdraw the drugs or remove the failed indication for the drug and requirements for confirmatory studies were compared. In addition, the dates of decisions by the two agencies were compared. Results Ten drug/indications were available for comparison. Regulatory decisions were similar in 4 cases, different in 1 case and could not be determined in the remaining 5, in 1 case because decisions were pending in both countries and in the other 4, because the NOC/c had not been fulfilled in Canada. The requirements for the confirmatory studies were similar in both countries. Decisions were made earlier in the United States. Conclusions This study shows that decisions made by Health Canada and the FDA about whether to withdraw a drug or remove a failed indication when drug/indications fail a confirmatory trial are usually similar, although the sample size on which this conclusion is made is small. The clinical implications of these similarities and differences should be explored.

Problems with the outcome measures in randomized controlled trials of traditional Chinese medicine in treating chronic heart failure caused by coronary heart disease: a systematic review

Background Traditional Chinese medicine (TCM) has gained widespread application in treating chronic heart failure (CHF) secondary to coronary heart disease (CHD). However, the sound clinical evidence is still lacking. Corresponding clinical trials vary considerably in the outcome measures assessing the efficacy of TCM, some that showed the improvement of clinical symptoms are not universally acknowledged. Rational outcome measures are the key to evaluate efficacy and safety of each treatment and significant elements of a convincing clinical trial. We aimed to summarize and analyze outcome measures in randomized controlled trials (RCTs) of TCM in treating CHF caused by CHD, subsequently identify the present problems and try to put forward solutions. Methods We systematically searched databases including Embase, PubMed, Cochrane Library, CBM, CNKI, VIP and Wanfang from inception to October 8, 2018, to identify eligible RCTs using TCM interventions for treating CHF patients caused by CHD. Cochrane Database of Systematic Reviews (CDSR) was searched to include Cochrane systematic reviews (CSRs) of CHF. Two authors independently assessed the risk of bias of the included RCTs according to the Cochrane Handbook. Outcome measures of each trial were extracted and analyzed those compared with the CSRs. We also evaluated the reporting quality of the outcome measures. Results A total of 31 RCTs were included and the methodology quality of the studies was generally low. Outcome measures in these RCTs were mortality, rehospitalization, efficacy of cardiac function, left ventricular ejection fraction (LVEF), 6 min’ walk distance (6MWD) and Brain natriuretic peptide (BNP), of which mortality and rehospitalization are clinical end points while the others are surrogate outcomes. The reporting rate of mortality and rehospitalization was 12.90% (4/31), the other included studies reported surrogate outcomes. As safety measure, 54.84% of the studies reported adverse drug reactions. Two trials were evaluated as high in reporting quality of outcomes and that of the other 29 studies was poor due to lack of necessary information for reporting. Conclusions The present RCTs of TCM in treating CHF secondary to CHD did not concentrate on the clinical end points of heart failure, which were generally small in size and short in duration. Moreover, these trials lacked adequate safety evaluation, had low quality in reporting outcomes and certain risk of bias in methodology. For objective assessment of the efficacy and safety of TCM in treating CHF secondary to CHD, future research should be rigorous designed, set end points as primary outcome measures and pay more attention to safety evaluation throughout the trial.

Efficacy and cost effectiveness of intravenous ferric carboxymaltose versus iron sucrose in adult patients with iron deficiency anaemia

Background Iron deficiency anaemia (IDA) is a major health issues and common type of nutritional deficiency worldwide. For IDA treatment, intravenous (IV) iron is a useful therapy. Objective To determine the efficacy and cost-effectiveness (CE) of intravenous (IV) Ferric Carboxymaltose (FCM) versus IV Iron Sucrose (IS) in treating IDA. Data sources Electronic medical record i.e. Cerner® system. Target population Adults patients with iron deficiency anaemia. Time horizon A 12-month period (01/01/2018–31/12/2018). Perspective Hamad Medical Corporation (HMC, a public hospital). Intervention IV Ferric Carboxymaltose versus IV Iron Sucrose. Outcome measures With regard to responses to treatment i.e., efficacy of treatment with FCM & IS in IDA patients, hemoglobin (Hgb), ferritin, and transferrin saturation (TSAT) levels were the primary outcomes. Additionally, the researchers also collected levels of iron, platelet, white blood cell (WBC), red blood cell (RBC), mean corpuscular hemoglobin (MCH), and mean corpuscular volume (MCV). The costs i.e. resources consumed (obtained from NCCCR-HMC) and the CE of FCM versus IS were the secondary outcomes. Results of base-case analysis There was a significant improvement in Hgb, RBC and MCH levels in the IS group than the FCM group. The overall cost of IS therapy was significantly higher than FCM. The medication cost for FCM was approximately 6.5 times higher than IS, nonetheless, it is cheaper in terms of bed cost and nursing cost. The cost effectiveness (CE) ratio illustrated that FCM and IS were significantly different in terms of Hgb, ferritin and MCH levels. Further, Incremental Cost Effectiveness Ratio (ICER) indicated that further justifications and decisions need to be made for FCM when using Hgb, iron, TSAT, MCH and MCV levels as surrogate outcomes. Results of sensitivity analysis Not applicable. Limitations The study did not consider the clinical or humanistic outcome. Conclusions The higher cost of FCM versus IS can be offset by savings in healthcare personnel time and bed space. ICER indicated that further justifications and decisions need to be made for FCM when using Hgb, iron, TSAT, MCH and MCV levels as surrogate outcomes.

Cells to the Rescue: Emerging Cell-Based Treatment Approaches for NMOSD and MOGAD

Cell-based therapies are gaining momentum as promising treatments for rare neurological autoimmune diseases, including neuromyelitis optica spectrum disorders and myelin oligodendrocyte glycoprotein antibody-associated disease. The development of targeted cell therapies is hampered by the lack of adequate animal models that mirror the human disease. Most cell-based treatments, including HSCT, CAR-T cell, tolerogenic dendritic cell and mesenchymal stem cell treatment have entered early stage clinical trials or have been used as rescue treatment in treatment-refractory cases. The development of antigen-specific cell-based immunotherapies for autoimmune diseases is slowed down by the rarity of the diseases, the lack of surrogate outcomes and biomarkers that are able to predict long-term outcomes and/or therapy effectiveness as well as challenges in the manufacturing of cellular products. These challenges are likely to be overcome by future research.

Study design flaws and statistical challenges in evaluating fertility treatments

Health interventions should be tested before being introduced into clinical practice, to find out whether they work and whether they are harmful. However, research studies will only provide reliable answers to these questions if they are appropriately designed and analysed. But these are not trivial tasks. We review some methodological challenges that arise when evaluating fertility interventions and explain the implications for a non-statistical audience. These include flexibility in outcomes and analyses; use of surrogate outcomes instead of live birth; use of inappropriate denominators; evaluating cumulative outcomes and time to live birth; allowing each patient or couple to contribute to a research study more than once. We highlight recurring errors and present solutions. We conclude by highlighting the importance of collaboration between clinical and methodological experts, as well as people with experience of subfertility, for realising high-quality research. Lay summary We do research to find out whether fertility treatments are beneficial and to make sure they don’t cause harm. However, research will only provide reliable answers if it is done properly. It is not unusual for researchers to make mistakes when they are designing research studies and analysing the data that we get from them. In this review, we describe some of the mistakes people make when they do research about fertility treatments and explain how to avoid them. These include challenges which arise due to the large number of things that can be measured and reported when looking to see if fertility treatments work; failure to check whether the treatment increases the number of live births; failing to include all study participants in calculations;challenges in studies where participants may have more than one treatment attempt. We conclude by highlighting the importance of collaboration between clinical and methodological experts, as well as people with experience of fertility problems.