Early warning score validation methodologies and performance metrics: A systematic review

Background Early warning scores (EWS) have been developed as clinical prognostication tools to identify acutely deteriorating patients. With recent advancements in machine learning, there has been a proliferation of studies that describe the development and validation of novel EWS. Systematic reviews of published studies which focus on evaluating performance of both well-established and novel EWS have shown conflicting conclusions. A possible reason for this is the lack of consistency in the validation methods used. In this review, we aim to examine the methodologies and performance metrics used in studies which describe EWS validation. Methods A systematic review of all eligible studies in the MEDLINE database from inception to 22-Feb-2019 was performed. Studies were eligible if they performed validation on at least one EWS and reported associations between EWS scores and mortality, intensive care unit (ICU) transfers, or cardiac arrest (CA) of adults within the inpatient setting. Two reviewers independently did a full-text review and performed data abstraction by using standardized data-worksheet based on the TRIPOD (Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis) checklist. Meta-analysis was not performed due to heterogeneity. Results The key differences in validation methodologies identified were (1) validation population characteristics, (2) outcomes of interest, (3) case definition, intended time of use and aggregation methods, and (4) handling of missing values in the validation dataset. In terms of case definition, among the 34 eligible studies, 22 used the patient episode case definition while 10 used the observation set case definition, and 2 did the validation using both case definitions. Of those that used the patient episode case definition, 11 studies used a single point of time score to validate the EWS, most of which used the first recorded observation. There were also more than 10 different performance metrics reported among the studies. Conclusions Methodologies and performance metrics used in studies performing validation on EWS were not consistent hence making it difficult to interpret and compare EWS performance. Standardizing EWS validation methodology and reporting can potentially address this issue.

Early warning score validation methodologies and performance metrics: A systematic review

Background Early warning scores (EWS) have been developed as clinical prognostication tools to identify acutely deteriorating patients. With recent advancements in machine learning, there has been a proliferation of studies that describe the development and validation of novel EWS. Systematic reviews of published studies which focus on evaluating performance of both well-established and novel EWS have shown conflicting conclusions. A possible reason for this is the lack of consistency in the validation methods used. In this review, we aim to examine the methodologies and performance metrics used in studies which describe EWS validation.Methods A systematic review of all eligible studies in the MEDLINE database from inception to 22-Feb-2019 was performed. Studies were eligible if they performed validation on at least one EWS and reported associations between EWS scores and mortality, intensive care unit (ICU) transfers, or cardiac arrest (CA) of adults within the inpatient setting. Two reviewers independently did a full-text review and performed data abstraction by using standardized data-worksheet based on the TRIPOD (Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis) checklist. Meta-analysis was not performed due to heterogeneity.Results The key differences in validation methodologies identified were (1) validation population characteristics, (2) outcomes of interest, (3) case definition, intended time of use and aggregation methods, and (4) handling of missing values in the validation dataset. In terms of case definition, among the 34 eligible studies, 22 used the patient episode case definition while 10 used the observation set case definition, and 2 did the validation using both case definitions. Of those that used the patient episode case definition, 11 studies used a single point of time score to validate the EWS, most of which used the first recorded observation. There were also more than 10 different performance metrics reported among the studies.Conclusions Methodologies and performance metrics used in studies performing validation on EWS were not consistent hence making it difficult to interpret and compare EWS performance. Standardizing EWS validation methodology and reporting can potentially address this issue.

Resource implications of bortezomib therapy in a large UK cohort: An evaluation study

Background Bortezomib is a cornerstone in the management of multiple myeloma. It remains an attractive treatment option because it is efficacious, reasonably well tolerated and easy to administer. However, data on resource implications in the UK for both patients and healthcare providers are limited. Methods We conducted a retrospective study of 127 patients to assess implications of bortezomib therapy on patients and healthcare resources. A patient-episode was defined as a patient attending the chemotherapy day treatment unit solely for bortezomib administration. Data were collected for the duration of therapy as follows: cost of drug calculated using the UK’s bortezomib indicative price as per British National Formulary, cost of drug administration in the chemotherapy day treatment unit calculated using the National Health Service’s schedule of service cost, time from check-in to drug administration, patient travel time and distance calculated using Google maps, and cost of travel. Results Median drug cost and administration cost per patient were £8336 (£2084–£108,368) and £4640 (£290–£15,080), respectively. Median time from check-in to administration was 63 min (range 5–433), median travel time was 90 min (range 8–270) and 80 min (range 8–280) during peak and off-peak periods, respectively. Median return travel distance was 33.4 miles (range 1.2–224) for travel cost per patient per trip was £8.35–£13.20. Conclusions Our real-world resource analysis demonstrated that delivering bortezomib therapy can be associated with significant cost and time implications for patients and healthcare providers. Our study method sets a basis for evaluating resource implications of other novel approaches to myeloma therapy.

Impact of telephone triage in managing patient demand for same-day appointments? A before-after study in a UK general practice

BackgroundTelephone consultations are increasingly being utilised to manage patient demand for GP appointments. It’s true impact on patient care remains to be answered as existing evidence has shown mixed results.AimTo report the impact of telephone triage on: number of face-to-face (F2F) appointments offered; unplanned re-consultations with GP or urgent care services within 24 hours; and total consultation duration per patient episode.MethodA retrospective cohort study in a single suburban practice in Kent, comparing outcomes between the new telephone triage and old walk-in F2F appointment systems in managing patient requests for same-day appointments. Data was sampled across 1 week, at 3-monthly intervals over each 12-month period before and after the system change-over in April 2016.Results1198 patient encounters matched the inclusion criteria. F2F appointments were offered to 34.7% of patients after telephone triage. Although unplanned re-consultations were four times more prevalent since the system changed compared to the previous year (P<0.001), no difference was seen between those consulted by telephone only, or offered a F2F appointment after telephone triage. Overall, patient consultations by telephone triage were 2.37 minutes shorter than consultations under its predecessor (P <0.001).ConclusionTelephone triage was able to manage majority of patient problems by telephone alone, with significant reduction in consulting time per patient episode. Telephone consulting was not shown accountable for the increase in unplanned re-consultations.

A retrospective audit on usage of Diatrizoate Meglumine (Gastrografin®) for intestinal obstruction or constipation in patients with advanced neoplasms

Background: Intestinal obstruction and constipation are common conditions in patients with advanced neoplasms. Diatrizoate Meglumine has been used in the management of both these conditions without good quality evidence of its effectiveness and safety. Aim: This audit aimed to assess the usage, effectiveness and adverse effects of Diatrizoate Meglumine for intestinal obstruction and constipation in patients with advanced neoplasms. Design: A retrospective chart review was undertaken. Descriptive statistics were utilised. Setting/participants: All patients with known advanced neoplasms admitted to Mater Health Services and St Vincent’s Private Hospital Brisbane between January 2013 and October 2015; who were administered Diatrizoate Meglumine were included. Results: Seventy-one patients received Diatrizoate Meglumine. The most common diagnoses were ovarian or primary peritoneal neoplasms (33.8%). Diatrizoate Meglumine was most commonly used for intestinal obstruction (59.2%). The median dose used per patient episode was 50 mL (range: 15–500 mL). Thirty-two patients (45%) had imaging 4–24 h post-dose with Diatrizoate Meglumine being present in the large intestine in 75% of these images. Intestinal obstruction or constipation resolved in 90% of patients post-dose. Conclusion: Most clinicians used 50 mL of Diatrizoate Meglumine as a single dose and repeated imaging after 4–24 h. Diatrizoate Meglumine was well tolerated and may be effective in resolving intestinal obstruction and constipation in patients with advanced neoplasms. Quality controlled studies are needed to further guide the use of Diatrizoate Meglumine in intestinal obstruction and constipation in patients with advanced neoplasms.

A decade of change in the uptake of parathyroidectomy in England and Wales

Introduction Since the late 1990s, a number of factors have reduced the threshold for parathyroidectomy in patients with primary hyperparathyroidism. This study examined whether this has translated into increased numbers of parathyroid operations over the last decade. Methods A retrospective analysis was performed of the Patient Episode Database for Wales and English Hospital Episode Statistics annual data from 2000 to 2010 for parathyroidectomy admissions per 100,000 population. Statistical analysis was by linear regression. Results Between 2000 and 2010 there were 24,247 parathyroid operations in England and Wales (0.005% of the population), with 3 times as many women treated as men. Overall, incidence of parathyroidectomy rose from 3.3/100,000 population in 2000 to 5.8/100,000 in 2010 (p<0.0001). In England, it increased from 3.3/100,000 population to 5.8/100,000 and in Wales, it increased from 2.4/100,000 population to 4.6/100,000. Despite similar population demographics, the difference in the rate of change between England and Wales was significant (p<0.05). Uptake also varied according to age; in those aged 0–14 years, incidence of parathyroidectomy remained static whereas in all other age groups, uptake of parathyroidectomy increased significantly from 2000 to 2010. Most notably, surgical intervention in those aged 60–74 and >75 years nearly doubled over the decade (p<0.0001). Conclusions The incidence of parathyroidectomy in adults has increased significantly in the last decade in England and Wales. This likely reflects changes in population demography, available guidelines, lower threshold for referral, changing surgical approach and the realisation that surgical morbidity is now infrequent.

Working from Claims Data

Claims data are more difficult to work with to extract the necessary information about patient conditions in relationship to costs. There can be multiple claims for the same patient episode from different sources. A physician visit after an inpatient claim can be followed up for the inpatient stay rather than to consider the inpatient stay as the start of a new patient episode or a new patient problem. Therefore, in addition to analyzing patient conditions as represented by ICD9 codes, we must also attempt to define an episode and to distinguish between new problems and follow up of old problems.