Predictive accuracy of enhanced versions of the on-admission National Early Warning Score in estimating the risk of COVID-19 for unplanned admission to hospital: a retrospective development and validation study

Abstract Background The novel coronavirus SARS-19 produces ‘COVID-19’ in patients with symptoms. COVID-19 patients admitted to the hospital require early assessment and care including isolation. The National Early Warning Score (NEWS) and its updated version NEWS2 is a simple physiological scoring system used in hospitals, which may be useful in the early identification of COVID-19 patients. We investigate the performance of multiple enhanced NEWS2 models in predicting the risk of COVID-19. Methods Our cohort included unplanned adult medical admissions discharged over 3 months (11 March 2020 to 13 June 2020 ) from two hospitals (YH for model development; SH for external model validation). We used logistic regression to build multiple prediction models for the risk of COVID-19 using the first electronically recorded NEWS2 within ± 24 hours of admission. Model M0’ included NEWS2; model M1’ included NEWS2 + age + sex, and model M2’ extends model M1’ with subcomponents of NEWS2 (including diastolic blood pressure + oxygen flow rate + oxygen scale). Model performance was evaluated according to discrimination (c statistic), calibration (graphically), and clinical usefulness at NEWS2 ≥ 5. Results The prevalence of COVID-19 was higher in SH (11.0 %=277/2520) than YH (8.7 %=343/3924) with a higher first NEWS2 scores ( SH 3.2 vs YH 2.8) but similar in-hospital mortality (SH 8.4 % vs YH 8.2 %). The c-statistics for predicting the risk of COVID-19 for models M0’,M1’,M2’ in the development dataset were: M0’: 0.71 (95 %CI 0.68–0.74); M1’: 0.67 (95 %CI 0.64–0.70) and M2’: 0.78 (95 %CI 0.75–0.80)). For the validation datasets the c-statistics were: M0’ 0.65 (95 %CI 0.61–0.68); M1’: 0.67 (95 %CI 0.64–0.70) and M2’: 0.72 (95 %CI 0.69–0.75) ). The calibration slope was similar across all models but Model M2’ had the highest sensitivity (M0’ 44 % (95 %CI 38-50 %); M1’ 53 % (95 %CI 47-59 %) and M2’: 57 % (95 %CI 51-63 %)) and specificity (M0’ 75 % (95 %CI 73-77 %); M1’ 72 % (95 %CI 70-74 %) and M2’: 76 % (95 %CI 74-78 %)) for the validation dataset at NEWS2 ≥ 5. Conclusions Model M2’ appears to be reasonably accurate for predicting the risk of COVID-19. It may be clinically useful as an early warning system at the time of admission especially to triage large numbers of unplanned hospital admissions.

Download Full-text

Predictive accuracy of computer-aided versions of the on-admission National Early Warning Score in estimating the risk of COVID-19 for unplanned admission to hospital: a retrospective development and validation study

10.1101/2020.11.30.20241257 ◽

2020 ◽

Author(s):

Muhammad Faisal ◽

Mohammed A Mohammed ◽

Donald Richardson ◽

Ewout W. Steyerberg ◽

Massimo Fiori ◽

...

Keyword(s):

Logistic Regression ◽

Model Validation ◽

Validation Study ◽

Early Warning ◽

Model Development ◽

Early Warning Score ◽

Validation Dataset ◽

Unplanned Admission ◽

National Early Warning Score ◽

Development And Validation

AbstractObjectivesTo consider the potential of the National Early Warning Score (NEWS2) for COVID-19 risk prediction on unplanned admission to hospital.DesignLogistic regression model development and validation study using a cohort of unplanned emergency medical admission to hospital.SettingYork Hospital (YH) as model development dataset and Scarborough Hospital (SH) as model validation dataset.ParticipantsUnplanned adult medical admissions discharged over 3 months (11 March 2020 to 13 June 2020) from two hospitals (YH for model development; SH for external model validation) based on admission NEWS2 electronically recorded within ±24 hours of admission. We used logistic regression modelling to predict the risk of COVID-19 using NEWS2 (Model M0’) versus enhanced cNEWS models which included age + sex (model M1’) + subcomponents (including diastolic blood pressure + oxygen flow rate + oxygen scale) of NEWS2 (model M2’). The ICD-10 code ‘U071’ was used to identify COVID-19 admissions. Model performance was evaluated according to discrimination (c statistic), calibration (graphically), and clinical usefulness at NEWS2 ≥5.ResultsThe prevalence of COVID-19 was higher in SH (11.0%=277/2520) than YH (8.7%=343/3924) with higher index NEWS2 (3.2 vs 2.8) but similar in-hospital mortality (8.4% vs 8.2%). The c-statistics for predicting COVID-19 for cNEWS models (M1’,M2’) was substantially better than NEWS2 alone (M0’) in development (M2’: 0.78 (95%CI 0.75-0.80) vs M0’ 0.71 (95%CI 0.68-0.74)) and validation datasets (M2’: 0.72 (95%CI 0.69-0.75) vs M0’ 0.65 (95%CI 0.61-0.68)). Model M2’ had better calibration than Model M0’ with improved sensitivity (M2’: 57% (95%CI 51%-63%) vs M0’ 44% (95%CI 38%-50%)) and similar specificity (M2’: 76% (95%CI 74%-78%) vs M0’ 75% (95%CI 73%-77%)) for validation dataset at NEWS2≥5.ConclusionsModel M2’ is reasonably accurate for predicting the on-admission risk of COVID-19. It may be clinically useful for an early warning system at the time of admission especially to triage large numbers of unplanned hospital admissions.

Download Full-text

Development of A Pharmaceutical Early Warning Scoring Model for Cardiac Arrest Using Deep Learning: Retrospective Cohort Study (Preprint)

10.2196/preprints.21720 ◽

2020 ◽

Author(s):

Hsiao-Ko Chang ◽

Hui-Chih Wang ◽

Chih-Fen Huang ◽

Feipei Lai

Keyword(s):

Time Series ◽

Cardiac Arrest ◽

Early Warning ◽

Time Series Data ◽

Predictive Accuracy ◽

Vital Signs ◽

Sliding Window ◽

Early Warning Score ◽

Series Data ◽

Dynamic Time

BACKGROUND In most of Taiwan’s medical institutions, congestion is a serious problem for emergency departments. Due to a lack of beds, patients spend more time in emergency retention zones, which make it difficult to detect cardiac arrest (CA). OBJECTIVE We seek to develop a pharmaceutical early warning model to predict cardiac arrest in emergency departments via drug classification and medical expert suggestion. METHODS We propose a new early warning score model for detecting cardiac arrest via pharmaceutical classification and by using a sliding window; we apply learning-based algorithms to time-series data for a Pharmaceutical Early Warning Scoring Model (PEWSM). By treating pharmaceutical features as a dynamic time-series factor for cardiopulmonary resuscitation (CPR) patients, we increase sensitivity, reduce false alarm rates and mortality, and increase the model’s accuracy. To evaluate the proposed model we use the area under the receiver operating characteristic curve (AUROC). RESULTS Four important findings are as follows: (1) We identify the most important drug predictors: bits, and replenishers and regulators of water and electrolytes. The best AUROC of bits is 85%; that of replenishers and regulators of water and electrolytes is 86%. These two features are the most influential of the drug features in the task. (2) We verify feature selection, in which accounting for drugs improve the accuracy: In Task 1, the best AUROC of vital signs is 77%, and that of all features is 86%. In Task 2, the best AUROC of all features is 85%, which demonstrates that thus accounting for the drugs significantly affects prediction. (3) We use a better model: For traditional machine learning, this study adds a new AI technology: the long short-term memory (LSTM) model with the best time-series accuracy, comparable to the traditional random forest (RF) model; the two AUROC measures are 85%. (4) We determine whether the event can be predicted beforehand: The best classifier is still an RF model, in which the observational starting time is 4 hours before the CPR event. Although the accuracy is impaired, the predictive accuracy still reaches 70%. Therefore, we believe that CPR events can be predicted four hours before the event. CONCLUSIONS This paper uses a sliding window to account for dynamic time-series data consisting of the patient’s vital signs and drug injections. In a comparison with NEWS, we improve predictive accuracy via feature selection, which includes drugs as features. In addition, LSTM yields better performance with time-series data. The proposed PEWSM, which offers 4-hour predictions, is better than the National Early Warning Score (NEWS) in the literature. This also confirms that the doctor’s heuristic rules are consistent with the results found by machine learning algorithms.

Download Full-text

Experiences of a National Early Warning Score (NEWS) intervention in care homes during the COVID-19 pandemic: a qualitative interview study

BMJ Open ◽

10.1136/bmjopen-2020-045469 ◽

2021 ◽

Vol 11 (7) ◽

pp. e045469

Author(s):

Rachel Stocker ◽

Siân Russell ◽

Jennifer Liddle ◽

Robert O Barker ◽

Adam Remmer ◽

...

Keyword(s):

Early Warning ◽

Healthcare Professionals ◽

Care Home ◽

Qualitative Interview ◽

Interview Study ◽

Care Homes ◽

Early Warning Score ◽

Service Utilisation ◽

Qualitative Interview Study ◽

National Early Warning Score

BackgroundThe COVID-19 pandemic has taken a heavy toll on the care home sector, with residents accounting for up to half of all deaths in Europe. The response to acute illness in care homes plays a particularly important role in the care of residents during a pandemic. Digital recording of a National Early Warning Score (NEWS), which involves the measurement of physical observations, started in care homes in one area of England in 2016. Implementation of a NEWS intervention (including equipment, training and support) was accelerated early in the pandemic, despite limited evidence for its use in the care home setting.ObjectivesTo understand how a NEWS intervention has been used in care homes in one area of North-East England during the COVID-19 pandemic, and how it has influenced resident care, from the perspective of stakeholders involved in care delivery and commissioning.MethodsA qualitative interview study with care home (n=10) and National Health Service (n=7) staff. Data were analysed using thematic analysis.ResultsUse of the NEWS intervention in care homes in this area accelerated during the COVID-19 pandemic. Stakeholders felt that NEWS, and its associated education and support package, improved the response of care homes and healthcare professionals to deterioration in residents’ health during the pandemic. Healthcare professionals valued the ability to remotely monitor resident observations, which facilitated triage and treatment decisions. Care home staff felt empowered by NEWS, providing a common clinical language to communicate concerns with external services, acting as an adjunct to staff intuition of resident deterioration.ConclusionsThe NEWS intervention formed an important part of the care home response to COVID-19 in the study area. Positive staff perceptions now need to be supplemented with data on the impact on resident health and well-being, workload, and service utilisation, during the pandemic and beyond.

Download Full-text

Corrigendum to “Risk prediction using the National Early Warning Score and the Worthing Physiological Scoring System in patients who were transported to the Intensive Care Unit from the Emergency Department: A cohort study” [Intens. Crit. Care Nurs. 64 (2021) 103015]

Intensive and Critical Care Nursing ◽

10.1016/j.iccn.2021.103085 ◽

2021 ◽

pp. 103085

Author(s):

Ying An ◽

Zi-Rong Tian ◽

Fei Li ◽

Ya-Mei Guan ◽

Zi-Feng Ma ◽

...

Keyword(s):

Intensive Care Unit ◽

Emergency Department ◽

Cohort Study ◽

Intensive Care ◽

Risk Prediction ◽

Early Warning ◽

Scoring System ◽

Early Warning Score ◽

National Early Warning Score

Download Full-text

8-Day and Daily Maximum and Minimum Air Temperature Estimation via Machine Learning Method on a Climate Zone to Global Scale

Remote Sensing ◽

10.3390/rs13122355 ◽

2021 ◽

Vol 13 (12) ◽

pp. 2355

Author(s):

Linglin Zeng ◽

Yuchao Hu ◽

Rui Wang ◽

Xiang Zhang ◽

Guozhang Peng ◽

...

Keyword(s):

Spatial Scale ◽

Air Temperature ◽

Spatial Scales ◽

Model Performance ◽

Global Scale ◽

Scale Model ◽

Validation Dataset ◽

Daily Maximum ◽

Climate Zone ◽

Temporal And Spatial

Air temperature (Ta) is a required input in a wide range of applications, e.g., agriculture. Land Surface Temperature (LST) products from Moderate Resolution Imaging Spectroradiometer (MODIS) are widely used to estimate Ta. Previous studies of these products in Ta estimation, however, were generally applied in small areas and with a small number of meteorological stations. This study designed both temporal and spatial experiments to estimate 8-day and daily maximum and minimum Ta (Tmax and Tmin) on three spatial scales: climate zone, continental and global scales from 2009 to 2018, using the Random Forest (RF) method based on MODIS LST products and other auxiliary data. Factors contributing to the relation between LST and Ta were determined based on physical models and equations. Temporal and spatial experiments were defined by the rules of dividing the training and validation datasets for the RF method, in which the stations selected in the training dataset were all included or not in the validation dataset. The RF model was first trained and validated on each spatial scale, respectively. On a global scale, model accuracy with a determination coefficient (R2) > 0.96 and root mean square error (RMSE) < 1.96 °C and R2 > 0.95 and RMSE < 2.55 °C was achieved for 8-day and daily Ta estimations, respectively, in both temporal and spatial experiments. Then the model was trained and cross-validated on each spatial scale. The results showed that the data size and station distribution of the study area were the main factors influencing the model performance at different spatial scales. Finally, the spatial patterns of the model performance and variable importance were analyzed. Both daytime and nighttime LST had a significant contribution in the 8-day Tmax estimation on all the three spatial scales; while their contribution in daily Tmax estimation varied over different continents or climate zones. This study was expected to improve our understanding of Ta estimation in terms of accuracy variations and influencing variables on different spatial and temporal scales. The future work mainly includes identifying underlying mechanisms of estimation errors and the uncertainty sources of Ta estimation from a local to a global scale.

Download Full-text

The Modified National Early Warning Score (m-NEWS) for COVID-19-Infected Patient Evaluation: a Proof-of-Concept

SN Comprehensive Clinical Medicine ◽

10.1007/s42399-020-00718-x ◽

2021 ◽

Author(s):

Luca Galassi ◽

Daniele Schena

Keyword(s):

Early Warning ◽

Infected Patient ◽

Early Warning Score ◽

Patient Evaluation ◽

Proof Of Concept ◽

National Early Warning Score

Download Full-text

Use of the first National Early Warning Score recorded within 24 hours of admission to estimate the risk of in-hospital mortality in unplanned COVID-19 patients: a retrospective cohort study

BMJ Open ◽

10.1136/bmjopen-2020-043721 ◽

2021 ◽

Vol 11 (2) ◽

pp. e043721

Author(s):

Donald Richardson ◽

Muhammad Faisal ◽

Massimo Fiori ◽

Kevin Beatson ◽

Mohammed Mohammed

Keyword(s):

Hospital Mortality ◽

Early Warning ◽

Absolute Risk ◽

Cross Sectional Study ◽

Early Warning Score ◽

Cross Sectional ◽

Risk Of Mortality ◽

Baseline Mortality ◽

Acute Hospitals ◽

National Early Warning Score

ObjectivesAlthough the National Early Warning Score (NEWS) and its latest version NEWS2 are recommended for monitoring deterioration in patients admitted to hospital, little is known about their performance in COVID-19 patients. We aimed to compare the performance of the NEWS and NEWS2 in patients with COVID-19 versus those without during the first phase of the pandemic.DesignA retrospective cross-sectional study.SettingTwo acute hospitals (Scarborough and York) are combined into a single dataset and analysed collectively.ParticipantsAdult (≥18 years) non-elective admissions discharged between 11 March 2020 and 13 June 2020 with an index or on-admission NEWS2 electronically recorded within ±24 hours of admission to predict mortality at four time points (in-hospital, 24 hours, 48 hours and 72 hours) in COVID-19 versus non-COVID-19 admissions.ResultsOut of 6480 non-elective admissions, 620 (9.6%) had a diagnosis of COVID-19. They were older (73.3 vs 67.7 years), more often male (54.7% vs 50.1%), had higher index NEWS (4 vs 2.5) and NEWS2 (4.6 vs 2.8) scores and higher in-hospital mortality (32.1% vs 5.8%). The c-statistics for predicting in-hospital mortality in COVID-19 admissions was significantly lower using NEWS (0.64 vs 0.74) or NEWS2 (0.64 vs 0.74), however, these differences reduced at 72hours (NEWS: 0.75 vs 0.81; NEWS2: 0.71 vs 0.81), 48 hours (NEWS: 0.78 vs 0.81; NEWS2: 0.76 vs 0.82) and 24hours (NEWS: 0.84 vs 0.84; NEWS2: 0.86 vs 0.84). Increasing NEWS2 values reflected increased mortality, but for any given value the absolute risk was on average 24% higher (eg, NEWS2=5: 36% vs 9%).ConclusionsThe index or on-admission NEWS and NEWS2 offers lower discrimination for COVID-19 admissions versus non-COVID-19 admissions. The index NEWS2 was not proven to be better than the index NEWS. For each value of the index NEWS/NEWS2, COVID-19 admissions had a substantially higher risk of mortality than non-COVID-19 admissions which reflects the increased baseline mortality risk of COVID-19.

Download Full-text

National Early Warning Score 2 (NEWS2) better predicts critical Coronavirus Disease 2019 (COVID-19) illness than COVID-GRAM, a multi-centre study

Infection ◽

10.1007/s15010-021-01620-x ◽

2021 ◽

Author(s):

Giuseppe Vittorio De Socio ◽

Anna Gidari ◽

Francesco Sicari ◽

Michele Palumbo ◽

Daniela Francisci

Keyword(s):

Early Warning ◽

Operating Characteristic ◽

Primary Outcome ◽

Characteristic Curve ◽

Early Warning Score ◽

Multi Centre Study ◽

Umbria Region ◽

Chinese Cohort ◽

National Early Warning Score ◽

Specific Score

Abstract Purpose Clinical scores to rapidly assess the severity illness of Coronavirus Disease 2019 (COVID-19) could be considered of help for clinicians. Recently, a specific score (named COVID-GRAM) for severe acute respiratory syndrome-coronavirus-2 (SARS-CoV-2) infection, based on a nationwide Chinese cohort, has been proposed. We routinely applied the National Early Warning Score 2 (NEWS2) to predict critical COVID-19. Aim of this study is to compare NEWS2 and COVID-GRAM score. Methods We retrospectively analysed data of 121 COVID-19 patients admitted in two Clinics of Infectious Diseases in the Umbria region, Italy. The primary outcome was critical COVID-19 illness defined as admission to the intensive care unit, invasive ventilation, or death. Accuracy of the scores was evaluated with the area under the receiver-operating characteristic curve (AUROC). Differences between scores were confirmed used Hanley–McNeil test. Results The NEWS2 AUROC curve measured 0.87 (standard error, SE 0.03; 95% CI 0.80–0.93; p < 0.0001). The COVID-GRAM score AUROC curve measured 0.77 (SE 0.04; 95% CI 0.68–0.85; p < 0.0001). Hanley–McNeil test showed that NEWS2 better predicted severe COVID-19 (Z = 2.03). Conclusions The NEWS2 showed superior accuracy to COVID-GRAM score for prediction of critical COVID-19 illness.

Download Full-text

Glucose as an additional parameter to National Early Warning Score (NEWS) in prehospital setting enhances identification of patients at risk of death: an observational cohort study

Emergency Medicine Journal ◽

10.1136/emermed-2018-208309 ◽

2020 ◽

pp. emermed-2018-208309

Author(s):

Hanna Vihonen ◽

Mitja Lääperi ◽

Markku Kuisma ◽

Jussi Pirneskoski ◽

Jouni Nurmi

Keyword(s):

At Risk ◽

Blood Glucose ◽

Regression Model ◽

Early Warning ◽

Prehospital Setting ◽

Early Warning Score ◽

Calibration Model ◽

Risk Of Death ◽

Patients At Risk ◽

National Early Warning Score

BackgroundTo determine if prehospital blood glucose could be added to National Early Warning Score (NEWS) for improved identification of risk of short-term mortality.MethodsRetrospective observational study (2008–2015) of adult patients seen by emergency medical services in Helsinki metropolitan area for whom all variables for calculation of NEWS and a blood glucose value were available. Survival of 24 hours and 30 days were determined. The NEWS parameters and glucose were tested by multivariate logistic regression model. Based on ORs we formed NEWSgluc model with hypoglycaemia (≤3.0 mmol/L) 3, normoglycaemia 0 and hyperglycaemia (≥11.1 mmol/L) 1 points. The scores from NEWS and NEWSgluc were compared using discrimination (area under the curve), calibration (Hosmer-Lemeshow test), likelihood ratio tests and reclassification (continuous net reclassification index (cNRI)).ResultsData of 27 141 patients were included in the study. Multivariable regression model for NEWSgluc parameters revealed a strong association with glucose disturbances and 24-hour and 30-day mortality. Likelihood ratios (LRs) for mortality at 24 hours using a cut-off point of 15 were for NEWSgluc: LR+ 17.78 and LR− 0.96 and for NEWS: LR+ 13.50 and LR− 0.92. Results were similar at 30 days. Risks per score point estimation and calibration model showed glucose added benefit to NEWS at 24 hours and at 30 days. Although areas under the curve were similar, reclassification test (cNRI) showed overall improvement of classification of survivors and non-survivors at 24 days and 30 days with NEWSgluc.ConclusionsIncluding glucose in NEWS in the prehospital setting seems to improve identification of patients at risk of death.

Download Full-text