Is the endoscopic third ventriculostomy score an appropriate tool to inform clinical decision-making? A national multi-institutional external validation study

Objectives of the study Demographic changes alongside medical advances have resulted in older adults accounting for an increasing proportion of emergency hospital admissions. Current measures of illness severity, limited to physiological parameters, have shortcomings in this cohort, partly due to patient complexity. This study aimed to derive and validate a risk score for acutely unwell older adults which may enhance risk stratification and support clinical decision-making. Methods Data was collected from emergency admissions in patients ≥65 years from two UK general hospitals (April 2017- April 2018). Variables underwent regression analysis for in-hospital mortality and independent predictors were used to create a risk score. Performance was assessed on external validation. Secondary outcomes included seven-day mortality and extended hospital stay. Results Derivation (n = 8,974) and validation (n = 8,391) cohorts were analysed. The model included the National Early Warning Score 2 (NEWS2), clinical frailty scale (CFS), acute kidney injury, age, sex, and Malnutrition Universal Screening Tool. For mortality, area under the curve for the model was 0.79 (95% CI 0.78–0.80), superior to NEWS2 0.65 (0.62–0.67) and CFS 0.76 (0.74–0.77) (P<0.0001). Risk groups predicted prolonged hospital stay: the highest risk group had an odds ratio of 9.7 (5.8–16.1) to stay >30 days. Conclusions Our simple validated model (Older Persons’ Emergency Risk Assessment [OPERA] score) predicts in-hospital mortality and prolonged length of stay and could be easily integrated into electronic hospital systems, enabling automatic digital generation of risk stratification within hours of admission. Future studies may validate the OPERA score in external populations and consider an impact analysis.

Download Full-text

923 CLINICAL AND PATHOLOGICAL NODAL STAGING SCORE FOR UROTHELIAL CARCINOMA OF THE BLADDER ARE VALID DECISION TOOLS FOR RISK ASSESSMENT AND CLINICAL DECISION-MAKING: AN EXTERNAL VALIDATION

The Journal of Urology ◽

10.1016/j.juro.2013.02.500 ◽

2013 ◽

Vol 189 (4S) ◽

Cited By ~ 1

Author(s):

Michael Gierth ◽

Hans Martin Fritsche ◽

Hannes Buchner ◽

Matthias May ◽

Attiqullah Aziz ◽

...

Keyword(s):

Risk Assessment ◽

Decision Making ◽

Urothelial Carcinoma ◽

Clinical Decision Making ◽

External Validation ◽

Clinical Decision ◽

Nodal Staging ◽

Decision Tools ◽

Carcinoma Of The Bladder

Download Full-text

Calibration: the Achilles heel of predictive analytics

BMC Medicine ◽

10.1186/s12916-019-1466-7 ◽

2019 ◽

Vol 17 (1) ◽

Cited By ~ 40

Author(s):

Ben Van Calster ◽

◽

David J. McLernon ◽

Maarten van Smeden ◽

Laure Wynants ◽

...

Keyword(s):

Decision Making ◽

Clinical Decision Making ◽

Prediction Models ◽

Predictive Analytics ◽

External Validation ◽

Clinical Decision ◽

Machine Learning Algorithms ◽

Model Complexity ◽

Patient Counseling ◽

Flexible Machine

Abstract Background The assessment of calibration performance of risk prediction models based on regression or more flexible machine learning algorithms receives little attention. Main text Herein, we argue that this needs to change immediately because poorly calibrated algorithms can be misleading and potentially harmful for clinical decision-making. We summarize how to avoid poor calibration at algorithm development and how to assess calibration at algorithm validation, emphasizing balance between model complexity and the available sample size. At external validation, calibration curves require sufficiently large samples. Algorithm updating should be considered for appropriate support of clinical practice. Conclusion Efforts are required to avoid poor calibration when developing prediction models, to evaluate calibration when validating models, and to update models when indicated. The ultimate aim is to optimize the utility of predictive analytics for shared decision-making and patient counseling.

Download Full-text

Sensitivity of proposed clinical decision rules for subarachnoid haemorrhage: An external validation study

Emergency Medicine Australasia ◽

10.1111/1742-6723.12325 ◽

2014 ◽

Vol 26 (6) ◽

pp. 556-560 ◽

Cited By ~ 1

Author(s):

Anne-Maree Kelly ◽

Sharon Klim ◽

Simon Edward ◽

Nicola Millar

Keyword(s):

Subarachnoid Haemorrhage ◽

Validation Study ◽

External Validation ◽

Decision Rules ◽

Clinical Decision ◽

Clinical Decision Rules ◽

External Validation Study

Download Full-text

286 Clinical and pathological nodal staging score for urothelial carcinoma of the bladder are valid decision tools for risk assessment and clinical decision-making: An external validation

European Urology Supplements ◽

10.1016/s1569-9056(13)60772-1 ◽

2013 ◽

Vol 12 (1) ◽

pp. e286-e287

Author(s):

M.S. Gierth ◽

H.M. Fritsche ◽

H. Buchner ◽

M. May ◽

A. Aziz ◽

...

Keyword(s):

Risk Assessment ◽

Decision Making ◽

Urothelial Carcinoma ◽

Clinical Decision Making ◽

External Validation ◽

Clinical Decision ◽

Nodal Staging ◽

Decision Tools ◽

Carcinoma Of The Bladder

Download Full-text

Automated Classification of Significant Prostate Cancer on MRI: A Systematic Review on the Performance of Machine Learning Applications

Cancers ◽

10.3390/cancers12061606 ◽

2020 ◽

Vol 12 (6) ◽

pp. 1606

Author(s):

Jose M. Castillo T. ◽

Muhammad Arif ◽

Wiro J. Niessen ◽

Ivo G. Schoots ◽

Jifke F. Veenland

Keyword(s):

Machine Learning ◽

Decision Making ◽

Deep Learning ◽

Clinical Decision Making ◽

External Validation ◽

Clinical Decision ◽

Interquartile Range ◽

Added Value ◽

Receiver Operating Curve ◽

Learning Approaches

Significant prostate carcinoma (sPCa) classification based on MRI using radiomics or deep learning approaches has gained much interest, due to the potential application in assisting in clinical decision-making. Objective: To systematically review the literature (i) to determine which algorithms are most frequently used for sPCa classification, (ii) to investigate whether there exists a relation between the performance and the method or the MRI sequences used, (iii) to assess what study design factors affect the performance on sPCa classification, and (iv) to research whether performance had been evaluated in a clinical setting Methods: The databases Embase and Ovid MEDLINE were searched for studies describing machine learning or deep learning classification methods discriminating between significant and nonsignificant PCa on multiparametric MRI that performed a valid validation procedure. Quality was assessed by the modified radiomics quality score. We computed the median area under the receiver operating curve (AUC) from overall methods and the interquartile range. Results: From 2846 potentially relevant publications, 27 were included. The most frequent algorithms used in the literature for PCa classification are logistic regression (22%) and convolutional neural networks (CNNs) (22%). The median AUC was 0.79 (interquartile range: 0.77–0.87). No significant effect of number of included patients, image sequences, or reference standard on the reported performance was found. Three studies described an external validation and none of the papers described a validation in a prospective clinical trial. Conclusions: To unlock the promising potential of machine and deep learning approaches, validation studies and clinical prospective studies should be performed with an established protocol to assess the added value in decision-making.

Download Full-text

M121. CLINICAL PREDICTION MODELS FOR TRANSITION TO PSYCHOSIS: AN EXTERNAL VALIDATION STUDY IN THE PRONIA SAMPLE

Schizophrenia Bulletin ◽

10.1093/schbul/sbaa030.433 ◽

2020 ◽

Vol 46 (Supplement_1) ◽

pp. S181-S181

Author(s):

Marlene Rosen ◽

Linda Betz ◽

Alessandro Bertolino ◽

Stefan Borgwardt ◽

Paolo Brambilla ◽

...

Keyword(s):

Validation Study ◽

Psychotic Disorders ◽

Clinical Decision Making ◽

Prediction Models ◽

External Validation ◽

Model Performance ◽

Discrimination Performance ◽

Data Driven ◽

Risk Models ◽

External Validation Study

Abstract Background A multitude of clinical models to predict transition to psychosis in individuals at clinical high risk (CHR) have been proposed. However, only limited efforts have been made to systematically compare these models and to validate their performance in independent samples. Therefore, in this study we identified psychosis risk models based on information readily obtainable in general clinical settings, such as clinical and neuropsychological data, and compared their performance in the PRONIA study (Personalised Prognostic Tools for Early Psychosis Management, www.pronia.eu) as an independent sample. Methods Of the 278 CHR participants in the PRONIA sample, 150 had available data until month 18 and were included in the validation of eleven psychosis prediction models identified through systematic literature search. Discrimination performance was assessed with the area under the receiver operating characteristic curve (AUC), and compared to the performance of the prognosis of clinical raters. Psychosocial functioning was explored as an alternative outcome. Results Discrimination performance varied considerably across models (AUC ranging from 0.42 to 0.79). High model performance was associated with the inclusion of neurocognitive variables as predictors. Low model performance was associated with predictors based on dichotomized variables. Clinical raters performed comparable to the best data-driven models (AUC = 0.75). Combining raters’ prognosis and model-based predictions improved discrimination performance (AUC = 0.84), particularly for less experienced raters. One of the tested models predicted transition to psychosis and psychosocial outcomes comparably well. Discussion The present external validation study highlights the benefit of enriching clinical information with neuropsychological data in predicting transition to psychosis satisfactorily and with good generalizability across samples. Integration of data-driven risk models and clinical expertise may improve clinical decision-making in CHR for psychosis, particularly for less experienced raters. This external validation study provides an important step toward early intervention and the personalized treatment of psychotic disorders.

Download Full-text