scholarly journals Minimum sample size for external validation of a clinical prediction model with a binary outcome

2021 ◽  
Author(s):  
Richard D. Riley ◽  
Thomas P. A. Debray ◽  
Gary S. Collins ◽  
Lucinda Archer ◽  
Joie Ensor ◽  
...  
2020 ◽  
Vol 40 (1) ◽  
pp. 133-146 ◽  
Author(s):  
Lucinda Archer ◽  
Kym I. E. Snell ◽  
Joie Ensor ◽  
Mohammed T. Hudda ◽  
Gary S. Collins ◽  
...  

2019 ◽  
Author(s):  
Matthias Gijsen ◽  
Chao-yuan Huang ◽  
Marine Flechet ◽  
Ruth Van Daele ◽  
Peter Declercq ◽  
...  

Abstract BackgroundAugmented renal clearance (ARC) might lead to subtherapeutic plasma levels of drugs with predominant renal clearance. Early identification of ARC remains challenging for the intensive care unit (ICU) physician. We developed and validated the ARC predictor, a clinical prediction model for ARC on the next day during ICU stay, and made it available via an online calculator. Its predictive performance was compared with that of two existing models for ARC.MethodsA large multicenter database including medical, surgical and cardiac ICU patients (n = 33258 ICU days) from three Belgian tertiary care academic hospitals was used for the development of the prediction model. Development was based on clinical information available during ICU stay. We assessed performance by measuring discrimination, calibration and net benefit. The final model was externally validated (n = 10259 ICU days) in a single-center population.ResultsARC was found on 19.6% of all ICU days in the development cohort. Six clinical variables were retained in the ARC predictor: day from ICU admission, age, sex, serum creatinine, trauma and cardiac surgery. External validation confirmed good performance with an area under the curve of 0.88 (95% CI 0.87 – 0.88), and a sensitivity and specificity of 84.1 (95% CI 82.5 – 85.7) and 76.3 (95% CI 75.4 – 77.2) at the default threshold probability of 0.2, respectively.ConclusionARC on the next day can be predicted with good performance during ICU stay, using routinely collected clinical information that is readily available at bedside. The ARC predictor is available at www.arcpredictor.com.


2020 ◽  
Author(s):  
Matthias Gijsen ◽  
Chao-yuan Huang ◽  
Marine Flechet ◽  
Ruth Van Daele ◽  
Peter Declercq ◽  
...  

Abstract Background Augmented renal clearance (ARC) might lead to subtherapeutic plasma levels of drugs with predominant renal clearance. Early identification of ARC remains challenging for the intensive care unit (ICU) physician. We developed and validated the ARC predictor, a clinical prediction model for ARC on the next day during ICU stay, and made it available via an online calculator. Its predictive performance was compared with that of two existing models for ARC. Methods A large multicenter database including medical, surgical and cardiac surgery ICU patients (n = 33258 ICU days) from three Belgian tertiary care academic hospitals was used for the development of the prediction model. Development was based on clinical information available during ICU stay. We assessed performance by measuring discrimination, calibration and net benefit. The final model was externally validated (n = 10259 ICU days) in a single-center population. Results ARC was found on 19.6% of all ICU days in the development cohort. Six clinical variables were retained in the ARC predictor: day from ICU admission, age, sex, serum creatinine, trauma and cardiac surgery. External validation confirmed good performance with an area under the curve of 0.88 (95% CI 0.87 – 0.88), and a sensitivity and specificity of 84.1 (95% CI 82.5 – 85.7) and 76.3 (95% CI 75.4 – 77.2) at the default threshold probability of 0.2, respectively. Conclusion ARC on the next day can be predicted with good performance during ICU stay, using routinely collected clinical information that is readily available at bedside. The ARC predictor is available at www.arcpredictor.com .


2018 ◽  
Vol 38 (7) ◽  
pp. 1262-1275 ◽  
Author(s):  
Richard D. Riley ◽  
Kym I.E. Snell ◽  
Joie Ensor ◽  
Danielle L. Burke ◽  
Frank E. Harrell ◽  
...  

2021 ◽  
Author(s):  
Steven J. Staffa ◽  
David Zurakowski

Summary Clinical prediction models in anesthesia and surgery research have many clinical applications including preoperative risk stratification with implications for clinical utility in decision-making, resource utilization, and costs. It is imperative that predictive algorithms and multivariable models are validated in a suitable and comprehensive way in order to establish the robustness of the model in terms of accuracy, predictive ability, reliability, and generalizability. The purpose of this article is to educate anesthesia researchers at an introductory level on important statistical concepts involved with development and validation of multivariable prediction models for a binary outcome. Methods covered include assessments of discrimination and calibration through internal and external validation. An anesthesia research publication is examined to illustrate the process and presentation of multivariable prediction model development and validation for a binary outcome. Properly assessing the statistical and clinical validity of a multivariable prediction model is essential for reassuring the generalizability and reproducibility of the published tool.


Blood ◽  
2019 ◽  
Vol 134 (Supplement_1) ◽  
pp. 2376-2376
Author(s):  
David Schmidt ◽  
Katja M.J. Heitink-Pollé ◽  
C. Ellen van der Schoot ◽  
Leendert Porcelijn ◽  
Gestur Vidarsson ◽  
...  

Background Newly diagnosed childhood immune thrombocytopenia (ITP), an acquired autoimmune bleeding disease, has a good prognosis: 60-70% of patients recover spontaneously 3 months after diagnosis (transient ITP), whereas 10-20% remain thrombocytopenic beyond 12 months (chronic ITP). A key clinical challenge is the early identification of a patient's disease course to counsel families, inform treatment decisions, and guide additional diagnostics, e.g. screening for systemic autoimmune diseases, immunodeficiencies or genetic thrombocytopenia. Several clinical predictors have been proposed (Heitink-Pollé et al. Blood 2014;142(22)), but it is unclear how they can be integrated to predict disease outcomes. Objective To develop and validate a clinical prediction model for transient vs. prolonged disease courses in children with newly diagnosed ITP, using clinical characteristics at diagnosis. Study design Model development and validation in a multinational prospective observational cohort; external validation in a multicenter randomized controlled trial. Methods Using modern statistical methods, we extended a score by Edslev et al. (BJH 2007; 138) into an updated, multivariate prediction score, using individual patient data from newly diagnosed childhood ITP patients included in the observational, prospective Nordic Pediatric Hematology-Oncology ITP study (NOPHO; N=377; data shared by original investigators). Transient ITP was defined as complete recovery by platelet count 3 months after diagnosis (NOPHO, ≥150x109/L; TIKI, ≥100x109/L). The model was developed by penalized regression (Ridge) with ten-fold cross-validation in patients included during the first half of the NOPHO study period (derivation cohort, N=233) and subsequently validated in the second half (validation cohort, N=144). External validation was performed on children with newly diagnosed ITP included in the Dutch randomized controlled trial Treatment With or Without IVIg for Kids With ITP (TIKI; N=200; N=100 randomized to IVIg and N=100 carefully observed; Heitink-Pollé et al.Blood 2018; 132(9)). Inclusion criteria of both studies included a diagnosis platelet count ≤20x109/L and age below 16 years. Results Case-mix analyses showed that TIKI and NOPHO cohorts had comparable baseline characteristics, considering age, gender, preceding infections and bleeding. The rate of transient ITP was 67% (NOPHO) and 73% (TIKI). Seven predictors were included in the model: age (years; penalized odds ratio [OR] for transient ITP, 0.97), male gender (OR, 1.07), presence of mucosal bleeding (OR, 1.27), preceding infection (OR, 1.27) or vaccination (OR, 0.99), insidious disease onset (> 14 days; OR, 0.41) and diagnosis platelet count (x109/L; OR, 0.99). We evaluated the clinical prediction model in two independent groups: patients with transient ITP were discriminated with a receiver-operating characteristic AUC of 0.72 (95% CI, 0.61 - 0.84) for the NOPHO validation cohort and 0.71 (95% CI, 0.62 - 0.80) for the TIKI cohort. Additional analyses in the TIKI cohort revealed a similar classification accuracy for patients randomized to IVIg and observation only. At a posterior probability ≥65%, the positive predictive value for transient ITP was 0.85 and negative predictive value was 0.41. For long-term follow-up, when patients were grouped in low, intermediate and high score terciles (obtained in the NOPHO derivation cohort), the rate of persistent ITP after six months was 60/36/14 % (low/medium/high scores; NOPHO) and 50/27/10 % (TIKI). Furthermore, the rate of chronic ITP at twelve months follow-up was 27/16/6 % for the same score terciles (TIKI). Finally, the model score correlated with cessation of mucosal bleeding as well as any bleeding during five clinical follow-up visits during twelve months after diagnosis. Conclusion The updated clinical prediction model for transient ITP, i.e. recovery from ITP three months after diagnosis, showed adequate performance in two independent validation cohorts. Next to short-term recovery, long-term recovery six and twelve months after diagnosis and bleeding symptoms were predicted. This prediction model may allow the targeting of intensive monitoring or additional diagnostic efforts to children with low scores. On the other hand, in children with high scores, the rate for persistent and chronic ITP is low and they could be monitored `hands-off`, if this is intended. Disclosures No relevant conflicts of interest to declare.


Sign in / Sign up

Export Citation Format

Share Document