scholarly journals Individual participant data validation of the PICNICC prediction model for febrile neutropenia

2019 ◽  
Vol 105 (5) ◽  
pp. 439-445 ◽  
Author(s):  
Bob Phillips ◽  
Jessica Elizabeth Morgan ◽  
Gabrielle M Haeusler ◽  
Richard D Riley
Keyword(s):  
Prediction Model ◽  
Prediction Models ◽  
Meta Analysis ◽  
Characteristic Curve ◽  
Area Under The Curve ◽  
Infectious Complications ◽  
Cancer Therapies ◽  
Individual Participant Data ◽  
Documented Infection ◽  
Individual Participant

BackgroundRisk-stratified approaches to managing cancer therapies and their consequent complications rely on accurate predictions to work effectively. The risk-stratified management of fever with neutropenia is one such very common area of management in paediatric practice. Such rules are frequently produced and promoted without adequate confirmation of their accuracy.MethodsAn individual participant data meta-analytic validation of the ‘Predicting Infectious ComplicatioNs In Children with Cancer’ (PICNICC) prediction model for microbiologically documented infection in paediatric fever with neutropenia was undertaken. Pooled estimates were produced using random-effects meta-analysis of the area under the curve-receiver operating characteristic curve (AUC-ROC), calibration slope and ratios of expected versus observed cases (E/O).ResultsThe PICNICC model was poorly predictive of microbiologically documented infection (MDI) in these validation cohorts. The pooled AUC-ROC was 0.59, 95% CI 0.41 to 0.78, tau2=0, compared with derivation value of 0.72, 95% CI 0.71 to 0.76. There was poor discrimination (pooled slope estimate 0.03, 95% CI −0.19 to 0.26) and calibration in the large (pooled E/O ratio 1.48, 95% CI 0.87 to 2.1). Three different simple recalibration approaches failed to improve performance meaningfully.ConclusionThis meta-analysis shows the PICNICC model should not be used at admission to predict MDI. Further work should focus on validating alternative prediction models. Validation across multiple cohorts from diverse locations is essential before widespread clinical adoption of such rules to avoid overtreating or undertreating children with fever with neutropenia.

scholarly journals Development and validation of a prediction model for fat mass in children and adolescents: meta-analysis using individual participant data

BMJ ◽  
2019 ◽  
pp. l4293 ◽  
Author(s):  
Mohammed T Hudda ◽  
Mary S Fewtrell ◽  
Dalia Haroun ◽  
Sooky Lum ◽  
Jane E Williams ◽  
...  
Keyword(s):  
Confidence Interval ◽  
Prediction Model ◽  
Fat Mass ◽  
Meta Analysis ◽  
External Validation ◽  
Predictive Performance ◽  
Individual Participant Data ◽  
Internal Validation ◽  
Individual Participant ◽  
Complex Forms

Abstract Objectives To develop and validate a prediction model for fat mass in children aged 4-15 years using routinely available risk factors of height, weight, and demographic information without the need for more complex forms of assessment. Design Individual participant data meta-analysis. Setting Four population based cross sectional studies and a fifth study for external validation, United Kingdom. Participants A pooled derivation dataset (four studies) of 2375 children and an external validation dataset of 176 children with complete data on anthropometric measurements and deuterium dilution assessments of fat mass. Main outcome measure Multivariable linear regression analysis, using backwards selection for inclusion of predictor variables and allowing non-linear relations, was used to develop a prediction model for fat-free mass (and subsequently fat mass by subtracting resulting estimates from weight) based on the four studies. Internal validation and then internal-external cross validation were used to examine overfitting and generalisability of the model’s predictive performance within the four development studies; external validation followed using the fifth dataset. Results Model derivation was based on a multi-ethnic population of 2375 children (47.8% boys, n=1136) aged 4-15 years. The final model containing predictor variables of height, weight, age, sex, and ethnicity had extremely high predictive ability (optimism adjusted R 2 : 94.8%, 95% confidence interval 94.4% to 95.2%) with excellent calibration of observed and predicted values. The internal validation showed minimal overfitting and good model generalisability, with excellent calibration and predictive performance. External validation in 176 children aged 11-12 years showed promising generalisability of the model (R 2 : 90.0%, 95% confidence interval 87.2% to 92.8%) with good calibration of observed and predicted fat mass (slope: 1.02, 95% confidence interval 0.97 to 1.07). The mean difference between observed and predicted fat mass was −1.29 kg (95% confidence interval −1.62 to −0.96 kg). Conclusion The developed model accurately predicted levels of fat mass in children aged 4-15 years. The prediction model is based on simple anthropometric measures without the need for more complex forms of assessment and could improve the accuracy of assessments for body fatness in children (compared with those provided by body mass index) for effective surveillance, prevention, and management of clinical and public health obesity.

A framework for developing, implementing, and evaluating clinical prediction models in an individual participant data meta-analysis

2013 ◽  
Vol 32 (18) ◽  
pp. 3158-3180 ◽  
Author(s):  
Thomas P.A. Debray ◽  
Karel G.M. Moons ◽  
Ikhlaaq Ahmed ◽  
Hendrik Koffijberg ◽  
Richard David Riley
Keyword(s):  
Prediction Models ◽  
Meta Analysis ◽  
Individual Participant Data ◽  
Clinical Prediction ◽  
Clinical Prediction Models ◽  
Individual Participant

scholarly journals External validation of prognostic models predicting pre-eclampsia: individual participant data meta-analysis

BMC Medicine ◽  
2020 ◽  
Vol 18 (1) ◽  
Author(s):  
Kym I. E. Snell ◽  
◽  
John Allotey ◽  
Melanie Smuk ◽  
Richard Hooper ◽  
...  
Keyword(s):  
Clinical Decision Making ◽  
Prediction Models ◽  
Meta Analysis ◽  
External Validation ◽  
Predictive Performance ◽  
Prognostic Models ◽  
Individual Participant Data ◽  
Net Benefit ◽  
Mortality And Morbidity ◽  
Individual Participant

Abstract Background Pre-eclampsia is a leading cause of maternal and perinatal mortality and morbidity. Early identification of women at risk during pregnancy is required to plan management. Although there are many published prediction models for pre-eclampsia, few have been validated in external data. Our objective was to externally validate published prediction models for pre-eclampsia using individual participant data (IPD) from UK studies, to evaluate whether any of the models can accurately predict the condition when used within the UK healthcare setting. Methods IPD from 11 UK cohort studies (217,415 pregnant women) within the International Prediction of Pregnancy Complications (IPPIC) pre-eclampsia network contributed to external validation of published prediction models, identified by systematic review. Cohorts that measured all predictor variables in at least one of the identified models and reported pre-eclampsia as an outcome were included for validation. We reported the model predictive performance as discrimination (C-statistic), calibration (calibration plots, calibration slope, calibration-in-the-large), and net benefit. Performance measures were estimated separately in each available study and then, where possible, combined across studies in a random-effects meta-analysis. Results Of 131 published models, 67 provided the full model equation and 24 could be validated in 11 UK cohorts. Most of the models showed modest discrimination with summary C-statistics between 0.6 and 0.7. The calibration of the predicted compared to observed risk was generally poor for most models with observed calibration slopes less than 1, indicating that predictions were generally too extreme, although confidence intervals were wide. There was large between-study heterogeneity in each model’s calibration-in-the-large, suggesting poor calibration of the predicted overall risk across populations. In a subset of models, the net benefit of using the models to inform clinical decisions appeared small and limited to probability thresholds between 5 and 7%. Conclusions The evaluated models had modest predictive performance, with key limitations such as poor calibration (likely due to overfitting in the original development datasets), substantial heterogeneity, and small net benefit across settings. The evidence to support the use of these prediction models for pre-eclampsia in clinical decision-making is limited. Any models that we could not validate should be examined in terms of their predictive performance, net benefit, and heterogeneity across multiple UK settings before consideration for use in practice. Trial registration PROSPERO ID: CRD42015029349.

Sign in / Sign up

Export Citation Format

Share Document