scholarly journals Validation and development of models using clinical, biochemical and ultrasound markers for predicting pre-eclampsia: an individual participant data meta-analysis

2020 ◽  
Vol 24 (72) ◽  
pp. 1-252
Author(s):  
John Allotey ◽  
Hannele Laivuori ◽  
Kym IE Snell ◽  
Melanie Smuk ◽  
Richard Hooper ◽  
...  
Keyword(s):  
Pregnancy Complications ◽  
Biochemical Markers ◽  
Clinical Utility ◽  
Clinical Characteristics ◽  
Prediction Models ◽  
Meta Analysis ◽  
Predictive Performance ◽  
Individual Participant Data ◽  
Net Benefit ◽  
Individual Participant

Background Pre-eclampsia is a leading cause of maternal and perinatal mortality and morbidity. Early identification of women at risk is needed to plan management. Objectives To assess the performance of existing pre-eclampsia prediction models and to develop and validate models for pre-eclampsia using individual participant data meta-analysis. We also estimated the prognostic value of individual markers. Design This was an individual participant data meta-analysis of cohort studies. Setting Source data from secondary and tertiary care. Predictors We identified predictors from systematic reviews, and prioritised for importance in an international survey. Primary outcomes Early-onset (delivery at < 34 weeks’ gestation), late-onset (delivery at ≥ 34 weeks’ gestation) and any-onset pre-eclampsia. Analysis We externally validated existing prediction models in UK cohorts and reported their performance in terms of discrimination and calibration. We developed and validated 12 new models based on clinical characteristics, clinical characteristics and biochemical markers, and clinical characteristics and ultrasound markers in the first and second trimesters. We summarised the data set-specific performance of each model using a random-effects meta-analysis. Discrimination was considered promising for C-statistics of ≥ 0.7, and calibration was considered good if the slope was near 1 and calibration-in-the-large was near 0. Heterogeneity was quantified using I 2 and τ2. A decision curve analysis was undertaken to determine the clinical utility (net benefit) of the models. We reported the unadjusted prognostic value of individual predictors for pre-eclampsia as odds ratios with 95% confidence and prediction intervals. Results The International Prediction of Pregnancy Complications network comprised 78 studies (3,570,993 singleton pregnancies) identified from systematic reviews of tests to predict pre-eclampsia. Twenty-four of the 131 published prediction models could be validated in 11 UK cohorts. Summary C-statistics were between 0.6 and 0.7 for most models, and calibration was generally poor owing to large between-study heterogeneity, suggesting model overfitting. The clinical utility of the models varied between showing net harm to showing minimal or no net benefit. The average discrimination for IPPIC models ranged between 0.68 and 0.83. This was highest for the second-trimester clinical characteristics and biochemical markers model to predict early-onset pre-eclampsia, and lowest for the first-trimester clinical characteristics models to predict any pre-eclampsia. Calibration performance was heterogeneous across studies. Net benefit was observed for International Prediction of Pregnancy Complications first and second-trimester clinical characteristics and clinical characteristics and biochemical markers models predicting any pre-eclampsia, when validated in singleton nulliparous women managed in the UK NHS. History of hypertension, parity, smoking, mode of conception, placental growth factor and uterine artery pulsatility index had the strongest unadjusted associations with pre-eclampsia. Limitations Variations in study population characteristics, type of predictors reported, too few events in some validation cohorts and the type of measurements contributed to heterogeneity in performance of the International Prediction of Pregnancy Complications models. Some published models were not validated because model predictors were unavailable in the individual participant data. Conclusion For models that could be validated, predictive performance was generally poor across data sets. Although the International Prediction of Pregnancy Complications models show good predictive performance on average, and in the singleton nulliparous population, heterogeneity in calibration performance is likely across settings. Future work Recalibration of model parameters within populations may improve calibration performance. Additional strong predictors need to be identified to improve model performance and consistency. Validation, including examination of calibration heterogeneity, is required for the models we could not validate. Study registration This study is registered as PROSPERO CRD42015029349. Funding This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 24, No. 72. See the NIHR Journals Library website for further project information.

scholarly journals External validation of prognostic models predicting pre-eclampsia: individual participant data meta-analysis

BMC Medicine ◽  
2020 ◽  
Vol 18 (1) ◽  
Author(s):  
Kym I. E. Snell ◽  
◽  
John Allotey ◽  
Melanie Smuk ◽  
Richard Hooper ◽  
...  
Keyword(s):  
Clinical Decision Making ◽  
Prediction Models ◽  
Meta Analysis ◽  
External Validation ◽  
Predictive Performance ◽  
Prognostic Models ◽  
Individual Participant Data ◽  
Net Benefit ◽  
Mortality And Morbidity ◽  
Individual Participant

Abstract Background Pre-eclampsia is a leading cause of maternal and perinatal mortality and morbidity. Early identification of women at risk during pregnancy is required to plan management. Although there are many published prediction models for pre-eclampsia, few have been validated in external data. Our objective was to externally validate published prediction models for pre-eclampsia using individual participant data (IPD) from UK studies, to evaluate whether any of the models can accurately predict the condition when used within the UK healthcare setting. Methods IPD from 11 UK cohort studies (217,415 pregnant women) within the International Prediction of Pregnancy Complications (IPPIC) pre-eclampsia network contributed to external validation of published prediction models, identified by systematic review. Cohorts that measured all predictor variables in at least one of the identified models and reported pre-eclampsia as an outcome were included for validation. We reported the model predictive performance as discrimination (C-statistic), calibration (calibration plots, calibration slope, calibration-in-the-large), and net benefit. Performance measures were estimated separately in each available study and then, where possible, combined across studies in a random-effects meta-analysis. Results Of 131 published models, 67 provided the full model equation and 24 could be validated in 11 UK cohorts. Most of the models showed modest discrimination with summary C-statistics between 0.6 and 0.7. The calibration of the predicted compared to observed risk was generally poor for most models with observed calibration slopes less than 1, indicating that predictions were generally too extreme, although confidence intervals were wide. There was large between-study heterogeneity in each model’s calibration-in-the-large, suggesting poor calibration of the predicted overall risk across populations. In a subset of models, the net benefit of using the models to inform clinical decisions appeared small and limited to probability thresholds between 5 and 7%. Conclusions The evaluated models had modest predictive performance, with key limitations such as poor calibration (likely due to overfitting in the original development datasets), substantial heterogeneity, and small net benefit across settings. The evidence to support the use of these prediction models for pre-eclampsia in clinical decision-making is limited. Any models that we could not validate should be examined in terms of their predictive performance, net benefit, and heterogeneity across multiple UK settings before consideration for use in practice. Trial registration PROSPERO ID: CRD42015029349.

scholarly journals Development and validation of a prediction model for fat mass in children and adolescents: meta-analysis using individual participant data

BMJ ◽  
2019 ◽  
pp. l4293 ◽  
Author(s):  
Mohammed T Hudda ◽  
Mary S Fewtrell ◽  
Dalia Haroun ◽  
Sooky Lum ◽  
Jane E Williams ◽  
...  
Keyword(s):  
Confidence Interval ◽  
Prediction Model ◽  
Fat Mass ◽  
Meta Analysis ◽  
External Validation ◽  
Predictive Performance ◽  
Individual Participant Data ◽  
Internal Validation ◽  
Individual Participant ◽  
Complex Forms

Abstract Objectives To develop and validate a prediction model for fat mass in children aged 4-15 years using routinely available risk factors of height, weight, and demographic information without the need for more complex forms of assessment. Design Individual participant data meta-analysis. Setting Four population based cross sectional studies and a fifth study for external validation, United Kingdom. Participants A pooled derivation dataset (four studies) of 2375 children and an external validation dataset of 176 children with complete data on anthropometric measurements and deuterium dilution assessments of fat mass. Main outcome measure Multivariable linear regression analysis, using backwards selection for inclusion of predictor variables and allowing non-linear relations, was used to develop a prediction model for fat-free mass (and subsequently fat mass by subtracting resulting estimates from weight) based on the four studies. Internal validation and then internal-external cross validation were used to examine overfitting and generalisability of the model’s predictive performance within the four development studies; external validation followed using the fifth dataset. Results Model derivation was based on a multi-ethnic population of 2375 children (47.8% boys, n=1136) aged 4-15 years. The final model containing predictor variables of height, weight, age, sex, and ethnicity had extremely high predictive ability (optimism adjusted R 2 : 94.8%, 95% confidence interval 94.4% to 95.2%) with excellent calibration of observed and predicted values. The internal validation showed minimal overfitting and good model generalisability, with excellent calibration and predictive performance. External validation in 176 children aged 11-12 years showed promising generalisability of the model (R 2 : 90.0%, 95% confidence interval 87.2% to 92.8%) with good calibration of observed and predicted fat mass (slope: 1.02, 95% confidence interval 0.97 to 1.07). The mean difference between observed and predicted fat mass was −1.29 kg (95% confidence interval −1.62 to −0.96 kg). Conclusion The developed model accurately predicted levels of fat mass in children aged 4-15 years. The prediction model is based on simple anthropometric measures without the need for more complex forms of assessment and could improve the accuracy of assessments for body fatness in children (compared with those provided by body mass index) for effective surveillance, prevention, and management of clinical and public health obesity.

A framework for developing, implementing, and evaluating clinical prediction models in an individual participant data meta-analysis

2013 ◽  
Vol 32 (18) ◽  
pp. 3158-3180 ◽  
Author(s):  
Thomas P.A. Debray ◽  
Karel G.M. Moons ◽  
Ikhlaaq Ahmed ◽  
Hendrik Koffijberg ◽  
Richard David Riley
Keyword(s):  
Prediction Models ◽  
Meta Analysis ◽  
Individual Participant Data ◽  
Clinical Prediction ◽  
Clinical Prediction Models ◽  
Individual Participant

scholarly journals Individual participant data validation of the PICNICC prediction model for febrile neutropenia

2019 ◽  
Vol 105 (5) ◽  
pp. 439-445 ◽  
Author(s):  
Bob Phillips ◽  
Jessica Elizabeth Morgan ◽  
Gabrielle M Haeusler ◽  
Richard D Riley
Keyword(s):  
Prediction Model ◽  
Prediction Models ◽  
Meta Analysis ◽  
Characteristic Curve ◽  
Area Under The Curve ◽  
Infectious Complications ◽  
Cancer Therapies ◽  
Individual Participant Data ◽  
Documented Infection ◽  
Individual Participant

BackgroundRisk-stratified approaches to managing cancer therapies and their consequent complications rely on accurate predictions to work effectively. The risk-stratified management of fever with neutropenia is one such very common area of management in paediatric practice. Such rules are frequently produced and promoted without adequate confirmation of their accuracy.MethodsAn individual participant data meta-analytic validation of the ‘Predicting Infectious ComplicatioNs In Children with Cancer’ (PICNICC) prediction model for microbiologically documented infection in paediatric fever with neutropenia was undertaken. Pooled estimates were produced using random-effects meta-analysis of the area under the curve-receiver operating characteristic curve (AUC-ROC), calibration slope and ratios of expected versus observed cases (E/O).ResultsThe PICNICC model was poorly predictive of microbiologically documented infection (MDI) in these validation cohorts. The pooled AUC-ROC was 0.59, 95% CI 0.41 to 0.78, tau2=0, compared with derivation value of 0.72, 95% CI 0.71 to 0.76. There was poor discrimination (pooled slope estimate 0.03, 95% CI −0.19 to 0.26) and calibration in the large (pooled E/O ratio 1.48, 95% CI 0.87 to 2.1). Three different simple recalibration approaches failed to improve performance meaningfully.ConclusionThis meta-analysis shows the PICNICC model should not be used at admission to predict MDI. Further work should focus on validating alternative prediction models. Validation across multiple cohorts from diverse locations is essential before widespread clinical adoption of such rules to avoid overtreating or undertreating children with fever with neutropenia.

Sign in / Sign up

Export Citation Format

Share Document