scholarly journals External validation of prognostic models predicting pre-eclampsia: individual participant data meta-analysis

BMC Medicine ◽  
2020 ◽  
Vol 18 (1) ◽  
Author(s):  
Kym I. E. Snell ◽  
◽  
John Allotey ◽  
Melanie Smuk ◽  
Richard Hooper ◽  
...  
Keyword(s):  
Clinical Decision Making ◽  
Prediction Models ◽  
Meta Analysis ◽  
External Validation ◽  
Predictive Performance ◽  
Prognostic Models ◽  
Individual Participant Data ◽  
Net Benefit ◽  
Mortality And Morbidity ◽  
Individual Participant

Abstract Background Pre-eclampsia is a leading cause of maternal and perinatal mortality and morbidity. Early identification of women at risk during pregnancy is required to plan management. Although there are many published prediction models for pre-eclampsia, few have been validated in external data. Our objective was to externally validate published prediction models for pre-eclampsia using individual participant data (IPD) from UK studies, to evaluate whether any of the models can accurately predict the condition when used within the UK healthcare setting. Methods IPD from 11 UK cohort studies (217,415 pregnant women) within the International Prediction of Pregnancy Complications (IPPIC) pre-eclampsia network contributed to external validation of published prediction models, identified by systematic review. Cohorts that measured all predictor variables in at least one of the identified models and reported pre-eclampsia as an outcome were included for validation. We reported the model predictive performance as discrimination (C-statistic), calibration (calibration plots, calibration slope, calibration-in-the-large), and net benefit. Performance measures were estimated separately in each available study and then, where possible, combined across studies in a random-effects meta-analysis. Results Of 131 published models, 67 provided the full model equation and 24 could be validated in 11 UK cohorts. Most of the models showed modest discrimination with summary C-statistics between 0.6 and 0.7. The calibration of the predicted compared to observed risk was generally poor for most models with observed calibration slopes less than 1, indicating that predictions were generally too extreme, although confidence intervals were wide. There was large between-study heterogeneity in each model’s calibration-in-the-large, suggesting poor calibration of the predicted overall risk across populations. In a subset of models, the net benefit of using the models to inform clinical decisions appeared small and limited to probability thresholds between 5 and 7%. Conclusions The evaluated models had modest predictive performance, with key limitations such as poor calibration (likely due to overfitting in the original development datasets), substantial heterogeneity, and small net benefit across settings. The evidence to support the use of these prediction models for pre-eclampsia in clinical decision-making is limited. Any models that we could not validate should be examined in terms of their predictive performance, net benefit, and heterogeneity across multiple UK settings before consideration for use in practice. Trial registration PROSPERO ID: CRD42015029349.

scholarly journals Validation and development of models using clinical, biochemical and ultrasound markers for predicting pre-eclampsia: an individual participant data meta-analysis

2020 ◽  
Vol 24 (72) ◽  
pp. 1-252
Author(s):  
John Allotey ◽  
Hannele Laivuori ◽  
Kym IE Snell ◽  
Melanie Smuk ◽  
Richard Hooper ◽  
...  
Keyword(s):  
Pregnancy Complications ◽  
Biochemical Markers ◽  
Clinical Utility ◽  
Clinical Characteristics ◽  
Prediction Models ◽  
Meta Analysis ◽  
Predictive Performance ◽  
Individual Participant Data ◽  
Net Benefit ◽  
Individual Participant

Background Pre-eclampsia is a leading cause of maternal and perinatal mortality and morbidity. Early identification of women at risk is needed to plan management. Objectives To assess the performance of existing pre-eclampsia prediction models and to develop and validate models for pre-eclampsia using individual participant data meta-analysis. We also estimated the prognostic value of individual markers. Design This was an individual participant data meta-analysis of cohort studies. Setting Source data from secondary and tertiary care. Predictors We identified predictors from systematic reviews, and prioritised for importance in an international survey. Primary outcomes Early-onset (delivery at < 34 weeks’ gestation), late-onset (delivery at ≥ 34 weeks’ gestation) and any-onset pre-eclampsia. Analysis We externally validated existing prediction models in UK cohorts and reported their performance in terms of discrimination and calibration. We developed and validated 12 new models based on clinical characteristics, clinical characteristics and biochemical markers, and clinical characteristics and ultrasound markers in the first and second trimesters. We summarised the data set-specific performance of each model using a random-effects meta-analysis. Discrimination was considered promising for C-statistics of ≥ 0.7, and calibration was considered good if the slope was near 1 and calibration-in-the-large was near 0. Heterogeneity was quantified using I 2 and τ2. A decision curve analysis was undertaken to determine the clinical utility (net benefit) of the models. We reported the unadjusted prognostic value of individual predictors for pre-eclampsia as odds ratios with 95% confidence and prediction intervals. Results The International Prediction of Pregnancy Complications network comprised 78 studies (3,570,993 singleton pregnancies) identified from systematic reviews of tests to predict pre-eclampsia. Twenty-four of the 131 published prediction models could be validated in 11 UK cohorts. Summary C-statistics were between 0.6 and 0.7 for most models, and calibration was generally poor owing to large between-study heterogeneity, suggesting model overfitting. The clinical utility of the models varied between showing net harm to showing minimal or no net benefit. The average discrimination for IPPIC models ranged between 0.68 and 0.83. This was highest for the second-trimester clinical characteristics and biochemical markers model to predict early-onset pre-eclampsia, and lowest for the first-trimester clinical characteristics models to predict any pre-eclampsia. Calibration performance was heterogeneous across studies. Net benefit was observed for International Prediction of Pregnancy Complications first and second-trimester clinical characteristics and clinical characteristics and biochemical markers models predicting any pre-eclampsia, when validated in singleton nulliparous women managed in the UK NHS. History of hypertension, parity, smoking, mode of conception, placental growth factor and uterine artery pulsatility index had the strongest unadjusted associations with pre-eclampsia. Limitations Variations in study population characteristics, type of predictors reported, too few events in some validation cohorts and the type of measurements contributed to heterogeneity in performance of the International Prediction of Pregnancy Complications models. Some published models were not validated because model predictors were unavailable in the individual participant data. Conclusion For models that could be validated, predictive performance was generally poor across data sets. Although the International Prediction of Pregnancy Complications models show good predictive performance on average, and in the singleton nulliparous population, heterogeneity in calibration performance is likely across settings. Future work Recalibration of model parameters within populations may improve calibration performance. Additional strong predictors need to be identified to improve model performance and consistency. Validation, including examination of calibration heterogeneity, is required for the models we could not validate. Study registration This study is registered as PROSPERO CRD42015029349. Funding This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 24, No. 72. See the NIHR Journals Library website for further project information.

scholarly journals Development and validation of a prediction model for fat mass in children and adolescents: meta-analysis using individual participant data

BMJ ◽  
2019 ◽  
pp. l4293 ◽  
Author(s):  
Mohammed T Hudda ◽  
Mary S Fewtrell ◽  
Dalia Haroun ◽  
Sooky Lum ◽  
Jane E Williams ◽  
...  
Keyword(s):  
Confidence Interval ◽  
Prediction Model ◽  
Fat Mass ◽  
Meta Analysis ◽  
External Validation ◽  
Predictive Performance ◽  
Individual Participant Data ◽  
Internal Validation ◽  
Individual Participant ◽  
Complex Forms

Abstract Objectives To develop and validate a prediction model for fat mass in children aged 4-15 years using routinely available risk factors of height, weight, and demographic information without the need for more complex forms of assessment. Design Individual participant data meta-analysis. Setting Four population based cross sectional studies and a fifth study for external validation, United Kingdom. Participants A pooled derivation dataset (four studies) of 2375 children and an external validation dataset of 176 children with complete data on anthropometric measurements and deuterium dilution assessments of fat mass. Main outcome measure Multivariable linear regression analysis, using backwards selection for inclusion of predictor variables and allowing non-linear relations, was used to develop a prediction model for fat-free mass (and subsequently fat mass by subtracting resulting estimates from weight) based on the four studies. Internal validation and then internal-external cross validation were used to examine overfitting and generalisability of the model’s predictive performance within the four development studies; external validation followed using the fifth dataset. Results Model derivation was based on a multi-ethnic population of 2375 children (47.8% boys, n=1136) aged 4-15 years. The final model containing predictor variables of height, weight, age, sex, and ethnicity had extremely high predictive ability (optimism adjusted R 2 : 94.8%, 95% confidence interval 94.4% to 95.2%) with excellent calibration of observed and predicted values. The internal validation showed minimal overfitting and good model generalisability, with excellent calibration and predictive performance. External validation in 176 children aged 11-12 years showed promising generalisability of the model (R 2 : 90.0%, 95% confidence interval 87.2% to 92.8%) with good calibration of observed and predicted fat mass (slope: 1.02, 95% confidence interval 0.97 to 1.07). The mean difference between observed and predicted fat mass was −1.29 kg (95% confidence interval −1.62 to −0.96 kg). Conclusion The developed model accurately predicted levels of fat mass in children aged 4-15 years. The prediction model is based on simple anthropometric measures without the need for more complex forms of assessment and could improve the accuracy of assessments for body fatness in children (compared with those provided by body mass index) for effective surveillance, prevention, and management of clinical and public health obesity.

scholarly journals A heart failure phenotype stratified model for predicting 1-year mortality in patients admitted with acute heart failure: results from an individual participant data meta-analysis of four prospective European cohorts

BMC Medicine ◽  
2021 ◽  
Vol 19 (1) ◽  
Author(s):  
Yuntao Chen ◽  
Adriaan A. Voors ◽  
Tiny Jaarsma ◽  
Chim C. Lang ◽  
Iziah E. Sama ◽  
...  
Keyword(s):  
Heart Failure ◽  
Clinical Decision Making ◽  
Cox Model ◽  
Meta Analysis ◽  
Prognostic Index ◽  
Individual Participant Data ◽  
Mortality Risks ◽  
Baseline Mortality ◽  
Individual Participant ◽  
Stratified Model

Abstract Background Prognostic models developed in general cohorts with a mixture of heart failure (HF) phenotypes, though more widely applicable, are also likely to yield larger prediction errors in settings where the HF phenotypes have substantially different baseline mortality rates or different predictor-outcome associations. This study sought to use individual participant data meta-analysis to develop an HF phenotype stratified model for predicting 1-year mortality in patients admitted with acute HF. Methods Four prospective European cohorts were used to develop an HF phenotype stratified model. Cox model with two rounds of backward elimination was used to derive the prognostic index. Weibull model was used to obtain the baseline hazard functions. The internal-external cross-validation (IECV) approach was used to evaluate the generalizability of the developed model in terms of discrimination and calibration. Results 3577 acute HF patients were included, of which 2368 were classified as having HF with reduced ejection fraction (EF) (HFrEF; EF < 40%), 588 as having HF with midrange EF (HFmrEF; EF 40–49%), and 621 as having HF with preserved EF (HFpEF; EF ≥ 50%). A total of 11 readily available variables built up the prognostic index. For four of these predictor variables, namely systolic blood pressure, serum creatinine, myocardial infarction, and diabetes, the effect differed across the three HF phenotypes. With a weighted IECV-adjusted AUC of 0.79 (0.74–0.83) for HFrEF, 0.74 (0.70–0.79) for HFmrEF, and 0.74 (0.71–0.77) for HFpEF, the model showed excellent discrimination. Moreover, there was a good agreement between the average observed and predicted 1-year mortality risks, especially after recalibration of the baseline mortality risks. Conclusions Our HF phenotype stratified model showed excellent generalizability across four European cohorts and may provide a useful tool in HF phenotype-specific clinical decision-making.

scholarly journals Arm Based on LEg blood pressures (ABLE-BP): can systolic leg blood pressure measurements predict systolic brachial blood pressure? Protocol for an individual participant data meta-analysis from the INTERPRESS-IPD Collaboration

BMJ Open ◽  
2021 ◽  
Vol 11 (3) ◽  
pp. e040481
Author(s):  
Sinead T J McDonagh ◽  
James P Sheppard ◽  
Fiona C Warren ◽  
Kate Boddy ◽  
Leon Farmer ◽  
...  
Keyword(s):  
Blood Pressure ◽  
Public Involvement ◽  
Meta Analysis ◽  
Patient And Public Involvement ◽  
Prognostic Models ◽  
Individual Participant Data ◽  
Dissemination Strategy ◽  
Cross Sectional ◽  
Individual Participant ◽  
Meta Analyses

IntroductionBlood pressure (BP) is normally measured on the upper arm, and guidelines for the diagnosis and treatment of high BP are based on such measurements. Leg BP measurement can be an alternative when brachial BP measurement is impractical, due to injury or disability. Limited data exist to guide interpretation of leg BP values for hypertension management; study-level systematic review findings suggest that systolic BP (SBP) is 17 mm Hg higher in the leg than the arm. However, uncertainty remains about the applicability of this figure in clinical practice due to substantial heterogeneity.AimsTo examine the relationship between arm and leg SBP, develop and validate a multivariable model predicting arm SBP from leg SBP and investigate the prognostic association between leg SBP and cardiovascular disease and mortality.Methods and analysisIndividual participant data (IPD) meta-analyses using arm and leg SBP measurements for 33 710 individuals from 14 studies within the Inter-arm blood pressure difference IPD (INTERPRESS-IPD) Collaboration. We will explore cross-sectional relationships between arm and leg SBP using hierarchical linear regression with participants nested by study, in multivariable models. Prognostic models will be derived for all-cause and cardiovascular mortality and cardiovascular events.Ethics and disseminationData originate from studies with prior ethical approval and consent, and data sharing agreements are in place—no further approvals are required to undertake the secondary analyses proposed in this protocol. Findings will be published in peer-reviewed journal articles and presented at conferences. A comprehensive dissemination strategy is in place, integrated with patient and public involvement.PROSPERO registration numberCRD42015031227.

A framework for developing, implementing, and evaluating clinical prediction models in an individual participant data meta-analysis

2013 ◽  
Vol 32 (18) ◽  
pp. 3158-3180 ◽  
Author(s):  
Thomas P.A. Debray ◽  
Karel G.M. Moons ◽  
Ikhlaaq Ahmed ◽  
Hendrik Koffijberg ◽  
Richard David Riley
Keyword(s):  
Prediction Models ◽  
Meta Analysis ◽  
Individual Participant Data ◽  
Clinical Prediction ◽  
Clinical Prediction Models ◽  
Individual Participant

scholarly journals Systematic review and network meta-analysis with individual participant data on cord management at preterm birth (iCOMP): study protocol

BMJ Open ◽  
2020 ◽  
Vol 10 (3) ◽  
pp. e034595
Author(s):  
Anna Lene Seidler ◽  
Lelia Duley ◽  
Anup C Katheria ◽  
Catalina De Paco Matallana ◽  
Eugene Dempsey ◽  
...  
Keyword(s):  
Preterm Birth ◽  
Primary Outcome ◽  
Meta Analysis ◽  
Management Strategies ◽  
Preterm Births ◽  
Individual Participant Data ◽  
Mortality And Morbidity ◽  
Cord Clamping ◽  
Individual Participant ◽  
Participant Characteristics

IntroductionTiming of cord clamping and other cord management strategies may improve outcomes at preterm birth. However, it is unclear whether benefits apply to all preterm subgroups. Previous and current trials compare various policies, including time-based or physiology-based deferred cord clamping, and cord milking. Individual participant data (IPD) enable exploration of different strategies within subgroups. Network meta-analysis (NMA) enables comparison and ranking of all available interventions using a combination of direct and indirect comparisons.Objectives(1) To evaluate the effectiveness of cord management strategies for preterm infants on neonatal mortality and morbidity overall and for different participant characteristics using IPD meta-analysis. (2) To evaluate and rank the effect of different cord management strategies for preterm births on mortality and other key outcomes using NMA.Methods and analysisSystematic searches of Medline, Embase, clinical trial registries, and other sources for all ongoing and completed randomised controlled trials comparing cord management strategies at preterm birth (before 37 weeks’ gestation) have been completed up to 13 February 2019, but will be updated regularly to include additional trials. IPD will be sought for all trials; aggregate summary data will be included where IPD are unavailable. First, deferred clamping and cord milking will be compared with immediate clamping in pairwise IPD meta-analyses. The primary outcome will be death prior to hospital discharge. Effect differences will be explored for prespecified participant subgroups. Second, all identified cord management strategies will be compared and ranked in an IPD NMA for the primary outcome and the key secondary outcomes. Treatment effect differences by participant characteristics will be identified. Inconsistency and heterogeneity will be explored.Ethics and disseminationEthics approval for this project has been granted by the University of Sydney Human Research Ethics Committee (2018/886). Results will be relevant to clinicians, guideline developers and policy-makers, and will be disseminated via publications, presentations and media releases.Registration numberAustralian New Zealand Clinical Trials Registry (ANZCTR) (ACTRN12619001305112) and International Prospective Register of Systematic Reviews (PROSPERO, CRD42019136640).

Individual participant data meta-analysis of prognosis studies

2019 ◽  
pp. 281-297
Author(s):  
Richard D Riley ◽  
Thomas PA Debray ◽  
Karel GM Moons
Keyword(s):  
Evidence Synthesis ◽  
Meta Analysis ◽  
Prognostic Models ◽  
Individual Participant Data ◽  
Lack Of Information ◽  
Alternative Approach ◽  
Individual Participant ◽  
Prognosis Research ◽  
Key Steps ◽  
Access Data

An alternative approach to meta-analysis of aggregate data from published prognosis research (as addressed in Chapter 9), with its challenges of heterogeneity and lack of information, is to conduct meta-analysis of individual participant data (IPD), that is, the original raw data of the individuals who are included in the primary prognosis studies. The approach is increasingly feasible as data sharing and open-access data become more popular, and the chapter highlights why they offer enormous advantages for a robust and meaningful evidence synthesis of prognosis studies. In particular, better prognostic models can be developed and directly validated across multiple settings, and power is increased to detect genuine predictors of treatment response. Key steps in such an IPD meta-analysis are explained, including practical guidance on how to obtain, handle, and synthesize data, and what potential challenges may be encountered.

Sign in / Sign up

Export Citation Format

Share Document