Prediction of Dementia in the General Population: Large-Scale External Validation of Prognostic Models in the AGES-Reykjavik Study

2020 ◽  
Author(s):  
Jet M. J. Vonk ◽  
Jacoba Greving ◽  
Vilmundur Guðnason ◽  
Lenore Launer ◽  
Mirjam Geerlings
Keyword(s):  
General Population ◽  
Large Scale ◽  
External Validation ◽  
Prognostic Models

scholarly journals Dementia risk in the general population: large-scale external validation of prediction models in the AGES-Reykjavik study

2021 ◽  
Author(s):  
Jet M. J. Vonk ◽  
Jacoba P. Greving ◽  
Vilmundur Gudnason ◽  
Lenore J. Launer ◽  
Mirjam I. Geerlings
Keyword(s):  
High Risk ◽  
General Population ◽  
Large Scale ◽  
Prediction Models ◽  
External Validation ◽  
Predictive Performance ◽  
Population Based ◽  
Cognitive Testing ◽  
Prognostic Models ◽  
Dementia Risk

AbstractWe aimed to evaluate the external performance of prediction models for all-cause dementia or AD in the general population, which can aid selection of high-risk individuals for clinical trials and prevention. We identified 17 out of 36 eligible published prognostic models for external validation in the population-based AGES-Reykjavik Study. Predictive performance was assessed with c statistics and calibration plots. All five models with a c statistic > .75 (.76–.81) contained cognitive testing as a predictor, while all models with lower c statistics (.67–.75) did not. Calibration ranged from good to poor across all models, including systematic risk overestimation or overestimation for particularly the highest risk group. Models that overestimate risk may be acceptable for exclusion purposes, but lack the ability to accurately identify individuals at higher dementia risk. Both updating existing models or developing new models aimed at identifying high-risk individuals, as well as more external validation studies of dementia prediction models are warranted.

scholarly journals Feasibility and Evaluation of a Large-Scale External Validation Approach for Patient-Level Prediction in an International Data Network: Validation of models predicting stroke in female patients newly diagnosed with atrial fibrillation.

2020 ◽  
Author(s):  
Jenna Marie Reps ◽  
Ross Williams ◽  
Seng Chan You ◽  
Thomas Falconer ◽  
Evan Minty ◽  
...  
Keyword(s):  
Atrial Fibrillation ◽  
Large Scale ◽  
Data Science ◽  
Prediction Models ◽  
External Validation ◽  
Scale Up ◽  
R Package ◽  
Prognostic Models ◽  
Healthcare Data ◽  
Patient Level

Abstract Objective: To demonstrate how the Observational Healthcare Data Science and Informatics (OHDSI) collaborative network and standardization can be utilized to scale-up external validation of patient-level prediction models by enabling validation across a large number of heterogeneous observational healthcare datasets.Materials & Methods: Five previously published prognostic models (ATRIA, CHADS2, CHADS2VASC, Q-Stroke and Framingham) that predict future risk of stroke in patients with atrial fibrillation were replicated using the OHDSI frameworks. A network study was run that enabled the five models to be externally validated across nine observational healthcare datasets spanning three countries and five independent sites. Results: The five existing models were able to be integrated into the OHDSI framework for patient-level prediction and they obtained mean c-statistics ranging between 0.57-0.63 across the 6 databases with sufficient data to predict stroke within 1 year of initial atrial fibrillation diagnosis for females with atrial fibrillation. This was comparable with existing validation studies. The validation network study was run across nine datasets within 60 days once the models were replicated. An R package for the study was published at https://github.com/OHDSI/StudyProtocolSandbox/tree/master/ExistingStrokeRiskExternalValidation.Discussion: This study demonstrates the ability to scale up external validation of patient-level prediction models using a collaboration of researchers and a data standardization that enable models to be readily shared across data sites. External validation is necessary to understand the transportability or reproducibility of a prediction model, but without collaborative approaches it can take three or more years for a model to be validated by one independent researcher. Conclusion : In this paper we show it is possible to both scale-up and speed-up external validation by showing how validation can be done across multiple databases in less than 2 months. We recommend that researchers developing new prediction models use the OHDSI network to externally validate their models.

scholarly journals Feasibility and Evaluation of a Large-Scale External Validation Approach for Patient-Level Prediction in an International Data Network: Validation of models predicting stroke in female patients newly diagnosed with atrial fibrillation.

2020 ◽  
Author(s):  
Jenna Marie Reps ◽  
Ross D Williams ◽  
Seng Chan You ◽  
Thomas Falconer ◽  
Evan Minty ◽  
...  
Keyword(s):  
Atrial Fibrillation ◽  
Large Scale ◽  
Data Science ◽  
Prediction Models ◽  
External Validation ◽  
Scale Up ◽  
R Package ◽  
Prognostic Models ◽  
Healthcare Data ◽  
Patient Level

Abstract Background: To demonstrate how the Observational Healthcare Data Science and Informatics (OHDSI) collaborative network and standardization can be utilized to scale-up external validation of patient-level prediction models by enabling validation across a large number of heterogeneous observational healthcare datasets.Methods: Five previously published prognostic models (ATRIA, CHADS2, CHADS2VASC, Q-Stroke and Framingham) that predict future risk of stroke in patients with atrial fibrillation were replicated using the OHDSI frameworks. A network study was run that enabled the five models to be externally validated across nine observational healthcare datasets spanning three countries and five independent sites. Results: The five existing models were able to be integrated into the OHDSI framework for patient-level prediction and they obtained mean c-statistics ranging between 0.57-0.63 across the 6 databases with sufficient data to predict stroke within 1 year of initial atrial fibrillation diagnosis for females with atrial fibrillation. This was comparable with existing validation studies. The validation network study was run across nine datasets within 60 days once the models were replicated. An R package for the study was published at https://github.com/OHDSI/StudyProtocolSandbox/tree/master/ExistingStrokeRiskExternalValidation.Conclusion : This study demonstrates the ability to scale up external validation of patient-level prediction models using a collaboration of researchers and a data standardization that enable models to be readily shared across data sites. External validation is necessary to understand the transportability or reproducibility of a prediction model, but without collaborative approaches it can take three or more years for a model to be validated by one independent researcher. In this paper we show it is possible to both scale-up and speed-up external validation by showing how validation can be done across multiple databases in less than 2 months. We recommend that researchers developing new prediction models use the OHDSI network to externally validate their models.

scholarly journals Feasibility and Evaluation of a Large-Scale External Validation Approach for Patient-Level Prediction in an International Data Network: Validation of models predicting stroke in female patients newly diagnosed with atrial fibrillation.

2020 ◽  
Author(s):  
Jenna Marie Reps ◽  
Ross D Williams ◽  
Seng Chan You ◽  
Thomas Falconer ◽  
Evan Minty ◽  
...  
Keyword(s):  
Atrial Fibrillation ◽  
Large Scale ◽  
Data Science ◽  
Prediction Models ◽  
External Validation ◽  
Scale Up ◽  
R Package ◽  
Prognostic Models ◽  
Healthcare Data ◽  
Patient Level

Abstract Background To demonstrate how the Observational Healthcare Data Science and Informatics (OHDSI) collaborative network and standardization can be utilized to scale-up external validation of patient-level prediction models by enabling validation across a large number of heterogeneous observational healthcare datasets.Methods Five previously published prognostic models (ATRIA, CHADS2, CHADS2VASC, Q-Stroke and Framingham) that predict future risk of stroke in patients with atrial fibrillation were replicated using the OHDSI frameworks. A network study was run that enabled the five models to be externally validated across nine observational healthcare datasets spanning three countries and five independent sites. Results The five existing models were able to be integrated into the OHDSI framework for patient-level prediction and they obtained mean c-statistics ranging between 0.57-0.63 across the 6 databases with sufficient data to predict stroke within 1 year of initial atrial fibrillation diagnosis for females with atrial fibrillation. This was comparable with existing validation studies. The validation network study was run across nine datasets within 60 days once the models were replicated. An R package for the study was published at https://github.com/OHDSI/StudyProtocolSandbox/tree/master/ExistingStrokeRiskExternalValidation .Conclusion This study demonstrates the ability to scale up external validation of patient-level prediction models using a collaboration of researchers and a data standardization that enable models to be readily shared across data sites. External validation is necessary to understand the transportability or reproducibility of a prediction model, but without collaborative approaches it can take three or more years for a model to be validated by one independent researcher. In this paper we show it is possible to both scale-up and speed-up external validation by showing how validation can be done across multiple databases in less than 2 months. We recommend that researchers developing new prediction models use the OHDSI network to externally validate their models.

scholarly journals Dementia prediction in the general population: External validation of 12 prognostic models in the AGES‐Reykjavik Study

2020 ◽  
Vol 16 (S5) ◽  
Author(s):  
Jet M.J. Vonk ◽  
Mirjam I. Geerlings ◽  
Vilmundur Gudnason ◽  
Lenore J. Launer ◽  
Jacoba P. Greving
Keyword(s):  
General Population ◽  
External Validation ◽  
Prognostic Models

scholarly journals Feasibility and Evaluation of a Large-Scale External Validation Approach for Patient-Level Prediction in an International Data Network

2019 ◽  
Author(s):  
Jenna Marie Reps ◽  
Ross Williams ◽  
Seng Chan You ◽  
Thomas Falconer ◽  
Evan Minty ◽  
...  
Keyword(s):  
Atrial Fibrillation ◽  
Large Scale ◽  
Data Science ◽  
Prediction Models ◽  
External Validation ◽  
Scale Up ◽  
R Package ◽  
Prognostic Models ◽  
Healthcare Data ◽  
Patient Level

Abstract Objective To demonstrate how the Observational Healthcare Data Science and Informatics (OHDSI) collaborative network and standardization can be utilized to externally validate patient-level prediction models at scale. Materials & Methods Five previously published prognostic models (ATRIA, CHADS2, CHADS2VASC, Q-Stroke and Framingham) that predict future risk of stroke in patients with atrial fibrillation were replicated using the OHDSI frameworks and a network study was run that enabled the five models to be externally validated across nine datasets spanning three countries and five independent sites. Results The five existing models were able to be integrated into the OHDSI framework for patient-level prediction and their performances in predicting stroke within 1 year of initial atrial fibrillation diagnosis for females were comparable with existing studies. The validation network study took 60 days once the models were replicated and an R package for the study was published to collaborators at https://github.com/OHDSI/StudyProtocolSandbox/tree/master/ExistingStrokeRiskExternalValidation. Discussion This study demonstrates the ability to scale up external validation of patient-level prediction models using a collaboration of researchers and data standardization that enable models to be readily shared across data sites. External validation is necessary to understand the transportability and reproducibility of prediction models, but without collaborative approaches it can take three or more years to be validated by one independent researcher. Conclusion In this paper we show it is possible to both scale-up and speed-up external validation by showing how validation can be done across multiple databases in less than 2 months.

scholarly journals Introducing Twitter Daily Estimates of Residents and Non-Residents at the County Level

2021 ◽  
Vol 10 (6) ◽  
pp. 227
Author(s):  
Yago Martín ◽  
Zhenlong Li ◽  
Yue Ge ◽  
Xiao Huang
Keyword(s):  
Large Scale ◽  
Local Population ◽  
External Validation ◽  
Population Estimates ◽  
County Level ◽  
Internal Validation ◽  
Population Movements ◽  
Dynamic Component ◽  
Home Location ◽  
Non Local

The study of migrations and mobility has historically been severely limited by the absence of reliable data or the temporal sparsity of available data. Using geospatial digital trace data, the study of population movements can be much more precisely and dynamically measured. Our research seeks to develop a near real-time (one-day lag) Twitter census that gives a more temporally granular picture of local and non-local population at the county level. Internal validation reveals over 80% accuracy when compared with users’ self-reported home location. External validation results suggest these stocks correlate with available statistics of residents/non-residents at the county level and can accurately reflect regular (seasonal tourism) and non-regular events such as the Great American Solar Eclipse of 2017. The findings demonstrate that Twitter holds the potential to introduce the dynamic component often lacking in population estimates. This study could potentially benefit various fields such as demography, tourism, emergency management, and public health and create new opportunities for large-scale mobility analyses.

scholarly journals External Validation of Surrogate Indices of Fatty Liver in the General Population: The Bagnacavallo Study

2021 ◽  
Vol 10 (3) ◽  
pp. 520
Author(s):  
Francesco Foschi ◽  
Fabio Conti ◽  
Marco Domenicali ◽  
Pierluigi Giacomoni ◽  
Alberto Borghi ◽  
...  
Keyword(s):  
General Population ◽  
Fatty Liver ◽  
External Validation ◽  
Strongly Correlated ◽  
Alcoholic Fatty Liver ◽  
Fatty Liver Index ◽  
C Statistic ◽  
Hepatic Steatosis Index ◽  
The Town ◽  
Small Advantage

We externally validated the fatty liver index (FLI), the lipid accumulation product (LAP), the hepatic steatosis index (HSI), and the Zhejiang University index (ZJU) for the diagnosis of fatty liver (FL) and non-alcoholic fatty liver disease (NAFLD) in the general population. The validation was performed on 2159 citizens of the town of Bagnacavallo (Ravenna, Italy). Calibration was evaluated by calculating the calibration slope and intercept and by inspecting calibration plots; discrimination was evaluated using the c-statistic. The average calibration slope was 1 and the average intercept was 0 for all combinations of outcomes and indices. For the diagnosis of FL, the c-statistic was 0.85 for FLI, 0.83 for ZJU, 0.82 for HSI, and 0.80 for LAP; for the diagnosis of NAFLD, the c-statistic was 0.77 for FLI, 0.76 for ZJU, 0.75 for HSI, and 0.74 for LAP. All indices were strongly correlated with each other. In conclusion, FLI, LAP, HSI, and ZJU perform similarly well to diagnose FL and NAFLD in the Bagnacavallo population, even if FLI has a small advantage as discrimination is concerned.

Sign in / Sign up

Export Citation Format

Share Document