scholarly journals A modular approach to integrating multiple data sources into real-time clinical prediction for pediatric diarrhea

2020 ◽  
Author(s):  
Ben J. Brintz ◽  
Benjamin Haaland ◽  
Joel Howard ◽  
Dennis L. Chao ◽  
Joshua L. Proctor ◽  
...  
Keyword(s):  
Real Time ◽  
Prediction Models ◽  
Predictive Performance ◽  
Data Availability ◽  
Clinical Prediction ◽  
Multiple Sources ◽  
Statistical Framework ◽  
Multiple Data ◽  
Post Test ◽  
The Individual

AbstractTraditional clinical prediction models focus on parameters of the individual patient. For infectious diseases, sources external to the patient, including characteristics of prior patients and seasonal factors, may improve predictive performance. We describe the development of a predictive model that integrates multiple sources of data in a principled statistical framework using a post-test odds formulation. Our method enables electronic real-time updating and flexibility, such that components can be included or excluded according to data availability. We apply this method to the prediction of etiology of pediatric diarrhea, where “pre-test” epidemiologic data may be highly informative. Diarrhea has a high burden in low-resource settings, and antibiotics are often over-prescribed. We demonstrate that our integrative method outperforms traditional prediction in accurately identifying cases with a viral etiology, and show that its clinical application, especially when used with an additional diagnostic test, could result in a 61% reduction in inappropriately prescribed antibiotics.

scholarly journals A modular approach to integrating multiple data sources into real-time clinical prediction for pediatric diarrhea

eLife ◽  
2021 ◽  
Vol 10 ◽  
Author(s):  
Ben J Brintz ◽  
Benjamin Haaland ◽  
Joel Howard ◽  
Dennis L Chao ◽  
Joshua L Proctor ◽  
...  
Keyword(s):  
Real Time ◽  
Prediction Models ◽  
Predictive Performance ◽  
Data Availability ◽  
Clinical Prediction ◽  
Multiple Sources ◽  
Statistical Framework ◽  
Multiple Data ◽  
Post Test ◽  
The Individual

Traditional clinical prediction models focus on parameters of the individual patient. For infectious diseases, sources external to the patient, including characteristics of prior patients and seasonal factors, may improve predictive performance. We describe the development of a predictive model that integrates multiple sources of data in a principled statistical framework using a post-test odds formulation. Our method enables electronic real-time updating and flexibility, such that components can be included or excluded according to data availability. We apply this method to the prediction of etiology of pediatric diarrhea, where 'pre-test’ epidemiologic data may be highly informative. Diarrhea has a high burden in low-resource settings, and antibiotics are often over-prescribed. We demonstrate that our integrative method outperforms traditional prediction in accurately identifying cases with a viral etiology, and show that its clinical application, especially when used with an additional diagnostic test, could result in a 61% reduction in inappropriately prescribed antibiotics.

scholarly journals Models Predicting Hospital Admission of Adult Patients Utilizing Prehospital Data: Systematic Review Using PROBAST and CHARMS

10.2196/30022 ◽  
2021 ◽  
Vol 9 (9) ◽  
pp. e30022
Author(s):  
Ann Corneille Monahan ◽  
Sue S Feldman
Keyword(s):  
Emergency Department ◽  
Hospital Admission ◽  
Real Time ◽  
Prediction Models ◽  
Assessment Tool ◽  
Predictive Performance ◽  
Patient Data ◽  
Adult Patients ◽  
Emergency Department Crowding ◽  
Exit Block

Background Emergency department boarding and hospital exit block are primary causes of emergency department crowding and have been conclusively associated with poor patient outcomes and major threats to patient safety. Boarding occurs when a patient is delayed or blocked from transitioning out of the emergency department because of dysfunctional transition or bed assignment processes. Predictive models for estimating the probability of an occurrence of this type could be useful in reducing or preventing emergency department boarding and hospital exit block, to reduce emergency department crowding. Objective The aim of this study was to identify and appraise the predictive performance, predictor utility, model application, and model utility of hospital admission prediction models that utilized prehospital, adult patient data and aimed to address emergency department crowding. Methods We searched multiple databases for studies, from inception to September 30, 2019, that evaluated models predicting adult patients’ imminent hospital admission, with prehospital patient data and regression analysis. We used PROBAST (Prediction Model Risk of Bias Assessment Tool) and CHARMS (Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies) to critically assess studies. Results Potential biases were found in most studies, which suggested that each model’s predictive performance required further investigation. We found that select prehospital patient data contribute to the identification of patients requiring hospital admission. Biomarker predictors may add superior value and advantages to models. It is, however, important to note that no models had been integrated with an information system or workflow, operated independently as electronic devices, or operated in real time within the care environment. Several models could be used at the site-of-care in real time without digital devices, which would make them suitable for low-technology or no-electricity environments. Conclusions There is incredible potential for prehospital admission prediction models to improve patient care and hospital operations. Patient data can be utilized to act as predictors and as data-driven, actionable tools to identify patients likely to require imminent hospital admission and reduce patient boarding and crowding in emergency departments. Prediction models can be used to justify earlier patient admission and care, to lower morbidity and mortality, and models that utilize biomarker predictors offer additional advantages.

scholarly journals Outcome Prediction Models for Endovascular Treatment of Ischemic Stroke: Systematic Review and External Validation

Stroke ◽  
2021 ◽  
Author(s):  
Femke Kremers ◽  
Esmee Venema ◽  
Martijne Duvekot ◽  
Lonneke Yo ◽  
Reinoud Bokkers ◽  
...  
Keyword(s):  
Ischemic Stroke ◽  
The Netherlands ◽  
Endovascular Treatment ◽  
Acute Ischemic Stroke ◽  
Functional Outcome ◽  
Prediction Models ◽  
Predictive Performance ◽  
Data Availability ◽  
Vascular Events ◽  
Discriminative Performance

Background and Purpose: Prediction models for outcome of patients with acute ischemic stroke who will undergo endovascular treatment have been developed to improve patient management. The aim of the current study is to provide an overview of preintervention models for functional outcome after endovascular treatment and to validate these models with data from daily clinical practice. Methods: We systematically searched within Medline, Embase, Cochrane, Web of Science, to include prediction models. Models identified from the search were validated in the MR CLEAN (Multicenter Randomized Clinical Trial of Endovascular Treatment for Acute Ischemic Stroke in the Netherlands) registry, which includes all patients treated with endovascular treatment within 6.5 hours after stroke onset in the Netherlands between March 2014 and November 2017. Predictive performance was evaluated according to discrimination (area under the curve) and calibration (slope and intercept of the calibration curve). Good functional outcome was defined as a score of 0–2 or 0–3 on the modified Rankin Scale depending on the model. Results: After screening 3468 publications, 19 models were included in this validation. Variables included in the models mainly addressed clinical and imaging characteristics at baseline. In the validation cohort of 3156 patients, discriminative performance ranged from 0.61 (SPAN-100 [Stroke Prognostication Using Age and NIH Stroke Scale]) to 0.80 (MR PREDICTS). Best-calibrated models were THRIVE (The Totaled Health Risks in Vascular Events; intercept −0.06 [95% CI, −0.14 to 0.02]; slope 0.84 [95% CI, 0.75–0.95]), THRIVE-c (intercept 0.08 [95% CI, −0.02 to 0.17]; slope 0.71 [95% CI, 0.65–0.77]), Stroke Checkerboard score (intercept −0.05 [95% CI, −0.13 to 0.03]; slope 0.97 [95% CI, 0.88–1.08]), and MR PREDICTS (intercept 0.43 [95% CI, 0.33–0.52]; slope 0.93 [95% CI, 0.85–1.01]). Conclusions: The THRIVE-c score and MR PREDICTS both showed a good combination of discrimination and calibration and were, therefore, superior in predicting functional outcome for patients with ischemic stroke after endovascular treatment within 6.5 hours. Since models used different predictors and several models had relatively good predictive performance, the decision on which model to use in practice may also depend on simplicity of the model, data availability, and the comparability of the population and setting.

scholarly journals Advanced cardiovascular risk prediction in the emergency department: updating a clinical prediction model – a large database study protocol

2021 ◽  
Vol 5 (1) ◽  
Author(s):  
Charles Reynard ◽  
Glen P. Martin ◽  
Evangelos Kontopantelis ◽  
David A. Jenkins ◽  
Anthony Heagerty ◽  
...  
Keyword(s):  
Myocardial Infarction ◽  
Acute Myocardial Infarction ◽  
Prediction Models ◽  
Predictive Performance ◽  
Modern Medicine ◽  
Clinical Prediction ◽  
Electronic Data ◽  
Cardiovascular Risk Prediction ◽  
Predicted Probability ◽  
Over Time

Abstract Background Patients presenting with chest pain represent a large proportion of attendances to emergency departments. In these patients clinicians often consider the diagnosis of acute myocardial infarction (AMI), the timely recognition and treatment of which is clinically important. Clinical prediction models (CPMs) have been used to enhance early diagnosis of AMI. The Troponin-only Manchester Acute Coronary Syndromes (T-MACS) decision aid is currently in clinical use across Greater Manchester. CPMs have been shown to deteriorate over time through calibration drift. We aim to assess potential calibration drift with T-MACS and compare methods for updating the model. Methods We will use routinely collected electronic data from patients who were treated using TMACS at two large NHS hospitals. This is estimated to include approximately 14,000 patient episodes spanning June 2016 to October 2020. The primary outcome of acute myocardial infarction will be sourced from NHS Digital’s admitted patient care dataset. We will assess the calibration drift of the existing model and the benefit of updating the CPM by model recalibration, model extension and dynamic updating. These models will be validated by bootstrapping and one step ahead prequential testing. We will evaluate predictive performance using calibrations plots and c-statistics. We will also examine the reclassification of predicted probability with the updated TMACS model. Discussion CPMs are widely used in modern medicine, but are vulnerable to deteriorating calibration over time. Ongoing refinement using routinely collected electronic data will inevitably be more efficient than deriving and validating new models. In this analysis we will seek to exemplify methods for updating CPMs to protect the initial investment of time and effort. If successful, the updating methods could be used to continually refine the algorithm used within TMACS, maintaining or even improving predictive performance over time. Trial registration ISRCTN number: ISRCTN41008456

scholarly journals Quantitative longitudinal predictions of Alzheimer's disease by multi-modal predictive learning

2020 ◽  
Author(s):  
Mithilesh Prakash ◽  
Mahmoud Abdelaziz ◽  
Linda Zhang ◽  
Bryan A Strange ◽  
Jussi Tohka
Keyword(s):  
Alzheimer’S Disease ◽  
Alzheimer's Disease ◽  
Prediction Models ◽  
Predictive Performance ◽  
Absolute Error ◽  
Disease Stage ◽  
Disease Category ◽  
Modal Data ◽  
Predictive Learning ◽  
The Individual

Abstract Background: Quantitatively predicting the progression of Alzheimer’s disease (AD) in an individual on a continuous scale, such as AD assessment scale-cognitive (ADAS-cog) scores, is informative for a personalized approach as opposed to qualitatively classifying the individual into a broad disease category. We hypothesize that multi-modal data and predictive learning models can be employed for longitudinally predicting ADAS-cog scores.Methods: Multivariate regression techniques were employed to model baseline multi-modal data (demographics, neuroimaging, and cerebrospinal fluid based markers, and genetic factors) and future ADAS-cog scores. Prediction models were subjected to repeated cross-validation and the resulting mean absolute error and cross-validated correlation of the model assessed.Results: Prediction models on multi-modal data outperformed single modal data up to 36 months. Incorporating baseline ADAS-cog scores to prediction models marginally improved predictive performance.Conclusions: Future ADAS-cog scores were successfully estimated via predictive learning aiding clinicians in identifying those at greater risk of decline and apply interventions at an earlier disease stage and inform likely future disease progression in individuals enrolled in AD clinical trials.

scholarly journals Effectiveness of a Novel HIV Self-Testing Service with Online Real-Time Counseling Support (HIVST-Online) in Increasing HIV Testing Rate and Repeated HIV Testing among Men Who Have Sex with Men in Hong Kong: Results of a Pilot Implementation Project

2021 ◽  
Vol 18 (2) ◽  
pp. 729
Author(s):  
Paul Shing-fong Chan ◽  
Andrew Chidgey ◽  
Jason Lau ◽  
Mary Ip ◽  
Joseph T.F. Lau ◽  
...  
Keyword(s):  
Hiv Testing ◽  
Real Time ◽  
Linkage To Care ◽  
Standard Of Care ◽  
Multiple Sources ◽  
Self Testing ◽  
Post Test ◽  
Pilot Implementation ◽  
Sex With Men ◽  
Testing Coverage

HIV self-testing (HIVST) with online real-time counseling (HIVST-online) is an evidence-based intervention to increase HIV testing coverage and to ensure linkage to care for men who have sex with men (MSM). A community-based organization (CBO) recruited 122 MSM who had ever used HIVST-online (ever-users) and another 228 new-users from multiple sources and promoted HIVST-online. A free oral fluid-based HIVST kit was sent to all the participants by mail. Experienced HIVST administrators implemented HIVST-online by providing real-time instruction, standard-of-care pre-test and post-test counseling via live-chat application. The number of HIVST-online sessions performed was documented by the administrators. The post-test evaluation was conducted 6 months after the pre-test survey. At month 6, 63.1% of ever-users and 40.4% of new-users received HIVST-online. Taking other types of HIV testing into account, 79.4% of ever-users and 58.6% of new-users being followed up at month 6 received any HIV testing during the project period. Ever-users were more likely to receive HIVST-online and any HIV testing as compared to new-users. Four HIVST-online users were screened to be HIV positive and linked to the treatment. The process evaluation of HIVST-online was positive. Implementation of HIVST-online was helpful to improve HIV testing coverage and repeated HIV testing among Chinese MSM. A larger scale implementation should be considered.

scholarly journals Quantitative longitudinal predictions of Alzheimer’s disease by multi-modal predictive learning

2020 ◽  
Author(s):  
M. Prakash ◽  
M. Abdelaziz ◽  
L. Zhang ◽  
B.A. Strange ◽  
J. Tohka ◽  
...  
Keyword(s):  
Alzheimer’S Disease ◽  
Alzheimer's Disease ◽  
Prediction Models ◽  
Predictive Performance ◽  
Absolute Error ◽  
Disease Stage ◽  
Disease Category ◽  
Modal Data ◽  
Predictive Learning ◽  
The Individual

AbstractBackgroundQuantitatively predicting the progression of Alzheimer’s disease (AD) in an individual on a continuous scale, such as AD assessment scale-cognitive (ADAS-cog) scores, is informative for a personalized approach as opposed to qualitatively classifying the individual into a broad disease category. We hypothesize that multi-modal data and predictive learning models can be employed for longitudinally predicting ADAS-cog scores.MethodsMultivariate regression techniques were employed to model baseline multi-modal data (demographics, neuroimaging, and cerebrospinal fluid based markers, and genetic factors) and future ADAS-cog scores. Prediction models were subjected to repeated cross-validation and the resulting mean absolute error and cross-validated correlation of the model assessed.ResultsPrediction models on multi-modal data outperformed single modal data up to 36 months. Incorporating baseline ADAS-cog scores to prediction models marginally improved predictive performance.ConclusionsFuture ADAS-cog scores were successfully estimated via predictive learning aiding clinicians in identifying those at greater risk of decline and apply interventions at an earlier disease stage and inform likely future disease progression in individuals enrolled in AD clinical trials.

scholarly journals Assessing prognosis and prediction of treatment response in early rheumatoid arthritis: systematic reviews

2018 ◽  
Vol 22 (66) ◽  
pp. 1-294 ◽  
Author(s):  
Rachel Archer ◽  
Emma Hock ◽  
Jean Hamilton ◽  
John Stevens ◽  
Munira Essat ◽  
...  
Keyword(s):  
Rheumatoid Arthritis ◽  
Prediction Models ◽  
Meta Analysis ◽  
External Validation ◽  
Model Development ◽  
Predictive Performance ◽  
Assessment Tools ◽  
Clinical Prediction ◽  
Clinical Prediction Models ◽  
Early Ra

Background Rheumatoid arthritis (RA) is a chronic, debilitating disease associated with reduced quality of life and substantial costs. It is unclear which tests and assessment tools allow the best assessment of prognosis in people with early RA and whether or not variables predict the response of patients to different drug treatments. Objective To systematically review evidence on the use of selected tests and assessment tools in patients with early RA (1) in the evaluation of a prognosis (review 1) and (2) as predictive markers of treatment response (review 2). Data sources Electronic databases (e.g. MEDLINE, EMBASE, The Cochrane Library, Web of Science Conference Proceedings; searched to September 2016), registers, key websites, hand-searching of reference lists of included studies and key systematic reviews and contact with experts. Study selection Review 1 – primary studies on the development, external validation and impact of clinical prediction models for selected outcomes in adult early RA patients. Review 2 – primary studies on the interaction between selected baseline covariates and treatment (conventional and biological disease-modifying antirheumatic drugs) on salient outcomes in adult early RA patients. Results Review 1 – 22 model development studies and one combined model development/external validation study reporting 39 clinical prediction models were included. Five external validation studies evaluating eight clinical prediction models for radiographic joint damage were also included. c-statistics from internal validation ranged from 0.63 to 0.87 for radiographic progression (different definitions, six studies) and 0.78 to 0.82 for the Health Assessment Questionnaire (HAQ). Predictive performance in external validations varied considerably. Three models [(1) Active controlled Study of Patients receiving Infliximab for the treatment of Rheumatoid arthritis of Early onset (ASPIRE) C-reactive protein (ASPIRE CRP), (2) ASPIRE erythrocyte sedimentation rate (ASPIRE ESR) and (3) Behandelings Strategie (BeSt)] were externally validated using the same outcome definition in more than one population. Results of the random-effects meta-analysis suggested substantial uncertainty in the expected predictive performance of models in a new sample of patients. Review 2 – 12 studies were identified. Covariates examined included anti-citrullinated protein/peptide anti-body (ACPA) status, smoking status, erosions, rheumatoid factor status, C-reactive protein level, erythrocyte sedimentation rate, swollen joint count (SJC), body mass index and vascularity of synovium on power Doppler ultrasound (PDUS). Outcomes examined included erosions/radiographic progression, disease activity, physical function and Disease Activity Score-28 remission. There was statistical evidence to suggest that ACPA status, SJC and PDUS status at baseline may be treatment effect modifiers, but not necessarily that they are prognostic of response for all treatments. Most of the results were subject to considerable uncertainty and were not statistically significant. Limitations The meta-analysis in review 1 was limited by the availability of only a small number of external validation studies. Studies rarely investigated the interaction between predictors and treatment. Suggested research priorities Collaborative research (including the use of individual participant data) is needed to further develop and externally validate the clinical prediction models. The clinical prediction models should be validated with respect to individual treatments. Future assessments of treatment by covariate interactions should follow good statistical practice. Conclusions Review 1 – uncertainty remains over the optimal prediction model(s) for use in clinical practice. Review 2 – in general, there was insufficient evidence that the effect of treatment depended on baseline characteristics. Study registration This study is registered as PROSPERO CRD42016042402. Funding The National Institute for Health Research Health Technology Assessment programme.

scholarly journals Models Predicting Hospital Admission of Adult Patients Utilizing Prehospital Data: Systematic Review Using PROBAST and CHARMS (Preprint)

2021 ◽  
Author(s):  
Ann Corneille Monahan ◽  
Sue S Feldman
Keyword(s):  
Emergency Department ◽  
Hospital Admission ◽  
Real Time ◽  
Prediction Models ◽  
Assessment Tool ◽  
Predictive Performance ◽  
Patient Data ◽  
Adult Patients ◽  
Emergency Department Crowding ◽  
Exit Block

BACKGROUND Emergency department boarding and hospital exit block are primary causes of emergency department crowding and have been conclusively associated with poor patient outcomes and major threats to patient safety. Boarding occurs when a patient is delayed or blocked from transitioning out of the emergency department because of dysfunctional transition or bed assignment processes. Predictive models for estimating the probability of an occurrence of this type could be useful in reducing or preventing emergency department boarding and hospital exit block, to reduce emergency department crowding. OBJECTIVE The aim of this study was to identify and appraise the predictive performance, predictor utility, model application, and model utility of hospital admission prediction models that utilized prehospital, adult patient data and aimed to address emergency department crowding. METHODS We searched multiple databases for studies, from inception to September 30, 2019, that evaluated models predicting adult patients’ imminent hospital admission, with prehospital patient data and regression analysis. We used PROBAST (Prediction Model Risk of Bias Assessment Tool) and CHARMS (Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies) to critically assess studies. RESULTS Potential biases were found in most studies, which suggested that each model’s predictive performance required further investigation. We found that select prehospital patient data contribute to the identification of patients requiring hospital admission. Biomarker predictors may add superior value and advantages to models. It is, however, important to note that no models had been integrated with an information system or workflow, operated independently as electronic devices, or operated in real time within the care environment. Several models could be used at the site-of-care in real time without digital devices, which would make them suitable for low-technology or no-electricity environments. CONCLUSIONS There is incredible potential for prehospital admission prediction models to improve patient care and hospital operations. Patient data can be utilized to act as predictors and as data-driven, actionable tools to identify patients likely to require imminent hospital admission and reduce patient boarding and crowding in emergency departments. Prediction models can be used to justify earlier patient admission and care, to lower morbidity and mortality, and models that utilize biomarker predictors offer additional advantages.

scholarly journals Developing clinical prediction models when adhering to minimum sample size recommendations: The importance of quantifying bootstrap variability in tuning parameters and predictive performance

2021 ◽  
pp. 096228022110463
Author(s):  
Glen P Martin ◽  
Richard D Riley ◽  
Gary S Collins ◽  
Matthew Sperrin
Keyword(s):  
Sample Size ◽  
Prediction Models ◽  
Model Development ◽  
Likelihood Estimation ◽  
Predictive Performance ◽  
Minimum Sample Size ◽  
Clinical Prediction ◽  
Clinical Prediction Models ◽  
Tuning Parameters ◽  
Minimum Sample

Recent minimum sample size formula (Riley et al.) for developing clinical prediction models help ensure that development datasets are of sufficient size to minimise overfitting. While these criteria are known to avoid excessive overfitting on average, the extent of variability in overfitting at recommended sample sizes is unknown. We investigated this through a simulation study and empirical example to develop logistic regression clinical prediction models using unpenalised maximum likelihood estimation, and various post-estimation shrinkage or penalisation methods. While the mean calibration slope was close to the ideal value of one for all methods, penalisation further reduced the level of overfitting, on average, compared to unpenalised methods. This came at the cost of higher variability in predictive performance for penalisation methods in external data. We recommend that penalisation methods are used in data that meet, or surpass, minimum sample size requirements to further mitigate overfitting, and that the variability in predictive performance and any tuning parameters should always be examined as part of the model development process, since this provides additional information over average (optimism-adjusted) performance alone. Lower variability would give reassurance that the developed clinical prediction model will perform well in new individuals from the same population as was used for model development.

Sign in / Sign up

Export Citation Format

Share Document