scholarly journals A tutorial on calibration measurements and calibration models for clinical prediction models

2020 ◽  
Vol 27 (4) ◽  
pp. 621-633 ◽  
Author(s):  
Yingxiang Huang ◽  
Wentao Li ◽  
Fima Macheret ◽  
Rodney A Gabriel ◽  
Lucila Ohno-Machado
Keyword(s):  
Predictive Model ◽  
Predictive Models ◽  
Prediction Models ◽  
Model Performance ◽  
Simulated Data ◽  
Primary Objective ◽  
Adequate Model ◽  
Starting Point ◽  
Calibration Models ◽  
Clinical Predictive Model

Abstract Our primary objective is to provide the clinical informatics community with an introductory tutorial on calibration measurements and calibration models for predictive models using existing R packages and custom implemented code in R on real and simulated data. Clinical predictive model performance is commonly published based on discrimination measures, but use of models for individualized predictions requires adequate model calibration. This tutorial is intended for clinical researchers who want to evaluate predictive models in terms of their applicability to a particular population. It is also for informaticians and for software engineers who want to understand the role that calibration plays in the evaluation of a clinical predictive model, and to provide them with a solid starting point to consider incorporating calibration evaluation and calibration models in their work. Covered topics include (1) an introduction to the importance of calibration in the clinical setting, (2) an illustration of the distinct roles that discrimination and calibration play in the assessment of clinical predictive models, (3) a tutorial and demonstration of selected calibration measurements, (4) a tutorial and demonstration of selected calibration models, and (5) a brief discussion of limitations of these methods and practical suggestions on how to use them in practice.

Bootstrap internal validation command for predictive logistic regression models

2021 ◽  
Vol 21 (2) ◽  
pp. 498-509
Author(s):  
B. M. Fernandez-Felix ◽  
E. García-Esquinas ◽  
A. Muriel ◽  
A. Royuela ◽  
J. Zamora
Keyword(s):  
Logistic Regression ◽  
Regression Model ◽  
Predictive Model ◽  
Performance Measures ◽  
Predictive Models ◽  
Regression Models ◽  
Logistic Regression Model ◽  
Model Performance ◽  
Internal Validation ◽  
Logistic Regression Models

Overfitting is a common problem in the development of predictive models. It leads to an optimistic estimation of apparent model performance. Internal validation using bootstrapping techniques allows one to quantify the optimism of a predictive model and provide a more realistic estimate of its performance measures. Our objective is to build an easy-to-use command, bsvalidation, aimed to perform a bootstrap internal validation of a logistic regression model.

scholarly journals Development and Validation of Diagnostic Models for Hand-Foot-and-Mouth Disease in Children

2021 ◽  
Vol 2021 ◽  
pp. 1-9
Author(s):  
Feng Zhuo ◽  
Mengjie Yu ◽  
Qiang Chen ◽  
Nuoya Li ◽  
Li Luo ◽  
...  
Keyword(s):  
Differential Diagnosis ◽  
Predictive Model ◽  
Predictive Models ◽  
Prediction Models ◽  
Multivariate Logistic Regression Analysis ◽  
Foot And Mouth Disease ◽  
Mouth Disease ◽  
Risk Markers ◽  
Failure Group ◽  
Foot And Mouth

Objective. To find risk markers and develop new clinical predictive models for the differential diagnosis of hand-foot-and-mouth disease (HFMD) with varying degrees of disease. Methods. 19766 children with HFMD and 64 clinical indexes were included in this study. The patients included in this study were divided into the mild patients’ group (mild) with 12292 cases, severe patients’ group (severe) with 6508 cases, and severe patients with respiratory failure group (severe-RF) with 966 cases. Single-factor analysis was carried out on 64 indexes collected from patients when they were admitted to the hospital, and the indexes with statistical differences were selected as the prediction factors. Binary multivariate logistic regression analysis was used to construct the prediction models and calculate the adjusted odds ratio (OR). Results. SP, DP, NEUT#, NEUT%, RDW-SD, RDW-CV, GGT, CK/CK-MB, and Glu were risk markers in mild/severe, mild/severe-RF, and severe/severe-RF. Glu was a diagnostic marker for mild/severe-RF ( AUROC = 0.80 , 95% CI: 0.78-0.82); the predictive model constructed by temperature, SP, MOMO%, EO%, RDW-SD, GLB, CRP, Glu, BUN, and Cl could be used for the differential diagnosis of mild/severe ( AUROC > 0.84 ); the predictive model constructed by SP, age, NEUT#, PCT, TBIL, GGT, Mb, β2MG, Glu, and Ca could be used for the differential diagnosis of severe/severe-RF ( AUROC > 0.76 ). Conclusion. By analyzing clinical indicators, we have found the risk markers of HFMD and established suitable predictive models.

scholarly journals A Clinical Predictive Model of Progesterone Elevation on the Day of HCG Administration Was Establish Based on the Basal P, BMI and Ovarian Induction Regimen

2021 ◽  
Author(s):  
chunmei yu ◽  
chao zhou ◽  
xiuliang dai ◽  
jianmei zhou ◽  
haiyan yang ◽  
...  
Keyword(s):  
Predictive Model ◽  
Prediction Models ◽  
Graphical Representation ◽  
Area Under The Curve ◽  
Care Hospital ◽  
Fresh Embryo Transfer ◽  
Ovarian Induction ◽  
Clinical Predictive Model ◽  
Infertility Patients ◽  
User Friendly

Abstract Background The progesterone elevation (PE) on the day of human chorionic gonadotrophin (hCG) is associated with a significant decrease in the probability of clinical pregnancy after fresh embryo transfer. The goal of this study was to develop a nomogram to explore the baseline indexes to predict the occurrence of PE (0.9ng/ml) on the day of HCG administration before IVF/ICSI treatment.Methods The patients who were performed a GnRH agonist or antagonist pituitary suppression protocol during controlled ovulation stimulation(COS) in reproductive center of Changzhou maternal and child health care hospital from 2017-2019 were included. The nomogram was built from all participants.Results Three variates significantly associated with the PE occurrence on the HCG administration of infertility women were the type ovulation regiment, basal P and BMI. This predictive model showed good calibration and discriminatory abilities, with an area under the curve (AUC) of 0.639（95%CI 0.596~0.681）. Hosmer and Leme show test were performed for evaluating both calibration and discrimination and confirmed our nomogram is a user-friendly graphical representation of the model (χ2= 4.750，P=0.784)Conclusion The nomogram presents graphically association factors (type ovulation regiment, basal P and BMI) and prediction models which can offer simple and useful guidance to avoid the PE occurrence on the HCG administration for clinicians and infertility patients.

Study of the route correction system for the on-board navigation system of an unmanned aerial vehicle on the grounds of radar images of the terrain

2020 ◽  
pp. 129-134
Keyword(s):  
Predictive Model ◽  
Unmanned Aerial Vehicle ◽  
Predictive Models ◽  
Navigation System ◽  
Error Compensation ◽  
Compensation Scheme ◽  
Radar Images ◽  
Aerial Vehicle ◽  
System Errors ◽  
Correction System

The system of route correction of an unmanned aerial vehicle (UAV) is considered. For the route correction the on-board radar complex is used. In conditions of active interference, it is impossible to use radar images for the route correction so it is proposed to use the on-board navigation system with algorithmic correction. An error compensation scheme of the navigation system in the output signal using the algorithm for constructing a predictive model of the system errors is applied. The predictive model is building using the genetic algorithm and the method of group accounting of arguments. The quality comparison of the algorithms for constructing predictive models is carried out using mathematical modeling.

A Historical Data based Ensemble System for Efficient Stock Price Prediction

2019 ◽  
Vol 12 ◽  
Author(s):  
Vijay Kumar Dwivedi ◽  
Manoj Madhava Gore
Keyword(s):  
Stock Price ◽  
Prediction Models ◽  
Historical Data ◽  
Stock Exchange ◽  
Primary Objective ◽  
Lasso Regression ◽  
Stock Price Prediction ◽  
Price Prediction ◽  
Consistent Manner ◽  
Selection Operator

Background: Stock price prediction is a challenging task. The social, economic, political, and various other factors cause frequent abrupt changes in the stock price. This article proposes a historical data-based ensemble system to predict the closing stock price with higher accuracy and consistency over the existing stock price prediction systems. Objective: The primary objective of this article is to predict the closing price of a stock for the next trading in more accurate and consistent manner over the existing methods employed for the stock price prediction. Method: The proposed system combines various machine learning-based prediction models employing least absolute shrinkage and selection operator (LASSO) regression regularization technique to enhance the accuracy of stock price prediction system as compared to any one of the base prediction models. Results: The analysis of results for all the eleven stocks (listed under Information Technology sector on the Bombay Stock Exchange, India) reveals that the proposed system performs best (on all defined metrics of the proposed system) for training datasets and test datasets comprising of all the stocks considered in the proposed system. Conclusion: The proposed ensemble model consistently predicts stock price with a high degree of accuracy over the existing methods used for the prediction.

Evaluation of predictive model performance of an existing model in the presence of missing data

2021 ◽  
Author(s):  
Pin Li ◽  
Jeremy M. G. Taylor ◽  
Daniel E. Spratt ◽  
R. Jeffery Karnes ◽  
Matthew J. Schipper
Keyword(s):  
Missing Data ◽  
Predictive Model ◽  
Model Performance

scholarly journals Prediction of COVID-19 severity using laboratory findings on admission: informative values, thresholds, ML model performance

BMJ Open ◽  
2021 ◽  
Vol 11 (2) ◽  
pp. e044500
Author(s):  
Yauhen Statsenko ◽  
Fatmah Al Zahmi ◽  
Tetiana Habuza ◽  
Klaus Neidl-Van Gorkom ◽  
Nazar Zaki
Keyword(s):  
Lymphocyte Count ◽  
Prediction Models ◽  
Model Performance ◽  
Area Under The Curve ◽  
Predictive Biomarkers ◽  
Supervised Machine Learning ◽  
Study Cohort ◽  
Threshold Values ◽  
Laboratory Findings ◽  
Study Results

BackgroundDespite the necessity, there is no reliable biomarker to predict disease severity and prognosis of patients with COVID-19. The currently published prediction models are not fully applicable to clinical use.ObjectivesTo identify predictive biomarkers of COVID-19 severity and to justify their threshold values for the stratification of the risk of deterioration that would require transferring to the intensive care unit (ICU).MethodsThe study cohort (560 subjects) included all consecutive patients admitted to Dubai Mediclinic Parkview Hospital from February to May 2020 with COVID-19 confirmed by the PCR. The challenge of finding the cut-off thresholds was the unbalanced dataset (eg, the disproportion in the number of 72 patients admitted to ICU vs 488 non-severe cases). Therefore, we customised supervised machine learning (ML) algorithm in terms of threshold value used to predict worsening.ResultsWith the default thresholds returned by the ML estimator, the performance of the models was low. It was improved by setting the cut-off level to the 25th percentile for lymphocyte count and the 75th percentile for other features. The study justified the following threshold values of the laboratory tests done on admission: lymphocyte count <2.59×109/L, and the upper levels for total bilirubin 11.9 μmol/L, alanine aminotransferase 43 U/L, aspartate aminotransferase 32 U/L, D-dimer 0.7 mg/L, activated partial thromboplastin time (aPTT) 39.9 s, creatine kinase 247 U/L, C reactive protein (CRP) 14.3 mg/L, lactate dehydrogenase 246 U/L, troponin 0.037 ng/mL, ferritin 498 ng/mL and fibrinogen 446 mg/dL.ConclusionThe performance of the neural network trained with top valuable tests (aPTT, CRP and fibrinogen) is admissible (area under the curve (AUC) 0.86; 95% CI 0.486 to 0.884; p<0.001) and comparable with the model trained with all the tests (AUC 0.90; 95% CI 0.812 to 0.902; p<0.001). Free online tool at https://med-predict.com illustrates the study results.

scholarly journals Enabling deeper learning on big data for materials informatics applications

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Dipendra Jha ◽  
Vishu Gupta ◽  
Logan Ward ◽  
Zijiang Yang ◽  
Christopher Wolverton ◽  
...  
Keyword(s):  
Neural Networks ◽  
Big Data ◽  
Deep Learning ◽  
Deep Neural Networks ◽  
Materials Science ◽  
Prediction Models ◽  
Model Performance ◽  
Materials Informatics ◽  
Learning Framework ◽  
Significant Attention

AbstractThe application of machine learning (ML) techniques in materials science has attracted significant attention in recent years, due to their impressive ability to efficiently extract data-driven linkages from various input materials representations to their output properties. While the application of traditional ML techniques has become quite ubiquitous, there have been limited applications of more advanced deep learning (DL) techniques, primarily because big materials datasets are relatively rare. Given the demonstrated potential and advantages of DL and the increasing availability of big materials datasets, it is attractive to go for deeper neural networks in a bid to boost model performance, but in reality, it leads to performance degradation due to the vanishing gradient problem. In this paper, we address the question of how to enable deeper learning for cases where big materials data is available. Here, we present a general deep learning framework based on Individual Residual learning (IRNet) composed of very deep neural networks that can work with any vector-based materials representation as input to build accurate property prediction models. We find that the proposed IRNet models can not only successfully alleviate the vanishing gradient problem and enable deeper learning, but also lead to significantly (up to 47%) better model accuracy as compared to plain deep neural networks and traditional ML techniques for a given input materials representation in the presence of big data.

scholarly journals Risk prediction in multicentre studies when there is confounding by cluster or informative cluster size

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Menelaos Pavlou ◽  
Gareth Ambler ◽  
Rumana Z. Omar
Keyword(s):  
Risk Prediction ◽  
Cluster Size ◽  
Linear Models ◽  
Prediction Models ◽  
Predictive Accuracy ◽  
Clustered Data ◽  
Predictor Variable ◽  
Simulated Data ◽  
Predictive Ability ◽  
Informative Cluster Size

Abstract Background Clustered data arise in research when patients are clustered within larger units. Generalised Estimating Equations (GEE) and Generalised Linear Models (GLMM) can be used to provide marginal and cluster-specific inference and predictions, respectively. Methods Confounding by Cluster (CBC) and Informative cluster size (ICS) are two complications that may arise when modelling clustered data. CBC can arise when the distribution of a predictor variable (termed ‘exposure’), varies between clusters causing confounding of the exposure-outcome relationship. ICS means that the cluster size conditional on covariates is not independent of the outcome. In both situations, standard GEE and GLMM may provide biased or misleading inference, and modifications have been proposed. However, both CBC and ICS are routinely overlooked in the context of risk prediction, and their impact on the predictive ability of the models has been little explored. We study the effect of CBC and ICS on the predictive ability of risk models for binary outcomes when GEE and GLMM are used. We examine whether two simple approaches to handle CBC and ICS, which involve adjusting for the cluster mean of the exposure and the cluster size, respectively, can improve the accuracy of predictions. Results Both CBC and ICS can be viewed as violations of the assumptions in the standard GLMM; the random effects are correlated with exposure for CBC and cluster size for ICS. Based on these principles, we simulated data subject to CBC/ICS. The simulation studies suggested that the predictive ability of models derived from using standard GLMM and GEE ignoring CBC/ICS was affected. Marginal predictions were found to be mis-calibrated. Adjusting for the cluster-mean of the exposure or the cluster size improved calibration, discrimination and the overall predictive accuracy of marginal predictions, by explaining part of the between cluster variability. The presence of CBC/ICS did not affect the accuracy of conditional predictions. We illustrate these concepts using real data from a multicentre study with potential CBC. Conclusion Ignoring CBC and ICS when developing prediction models for clustered data can affect the accuracy of marginal predictions. Adjusting for the cluster mean of the exposure or the cluster size can improve the predictive accuracy of marginal predictions.

Sign in / Sign up

Export Citation Format

Share Document