Post-analysis Procedures

2021 ◽  
pp. 175-192
Author(s):  
Joseph A. Veech
Keyword(s):  
Statistical Model ◽  
Model Comparison ◽  
Predictive Accuracy ◽  
False Negative ◽  
Area Under The Curve ◽  
Predictive Performance ◽  
Error Rates ◽  
True Negative ◽  
Habitat Analysis ◽  
Formal Procedure

There are several additional statistical procedures that can be conducted after a habitat analysis. The statistical model produced by a habitat analysis can be assessed for fit to the data. Model fit describes how well the predictor variables explain the variance in the response variable, typically species presence–absence or abundance. When more than one statistical model has been produced by the habitat analysis, these can be compared by a formal procedure called model comparison. This usually involves identifying the model with the lowest Akaike information criterion (AIC) value. If the statistical model is considered a predictive tool then its predictive accuracy needs to be assessed. There are many metrics for assessing the predictive performance of a model and quantifying rates of correct and incorrect classification; the latter are error rates. Many of these metrics are based on the numbers of true positive, true negative, false positive, and false negative observations in an independent dataset. “True” and “false” refer to whether species presence–absence was correctly predicted or not. Predictive performance can also be assessed by constructing a receiver operating characteristic (ROC) curve and calculating area under the curve (AUC) values. High AUC values approaching 1 indicate good predictive performance, whereas a value near 0.5 indicates a poor model that predicts species presence–absence no better than a random guess.

A proof-of-concept study applying machine learning methods to putative risk factors for eating disorders: results from the multi-centre European project on healthy eating

2021 ◽  
pp. 1-10
Author(s):  
I. Krug ◽  
J. Linardon ◽  
C. Greenwood ◽  
G. Youssef ◽  
J. Treasure ◽  
...  
Keyword(s):  
Machine Learning ◽  
Risk Factors ◽  
Logistic Regression ◽  
Predictive Accuracy ◽  
Area Under The Curve ◽  
Prediction Rule ◽  
Predictive Performance ◽  
Individual Risk ◽  
European Project ◽  
Wide Range

Abstract Background Despite a wide range of proposed risk factors and theoretical models, prediction of eating disorder (ED) onset remains poor. This study undertook the first comparison of two machine learning (ML) approaches [penalised logistic regression (LASSO), and prediction rule ensembles (PREs)] to conventional logistic regression (LR) models to enhance prediction of ED onset and differential ED diagnoses from a range of putative risk factors. Method Data were part of a European Project and comprised 1402 participants, 642 ED patients [52% with anorexia nervosa (AN) and 40% with bulimia nervosa (BN)] and 760 controls. The Cross-Cultural Risk Factor Questionnaire, which assesses retrospectively a range of sociocultural and psychological ED risk factors occurring before the age of 12 years (46 predictors in total), was used. Results All three statistical approaches had satisfactory model accuracy, with an average area under the curve (AUC) of 86% for predicting ED onset and 70% for predicting AN v. BN. Predictive performance was greatest for the two regression methods (LR and LASSO), although the PRE technique relied on fewer predictors with comparable accuracy. The individual risk factors differed depending on the outcome classification (EDs v. non-EDs and AN v. BN). Conclusions Even though the conventional LR performed comparably to the ML approaches in terms of predictive accuracy, the ML methods produced more parsimonious predictive models. ML approaches offer a viable way to modify screening practices for ED risk that balance accuracy against participant burden.

scholarly journals A predictive model to estimate survival of hospitalized COVID-19 patients from admission data

2020 ◽  
Author(s):  
Todd J. Levy ◽  
Safiya Richardson ◽  
Kevin Coppa ◽  
Douglas P. Barnaby ◽  
Thomas McGinn ◽  
...  
Keyword(s):  
Survival Probability ◽  
Predictive Accuracy ◽  
Area Under The Curve ◽  
Predictive Performance ◽  
Roc Curves ◽  
Severity Index ◽  
Primary Objective ◽  
Lasso Regression ◽  
Distribution Width ◽  
Serum Blood Urea Nitrogen

AbstractObjectiveOur primary objective was to use initial data available to clinicians to characterize and predict survival for hospitalized coronavirus disease 2019 (COVID-19) patients. While clinical characteristics and mortality risk factors of COVID-19 patients have been reported, a practical survival calculator based on data from a diverse group of U.S. patients has not yet been introduced. Such a tool would provide timely and valuable guidance in decision-making during this global pandemic.DesignWe extracted demographic, laboratory, clinical, and treatment data from electronic health records and used it to build and test the predictive accuracy of a survival probability calculator referred to as “the Northwell COVID-19 Survival (‘NOCOS’) calculator.”Setting13 acute care facilities at Northwell Health served as the setting for this study.Participants5,233 hospitalized COVID-19–positive patients served as the participants for this study.Main outcome measuresThe NOCOS calculator was constructed using multivariate regression with L1 regularization (LASSO) to predict survival during hospitalization. Model predictive performance was measured using Receiver Operating Characteristic (ROC) curves and the Area Under the Curve (AUC) of the calculators.ResultsPatient age, serum blood urea nitrogen, Emergency Severity Index, red cell distribution width, absolute neutrophil count, serum bicarbonate, and glucose were identified as the optimal predictors of survival by multivariate LASSO regression. The predictive performance of the NOCOS calculator had an AUC of 0.832, reaching 0.91 when updated for each patient daily, with stability assessed and maintained for 14 consecutive days. This outperformed other established models, including the Sequential Organ Failure Assessment (SOFA) score (0.732).ConclusionsWe present a practical estimate of survival probability that outperforms other general risk models. The seven early predictors of in-hospital survival can help clinicians identify patients with increased probabilities of survival and provide critical decision support as COVID-19 spreads across the U.S.Trial registrationN/A

27 Machine Learning Algorithms Based on Haplotype Libraries for Classification of Stillbirth Susceptibility in Holstein Cows

2021 ◽  
Vol 99 (Supplement_3) ◽  
pp. 15-16
Author(s):  
Pablo A S Fonseca ◽  
Massimo Tornatore ◽  
Angela Cánovas
Keyword(s):  
Machine Learning ◽  
Genome Wide Association Study ◽  
Linear Models ◽  
Predictive Accuracy ◽  
Area Under The Curve ◽  
Predictive Performance ◽  
Machine Learning Algorithms ◽  
Economic Losses ◽  
Nucleotide Polymorphisms ◽  
Birth Records

Abstract Reduced fertility is one of the main causes of economic losses in dairy farms. The cost of a stillbirth is estimated in US$ 938 per case in Holstein herds. Machine learning (ML) is gaining popularity in the livestock sector as a mean to identify hidden patterns and due to its potential to address dimensionality problems. Here we investigate the application of ML algorithms for the prediction of cows with higher stillbirth susceptibility in two scenarios: cows with >25% and >33.33% of stillbirths among birth records. These thresholds correspond to percentiles 75 (still_75) and 90 (still_90), respectively. A total of 10,570 cows and 50,541 birth records were collected to perform a haplotype-based genome-wide association study. Five-hundred significant pseudo single nucleotide polymorphisms (pseudo-SNPs) (False-Discovery Rate< 0.05) were used as input features of ML-based predictions to determine if the cow is in the top-75 and top-90 percentiles. Table 1 shows the classification performance of the investigated ML and linear models. The ML models outperformed linear models for both thresholds. In general, still_75 showed higher F1 values compared to still_90, suggesting a lower misclassification ratio when a less stringent threshold is used. We observe that accuracy of the models in our study is higher when compared to ML-based prediction accuracies in other breeds, e.g. compared to the accuracies of 0.46 and 0.67 that were achieved using SNPs for body weight in Brahman and fertility traits in Nellore, respectively. Xgboost algorithm shows the highest balanced accuracy (BA; 0.625), F1-score (0.588) and area under the curve (AUC; 0.688), suggesting that xgboost can achieve the highest predictive performance and the lowest difference in misclassification ratio between classes. The ML applied over haplotype libraries is an interesting approach for the detection of animals with higher susceptibility to stillbirths due to highest predictive accuracy and relatively lower misclassification ratio.

scholarly journals Prediction of Inpatient Pressure Ulcers Based on Routine Healthcare Data Using Machine Learning Methodology

2021 ◽  
Author(s):  
Felix Walther ◽  
Luise Heinrich ◽  
Jochen Schmitt ◽  
Maria Eberlein-Gonska ◽  
Martin Roessler
Keyword(s):  
Intensive Care ◽  
Pressure Ulcers ◽  
Predictive Accuracy ◽  
False Negative ◽  
Area Under The Curve ◽  
University Hospital ◽  
Healthcare Data ◽  
Related Risk ◽  
Additive Regression ◽  
Confusion Matrices

Abstract Despite the relevance of pressure ulcers (PU) in inpatient care, the predictive power and role of care-related risk factors (e.g. surgical anesthesia) remain unclear. We investigated the predictability of PU incidence and its association with multiple care variables. We included all somatic cases between 2014 and 2018 with length of stay ≥2 days in a German university hospital. For regression analyses and prediction we used Bayesian Additive Regression Trees (BART) as nonparametric modeling approach. To assess predictive accuracy, we compared BART and logistic regression (LR) using area under the curve (AUC) and confusion matrices. The analysis of 149,006 cases revealed high predictive variable importance and associations between incident PU and intensive care with ventilation, age, surgical anesthesia (≥1 hour) and number of care-involved wards. Despite high AUCs (LR: 0.89; BART: 0.9), the confusion matrices showed a higher number of false negative (LR: 816; BART: 826) than true positive (LR: 138; BART: 68) predictions. In summary, particularly intensive care with ventilation, age, anesthesia and number of care-involved wards were associated with incident PU. Using surgical anesthesia as a proxy for immobility, our results suggest hourly repositioning. High rates of false negative predictions indicate a general challenge in the predictability of PU.

scholarly journals Prospective predictive performance comparison between clinical gestalt and validated COVID-19 mortality scores

2021 ◽  
pp. jim-2021-002037
Author(s):  
Adrian Soto-Mota ◽  
Braulio Alejandro Marfil-Garza ◽  
Santiago Castiello-de Obeso ◽  
Erick Jose Martinez Rodriguez ◽  
Daniel Alberto Carrillo Vazquez ◽  
...  
Keyword(s):  
Clinical Outcomes ◽  
Mexico City ◽  
Predictive Accuracy ◽  
Area Under The Curve ◽  
Predictive Performance ◽  
Tertiary Hospital ◽  
Performance Comparison ◽  
Evidence Based ◽  
Likelihood Ratios ◽  
And Performance

Most COVID-19 mortality scores were developed at the beginning of the pandemic and clinicians now have more experience and evidence-based interventions. Therefore, we hypothesized that the predictive performance of COVID-19 mortality scores is now lower than originally reported. We aimed to prospectively evaluate the current predictive accuracy of six COVID-19 scores and compared it with the accuracy of clinical gestalt predictions. 200 patients with COVID-19 were enrolled in a tertiary hospital in Mexico City between September and December 2020. The area under the curve (AUC) of the LOW-HARM, qSOFA, MSL-COVID-19, NUTRI-CoV, and NEWS2 scores and the AUC of clinical gestalt predictions of death (as a percentage) were determined. In total, 166 patients (106 men and 60 women aged 56±9 years) with confirmed COVID-19 were included in the analysis. The AUC of all scores was significantly lower than originally reported: LOW-HARM 0.76 (95% CI 0.69 to 0.84) vs 0.96 (95% CI 0.94 to 0.98), qSOFA 0.61 (95% CI 0.53 to 0.69) vs 0.74 (95% CI 0.65 to 0.81), MSL-COVID-19 0.64 (95% CI 0.55 to 0.73) vs 0.72 (95% CI 0.69 to 0.75), NUTRI-CoV 0.60 (95% CI 0.51 to 0.69) vs 0.79 (95% CI 0.76 to 0.82), NEWS2 0.65 (95% CI 0.56 to 0.75) vs 0.84 (95% CI 0.79 to 0.90), and neutrophil to lymphocyte ratio 0.65 (95% CI 0.57 to 0.73) vs 0.74 (95% CI 0.62 to 0.85). Clinical gestalt predictions were non-inferior to mortality scores, with an AUC of 0.68 (95% CI 0.59 to 0.77). Adjusting scores with locally derived likelihood ratios did not improve their performance; however, some scores outperformed clinical gestalt predictions when clinicians’ confidence of prediction was <80%. Despite its subjective nature, clinical gestalt has relevant advantages in predicting COVID-19 clinical outcomes. The need and performance of most COVID-19 mortality scores need to be evaluated regularly.

The role of serum periostin in the diagnosis of asthma: A meta-analysis

2020 ◽  
Vol 41 (4) ◽  
pp. 240-247
Author(s):  
Lei Yang ◽  
Qingtao Zhao ◽  
Shuyu Wang
Keyword(s):  
Diagnostic Accuracy ◽  
Meta Analysis ◽  
Characteristic Curve ◽  
False Negative ◽  
Summary Receiver Operating Characteristic ◽  
True Negative ◽  
Negative Results ◽  
Noninvasive Biomarker ◽  
Sensitivity Specificity ◽  
Serum Periostin

Background: Serum periostin has been proposed as a noninvasive biomarker for asthma diagnosis and management. However, its accuracy for the diagnosis of asthma in different populations is not completely clear. Methods: This meta-analysis aimed to evaluate the diagnostic accuracy of periostin level in the clinical determination of asthma. Several medical literature data bases were searched for relevant studies through December 1, 2019. The numbers of patients with true-positive, false-positive, false-negative, and true-negative results for the periostin level were extracted from each individual study. We assessed the risk of bias by using Quality Assessment of Diagnostic Accuracy Studies 2. We used the meta-analysis to produce summary estimates of accuracy. Results: In total, nine studies with 1757 subjects met the inclusion criteria. The pooled estimates of sensitivity, specificity, and diagnostic odds ratios for the detection of asthma were 0.58 (95% confidence interval [CI], 0.38‐0.76), 0.86 (95% CI, 0.74‐0.93), and 8.28 (95% CI, 3.67‐18.68), respectively. The area under the summary receiver operating characteristic curve was 0.82 (95% CI, 0.79‐0.85). And significant publication bias was found in this meta‐analysis (p = 0.39). Conclusion: Serum periostin may be used for the diagnosis of asthma, with moderate diagnostic accuracy.

scholarly journals A multicenter, hospital-based and non-inferiority study for diagnostic efficacy of automated whole breast ultrasound for breast cancer in China

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Yujing Xin ◽  
Xinyuan Zhang ◽  
Yi Yang ◽  
Yi Chen ◽  
Yanan Wang ◽  
...  
Keyword(s):  
Breast Cancer ◽  
False Negative ◽  
Final Analysis ◽  
Age Group ◽  
Surgical Biopsy ◽  
True Negative ◽  
Diagnostic Efficacy ◽  
Negative Results ◽  
Informed Consent Form ◽  
Magnetic Resonance Imaging Mri

AbstractThis study is the first multi-center non-inferiority study that aims to critically evaluate the effectiveness of HHUS/ABUS in China breast cancer detection. This was a multicenter hospital-based study. Five hospitals participated in this study. Women (30–69 years old) with defined criteria were invited for breast examination by HHUS, ABUS or/and mammography. For BI-RADS category 3, an additional magnetic resonance imaging (MRI) test was provided to distinguish the true negative results from false negative results. For women classified as BI-RADS category 4 or 5, either core aspiration biopsy or surgical biopsy was done to confirm the diagnosis. Between February 2016 and March 2017, 2844 women signed the informed consent form, and 1947 of them involved in final analysis (680 were 30 to 39 years old, 1267 were 40 to 69 years old).For all participants, ABUS sensitivity (91.81%) compared with HHUS sensitivity (94.70%) with non-inferior Z tests, P = 0.015. In the 40–69 age group, non-inferior Z tests showed that ABUS sensitivity (93.01%) was non-inferior to MG sensitivity (86.02%) with P < 0.001 and HHUS sensitivity (95.44%) was non-inferior to MG sensitivity (86.02%) with P < 0.001. Sensitivity of ABUS and HHUS are all superior to that of MG with P < 0.001 by superior test.For all participants, ABUS specificity (92.89%) was non-inferior to HHUS specificity (89.36%) with P < 0.001. Superiority test show that specificity of ABUS was superior to that of HHUS with P < 0.001. In the 40–69 age group, ABUS specificity (92.86%) was non-inferior to MG specificity (91.68%) with P < 0.001 and HHUS specificity (89.55%) was non-inferior to MG specificity (91.68%) with P < 0.001. ABUS is not superior to MG with P = 0.114 by superior test. The sensitivity of ABUS/HHUS is superior to that of MG. The specificity of ABUS/HHUS is non-inferior to that of MG. In China, for an experienced US radiologist, both HHUS and ABUS have better diagnostic efficacy than MG in symptomatic individuals.

scholarly journals Magnetic resonance cisternography imaging findings related to the leakage of Gadolinium into the subarachnoid space

2021 ◽  
Author(s):  
Rei Nakamichi ◽  
Toshiaki Taoka ◽  
Hisashi Kawai ◽  
Tadao Yoshida ◽  
Michihiko Sone ◽  
...  
Keyword(s):  
Magnetic Resonance ◽  
Subarachnoid Space ◽  
Area Under The Curve ◽  
Rank Correlation ◽  
Predictive Performance ◽  
Roc Curves ◽  
Imaging Findings ◽  
Spearman’S Rank Correlation ◽  
Sagittal Sinus ◽  
Magnetic Resonance Cisternography

Abstract Purpose To identify magnetic resonance cisternography (MRC) imaging findings related to Gadolinium-based contrast agent (GBCA) leakage into the subarachnoid space. Materials and methods The number of voxels of GBCA leakage (V-leak) on 3D-real inversion recovery images was measured in 56 patients scanned 4 h post-intravenous GBCA injection. Bridging veins (BVs) were identified on MRC. The numbers of BVs with surrounding cystic structures (BV-cyst), with arachnoid granulations protruding into the superior sagittal sinus (BV-AG-SSS) and the skull (BV-AG-skull), and including any of these factors (BV-incl) were recorded. Correlations between these variables and V-leak were examined based on the Spearman’s rank correlation coefficient. Receiver-operating characteristic (ROC) curves were generated to investigate the predictive performance of GBCA leakage. Results V-leak and the number of BV-incl were strongly correlated (r = 0.609, p < 0.0001). The numbers of BV-cyst and BV-AG-skull had weaker correlations with V-leak (r = 0.364, p = 0.006; r = 0.311, p = 0.020, respectively). The number of BV-AG-SSS was not correlated with V-leak. The ROC curve for contrast leakage exceeding 1000 voxels and the number of BV-incl had moderate accuracy, with an area under the curve of 0.871. Conclusion The number of BV-incl may be a predictor of GBCA leakage and a biomarker for waste drainage function without using GBCA.

scholarly journals Validation and verification of predictive salivary biomarkers for oral health

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Nagihan Bostanci ◽  
Konstantinos Mitsakakis ◽  
Beral Afacan ◽  
Kai Bao ◽  
Benita Johannsen ◽  
...  
Keyword(s):  
Oral Health ◽  
Predictive Accuracy ◽  
Statistical Tests ◽  
Area Under The Curve ◽  
Regression Tree ◽  
High Sensitivity ◽  
Oral Health Status ◽  
Classification And Regression Tree ◽  
Cart Analysis ◽  
Salivary Biomarkers

AbstractOral health is important not only due to the diseases emerging in the oral cavity but also due to the direct relation to systemic health. Thus, early and accurate characterization of the oral health status is of utmost importance. There are several salivary biomarkers as candidates for gingivitis and periodontitis, which are major oral health threats, affecting the gums. These need to be verified and validated for their potential use as differentiators of health, gingivitis and periodontitis status, before they are translated to chair-side for diagnostics and personalized monitoring. We aimed to measure 10 candidates using high sensitivity ELISAs in a well-controlled cohort of 127 individuals from three groups: periodontitis (60), gingivitis (31) and healthy (36). The statistical approaches included univariate statistical tests, receiver operating characteristic curves (ROC) with the corresponding Area Under the Curve (AUC) and Classification and Regression Tree (CART) analysis. The main outcomes were that the combination of multiple biomarker assays, rather than the use of single ones, can offer a predictive accuracy of > 90% for gingivitis versus health groups; and 100% for periodontitis versus health and periodontitis versus gingivitis groups. Furthermore, ratios of biomarkers MMP-8, MMP-9 and TIMP-1 were also proven to be powerful differentiating values compared to the single biomarkers.

Sign in / Sign up

Export Citation Format

Share Document