scholarly journals Recalibration Methods for Improved Clinical Utility of Risk Scores

2021 ◽  
pp. 0272989X2110446
Author(s):  
Anu Mishra ◽  
Robyn L. McClelland ◽  
Lurdes Y. T. Inoue ◽  
Kathleen F. Kerr
Keyword(s):  
Constrained Optimization ◽  
Risk Score ◽  
Clinical Utility ◽  
Risk Model ◽  
Net Benefit ◽  
New Methods ◽  
Risk Threshold ◽  
Graphical Tool ◽  
Ascvd Risk ◽  
Good Calibration

Background An established risk model may demonstrate miscalibration, meaning predicted risks do not accurately capture event rates. In some instances, investigators can identify and address the cause of miscalibration. In other circumstances, it may be appropriate to recalibrate the risk model. Existing recalibration methods do not account for settings in which the risk score will be used for risk-based clinical decision making. Methods We propose 2 new methods for risk model recalibration when the intended purpose of the risk model is to prescribe an intervention to high-risk individuals. Our measure of risk model clinical utility is standardized net benefit. The first method is a weighted strategy that prioritizes good calibration at or around the critical risk threshold. The second method uses constrained optimization to produce a recalibrated risk model with maximum possible net benefit, thereby prioritizing good calibration around the critical risk threshold. We also propose a graphical tool for assessing the potential for recalibration to improve the net benefit of a risk model. We illustrate these methods by recalibrating the American College of Cardiology (ACC)–American Heart Association (AHA) atherosclerotic cardiovascular disease (ASCVD) risk score within the Multi-Ethnic Study of Atherosclerosis (MESA) cohort. Results New methods are implemented in the R package ClinicalUtilityRecal. Recalibrating the ACC-AHA-ASCVD risk score for a MESA subcohort results in higher estimated net benefit using the proposed methods compared with existing methods, with improved calibration in the most clinically impactful regions of risk. Conclusion The proposed methods target good calibration for critical risks and can improve the net benefit of a risk model. We recommend constrained optimization when the risk model net benefit is paramount. The weighted approach can be considered when good calibration over an interval of risks is important.

scholarly journals Assessing the clinical utility of genetic risk scores for targeted cancer screening

2021 ◽  
Vol 19 (1) ◽  
Author(s):  
Carly A. Conran ◽  
Zhuqing Shi ◽  
William Kyle Resurreccion ◽  
Rong Na ◽  
Brian T. Helfand ◽  
...  
Keyword(s):  
Colorectal Cancer ◽  
Primary Care ◽  
High Risk ◽  
Risk Score ◽  
Genetic Risk ◽  
Clinical Utility ◽  
Low Risk ◽  
Risk Scores ◽  
Average Risk ◽  
Genetic Risk Scores

Abstract Background Genome-wide association studies have identified thousands of disease-associated single nucleotide polymorphisms (SNPs). A subset of these SNPs may be additively combined to generate genetic risk scores (GRSs) that confer risk for a specific disease. Although the clinical validity of GRSs to predict risk of specific diseases has been well established, there is still a great need to determine their clinical utility by applying GRSs in primary care for cancer risk assessment and targeted intervention. Methods This clinical study involved 281 primary care patients without a personal history of breast, prostate or colorectal cancer who were 40–70 years old. DNA was obtained from a pre-existing biobank at NorthShore University HealthSystem. GRSs for colorectal cancer and breast or prostate cancer were calculated and shared with participants through their primary care provider. Additional data was gathered using questionnaires as well as electronic medical record information. A t-test or Chi-square test was applied for comparison of demographic and key clinical variables among different groups. Results The median age of the 281 participants was 58 years and the majority were female (66.6%). One hundred one (36.9%) participants received 2 low risk scores, 99 (35.2%) received 1 low risk and 1 average risk score, 37 (13.2%) received 1 low risk and 1 high risk score, 23 (8.2%) received 2 average risk scores, 21 (7.5%) received 1 average risk and 1 high risk score, and no one received 2 high risk scores. Before receiving GRSs, younger patients and women reported significantly more worry about risk of developing cancer. After receiving GRSs, those who received at least one high GRS reported significantly more worry about developing cancer. There were no significant differences found between gender, age, or GRS with regards to participants’ reported optimism about their future health neither before nor after receiving GRS results. Conclusions Genetic risk scores that quantify an individual’s risk of developing breast, prostate and colorectal cancers as compared with a race-defined population average risk have potential clinical utility as a tool for risk stratification and to guide cancer screening in a primary care setting.

scholarly journals POS0261 DIAGNOSTIC PERFORMANCE AND CLINICAL UTILITY OF TWO REFERRAL RULES FOR INFLAMMATORY ARTHRITIS

2021 ◽  
Vol 80 (Suppl 1) ◽  
pp. 353.1-353
Author(s):  
E. Van Delft ◽  
D. Lopes Barreto ◽  
A. Van der Helm - van Mil ◽  
C. Alves ◽  
J. Hazes ◽  
...  
Keyword(s):  
Primary Care ◽  
Usual Care ◽  
Inflammatory Arthritis ◽  
Clinical Utility ◽  
Diagnostic Performance ◽  
Youden Index ◽  
Care Model ◽  
Net Benefit ◽  
Independent Population ◽  
The Right

Background:The Rotterdam Early Arthritis Cohort (REACH) rule [1] and Clinical Arthritis RulE (CARE) [2] are both evidence-based and easy-to-use methods developed to identify the presence of inflammatory arthritis (IA) in patients suspected by their general practitioner (GP). However, the clinical utility of both models in daily clinical practice in an independent primary care setting has not yet been established. While developed for recognizing IA, we believe that it is also important that the broader spectrum of inflammatory rheumatic diseases (IRDs) is correctly classified from primary care, to facilitate appropriate referral towards outpatient rheumatology clinics.Objectives:The primary objective was to determine the diagnostic performance and clinical utility of the REACH and CARE referral rules in identifying IA in an independent population of unselected suspected patients from primary care. Secondly we will assess the diagnostic performance and clinical utility of both models in identifying IRDs.Methods:This prospective observational diagnostic study consisted of adults newly suspected by their GP for the need of referral to the rheumatology outpatient clinic of the Maasstad Hospital in Rotterdam. Primary outcome was IA, consisting of rheumatoid arthritis, axial spondylitis and psoriatic arthritis. Secondary outcome was IRD, defined as IA plus arthritis in systemic disorders such as systemic lupus erythematosus, systemic sclerosis and morbus sjögren. Rheumatologist diagnosis was used as gold standard. To evaluate the clinical performance of the REACH and CARE referral rules in this population, diagnostic accuracy measures were investigated using the Youden index (J) [3]. Moreover, a net benefit approach [4] was used to determine clinical utility of both rules when compared to usual care.Results:This study consisted of 250 patients (22.8% male) with a mean age of 50.8 years (SD 13.9 years). In total 42 (17%) patients were diagnosed with IA and 55 (22%) with an IRD. Figure 1 presents the diagnostic performance in IA (Figure 1A) and in IRD (Figure 1B). For the primary outcome, the REACH model shows an AUC of 0.72 (95% CI 0.64-0.80) and the optimal cut-off point is indicated (J). The CARE model shows an AUC of 0.82 (95% CI 0.75-0.88) and at J there is a somewhat higher sensitivity and specificity. When taking the broader spectrum of IRDs as outcome, the AUC was 0.66 (95% CI 0.58-0.74) for the REACH and 0.76 (95% CI 0.69-0.83) for the CARE model. The net benefit analysis with either IA or IRD as outcome showed that the CARE was of the highest clinical value when compared to usual care.Conclusion:Both the REACH and CARE model showed a good diagnostic performance for detecting IA in an independent population of unselected suspected patients from primary care. Although not specifically developed to recognize the entire spectrum of IRDs, the CARE shows a good performance in doing so. When evaluating clinical utility, we see that both rules have a net benefit in recognizing IA as well as IRDs compared to usual care, however the CARE shows superiority over the REACH. By using the CARE, over half of all suspected patients can be withheld from expensive outpatient rheumatology care, implied by the high specificity of 70%. These results support the idea that incorporating these easy-to-use methods into primary care could lead to providing patients the right care at the right place and improving value based health care.References:[1]ten Brinck RM, van Dijk BT, van Steenbergen HW, le Cessie S, Numans ME. Development and validation of a clinical rule for recognition of early inflammatory arthritis. BMJ Open; 2018: 8[2]Alves, C. Improving early referral of inflammatory arthritis. In Early detection of patients at risk for rheumatoid arthritis – a challenge for primary and secondary care; 2015: 27-38 Ridderkerk, the Netherlands.[3]Fluss R, Faraggi D, Reiser B. Estimation of the Youden Index and its associated cutoff point. Biom J; 2005: 47(4): 458-472[4]Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making; 2006: 26(6): 565-574Disclosure of Interests:None declared

scholarly journals Integration of Urinary EN2 Protein & Cell-Free RNA Data in the Development of a Multivariable Risk Model for the Detection of Prostate Cancer Prior to Biopsy

Cancers ◽  
2021 ◽  
Vol 13 (9) ◽  
pp. 2102
Author(s):  
Shea Connell ◽  
Robert Mills ◽  
Hardev Pandha ◽  
Richard Morgan ◽  
Colin Cooper ◽  
...  
Keyword(s):  
Prostate Cancer ◽  
Risk Model ◽  
Action Plan ◽  
Digital Rectal Examination ◽  
Standards Of Care ◽  
Net Benefit ◽  
Protein Levels ◽  
Non Invasive ◽  
Trus Biopsy ◽  
Diagnostic Potential

The objective is to develop a multivariable risk model for the non-invasive detection of prostate cancer prior to biopsy by integrating information from clinically available parameters, Engrailed-2 (EN2) whole-urine protein levels and data from urinary cell-free RNA. Post-digital-rectal examination urine samples collected as part of the Movember Global Action Plan 1 study which has been analysed for both cell-free-RNA and EN2 protein levels were chosen to be integrated with clinical parameters (n = 207). A previously described robust feature selection framework incorporating bootstrap resampling and permutation was applied to the data to generate an optimal feature set for use in Random Forest models for prediction. The fully integrated model was named ExoGrail, and the out-of-bag predictions were used to evaluate the diagnostic potential of the risk model. ExoGrail risk (range 0–1) was able to determine the outcome of an initial trans-rectal ultrasound guided (TRUS) biopsy more accurately than clinical standards of care, predicting the presence of any cancer with an area under the receiver operator curve (AUC) = 0.89 (95% confidence interval(CI): 0.85–0.94), and discriminating more aggressive Gleason ≥ 3 + 4 disease returning an AUC = 0.84 (95% CI: 0.78–0.89). The likelihood of more aggressive disease being detected significantly increased as ExoGrail risk score increased (Odds Ratio (OR) = 2.21 per 0.1 ExoGrail increase, 95% CI: 1.91–2.59). Decision curve analysis of the net benefit of ExoGrail showed the potential to reduce the numbers of unnecessary biopsies by 35% when compared to current standards of care. Integration of information from multiple, non-invasive biomarker sources has the potential to greatly improve how patients with a clinical suspicion of prostate cancer are risk-assessed prior to an invasive biopsy.

Abstract 14677: Distribution of 10- and 30-year Predicted Risks for Atherosclerotic Cardiovascular Disease in the United States: NHANES 2015 to 2018

Circulation ◽  
2020 ◽  
Vol 142 (Suppl_3) ◽  
Author(s):  
Nilay S Shah ◽  
Hongyan Ning ◽  
Amanda Perak ◽  
Norrina B Allen ◽  
John T Wilkins ◽  
...  
Keyword(s):  
Cardiovascular Disease ◽  
Life Course ◽  
Risk Model ◽  
The United States ◽  
Atherosclerotic Cardiovascular Disease ◽  
Middle Aged ◽  
Ascvd Risk ◽  
The Life Course ◽  
Predicted Risk

Introduction: Premature fatal cardiovascular disease rates have plateaued in the US. Identifying population distributions of short- and long-term predicted risk for atherosclerotic cardiovascular disease (ASCVD) can inform interventions and policy to improve cardiovascular health over the life course. Methods: Among nonpregnant participants age 30-59 years without prevalent CVD from the National Health and Nutrition Examination Surveys 2015-18, continuous 10 year (10Y) and 30 year (30Y) predicted ASCVD risk were assigned using the Pooled Cohort Equations and a 30-year competing risk model, respectively. Intermediate/high 10Y risk was defined as ≥7.5%, and high 30Y risk was chosen a priori as ≥20%, based on 2019 guideline levels for risk stratification. Participants were combined into low 10Y/low 30Y, low 10Y/high 30Y, and intermediate/high 10Y categories. We calculated and compared risk distributions overall and across race-sex, age, body mass index (BMI), and education using chi-square tests. Results: In 1495 NHANES participants age 30-59 years (representing 53,022,413 Americans), median 10Y risk was 2.3% and 30Y risk was 15.5%. Approximately 12% of individuals were already estimated to have intermediate/high 10Y risk. Of those at low 10Y risk, 30% had high 30Y predicted risk. Distributions differed significantly by sex, race, age, BMI, and education (P<0.01, Figure ). Black males more frequently had high 10Y risk compared with other race-sex groups. Older individuals, those with BMI ≥30 kg/m 2 , and with ≤high school education had a higher frequency of low 10Y/high 30Y risk. Conclusions: More than one-third of middle-aged U.S. adults have elevated short- or long-term predicted risk for ASCVD. While the majority of middle-aged US adults are at low 10Y risk, a large proportion among this subgroup are at high 30Y ASCVD risk, indicating a substantial need for enhanced clinical and population level prevention earlier in the life course.

scholarly journals Machine Learning for Building Immune Genetic Model in Hepatocellular Carcinoma Patients

2021 ◽  
Vol 2021 ◽  
pp. 1-15
Author(s):  
Jun Liu ◽  
Zheng Chen ◽  
Wenli Li
Keyword(s):  
Hepatocellular Carcinoma ◽  
Risk Score ◽  
Risk Model ◽  
Cox Model ◽  
Cancer Genome ◽  
Algorithm Analysis ◽  
Immune Genes ◽  
Normal Tissues ◽  
Model Based ◽  
Tumor Stages

Background. Hepatocellular carcinoma (HCC) is the leading liver cancer with special immune microenvironment, which played vital roles in tumor relapse and poor drug responses. In this study, we aimed to explore the prognostic immune signatures in HCC and tried to construct an immune-risk model for patient evaluation. Methods. RNA sequencing profiles of HCC patients were collected from the cancer genome Atlas (TCGA), international cancer genome consortium (ICGC), and gene expression omnibus (GEO) databases (GSE14520). Differentially expressed immune genes, derived from ImmPort database and MSigDB signaling pathway lists, between tumor and normal tissues were analyzed with Limma package in R environment. Univariate Cox regression was performed to find survival-related immune genes in TCGA dataset, and in further random forest algorithm analysis, significantly changed immune genes were used to generate a multivariate Cox model to calculate the corresponding immune-risk score. The model was examined in the other two datasets with recipient operation curve (ROC) and survival analysis. Risk effects of immune-risk score and clinical characteristics of patients were individually evaluated, and significant factors were then used to generate a nomogram. Results. There were 52 downregulated and 259 upregulated immune genes between tumor and relatively normal tissues, and the final immune-risk model (based on SPP1, BRD8, NDRG1, KITLG, HSPA4, TRAF3, ITGAV and MAP4K2) can better differentiate patients into high and low immune-risk subpopulations, in which high score patients showed worse outcomes after resection ( p < 0.05 ). The differentially enriched pathways between the two groups were mainly about cell proliferation and cytokine production, and calculated immune-risk score was also highly correlated with immune infiltration levels. The nomogram, constructed with immune-risk score and tumor stages, showed high accuracy and clinical benefits in prediction of 1-, 3- and 5-year overall survival, which is useful in clinical practice. Conclusion. The immune-risk model, based on expression of SPP1, BRD8, NDRG1, KITLG, HSPA4, TRAF3, ITGAV, and MAP4K2, can better differentiate patients into high and low immune-risk groups. Combined nomogram, using immune-risk score and tumor stages, could make accurate prediction of 1-, 3- and 5-year survival in HCC patients.

scholarly journals TYROBP, TLR4 and ITGAM regulated macrophages polarization and immune checkpoints expression in osteosarcoma

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Tuo Liang ◽  
Jiarui Chen ◽  
GuoYong Xu ◽  
Zide Zhang ◽  
Jiang Xue ◽  
...  
Keyword(s):  
Risk Score ◽  
Immune Cell ◽  
Risk Model ◽  
Low Risk ◽  
Risk Scores ◽  
Immune Checkpoints ◽  
Kegg Pathways ◽  
Gene Sets ◽  
Macrophages Polarization ◽  
Immune Related Genes

AbstractWe established a relationship among the immune-related genes, tumor-infiltrating immune cells (TIICs), and immune checkpoints in patients with osteosarcoma. The gene expression data for osteosarcoma were downloaded from UCSC Xena and GEO database. Immune-related differentially expressed genes (DEGs) were detected to calculate the risk score. “Estimate” was used for immune infiltrating estimation and “xCell” was used to obtain 64 immune cell subtypes. Furthermore, the relationship among the risk scores, immune cell subtypes, and immune checkpoints was evaluated. The three immune-related genes (TYROBP, TLR4, and ITGAM) were selected to establish a risk scoring system based on their integrated prognostic relevance. The GSEA results for the Hallmark and KEGG pathways revealed that the low-risk score group exhibited the most gene sets that were related to immune-related pathways. The risk score significantly correlated with the xCell score of macrophages, M1 macrophages, and M2 macrophages, which significantly affected the prognosis of osteosarcoma. Thus, patients with low-risk scores showed better results with the immune checkpoints inhibitor therapy. A three immune-related, gene-based risk model can regulate macrophage activation and predict the treatment outcomes the survival rate in osteosarcoma.

scholarly journals Pathways by Which Food Insecurity Is Associated With Atherosclerotic Cardiovascular Disease Risk

2021 ◽  
Author(s):  
Deepak Palakshappa ◽  
Edward H. Ip ◽  
Seth A. Berkowitz ◽  
Alain G. Bertoni ◽  
Kristie L. Foley ◽  
...  
Keyword(s):  
Mental Health ◽  
Cardiovascular Disease ◽  
High Risk ◽  
Food Insecurity ◽  
Access To Care ◽  
Risk Score ◽  
Equation Modeling ◽  
Care Pathways ◽  
Atherosclerotic Cardiovascular Disease ◽  
Ascvd Risk

Background Food insecurity (FI) has been associated with an increased atherosclerotic cardiovascular disease (ASCVD) risk; however, the pathways by which FI leads to worse cardiovascular health are unknown. We tested the hypothesis that FI is associated with ASCVD risk through nutritional/anthropometric (eg, worse diet quality and increased weight), psychological/mental health (eg, increased depressive symptoms and risk of substance abuse), and access to care pathways. Methods and Results We conducted a cross‐sectional study of adults (aged 40–79 years) using the 2007 to 2016 National Health and Nutrition Examination Survey. Our primary exposure was household FI, and our outcome was 10‐year ASCVD risk categorized as low (<5%), borderline (≥5% –<7.5%), intermediate (≥7.5%–<20%), and high risk (≥20%). We used structural equation modeling to evaluate the pathways and multiple mediation analysis to determine direct and indirect effects. Of the 12 429 participants, 2231 (18.0%) reported living in a food‐insecure household; 5326 (42.9%) had a low ASCVD risk score, 1402 (11.3%) borderline, 3606 (29.0%) intermediate, and 2095 (16.9%) had a high‐risk score. In structural models, we found significant path coefficients between FI and the nutrition/anthropometric (β, 0.130; SE, 0.027; P <0.001), psychological/mental health (β, 0.612; SE, 0.043; P <0.001), and access to care (β, 0.110; SE, 0.036; P =0.002) pathways. We did not find a significant direct effect of FI on ASCVD risk, and the nutrition, psychological, and access to care pathways accounted for 31.6%, 43.9%, and 15.8% of the association, respectively. Conclusions We found that the association between FI and ASCVD risk category was mediated through the nutrition/anthropometric, psychological/mental health, and access to care pathways. Interventions that address all 3 pathways may be needed to mitigate the negative impact of FI on cardiovascular disease.

scholarly journals Comprehensive Analysis of Cell Cycle-Related Genes in Patients With Prostate Cancer

2022 ◽  
Vol 11 ◽  
Author(s):  
Zehua Liu ◽  
Rongfang Pan ◽  
Wenxian Li ◽  
Yanjiang Li
Keyword(s):  
Prostate Cancer ◽  
Cell Cycle ◽  
Risk Score ◽  
Risk Model ◽  
Cox Regression ◽  
Expression Profiles ◽  
Functional Enrichment ◽  
Cancer Center ◽  
Clinical Prognosis ◽  
Normal Tissues

This study aimed to identify critical cell cycle-related genes (CCRGs) in prostate cancer (PRAD) and to evaluate the clinical prognostic value of the gene panel selected. Gene set variation analysis (GSVA) of dysregulated genes between PRAD and normal tissues demonstrated that the cell cycle-related pathways played vital roles in PRAD. Patients were classified into four clusters, which were associated with recurrence-free survival (RFS). Moreover, 200 prognostic-related genes were selected using the Kaplan–Meier (KM) survival analysis and univariable Cox regression. The prognostic CCRG risk score was constructed using random forest survival and multivariate regression Cox methods, and their efficiency was validated in Memorial Sloan Kettering Cancer Center (MSKCC) and GSE70770. We identified nine survival-related genes: CCNL2, CDCA5, KAT2A, CHTF18, SPC24, EME2, CDK5RAP3, CDC20, and PTTG1. Based on the median risk score, the patients were divided into two groups. Then the functional enrichment analyses, mutational profiles, immune components, estimated half-maximal inhibitory concentration (IC50), and candidate drugs were screened of these two groups. In addition, the characteristics of nine hub CCRGs were explored in Oncomine, cBioPortal, and the Human Protein Atlas (HPA) datasets. Finally, the expression profiles of these hub CCRGs were validated in RWPE-1 and three PRAD cell lines (PC-3, C4-2, and DU-145). In conclusion, our study systematically explored the role of CCRGs in PRAD and constructed a risk model that can predict the clinical prognosis and immunotherapeutic benefits.

scholarly journals Identification and Validation of a Predictive Immune-related Genes Signature for the Prognosis of Cholangiocarcinoma

2020 ◽  
Author(s):  
Rui Zhang ◽  
Chen Chen ◽  
Qi Li ◽  
Jialu Fu ◽  
Dong Zhang ◽  
...  
Keyword(s):  
Gene Expression ◽  
T Cells ◽  
High Risk ◽  
Risk Score ◽  
Risk Model ◽  
Predictive Accuracy ◽  
Expression Profiles ◽  
Gene Expression Profiles ◽  
The Cancer Genome Atlas ◽  
Immune Related Genes

Abstract Background: Immune-related genes (IRGs) play a crucial role in the initiation and progression of cholangiocarcinoma (CCA). However, immune signatures have rarely been used to predict prognosis of CCA. The aim of this study was to construct a novel model for CCA to predict survival based on IRGs expression data.Methods: The gene expression profiles and clinical data of CCA patients from The Cancer Genome Atlas (TCGA) and Gene Expression Omnibus (GEO) database were integrated to establish and validate prognostic IRG signatures. Differentially expressed immune-related genes were screened. Univariate and multivariate Cox analysis were performed to identify prognostic IRGs, and the risk model that predicts outcomes was constructed. Furthermore, receiver operating characteristic (ROC) and Kaplan-Meier curve were plotted to examine predictive accuracy of the model, and a nomogram was constructed based on IRGs signature, combining with other clinical characteristics. Finally, CIBERSORT was used to analyze the association of immune cells infiltration with risk score.Results: We identified that 223 IRGs were significantly dysregulated in patients with CCA, among which five IRGs (AVPR1B, CST4, TDGF1, RAET1E and IL9R) were identified as robust indicators for overall survival (OS), and a prognostic model was built based on the IRGs signature. Meanwhile, patients with high risk had worse OS in training and validation cohort, and the area under the ROC was 0.898 and 0.846, respectively. Nomogram demonstrated that immune risk score contributed much more points than other clinicopathological variables, with a C-index of 0.819 (95% CI, 0.727-0.911). Finally, we found that IRGs signature was positively correlated with the proportion of CD8+ T cells, neurophils and T gamma delta, while negatively with that of CD4+ memory resting T cells.Conclusions: We established and validated an effective five IRGs-based prediction model for CCA, which could accurately classify patients into groups with low and high risk of poor prognosis.

Sign in / Sign up

Export Citation Format

Share Document