Internal Validation
Recently Published Documents





2021 ◽  
Vol 11 ◽  
Xiangtian Zhao ◽  
Yukun Zhou ◽  
Yuan Zhang ◽  
Lujun Han ◽  
Li Mao ◽  

ObjectiveThis study aims to develop and externally validate a contrast-enhanced magnetic resonance imaging (CE-MRI) radiomics-based model for preoperative differentiation between fat-poor angiomyolipoma (fp-AML) and hepatocellular carcinoma (HCC) in patients with noncirrhotic livers and to compare the diagnostic performance with that of two radiologists.MethodsThis retrospective study was performed with 165 patients with noncirrhotic livers from three medical centers. The dataset was divided into a training cohort (n = 99), a time-independent internal validation cohort (n = 24) from one center, and an external validation cohort (n = 42) from the remaining two centers. The volumes of interest were contoured on the arterial phase (AP) images and then registered to the venous phase (VP) and delayed phase (DP), and a total of 3,396 radiomics features were extracted from the three phases. After the joint mutual information maximization feature selection procedure, four radiomics logistic regression classifiers, including the AP model, VP model, DP model, and combined model, were built. The area under the receiver operating characteristic curve (AUC), diagnostic accuracy, sensitivity, and specificity of each radiomics model and those of two radiologists were evaluated and compared.ResultsThe AUCs of the combined model reached 0.789 (95%CI, 0.579–0.999) in the internal validation cohort and 0.730 (95%CI, 0.563–0.896) in the external validation cohort, higher than the AP model (AUCs, 0.711 and 0.638) and significantly higher than the VP model (AUCs, 0.594 and 0.610) and the DP model (AUCs, 0.547 and 0.538). The diagnostic accuracy, sensitivity, and specificity of the combined model were 0.708, 0.625, and 0.750 in the internal validation cohort and 0.619, 0.786, and 0.536 in the external validation cohort, respectively. The AUCs for the two radiologists were 0.656 and 0.594 in the internal validation cohort and 0.643 and 0.500 in the external validation cohort. The AUCs of the combined model surpassed those of the two radiologists and were significantly higher than that of the junior one in both validation cohorts.ConclusionsThe proposed radiomics model based on triple-phase CE-MRI images was proven to be useful for differentiating between fp-AML and HCC and yielded comparable or better performance than two radiologists in different centers, with different scanners and different scanning parameters.

2021 ◽  
Sang-Heon Lim ◽  
Young Jae Kim ◽  
Yeon-Ho Park ◽  
Doojin Kim ◽  
Kwang Gi Kim ◽  

Abstract Pancreas segmentation is necessary for observing lesions, analyzing anatomical structures, and predicting patient prognosis. Therefore, various studies have designed segmentation models based on convolutional neural networks for pancreas segmentation. However, the deep learning approach is limited by a lack of data, and studies conducted on a large computed tomography dataset are scarce. Therefore, this study aims to perform deep-learning-based semantic segmentation on 1,006 participants and evaluate the automatic segmentation performance of the pancreas via four individual three-dimensional segmentation networks. In this study, we performed internal validation with 1,006 patients and external validation using the Cancer Imaging Archive (TCIA) pancreas dataset. We obtained mean precision, recall, and dice similarity coefficients of 0.869, 0.842, and 0.842, respectively, for internal validation via a relevant approach among the four deep learning networks. Using the external dataset, the deep learning network achieved mean precision, recall, and dice similarity coefficients of 0.779, 0.749, and 0.735, respectively. We expect that generalized deep-learning-based systems can assist clinical decisions by providing accurate pancreatic segmentation and quantitative information of the pancreas for abdominal computed tomography.

2021 ◽  
Vol 8 ◽  
Meng Li ◽  
Ke Wang ◽  
Yanpeng Zhang ◽  
Meng Fan ◽  
Anqi Li ◽  

Background: Idiopathic pulmonary fibrosis (IPF) is a chronic progressive disease with unknown etiology and unfavorable prognosis. Ferroptosis is a form of regulated cell death with an iron-dependent way that is involved in the development of various diseases. Whereas the prognostic value of ferroptosis-related genes (FRGs) in IPF remains uncertain and needs to be further elucidated.Methods: The FerrDb database and the previous studies were screened to explore the FRGs. The data of patients with IPF were obtained from the GSE70866 dataset. Wilcoxon's test and univariate Cox regression analysis were applied to identify the FRGs that are differentially expressed between normal and patients with IPF and associated with prognosis. Next, a multigene signature was constructed by the least absolute shrinkage and selection operator (LASSO)-penalized Cox model in the training cohort and evaluated by using calibration and receiver operating characteristic (ROC) curves. Then, 30% of the dataset samples were randomly selected for internal validation. Finally, the potential function and pathways that might be affected by the risk score-related differently expressed genes (DEGs) were further explored.Results: A total of 183 FRGs were identified by the FerrDb database and the previous studies, and 19 of them were differentially expressed in bronchoalveolar lavage fluid (BALF) between IPF and healthy controls and associated with prognosis (p < 0.05). There were five FRGs (aconitase 1 [ACO1], neuroblastoma RAS viral (v-ras) oncogene homolog [NRAS], Ectonucleotide pyrophosphatase/phosphodiesterase 2 [ENPP2], Mucin 1 [MUC1], and ZFP36 ring finger protein [ZFP36]) identified as risk signatures and stratified patients with IPF into the two risk groups. The overall survival rate in patients with high risk was significantly lower than that in patients with low risk (p < 0.001). The calibration and ROC curve analysis confirmed the predictive capacity of this signature, and the results were further verified in the validation group. Risk score-related DEGs were found enriched in ECM-receptor interaction and focal adhesion pathways.Conclusion: The five FRGs in BALF can be used for prognostic prediction in IPF, which may contribute to improving the management strategies of IPF.

Joon-myoung Kwon ◽  
Ye Rang Lee ◽  
Min-Seung Jung ◽  
Yoon-Ji Lee ◽  
Yong-Yeon Jo ◽  

Abstract Background Sepsis is a life-threatening organ dysfunction and a major healthcare burden worldwide. Although sepsis is a medical emergency that requires immediate management, screening for the occurrence of sepsis is difficult. Herein, we propose a deep learning-based model (DLM) for screening sepsis using electrocardiography (ECG). Methods This retrospective cohort study included 46,017 patients who were admitted to two hospitals. A total of 1,548 and 639 patients had sepsis and septic shock, respectively. The DLM was developed using 73,727 ECGs from 18,142 patients, and internal validation was conducted using 7774 ECGs from 7,774 patients. Furthermore, we conducted an external validation with 20,101 ECGs from 20,101 patients from another hospital to verify the applicability of the DLM across centers. Results During the internal and external validations, the area under the receiver operating characteristic curve (AUC) of the DLM using 12-lead ECG was 0.901 (95% confidence interval, 0.882–0.920) and 0.863 (0.846–0.879), respectively, for screening sepsis and 0.906 (95% confidence interval (CI), 0.877–0.936) and 0.899 (95% CI, 0.872–0.925), respectively, for detecting septic shock. The AUC of the DLM for detecting sepsis using 6-lead and single-lead ECGs was 0.845–0.882. A sensitivity map revealed that the QRS complex and T waves were associated with sepsis. Subgroup analysis was conducted using ECGs from 4,609 patients who were admitted with an infectious disease, and the AUC of the DLM for predicting in-hospital mortality was 0.817 (0.793–0.840). There was a significant difference in the prediction score of DLM using ECG according to the presence of infection in the validation dataset (0.277 vs. 0.574, p < 0.001), including severe acute respiratory syndrome coronavirus 2 (0.260 vs. 0.725, p = 0.018). Conclusions The DLM delivered reasonable performance for sepsis screening using 12-, 6-, and single-lead ECGs. The results suggest that sepsis can be screened using not only conventional ECG devices but also diverse life-type ECG machines employing the DLM, thereby preventing irreversible disease progression and mortality.

2021 ◽  
Vol 11 ◽  
Okechinyere J. Achilonu ◽  
June Fabian ◽  
Brendan Bebington ◽  
Elvira Singh ◽  
Gideon Nimako ◽  

The aim of this pilot study was to develop logistic regression (LR) and support vector machine (SVM) models that differentiate low from high risk for prolonged hospital length of stay (LOS) in a South African cohort of 383 colorectal cancer patients who underwent surgical resection with curative intent. Additionally, the impact of 10-fold cross-validation (CV), Monte Carlo CV, and bootstrap internal validation methods on the performance of the two models was evaluated. The median LOS was 9 days, and prolonged LOS was defined as greater than 9 days post-operation. Preoperative factors associated with prolonged LOS were a prior history of hypertension and an Eastern Cooperative Oncology Group score between 2 and 4. Postoperative factors related to prolonged LOS were the need for a stoma as part of the surgical procedure and the development of post-surgical complications. The risk of prolonged LOS was higher in male patients and in any patient with lower preoperative hemoglobin. The highest area under the receiving operating characteristics (AU-ROC) was achieved using LR of 0.823 (CI = 0.798–0.849) and SVM of 0.821 (CI = 0.776–0.825), with each model using the Monte Carlo CV method for internal validation. However, bootstrapping resulted in models with slightly lower variability. We found no significant difference between the models across the three internal validation methods. The LR and SVM algorithms used in this study required incorporating important features for optimal hospital LOS predictions. The factors identified in this study, especially postoperative complications, can be employed as a simple and quick test clinicians may flag a patient at risk of prolonged LOS.

2021 ◽  
Vol 11 ◽  
Tongtong Zhang ◽  
Xiebing Bao ◽  
Huiying Qiu ◽  
Xiaowen Tang ◽  
Yue Han ◽  

Using targeted exome sequencing, we studied correlations between mutations at diagnosis and transplant outcomes in 332 subjects with acute myeloid leukemia (AML) receiving allotransplantation. A total of 299 patients (299/332, 90.1%) had at least one oncogenic point mutation. In multivariable analyses, pretransplant disease status, minimal residual disease (MRD) before transplantation (pre-MRD), cytogenetic risk classification, and TP53 and FLT3-ITDhigh ratio mutations were independent risk factors for AML recurrence after allotransplantation (p &lt; 0.05). A nomogram for the cumulative incidence of relapse (CIR) that integrated all the predictors in the multivariable model was then constructed, and the concordance index (C-index) values at 6, 12, 18, and 24 months for CIR prediction were 0.754, 0.730, 0.715, and 0.690, respectively. Moreover, calibration plots showed good agreements between the actual observation and the nomogram prediction for the 6, 12, 18, and 24 months posttransplantation CIR in the internal validation. The integrated calibration index (ICI) values were 0.008, 0.055, 0.094, and 0.136 at 6, 12, 18, and 24 months posttransplantation, respectively. With a median cutoff score of 9.73 from the nomogram, all patients could be divided into two groups, and the differences in 2-year CIR, disease-free survival (DFS), and overall survival (OS) between these two groups were significant (p &lt; 0.05). Taken together, the results of our study indicate that gene mutations could help to predict the outcomes of patients with AML receiving allotransplantation.

2021 ◽  
Vol 22 (1) ◽  
Huizhen Ye ◽  
Youyuan Chen ◽  
Peiyi Ye ◽  
Yu Zhang ◽  
Xiaoyi Liu ◽  

Abstract Background Chronic kidney disease (CKD) is a common health challenge. There are some risk models predicting CKD adverse outcomes, but seldom focus on the Mongoloid population in East Asian. So, we developed a simple but intuitive nomogram model to predict 3-year CKD adverse outcomes for East Asian patients with CKD. Methods The development and internal validation of prediction models used data from the CKD-ROUTE study in Japan, while the external validation set used data collected at the First People’s Hospital of Foshan in southern China from January 2013 to December 2018. Models were developed using the cox proportional hazards model and nomogram with SPSS and R software. Finally, the model discrimination, calibration and clinical value were tested by R software. Results The development and internal validation data-sets included 797 patients (191 with progression [23.96%]) and 341 patients (89 with progression [26.10%]), respectively, while 297 patients (108 with progression [36.36%]) were included in the external validation data set. The nomogram model was developed with age, eGFR, haemoglobin, blood albumin and dipstick proteinuria to predict three-year adverse-outcome-free probability. The C-statistics of this nomogram were 0.90(95% CI, 0.89–0.92) for the development data set, 0.91(95% CI, 0.89–0.94) for the internal validation data set and 0.83(95% CI, 0.78–0.88) for the external validation data-set. The calibration and decision curve analyses were good in this model. Conclusion This visualized predictive nomogram model could accurately predict CKD three-year adverse outcomes for East Asian patients with CKD, providing an easy-to-use and widely applicable tool for clinical practitioners.

2021 ◽  
Vol 12 ◽  
Fabio Bioletto ◽  
Mirko Parasiliti-Caprino ◽  
Alessandro Maria Berton ◽  
Nunzia Prencipe ◽  
Valeria Cambria ◽  

BackgroundThe diagnosis of adult GH deficiency (GHD) relies on a reduced GH response to provocative tests. Their diagnostic accuracy, however, is not perfect, and a reliable estimation of pre-test GHD probability could be helpful for a better interpretation of their results.MethodsEighty patients showing concordant GH response to two provocative tests, i.e. the insulin tolerance test and the GHRH + arginine test, were enrolled. Data on IGF-I values and on the presence/absence of other pituitary deficits were collected and integrated for the estimation of GHD probability prior to stimulation tests.ResultsAn independent statistically significant association with the diagnosis of GHD was found both for IGF-I SDS (OR 0.34, 95%-CI 0.18-0.65, p=0.001) and for the presence of other pituitary deficits (OR 6.55, 95%-CI 2.06-20.83, p=0.001). A low (&lt;25%) pre-test GHD probability could be predicted when IGF-I SDS &gt; +0.91 in the presence of other pituitary deficits or IGF-I SDS &gt; -0.52 in the absence of other pituitary deficits. A high (&gt;75%) pre-test GHD probability could be predicted when IGF-I SDS &lt; -0.82 in the presence of other pituitary deficits or IGF-I SDS &lt; -2.26 in the absence of other pituitary deficits.ConclusionThis is the first study that proposes a quantitative estimation of GHD probability prior to stimulation tests. Our risk class stratification represents a simple tool that could be adopted for a Bayesian interpretation of stimulation test results, selecting patients who may benefit from a second stimulation test and possibly reducing the risk of wrong GHD diagnosis.

2021 ◽  
Vol 39 (28_suppl) ◽  
pp. 333-333
Kevin Miao ◽  
Justice Dahle ◽  
Sasha Yousefi ◽  
Bilwa Buchake ◽  
Parambir Kaur ◽  

333 Background: Patients undergoing outpatient infusion chemotherapy for cancer are at risk for potentially preventable, unplanned acute care in the form of emergency department (ED) visits and hospital admissions. This can impact outcomes, patient decisions, and costs to the patient and healthcare system. To address this need, the Centers for Medicare & Medicaid Services developed the Chemotherapy Measure (OP-35). Recent randomized controlled data indicate that electronic health record (EHR)-based machine learning (ML) approaches accurately direct supportive care to reduce acute care during radiotherapy. As this may extend to systemic therapy, this study aims to develop and evaluate ML approaches to predict the risk of OP-35 qualifying, potentially preventable acute care within 30 days of infusional systemic therapy. Methods: This study included data from UCSF cancer patients receiving infusional chemotherapy from July 1, 2017, to February 11, 2021, (total 7,068 patients over 84,174 treatments). The data incorporated into the ML included 430 EHR-derived variables, including cancer diagnosis, therapeutic agents, laboratory values, vital signs, medications, and encounter history. Three ML approaches were trained to predict an OP-35 acute care risk following a systemic therapy infusion with least absolute shrinkage selection operator (LASSO), random forest, and gradient boosted trees (GBT; XGBoost) approaches. The models were trained on a subset (75% of patients; before October 12, 2019) of the dataset and validated on a mutually exclusive subset (25% patients; after October 12, 2019) based on the receiver operating characteristic (ROC) curves and calibration plots. Results: There were 1,651 total acute care visits (244 ED visits and 1,407 ED visits converted into hospitalization); 1,310 infusions included a qualifying acute care visit (200 with ED visits only, 0 direct hospital admissions, and 1,110 with both ED visit and hospitalization). Each ML approach demonstrated good performance in the internal validation cohort, with GBT (AUC 0.805) outpacing the random forest (0.750) and LASSO logistic regression (0.755) approaches. Visualization of calibration plots verified concordance between predicted and observed rates of acute care. All three models shared patient age and days elapsed since last treatment as important contributors. Conclusions: EHR-based ML approaches demonstrate high predictive ability for OP-35 qualifying acute care rates on a per-infusion basis, identifying 30-day potentially preventable acute care risk for patients undergoing chemotherapy. Prospective validation of these models is ongoing. Early prediction can facilitate interventional strategies which may reduce acute care, improve health outcomes, and reduce costs.

2021 ◽  
Yijie Yan ◽  
Yue Li ◽  
Chunlei Fan ◽  
Yuening Zhang ◽  
Shibin Zhang ◽  

Abstract Background & aims: To develop and validate a novel machine learning-based radiomic model (RM) for diagnosing high bleeding risk esophageal varices (HREV) in cirrhosis. Methods: In training cohort, total 218 cirrhotic patients for mild esophageal varices (EV) and 240 for HREV RM were enrolled for training and internal validation. In external validation cohort, 159 and 340 cirrhotic patients were respectively used for mild EV and HREV RM validation. Interesting regions of liver, spleen, and esophagus were labeled on the portal venous-phase enhanced CT images. RM was assessed by area under the receiver operating characteristic curves (AUROC), sensitivity, specificity, calibration and decision curve analysis (DCA). Results: The AUROC of mild EV RM in training and internal validation was 0.943 and 0.732, sensitivity and specificity was 0.863, 0.773 and 0.763, 0.763. The AUROC, sensitivity and specificity was 0.654, 0.773 and 0.632 in external validation. Interestingly, the AUROC of HREV RM in training and internal validation was 0.983 and 0.834, sensitivity and specificity was 0.948, 0.916 and 0.977, 0.969. The AUROC, sensitivity and specificity was 0.736, 0.690 and 0.762 in external validation. Calibration and DCA indicated RM had good performance in clinical practice. Compared with Baveno VI and its expanded criteria, HREV RM had a higher accuracy and net reclassification improvement reached 49.0% and 32.8%. Conclusion: A novel non-invasive RM for diagnosing HREV in cirrhotic patients with highly accuracy was developed. However, this RM still needs to be validated by a multi-center large cohort.

Sign in / Sign up

Export Citation Format

Share Document