Prediction of distant metastasis of spinal and pelvic osteosarcoma by constructing and validating nomogram:a retrospective study based on SEER database

Background:Spinal and pelvic osteosarcoma is a rare type of all osteosarcomas,and distant metastasis is an important factor for poor prognosis of this disease. There are no similar studies on prediction of distant metastasis of spinal and pelvic osteosarcoma. We aim to construct and validate a nomogram to predict the risk of distant metastasis of spinal and pelvic osteosarcoma.Methods:We collected the data on patients with spinal and pelvic osteosarcoma from the Surveillance, Epidemiology, and End Results(SEER) database retrospectively. The Kaplan-Meier curve was used to compare differences in survival time between patients with metastasis and non-metastasis. Total patients were randomly divided into training cohort and validation cohort. The risk factor of distant metastasis were identified via the least absolute shrinkage and selection operator(LASSO) regression and multivariate logistic analysis. The nomogram we constructed were validated internally and externally by C-index, calibration curves,receiver operating characteristic(ROC) curve and Decision curve analysis (DCA).Results:The Kaplan-Meier curve showed that the survival time of non-metastatic patients was longer than that of metastatic patients(P＜0.001).All patients(n=358) were divided into training cohort(n=269) and validation cohort(n=89).The LASSO regression selected five meaningful variables in the training cohort. The multivariate logistic regression analysis demonstrated that surgery(yes,OR=0.175, 95%CI=0.095-0.321,p=0.000) was the independent risk factors for distant metastasis of patients with spinal and pelvic osteosarcoma. The C-index and calibration curves showed the good agreement between the predicted results and the actual results. The area under the receiver operating characteristic curve(AUC) values were 0.748(95%CI=0.687-0.817) and 0.758(95%CI=0.631-0.868) in the training and validation cohorts respectively. The DCA showed that the nomogram has a good clinical usefulness and net benefit.Conclusion:No surgery is the independent risk factor of distant metastasis of spinal and pelvic osteosarcoma. The nomogram we constructed to predict the probability of distant metastasis of patients with spinal and pelvic osteosarcoma is reliable and effective by internal and external verification.

Clinical Significance and Potential Role of LSM4 Overexpression in Hepatocellular Carcinoma: An Integrated Analysis Based on Multiple Databases

Background: Hepatocellular carcinoma (HCC) is a solid tumor with high recurrence rate and high mortality. It is crucial to discover available biomarkers to achieve early diagnosis and improve the prognosis. The effect of LSM4 in HCC still remains unrevealed. Our study is dedicated to exploring the expression of LSM4 in HCC, demonstrating its clinical significance and potential molecular mechanisms.Methods: Clinical information and LSM4 expression values of HCC were obtained from Gene Expression Omnibus (GEO) and The Cancer Genome Atlas (TCGA) databases. Survival analysis and receiver operating characteristic (ROC) curve analysis were applied to evaluate the prognostic and diagnostic significance of LSM4. Calculating pooled standardized mean difference (SMD) and performing summary receiver operating characteristic (sROC) curve analysis to further determine its expression status and diagnostic significance. LSM4-related co-expressed genes (CEGs) were obtained and explored their clinical significance in HCC. LSM4-associated pathways were identified through Gene set enrichment analysis (GSEA).Results: Up-regulated LSM4 was detected in HCC tissues (SMD = 1.56, 95% CI: 1.29–1.84) and overexpressed LSM4 had excellent distinguishing ability (AUC = 0.91, 95% CI: 0.88–0.93). LSM4 was associated with clinical stage, tumor grade, and lymph node metastasis status (p < 0.05). Survival analysis showed that high LSM4 expression was related to poor overall survival (OS) of HCC patients. Cox regression analysis suggested that high LSM4 expression may be an independent risk factor for HCC. We obtained nine up-regulated CEGs of LSM4 in HCC tissues, and six CEGs had good prognostic and diagnostic significance. GSEA analysis showed that up-regulated LSM4 was closely related to the cell cycle, cell replication, focal adhesion, and several metabolism-associated pathways, including fatty acid metabolism.Conclusion: Overexpressed LSM4 may serve as a promising diagnostic and prognostic biomarker of HCC. Besides, LSM4 may play a synergistic effect with CEGs in promoting the growth and metastasis of HCC cells via regulating crucial pathways such as cell cycle, focal adhesion, and metabolism-associated pathways.

Preoperative prediction of microvascular invasion (MVI) in hepatocellular carcinoma based on kupffer phase radiomic features of sonazoid contrast-enhanced ultrasound (SCEUS): A prospective study

OBJECTIVES: To establish and evaluate a machine learning radiomics model based on grayscale and Sonazoid contrast enhanced ultrasound images for the preoperative prediction of microvascular invasion (MVI) in hepatocellular carcinoma (HCC) patients. METHODS: 100 cases of histopathological confirmed HCC lesions were prospectively included. Regions of interest were segmented on both grayscale and Kupffer phase of Sonazoid contrast enhanced (CEUS) images. Radiomic features were extracted from tumor region and region containing 5 mm of peritumoral liver tissues. Maximum relevance minimum redundancy (MRMR) and Least Absolute Shrinkage and Selection Operator (LASSO) were used for feature selection and Support Vector Machine (SVM) classifier was trained for radiomic signature calculation. Radiomic signatures were incorporated with clinical variables using univariate-multivariate logistic regression for the final prediction of MVI. Receiver operating characteristic curves, calibration curves and decision curve analysis were used to evaluate model’s predictive performance of MVI. RESULTS: Age were the only clinical variable significantly associated with MVI. Radiomic signature derived from Kupffer phase images of peritumoral liver tissues (kupfferPT) displayed a significantly better performance with an area under the receiver operating characteristic curve (AUROC) of 0.800 (95% confidence interval: 0.667, 0.834), the final prediction model using Age and kupfferPT achieved an AUROC of 0.804 (95% CI: 0.723, 0.878), accuracy of 75.0%, sensitivity of 87.5% and specificity of 69.1%. CONCLUSIONS: Radiomic model based on Kupffer phase ultrasound images of tissue adjacent to HCC lesions showed an observable better predictive value compared to grayscale images and has potential value to facilitate preoperative identification of HCC patients at higher risk of MVI.

“Emergency Room Evaluation and Recommendations” (ER2) Tool for the Screening of Older Emergency Department Visitors With Major Neurocognitive Disorders: Results From the ER2 Database

Purpose: The Emergency Room Evaluation and Recommendation (ER2) is an application in the electronic medical file of patients visiting the Emergency Department (ED) of the Jewish General Hospital (JGH; Montreal, Quebec, Canada). It screens for older ED visitors at high risk of undesirable events. The aim of this study is to examine the performance criteria (i.e., sensitivity, specificity, positive predictive value [PPV], negative predictive value [NPV], positive likelihood ratio [LR+], negative likelihood ratio [LR-] and area under the receiver operating characteristic curve [AUROC]) of the ER2 high-risk level and its “temporal disorientation” item alone to screen for major neurocognitive disorders in older ED visitors at the JGH.Methods: Based on a cross-sectional design, 999 older adults (age 84.9 ± 5.6, 65.1% female) visiting the ED of the JGH were selected from the ER2 database. ER2 was completed upon the patients' arrival at the ED. The outcomes were ER2's high-risk level, the answer to ER2's temporal disorientation item (present vs. absent), and the diagnosis of major neurocognitive disorders (yes vs. no) which was confirmed when it was present in a letter or other files signed by a physician.Results: The sensitivities of both ER2's high-risk level and temporal disorientation item were high (≥0.91). Specificity, the PPV, LR+, and AROC were higher for the temporal disorientation item compared to ER2's high-risk level, whereas a highest sensitivity, LR-, and NPV were obtained with the ER2 high-risk level. Both area under the receiver operating characteristic curves were high (0.71 for ER2's high-risk level and 0.82 for ER2 temporal disorientation item). The odds ratios (OR) of ER2's high-risk level and of temporal disorientation item for the diagnosis of major neurocognitive disorders were positive and significant with all OR above 18, the highest OR being reported for the temporal disorientation item in the unadjusted model [OR = 26.4 with 95% confidence interval (CI) = 17.7–39.3].Conclusion: Our results suggest that ER2 and especially its temporal disorientation item may be used to screen for major neurocognitive disorders in older ED users.

Human Versus Machine: How Do We Know Who Is Winning? ROC Analysis for Comparing Human and Machine Performance under Varying Cost-Prevalence Assumptions

Background Receiver operating characteristic (ROC) analysis is commonly used for comparing models and humans; however, the exact analytical techniques vary and some are flawed. Objectives The aim of the study is to identify common flaws in ROC analysis for human versus model performance, and address them. Methods We review current use and identify common errors. We also review the ROC analysis literature for more appropriate techniques. Results We identify concerns in three techniques: (1) using mean human sensitivity and specificity; (2) assuming humans can be approximated by ROCs; and (3) matching sensitivity and specificity. We identify a technique from Provost et al using dominance tables and cost-prevalence gradients that can be adapted to address these concerns. Conclusion Dominance tables and cost-prevalence gradients provide far greater detail when comparing performances of models and humans, and address common failings in other approaches. This should be the standard method for such analyses moving forward.

Score for the Overall Survival Probability Scores of Fibrosarcoma Patients after Surgery: A Novel Nomogram-Based Risk Assessment System

Background. The primary purpose of this study was to determine the risk factors affecting overall survival (OS) in patients with fibrosarcoma after surgery and to develop a prognostic nomogram in these patients. Methods. Data were collected from the Surveillance, Epidemiology, and End Results database on 439 postoperative patients with fibrosarcoma who underwent surgical resection from 2004 to 2015. Independent risk factors were identified by performing Cox regression analysis on the training set, and based on this, a prognostic nomogram was created. The accuracy of the prognostic model in terms of survival was demonstrated by the area under the curve (AUC) of the receiver operating characteristic curves. In addition, the prediction consistency and clinical value of the nomogram were validated by calibration curves and decision curve analysis. Results. All included patients were divided into a training set (n = 308) and a validation set (n = 131). Based on univariate and multivariate analyses, we determined that age, race, grade, and historic stage were independent risk factors for overall survival after surgery in patients with fibrosarcoma. The AUC of the receiver operating characteristic curves demonstrated the high predictive accuracy of the prognostic nomogram, while the decision curve analysis revealed the high clinical application of the model. The calibration curves showed good agreement between predicted and observed survival rates. Conclusion. We developed a new nomogram to estimate 1-year, 3-year, and 5-year OS based on the independent risk factors. The model has good discriminatory performance and calibration ability for predicting the prognosis of patients with fibrosarcoma after surgery.

Artificial Neural Network Analysis-Based Immune-Related Signatures of Primary Non-Response to Infliximab in Patients With Ulcerative Colitis

Infliximab (IFX) is an effective medication for ulcerative colitis (UC) patients. However, one-third of UC patients show primary non-response (PNR) to IFX. Our study analyzed three Gene Expression Omnibus (GEO) datasets and used the RobustRankAggreg (RRA) algorithm to assist in identifying differentially expressed genes (DEGs) between IFX responders and non-responders. Then, an artificial intelligence (AI) technology, artificial neural network (ANN) analysis, was applied to validate the predictive value of the selected genes. The results showed that the combination of CDX2, CHP2, HSD11B2, RANK, NOX4, and VDR is a good predictor of patients’ response to IFX therapy. The range of repeated overall area under the receiver-operating characteristic curve (AUC) was 0.850 ± 0.103. Moreover, we used an independent GEO dataset to further verify the value of the six DEGs in predicting PNR to IFX, which has a range of overall AUC of 0.759 ± 0.065. Since protein detection did not require fresh tissue and can avoid multiple biopsies, our study tried to discover whether the key information, analyzed by RNA levels, is suitable for protein detection. Therefore, immunohistochemistry (IHC) staining of colonic biopsy tissues from UC patients treated with IFX and a receiver-operating characteristic (ROC) analysis were used to further explore the clinical application value of the six DEGs at the protein level. The IHC staining of colon tissues from UC patients confirmed that VDR and RANK are significantly associated with IFX efficacy. Total IHC scores lower than 5 for VDR and lower than 7 for RANK had an AUC of 0.828 (95% CI: 0.665–0.991, p = 0.013) in predicting PNR to IFX. Collectively, we identified a predictive RNA model for PNR to IFX and explored an immune-related protein model based on the RNA model, including VDR and RANK, as a predictor of IFX non-response, and determined the cutoff value. The result showed a connection between the RNA and protein model, and both two models were available. However, the composite signature of VDR and RANK is more conducive to clinical application, which could be used to guide the preselection of patients who might benefit from pharmacological treatment in the future.

Considerations on the region of interest in the ROC space

Receiver Operating Characteristic curves have been widely used to represent the performance of diagnostic tests. The corresponding area under the curve, widely used to evaluate their performance quantitatively, has been criticized in several respects. Several proposals have been introduced to improve area under the curve by taking into account only specific regions of the Receiver Operating Characteristic space, that is, the plane to which Receiver Operating Characteristic curves belong. For instance, a region of interest can be delimited by setting specific thresholds for the true positive rate or the false positive rate. Different ways of setting the borders of the region of interest may result in completely different, even opposing, evaluations. In this paper, we present a method to define a region of interest in a rigorous and objective way, and compute a partial area under the curve that can be used to evaluate the performance of diagnostic tests. The method was originally conceived in the Software Engineering domain to evaluate the performance of methods that estimate the defectiveness of software modules. We compare this method with previous proposals. Our method allows the definition of regions of interest by setting acceptability thresholds on any kind of performance metric, and not just false positive rate and true positive rate: for instance, the region of interest can be determined by imposing that [Formula: see text] (also known as the Matthews Correlation Coefficient) is above a given threshold. We also show how to delimit the region of interest corresponding to acceptable costs, whenever the individual cost of false positives and false negatives is known. Finally, we demonstrate the effectiveness of the method by applying it to the Wisconsin Breast Cancer Data. We provide Python and R packages supporting the presented method.

Prediction of extubation failure in the paediatric cardiac ICU using machine learning and high-frequency physiologic data

Background: Cardiac intensivists frequently assess patient readiness to wean off mechanical ventilation with an extubation readiness trial despite it being no more effective than clinician judgement alone. We evaluated the utility of high-frequency physiologic data and machine learning for improving the prediction of extubation failure in children with cardiovascular disease. Methods: This was a retrospective analysis of clinical registry data and streamed physiologic extubation readiness trial data from one paediatric cardiac ICU (12/2016-3/2018). We analysed patients’ final extubation readiness trial. Machine learning methods (classification and regression tree, Boosting, Random Forest) were performed using clinical/demographic data, physiologic data, and both datasets. Extubation failure was defined as reintubation within 48 hrs. Classifier performance was assessed on prediction accuracy and area under the receiver operating characteristic curve. Results: Of 178 episodes, 11.2% (N = 20) failed extubation. Using clinical/demographic data, our machine learning methods identified variables such as age, weight, height, and ventilation duration as being important in predicting extubation failure. Best classifier performance with this data was Boosting (prediction accuracy: 0.88; area under the receiver operating characteristic curve: 0.74). Using physiologic data, our machine learning methods found oxygen saturation extremes and descriptors of dynamic compliance, central venous pressure, and heart/respiratory rate to be of importance. The best classifier in this setting was Random Forest (prediction accuracy: 0.89; area under the receiver operating characteristic curve: 0.75). Combining both datasets produced classifiers highlighting the importance of physiologic variables in determining extubation failure, though predictive performance was not improved. Conclusion: Physiologic variables not routinely scrutinised during extubation readiness trials were identified as potential extubation failure predictors. Larger analyses are necessary to investigate whether these markers can improve clinical decision-making.

Receiver operating characteristic (ROC) curves: equivalences, beta model, and minimum distance estimation

Receiver operating characteristic (ROC) curves are used ubiquitously to evaluate scores, features, covariates or markers as potential predictors in binary problems. We characterize ROC curves from a probabilistic perspective and establish an equivalence between ROC curves and cumulative distribution functions (CDFs). These results support a subtle shift of paradigms in the statistical modelling of ROC curves, which we view as curve fitting. We propose the flexible two-parameter beta family for fitting CDFs to empirical ROC curves and derive the large sample distribution of minimum distance estimators in general parametric settings. In a range of empirical examples the beta family fits better than the classical binormal model, particularly under the vital constraint of the fitted curve being concave.