Measuring diagnostic accuracy for biomarkers under tree-ordering

2018 ◽  
Vol 28 (5) ◽  
pp. 1328-1346
Author(s):  
Yingdong Feng ◽  
Lili Tian
Keyword(s):  
Diagnostic Accuracy ◽  
Microarray Data ◽  
Roc Curve ◽  
Confidence Region ◽  
False Negative ◽  
False Negative Rate ◽  
Simulation Studies ◽  
Data Set ◽  
Diagnostic Measures ◽  
Joint Confidence Region

In the field of diagnostic studies for tree or umbrella ordering, under which the marker measurement for one class is lower or higher than those for the rest unordered classes, there exist a few diagnostic measures such as the naive AUC ( NAUC), the umbrella volume ( UV), and the recently proposed TAUC, i.e. area under a ROC curve for tree or umbrella ordering (TROC). However, an important characteristic about tree or umbrella ordering has been neglected. This paper mainly focuses on promoting the use of the integrated false negative rate under tree ordering ( ITFNR) as an additional diagnostic measure besides TAUC, and proposing the idea of using ( TAUC, ITFNR) instead of TAUC to evaluate the diagnostic accuracy of a biomarker under tree or umbrella ordering. Parametric and non-parametric approaches for constructing joint confidence region of ( TAUC, ITFNR) are proposed. Simulation studies under a variety of settings are carried out to assess and compare the performance of these methods. In the end, a published microarray data set is analyzed.

One- year mortality in patients with advanced hepatocellular carcinoma on immunotherapy: Prediction using machine learning models (Preprint)

2021 ◽  
Author(s):  
Thomas Ka-Luen Lui ◽  
Ka Shing, Michael Cheung ◽  
Wai Keung Leung
Keyword(s):  
Machine Learning ◽  
Hepatocellular Carcinoma ◽  
Characteristic Curve ◽  
False Negative ◽  
False Negative Rate ◽  
Absolute Error ◽  
Advanced Hepatocellular Carcinoma ◽  
Data Set ◽  
One Year ◽  
Related Mortality

BACKGROUND Immunotherapy is a new promising treatment for patients with advanced hepatocellular carcinoma (HCC), but is costly and potentially associated with considerable side effects. OBJECTIVE This study aimed to evaluate the role of machine learning (ML) models in predicting the one-year cancer-related mortality in advanced HCC patients treated with immunotherapy METHODS 395 HCC patients who had received immunotherapy (including nivolumab, pembrolizumab or ipilimumab) in 2014 - 2019 in Hong Kong were included. The whole data set were randomly divided into training (n=316) and validation (n=79) set. The data set, including 45 clinical variables, was used to construct six different ML models in predicting the risk of one-year mortality. The performances of ML models were measured by the area under receiver operating characteristic curve (AUC) and the mean absolute error (MAE) using calibration analysis. RESULTS The overall one-year cancer-related mortality was 51.1%. Of the six ML models, the random forest (RF) has the highest AUC of 0.93 (95%CI: 0.86-0.98), which was better than logistic regression (0.82, p=0.01) and XGBoost (0.86, p=0.04). RF also had the lowest false positive (6.7%) and false negative rate (2.8%). High baseline AFP, bilirubin and alkaline phosphatase were three common risk factors identified by all ML models. CONCLUSIONS ML models could predict one-year cancer-related mortality of HCC patients treated with immunotherapy, which may help to select patients who would most benefit from this new treatment option.

scholarly journals An assessment of the accuracy of contrast enema for the diagnosis of Hirschsprung disease at a South African tertiary hospital

2017 ◽  
Vol 21 (1) ◽  
Author(s):  
Monica S. Msomi ◽  
Hansraj Mangray ◽  
Vicci Du Plessis
Keyword(s):  
Diagnostic Accuracy ◽  
South African ◽  
False Negative ◽  
False Negative Rate ◽  
Hirschsprung Disease ◽  
Contrast Enema ◽  
Initial Screening ◽  
Tertiary Referral Hospital ◽  
Predictive Values ◽  
Picture Archiving And Communication

Objectives: To compare radiological findings with the histological diagnosis of Hirschsprung disease (HD) to establish the usefulness of contrast enema as an initial screening and diagnostic tool. To correlate accuracy of radiological diagnosis at Grey’s Hospital with international standards.Materials and methods: Systematic searches were conducted through the Picture Archiving and Communication System and the National Health Laboratory Service records for patients aged 0–12 years, with clinically suspected HD, for whom both contrast enemas and rectal biopsies were performed between 01 January 2011 and 31 August 2015 in a tertiary-level hospital. A total of 54 such patients were identified. Diagnostic accuracy levels were calculated by comparing radiological results with histology results, which is the gold standard.Results: Diagnostic accuracy of contrast enema was 78%, sensitivity was 94.4% and the negative predictive value was 95.7%. Specificity (68.8%) and positive predictive values (63%) were considerably lower. A lower false-negative rate of 5.6% was obtained at Grey’s Hospital as compared with the international reports of up to 30%.Conclusion: Contrast enema remains useful as an initial screening and diagnostic test for HD. Results of this South African tertiary referral hospital were consistent with the best international results for sensitivity of the contrast enema (approximately 80% – 88% in excluding the disease).

Correlation of decreased expression of PD-L1 on circulating tumor cells and clinical benefit in SCLC Patients treated with RRx-001, a CD47 downregulator, in a phase II trial.

2020 ◽  
Vol 38 (15_suppl) ◽  
pp. 9062-9062
Author(s):  
Corey Carter ◽  
Yusuke Tomita ◽  
Akira Yuno ◽  
Jonathan Baker ◽  
Min-Jung Lee ◽  
...  
Keyword(s):  
Tumor Cells ◽  
Circulating Tumor Cells ◽  
Clinical Benefit ◽  
Logistic Model ◽  
Goodness Of Fit ◽  
False Negative ◽  
False Negative Rate ◽  
Cycle Duration ◽  
Data Set ◽  
Platinum Doublet

9062 Background: In a Phase 2 trial called QUADRUPLE THREAT (QT) (NCT02489903), where 2nd line+ small cell lung cancer (SCLC) patients were treated with RRx-001 and a platinum doublet, the programmed death-ligand 1 (PD-L1) status of circulating tumor cells (CTCs) in 14 patient samples was evaluated. Methods: 26 consented patients received weekly RRx-001 4 mg followed by a reintroduced platinum doublet; epithelial cell adhesion molecule (EPCAM+) CTCs from 10 ml of blood on two consecutive timepoints cycle 1 day 1 and cycle 3 day 8 (cycle duration = 1 week) were detected by EpCAM-based immunomagnetic capture and flow cytometric analysis. CTCs were further characterized for protein expression of PD-L1. Tumor response was classified as partial or complete response based on the response evaluation criteria in solid tumors (RECISTv1.1) measured every 6 weeks. Results: The analyzed clinical data set comprised 14 RECIST-evaluable patients. 50% were females (7/14) and the median age (years) at baseline was 64.5 (Min = 48.5, Max = 84.2, SD = 10.3). The logistic model McFadden goodness of fit score (0 to 100) is 0.477, which is a strong correlation value. The logistic model analyzing the association of CTC PD-L1 expression at two timepoints and response had an approximate 92.8% accuracy in its prediction of clinical benefit (SD/PR/CR). Accuracy is defined in the standard way as 1- (False positive rate + False negative rate). The estimated ROC displayed in Figure 1 suggests a ROC AUC of 0.93 (95% CI: 0.78, 0.99), an excellent measure of performance. Conclusions: Reduction of PD-L1 expression was correlated with good clinical outcome after RRx-001 + platinum doublet treatment. PD-L1 expression reduction in favor of RRx-001 RECIST clinical benefit was clinically significant as compared to non-responders with progressive disease (PD). In the ongoing SCLC Phase 3 study called REPLATINUM (NCT03699956), analyses are planned to correlate response and survival with expression of CD47 and PD-L1 on CTCs. Clinical trial information: NCT02489903.

scholarly journals ED misdiagnosis of cerebrovascular events in the era of modern neuroimaging

Neurology ◽  
2017 ◽  
Vol 88 (15) ◽  
pp. 1468-1477 ◽  
Author(s):  
Alexander Andrea Tarnutzer ◽  
Seung-Han Lee ◽  
Karen A. Robinson ◽  
Zheyu Wang ◽  
Jonathan A. Edlow ◽  
...  
Keyword(s):  
Ischemic Stroke ◽  
Diagnostic Accuracy ◽  
Clinical Presentation ◽  
False Negative ◽  
False Negative Rate ◽  
Diagnostic Errors ◽  
Risk Of Bias ◽  
Future Research ◽  
Negative Rate ◽  
Cerebrovascular Events

Objective:With the emergency department (ED) being a high-risk site for diagnostic errors, we sought to estimate ED diagnostic accuracy for identifying acute cerebrovascular events.Methods:MEDLINE and Embase were searched for studies (1995–2016) reporting ED diagnostic accuracy for ischemic stroke, TIA, or subarachnoid hemorrhage (SAH). Two independent reviewers determined inclusion. We identified 1,693 unique citations, examined 214 full articles, and analyzed 23 studies. Studies were rated on risk of bias (QUADAS-2). Diagnostic data were extracted. We prospectively defined clinical presentation subgroups to compare odds of misdiagnosis.Results:Included studies reported on 15,721 patients. Studies were at low risk of bias. Overall sensitivity (91.3% [95% confidence interval (CI) 90.7–92.0]) and specificity (92.7% [91.7–93.7]) for a cerebrovascular etiology was high, but there was significant variation based on clinical presentation. Misdiagnosis was more frequent among subgroups with milder (SAH with normal vs abnormal mental state; false-negative rate 23.8% vs 4.2%, odds ratio [OR] 7.03 [4.80–10.31]), nonspecific (dizziness vs motor findings; false-negative rate 39.4% vs 4.4%, OR 14.22 [9.76–20.74]), or transient (TIA vs ischemic stroke; false discovery rate 59.7% vs 11.7%, OR 11.21 [6.66–18.89]) symptoms.Conclusions:Roughly 9% of cerebrovascular events are missed at initial ED presentation. Risk of misdiagnosis is much greater when presenting neurologic complaints are mild, nonspecific, or transient (range 24%–60%). This difference suggests that many misdiagnoses relate to symptom-specific factors. Future research should emphasize studying causes and designing error-reduction strategies in symptom-specific subgroups at greatest risk of misdiagnosis.

scholarly journals Sensitivity, Specificity and Accuracy of Color Doppler Ultrasonography for Diagnosis of Retinal Detachment

2020 ◽  
Author(s):  
Mohammadreza Akhlaghi ◽  
Masoomeh Zarei ◽  
Majid Ziaei ◽  
Mohsen Pourazizi
Keyword(s):  
Retinal Detachment ◽  
Diagnostic Accuracy ◽  
Doppler Ultrasonography ◽  
Color Doppler ◽  
False Positive Rate ◽  
False Negative ◽  
False Negative Rate ◽  
Color Doppler Ultrasonography ◽  
Promising Tool ◽  
Sensitivity Specificity

Purpose: This study evaluated the sensitivity, specificity, and diagnostic accuracy of Color Doppler Ultrasonography (CDUS) in patients with suspected retinal detachment (RD) who underwent surgery. Methods: In this prospective, observational clinical study, 65 eyes of 65 consecutive patients with suspected RD with opaque media were included. Following a standardized protocol, CDUS of the retina of the affected eye was performed. The sensitivity, specificity, and diagnostic accuracy of CDUS were determined and compared to the findings during surgery. Results: The mean age of patients (18 men and 47 women) was 52.36 years (range: 8–77 years). The sensitivity, specificity, and overall accuracy of ocular CDUS were 91.3%, 88.1%, and 89.2%, respectively. The false-negative rate (negative CDUS images but presence of RD at operation) was 3.1% (2/65) and the false-positive rate (positive CDUS images but absence of RD at operation) was 7.7% (5/65). Conclusion: CDUS of the retina could be considered as a promising tool in the diagnosis of RD in patients with opaque media.

Prevalence and diagnostic accuracy of in-toeing and out-toeing of the foot for patients with abnormal femoral torsion and femoroacetabular impingement

2019 ◽  
Vol 101-B (10) ◽  
pp. 1218-1229 ◽  
Author(s):  
Till D. Lerch ◽  
Patric Eichelberger ◽  
Heiner Baur ◽  
Florian Schmaranzer ◽  
Emanuel F. Liechti ◽  
...  
Keyword(s):  
Diagnostic Accuracy ◽  
Ct Scan ◽  
Femoroacetabular Impingement ◽  
Hip Dysplasia ◽  
False Positive Rate ◽  
False Negative ◽  
False Negative Rate ◽  
Controlled Study ◽  
Femoral Torsion ◽  
Foot Position

Aims Abnormal femoral torsion (FT) is increasingly recognized as an additional cause for femoroacetabular impingement (FAI). It is unknown if in-toeing of the foot is a specific diagnostic sign for increased FT in patients with symptomatic FAI. The aims of this study were to determine: 1) the prevalence and diagnostic accuracy of in-toeing to detect increased FT; 2) if foot progression angle (FPA) and tibial torsion (TT) are different among patients with abnormal FT; and 3) if FPA correlates with FT. Patients and Methods A retrospective, institutional review board (IRB)-approved, controlled study of 85 symptomatic patients (148 hips) with FAI or hip dysplasia was performed in the gait laboratory. All patients had a measurement of FT (pelvic CT scan), TT (CT scan), and FPA (optical motion capture system). We allocated all patients to three groups with decreased FT (< 10°, 37 hips), increased FT (> 25°, 61 hips), and normal FT (10° to 25°, 50 hips). Cluster analysis was performed. Results We found a specificity of 99%, positive predictive value (PPV) of 93%, and sensitivity of 23% for in-toeing (FPA < 0°) to detect increased FT > 25°. Most of the hips with normal or decreased FT had no in-toeing (false-positive rate of 1%). Patients with increased FT had significantly (p < 0.001) more in-toeing than patients with decreased FT. The majority of the patients (77%) with increased FT walk with a normal foot position. The correlation between FPA and FT was significant (r = 0.404, p < 0.001). Five cluster groups were identified. Conclusion In-toeing has a high specificity and high PPV to detect increased FT, but increased FT can be missed because of the low sensitivity and high false-negative rate. These results can be used for diagnosis of abnormal FT in patients with FAI or hip dysplasia undergoing hip arthroscopy or femoral derotation osteotomy. However, most of the patients with increased FT walk with a normal foot position. This can lead to underestimation or misdiagnosis of abnormal FT. We recommend measuring FT with CT/MRI scans in all patients with FAI. Cite this article: Bone Joint J 2019;101-B:1218–1229

scholarly journals False-Negative Rate of Gram-Stain Microscopy for Diagnosis of Septic Arthritis: Suggestions for Improvement

2014 ◽  
Vol 2014 ◽  
pp. 1-4 ◽  
Author(s):  
Paul Stirling ◽  
Radwane Faroug ◽  
Suheil Amanat ◽  
Abdulkhaled Ahmed ◽  
Malcolm Armstrong ◽  
...  
Keyword(s):  
Synovial Fluid ◽  
Septic Arthritis ◽  
False Negative ◽  
False Negative Rate ◽  
Diagnostic Value ◽  
Royal Infirmary ◽  
Gram Stain ◽  
Data Set ◽  
Ongoing Research ◽  
Negative Rate

We quantify the false-negative diagnostic rate of septic arthritis using Gram-stain microscopy of synovial fluid and compare this to values reported in the peer-reviewed literature. We propose a method of improving the diagnostic value of Gram-stain microscopy using Lithium Heparin containers that prevent synovial fluid coagulation. Retrospective study of the Manchester Royal Infirmary microbiology database of patients undergoing synovial fluid Gram-stain and culture between December 2003 and March 2012 was undertaken. The initial cohort of 1896 synovial fluid analyses for suspected septic arthritis was reduced to 143 after exclusion criteria were applied. Analysis of our Gram-stain microscopy yielded 111 false-negative results from a cohort size of 143 positive synovial fluid cultures, giving a false-negative rate of 78%. We report a false-negative rate of Gram-stain microscopy for septic arthritis of 78%. Clinicians should therefore avoid the investigation until a statistically significant data set confirms its efficacy. The investigation's value could be improved by using Lithium Heparin containers to collect homogenous synovial fluid samples. Ongoing research aims to establish how much this could reduce the false-negative rate.

scholarly journals Using an artificial neural network to predict traumatic brain injury

2019 ◽  
Vol 23 (2) ◽  
pp. 219-226 ◽  
Author(s):  
Andrew T. Hale ◽  
David P. Stonko ◽  
Jaims Lim ◽  
Oscar D. Guillamondegui ◽  
Chevis N. Shannon ◽  
...  
Keyword(s):  
Neural Network ◽  
Traumatic Brain Injury ◽  
Artificial Neural Network ◽  
Brain Injury ◽  
False Negative ◽  
False Negative Rate ◽  
Research Network ◽  
Direct Result ◽  
Data Set ◽  
Artificial Neural

OBJECTIVEPediatric traumatic brain injury (TBI) is common, but not all injuries require hospitalization. A computational tool for ruling in patients who will have a clinically relevant TBI (CRTBI) would be valuable, providing an evidence-based way to safely discharge children who are at low risk for a CRTBI. The authors hypothesized that an artificial neural network (ANN) trained on clinical and radiologist-interpreted imaging metrics could provide a tool for identifying patients likely to suffer from a CRTBI.METHODSThe authors used the prospectively collected, publicly available, multicenter Pediatric Emergency Care Applied Research Network (PECARN) TBI data set. All patients under the age of 18 years with TBI and admission head CT imaging data were included. The authors constructed an ANN using clinical and radiologist-interpreted imaging metrics in order to predict a CRTBI, as previously defined by PECARN: 1) neurosurgical procedure, 2) intubation > 24 hours as direct result of the head trauma, 3) hospitalization ≥ 48 hours and evidence of TBI on a CT scan, or 4) death due to TBI.RESULTSAmong 12,902 patients included in this study, 480 were diagnosed with CRTBI. The authors’ ANN had a sensitivity of 99.73% with precision of 98.19%, accuracy of 97.98%, negative predictive value of 91.23%, false-negative rate of 0.0027%, and specificity for CRTBI of 60.47%. The area under the receiver operating characteristic curve was 0.9907.CONCLUSIONSThe authors are the first to utilize artificial intelligence to predict a CRTBI in a clinically meaningful manner, using radiologist-interpreted CT information, in order to identify pediatric patients likely to suffer from a CRTBI. This proof-of-concept study lays the groundwork for future studies incorporating iterations of this algorithm directly into the electronic medical record for real-time, data-driven predictive assistance to physicians.

Sign in / Sign up

Export Citation Format

Share Document