A bivariate contaminated binormal model for robust fitting of proper ROC curves to a pair of correlated, possibly degenerate, ROC datasets

Receiver operating characteristic (ROC) curves are a frequent tool to study the discriminating ability of a certain characteristic. The area under the ROC curve (AUC) is a widely used measure of statistical accuracy of continuous markers for diagnostic tests, and has the advantage of providing a single summary index of overall performance of the test. Recent studies have shown some critical issues related to traditional point and interval estimates for the AUC, especially for small samples, more complex models, unbalanced samples or values near the boundary of the parameter space, i.e., when the AUC approaches the values 0.5 or 1.Parametric models for the AUC have shown to be powerful when the underlying distributional assumptions are not misspecified. However, in the above circumstances parametric inference may be not accurate, sometimes yielding misleading conclusions. The objective of the paper is to propose an alternative inferential approach based on modified profile likelihoods, which provides more accurate statistical results in any parametric settings, including the above circumstances. The proposed method is illustrated for the binormal model, but can potentially be used in any other complex model and for any other parametric distribution. We report simulation studies to show the improved performance of the proposed approach, when compared to classical first-order likelihood theory. An application to real-life data in a small sample setting is also discussed, to provide practical guidelines.

Lehmann Family of ROC Curves

Medical Decision Making ◽

10.1177/0272989x09360067 ◽

2010 ◽

Vol 30 (4) ◽

pp. 509-517 ◽

Cited By ~ 16

Author(s):

Mithat Gönen ◽

Glenn Heller

Keyword(s):

Roc Curve ◽

Operating Characteristic ◽

Proportional Hazards ◽

Model Fitting ◽

Area Under The Curve ◽

Roc Curves ◽

Binary Outcome ◽

Monotone Transformation ◽

Binormal Model ◽

Asymptotic Variances

Receiver operating characteristic (ROC) curves evaluate the discriminatory power of a continuous marker to predict a binary outcome. The most popular parametric model for an ROC curve is the binormal model, which assumes that the marker, after a monotone transformation, is normally distributed conditional on the outcome. Here, the authors present an alternative to the binormal model based on the Lehmann family, also known as the proportional hazards specification. The resulting ROC curve and its functionals (such as the area under the curve and the sensitivity at a given level of specificity) have simple analytic forms. Closed-form expressions for the functional estimates and their corresponding asymptotic variances are derived. This family accommodates the comparison of multiple markers, covariate adjustments, and clustered data through a regression formulation. Evaluation of the underlying assumptions, model fitting, and model selection can be performed using any off-the-shelf proportional hazards statistical software package.

Comparison of diagnostic accuracy of 18F-FDG PET, 123I-IMT- and 99mTc-MIBI SPECT

Nuklearmedizin ◽

10.1055/s-0038-1623934 ◽

2006 ◽

Vol 45 (01) ◽

pp. 49-56 ◽

Cited By ~ 2

Author(s):

N. Özdemir-Sahin ◽

P. Hipp ◽

W. Mier ◽

M. Eisenhut ◽

J. Debus ◽

...

Keyword(s):

Diagnostic Accuracy ◽

Fdg Pet ◽

Tumour Progression ◽

Roc Curves ◽

Low Grade ◽

Who Grade ◽

Pet Tracer ◽

Patient Groups ◽

99Mtc Mibi ◽

Mibi Spect

Summary Aim was to evaluates the diagnostic accuracy of the SPECTtracers 3-123I-α-methyl-L-tyrosine (IMT) and 99mTc(I)- hexakis(2-methoxyisobutylisonitrile) (MIBI) as well as the PET-tracer 2-18F-2-deoxyglucose (FDG) for detecting tumour progression in irradiated low grade astrocytomas (LGA). Patients, methods: We examined 91 patients (56 males; 35 females; 44.7 ± 11.5 years), initially suffering from histologically proven LGAs (mean WHO grade II) and treated by stereotactic radiotherapy (59.0 ± 4.6 Gy). On average 21.9 ± 11.2 months after radiotherapy, patients presented new Gd-DTPA enhancing lesions on MRI, which did not allow a differentiation between progressive tumour (PT) and non-PT (nPT) at this point of time. PET scans (n=82) were acquired 45 min after injection of 208 ± 32 MBq FDG. SPECT scans started 10 min after injection of 269 ± 73 MBq IMT (n=68) and 15 min after injection of 706 ± 63 MBq MIBI (n=34). Lesions were classified as PT and nPT based on prospective follow-up (clinically, MRI) for 17.2 ± 9.9 months after PET/SPECT. Lesion-to-normal ratios (L/N) were calculated using contra lateraly mirrored reference regions for the SPECT examinations and reference regions in the contra lateral grey (GM) and white matter (WM) for FDG PET. Ratios were evaluated by Receiver Operating Characteristic (ROC) analysis. Results: In the patient groups nPT and PT, L/N ratios for FDG (GS) were 0.6 ± 0.3 vs. 1.2 ± 0.5 (p = 0.003), for FDG (WS) 1.2 ± 0.4 vs. 2.6 ± 0.4 (p <0.001), for IMT 1.1 ± 0.1 vs. 1.8 ± 0.4 (p <0.001) and for MIBI 1.6 ± 0.7 vs. 2.6 ± 2.2 (p = 0.554). Areas under the non-parametric ROC-curves were: 0.738 ± 0.059 for FDG (GS), 0.790 ± 0.057 for FDG (WS), 0.937 ± 0.037 for IMT and 0.564 ± 0.105 for MIBI. Conclusion: MIBI-SPECT examinations resulted in a low accuracy and especially in a poor sensitivity even at modest specificity values. A satisfying diagnostic accuracy was reached with FDG PET. Using WM as reference region for FDG PET, a slightly higher AUC as compared to GM was calculated. IMT yielded the best ROC characteristics and the highest diagnostic accuracy for differentiating between PT and nPT in irradiated LGA.

57Co-Bleomycin Scintigraphy in Staging of Carcinoma of the Cervix

Nuklearmedizin ◽

10.1055/s-0038-1624279 ◽

1985 ◽

Vol 24 (01) ◽

pp. 48-51 ◽

Cited By ~ 2

Author(s):

K. Dursk ◽

T. Pertyński

Keyword(s):

Cervical Cancer ◽

Roc Curves ◽

Selection Of

Summary65 patients with cervical cancer at various stages of the disease were studied scintigraphically after 57Co-bleomycin administration, active urine being flushed out from the bladder continuously. The patients were submitted to other auxiliary tests and the interpretation of scintigraphic images could thus be verified. Scintigraphic data were classified semiquantitatively by scoring in the range from zero to 5. Results were evaluated in terms of ROC-curves. This method of analysis permitted selection of cut-off levels for binary decisions that yielded satisfactory sensitivity and acceptable specificity of the test.

Use of Receiver Operating Characteristic (ROC) Curves to Evaluate Computer Confidence Threshold and Clinical Performance in the Diagnosis of Appendicitis

Methods of Information in Medicine ◽

10.1055/s-0038-1636435 ◽

1978 ◽

Vol 17 (03) ◽

pp. 157-161 ◽

Cited By ~ 12

Author(s):

F. T. De Dombal ◽

Jane C. Horrocks

Keyword(s):

Receiver Operating Characteristic ◽

Operating Characteristic ◽

Clinical Performance ◽

Roc Curves ◽

Final Diagnosis ◽

Confidence Threshold ◽

Computer Aided ◽

Computer Confidence ◽

Overall Performance ◽

Receiver Operating

This paper uses simple receiver operating characteristic (ROC) curves (i) to study the effect of varying computer confidence of threshold levels and (ii) to evaluate clinical performance in the diagnosis of acute appendicitis. Over 1300 patients presenting to five centres with abdominal pain of short duration were studied in varying detail. Clinical and computer-aided diagnostic predictions were compared with the »final« diagnosis. From these studies it is concluded the simplistic setting of a 50/50 confidence threshold for the computer program is as »good« as any other. The proximity of a computer-aided system changed clinical behaviour patterns; a higher overall performance level was achieved and clinicians performance levels became associated with the »mildly conservative« end of the computers ROC curve. Prior forecasts of over-confidence or ultra-caution amongst clinicians using the computer-aided system have not been fulfilled.

Evaluation of the clot burden score (cbs) for acute ischemic stroke (ais) in intent-to-treat patients.

Journal Of Cardiovascular, Neurovascular & Stroke ◽

10.32896/cvns.v1n1.11-15 ◽

2019 ◽

Vol 1 (1) ◽

pp. 11-15 ◽

Cited By ~ 1

Author(s):

Sarah Yaziz ◽

Ahmad Sobri Muda ◽

Wan Asyraf Wan Zaidi ◽

Nik Azuan Nik Ismail

Keyword(s):

Ischemic Stroke ◽

Clinical Outcome ◽

Acute Ischemic Stroke ◽

Predictive Value ◽

Area Under The Curve ◽

Rank Correlation ◽

Treatment Decision ◽

Roc Curves ◽

Significant Negative Correlation ◽

Clot Burden

Background : The clot burden score (CBS) is a scoring system used in acute ischemic stroke (AIS) to predict patient outcome and guide treatment decision. However, CBS is not routinely practiced in many institutions. This study aimed to investigate the feasibility of CBS as a relevant predictor of good clinical outcome in AIS cases. Methods: A retrospective data collection and review of AIS patients in a teaching hospital was done from June 2010 until June 2015. Patients were selected following the inclusion and exclusion criteria. These patients were followed up after 90 days of discharge. The Modified Rankin scale (mRS) was used to assess their outcome (functional status). Linear regression Spearman Rank correlation was performed between the CBS and mRS. The quality performance of the correlations was evaluated using Receiver operating characteristic (ROC) curves. Results: A total of 89 patients with AIS were analysed, 67.4% (n=60) male and 32.6% (n=29) female. Twenty-nine (29) patients (33.7%) had a CBS ?6, 6 patients (6.7%) had CBS <6, while 53 patients (59.6%) were deemed clot free. Ninety (90) days post insult, clinical assessment showed that 57 (67.6%) patients were functionally independent, 27 (30.3%) patients functionally dependent, and 5 (5.6%) patients were deceased. Data analysis reported a significant negative correlation (r= -0.611, p<0.001). ROC curves analysis showed an area under the curve of 0.81 at the cut-off point of 6.5. This showed that a CBS of more than 6 predicted a good mRS clinical outcome in AIS patients; with sensitivity of 98.2%, specificity of 53.1%, positive predictive value (PPV) of 76%, and negative predictive value (NPV) of 21%. Conclusion: CBS is a useful additional variable for the management of AIS cases, and should be incorporated into the routine radiological reporting for acute ischemic stroke (AIS) cases.

Conscious access to fear-relevant information is mediated by threshold

Polish Psychological Bulletin ◽

10.2478/v10059-011-0009-7 ◽

2011 ◽

Vol 42 (2) ◽

pp. 56-64 ◽

Cited By ~ 2

Author(s):

Remigiusz Szczepanowski

Keyword(s):

Present Report ◽

Intrinsic Property ◽

Roc Curves ◽

Relevant Information ◽

Perceptual Processing ◽

High Threshold ◽

Operating Characteristics ◽

Emotional Perception ◽

Fearful Faces ◽

The Subject

Conscious access to fear-relevant information is mediated by thresholdThe present report proposed a model of access consciousness to fear-relevant information according to which there is a threshold for emotional perception beyond that the subject makes hits with no false alarm. The model was examined by having the participants performed a confidence-ratings masking task with fearful faces. Measures of the thresholds for conscious access were taken by looking at the receiver operating characteristics (ROC) curves generated from a three-state low- and high-threshold (3-LHT) model by Krantz. Indeed, the analysis of the masking data revealed that the ROCs had threshold-like-nature (a two-limb shape) rather continuous (a curvilinear shape) challenging in this fashion the classical signal-detection view on perceptual processing. Moreover, the threshold ROC curve exhibited the specific y-intercepts relevant to conscious access performance. The study suggests that the threshold can be an intrinsic property of conscious access, mediating emotional contents between perceptual states and consciousness.

MicroRNAs-1299, -126-3p and -30e-3p as Potential Diagnostic Biomarkers for Prediabetes

Diagnostics ◽

10.3390/diagnostics11060949 ◽

2021 ◽

Vol 11 (6) ◽

pp. 949

Author(s):

Cecil J. Weale ◽

Don M. Matshazi ◽

Saarah F. G. Davids ◽

Shanel Raghubeer ◽

Rajiv T. Erasmus ◽

...

Keyword(s):

Receiver Operating Characteristic ◽

Operating Characteristic ◽

Characteristic Curve ◽

Predictive Ability ◽

Roc Curves ◽

Tolerance Test ◽

Cross Sectional Study ◽

Oral Glucose ◽

Cross Sectional ◽

Receiver Operating

This cross-sectional study investigated the association of miR-1299, -126-3p and -30e-3p with and their diagnostic capability for dysglycaemia in 1273 (men, n = 345) South Africans, aged >20 years. Glycaemic status was assessed by oral glucose tolerance test (OGTT). Whole blood microRNA (miRNA) expressions were assessed using TaqMan-based reverse transcription quantitative-PCR (RT-qPCR). Receiver operating characteristic (ROC) curves assessed the ability of each miRNA to discriminate dysglycaemia, while multivariable logistic regression analyses linked expression with dysglycaemia. In all, 207 (16.2%) and 94 (7.4%) participants had prediabetes and type 2 diabetes mellitus (T2DM), respectively. All three miRNAs were significantly highly expressed in individuals with prediabetes compared to normotolerant patients, p < 0.001. miR-30e-3p and miR-126-3p were also significantly more expressed in T2DM versus normotolerant patients, p < 0.001. In multivariable logistic regressions, the three miRNAs were consistently and continuously associated with prediabetes, while only miR-126-3p was associated with T2DM. The ROC analysis indicated all three miRNAs had a significant overall predictive ability to diagnose prediabetes, diabetes and the combination of both (dysglycaemia), with the area under the receiver operating characteristic curve (AUC) being significantly higher for miR-126-3p in prediabetes. For prediabetes diagnosis, miR-126-3p (AUC = 0.760) outperformed HbA1c (AUC = 0.695), p = 0.042. These results suggest that miR-1299, -126-3p and -30e-3p are associated with prediabetes, and measuring miR-126-3p could potentially contribute to diabetes risk screening strategies.

Impulse oscillometry for detection of small airway dysfunction in subjects with chronic respiratory symptoms and preserved pulmonary function

Respiratory Research ◽

10.1186/s12931-021-01662-7 ◽

2021 ◽

Vol 22 (1) ◽

Cited By ~ 2

Author(s):

Liang-Yuan Li ◽

Tian-Sheng Yan ◽

Jing Yang ◽

Yu-Qi Li ◽

Lin-Xi Fu ◽

...

Keyword(s):

Pulmonary Function ◽

Respiratory Symptoms ◽

Early Stage ◽

Roc Curves ◽

Diagnostic Value ◽

Small Airway ◽

Impulse Oscillometry ◽

Additional Method ◽

Cutoff Values ◽

Higher Sensitivity

Abstract Background Subjects with chronic respiratory symptoms and preserved pulmonary function (PPF) may have small airway dysfunction (SAD). As the most common means to detect SAD, spirometry needs good cooperation and its reliability is controversial. Impulse oscillometry (IOS) may complete the deficiency of spirometry and have higher sensitivity. We aimed to explore the diagnostic value of IOS to detect SAD in symptomatic subjects with PPF. Methods The evaluation of symptoms, spirometry and IOS results in 209 subjects with chronic respiratory symptoms and PPF were assessed. ROC curves of IOS to detect SAD were analyzed. Results 209 subjects with chronic respiratory symptoms and PPF were included. Subjects who reported sputum had higher R5–R20 and Fres than those who didn’t. Subjects with dyspnea had higher R5, R5–R20 and AX than those without. CAT and mMRC scores correlated better with IOS parameters than with spirometry. R5, R5–R20, AX and Fres in subjects with SAD (n = 42) significantly increased compared to those without. Cutoff values for IOS parameters to detect SAD were 0.30 kPa/L s for R5, 0.015 kPa/L s for R5–R20, 0.30 kPa/L for AX and 11.23 Hz for Fres. Fres has the largest AUC (0.665, P = 0.001) among these parameters. Compared with spirometry, prevalence of SAD was higher when measured with IOS. R5 could detect the most SAD subjects with a prevalence of 60.77% and a sensitivity of 81% (AUC = 0.659, P = 0.002). Conclusion IOS is more sensitive to detect SAD than spirometry in subjects with chronic respiratory symptoms and PPF, and it correlates better with symptoms. IOS could be an additional method for SAD detection in the early stage of diseases.

Diagnosis of left ventricular hypertrophy using non-ECG-gated 15O-water PET

Journal of Nuclear Cardiology ◽

10.1007/s12350-021-02734-3 ◽

2021 ◽

Author(s):

Jens Sörensen ◽

Jonny Nordström ◽

Tomasz Baron ◽

Stellan Mörner ◽

Sven-Olof Granstam ◽

...

Keyword(s):

Method Development ◽

Intraclass Correlation ◽

Correlation Coefficients ◽

Roc Curves ◽

Left Ventricular ◽

Intraclass Correlation Coefficients ◽

Concentric Hypertrophy ◽

2D Echocardiography ◽

Positron Emission ◽

Septal Wall Thickness

Abstract Aim To develop a method for diagnosing left ventricular (LV) hypertrophy from cardiac perfusion 15O-water positron emission tomography (PET). Methods We retrospectively pooled data from 139 subjects in four research cohorts. LV remodeling patterns ranged from normal to severe eccentric and concentric hypertrophy. 15O-water PET scans (n = 197) were performed with three different PET devices. A low-end scanner (66 scans) was used for method development, and remaining scans with newer devices for a blinded evaluation. Dynamic data were converted into parametric images of perfusable tissue fraction for semi-automatic delineation of the LV wall and calculation of LV mass (LVM) and septal wall thickness (WT). LVM and WT from PET were compared to cardiac magnetic resonance (CMR, n = 47) and WT to 2D-echocardiography (2DE, n = 36). PET accuracy was tested using linear regression, Bland–Altman plots, and ROC curves. Observer reproducibility were evaluated using intraclass correlation coefficients. Results High correlations were found in the blinded analyses (r ≥ 0.87, P < 0.0001 for all). AUC for detecting increased LVM and WT (> 12 mm and > 15 mm) was ≥ 0.95 (P < 0.0001 for all). Reproducibility was excellent (ICC ≥ 0.93, P < 0.0001). Conclusion 15O-water PET might detect LV hypertrophy with high accuracy and precision.