Sensitivity, Specificity, and Receiver Operating Characteristics

AbstractIt has been shown that there are differences in diagnostic accuracy of cancer detection on mammograms, from below 50% in developing countries to over 80% in developed world. One previous study reported that radiologists from a population in Asia displayed a low mammographic cancer detection of 48% compared with over 80% in developed countries, and more importantly, that most lesions missed by these radiologists were spiculated masses or stellate lesions. The aim of this study was to explore the performance of radiologists after undertaking a training test set which had been designed to improve the capability in detecting a specific type of cancers on mammograms. Twenty-five radiologists read two sets of 60 mammograms in a standardized mammogram reading room. The first test set focused on stellate or spiculated masses. When radiologists completed the first set, the system displayed immediate feedback to the readers comparing their performances in each case with the truth of cancer cases and cancer types so that the readers could identify individual-based errors. Later radiologists were asked to read the second set of mammograms which contained different types of cancers including stellate/spiculated masses, asymmetric density, calcification, discrete mass and architectural distortion. Case sensitivity, lesion sensitivity, specificity, receiver operating characteristics (ROC) and Jackknife alternative free-response receiver operating characteristics (JAFROC) were calculated for each participant and their diagnostic accuracy was compared between two sessions. Results showed significant improvement among radiologists in case sensitivity (+ 11.4%; P < 0.05), lesion sensitivity (+ 18.7%; P < 0.01) and JAFROC (+ 11%; P < 0.01) in the second set compared with the first set. The increase in diagnostic accuracy was also recorded in the detection of stellate/spiculated mass (+ 20.6%; P < 0.05). This indicated that the performance of radiologists in detecting malignant lesions on mammograms can be improved if an appropriate training intervention is applied after the readers’ weakness and strength are identified.

Download Full-text

Screening for Personality Disorders: A Three-Item Screener From the Inventory of Interpersonal Problems (IIP-3)

Journal of Personality Disorders ◽

10.1521/pedi_2019_33_369 ◽

2019 ◽

Vol 33 (6) ◽

pp. 832-845

Author(s):

Paul A. Pilkonis ◽

Suzanne M. Lawrence ◽

Kelly L. Johnston ◽

Nathan E. Dodds

Keyword(s):

Personality Disorders ◽

Interpersonal Problems ◽

Self Report ◽

Receiver Operating Characteristics ◽

Operating Characteristics ◽

Predictive Values ◽

Inventory Of Interpersonal Problems ◽

Sensitivity Specificity ◽

Receiver Operating

To encourage screening for personality disorders (PDs), we developed (in previous work) self-report scales for PDs using the Inventory of Interpersonal Problems (IIP). The combined score from three of the scales—inter-personal sensitivity, interpersonal ambivalence, and aggression—requiring 15 items (IIP-15) did the best job of distinguishing between respondents with any versus no PD. The goals for the present work were (a) to cross-validate the IIP-15 by examining its performance using receiver operating characteristics (ROC) analyses in a new sample (N = 410), and (b) to investigate the utility of a brief three-item variant (IIP-3). The present results again documented the good operating characteristics of the IIP-15. Sensitivity, specificity, and positive and negative predictive values were all above. 70. The operating characteristics of the IIP-3 were nearly as good despite its brevity and support its use as an initial screen for PDs.

Download Full-text

Kriteriumsbezogene Cut-Off-Werte für Tätigkeitsspielraum und Arbeitsintensität

Diagnostica ◽

10.1026/0012-1924/a000226 ◽

2019 ◽

Vol 65 (3) ◽

pp. 179-190 ◽

Cited By ~ 1

Author(s):

Vincent Mustapha ◽

Renate Rau

Keyword(s):

Receiver Operating Characteristics ◽

Operating Characteristics ◽

Receiver Operating

Zusammenfassung. Cut-Off-Werte ermöglichen eine ökonomische, binäre Beurteilung von Summenscores. Für Beanspruchungsfragebögen, die personenbezogene Merkmale erfragen, sind Cut-Off-Werte häufig vorhanden und in der klinischen Diagnostik unerlässlich. Für die Bewertung von Arbeitsmerkmalen sind Cut-Off-Werte ebenfalls wünschenswert. Bislang fehlen sie jedoch für die Beurteilung von Arbeitsmerkmalen wie Arbeitsintensität und Tätigkeitsspielraum. Zwischen 2006 und 2016 wurden daher in verschiedenen Branchen 801 objektive Arbeitsplatzanalysen durchgeführt, welche eine Unterteilung in gut und schlecht gestalteten Tätigkeitsspielraum sowie gut und schlecht gestaltete Arbeitsintensität nach DIN EN ISO 6385 (2016) ermöglichen. Anhand dieser Unterteilung wurden mit der Receiver-Operating-Characteristics-Analyse Cut-Off-Werte für den subjektiv-bedingungsbezogen Fragebogen zum Erleben von Arbeitsintensität und Tätigkeitsspielraum (FIT; Richter et al., 2000 ) ermittelt. Für den Tätigkeitsspielraum weisen Summenscores ≤ 22 und für die Arbeitsintensität Summenscores ≥ 15 auf eine schlechte Gestaltung des jeweiligen Arbeitsmerkmals hin. Anhand einer weiteren Stichprobe von 1 076 Arbeitenden konnte gezeigt werden, dass Arbeitende mit schlecht gestaltetem Tätigkeitspielraum vital erschöpfter sowie weniger engagiert sind und Arbeitende mit schlecht gestalteter Arbeitsintensität eine höhere Erholungsunfähigkeit sowie vitale Erschöpfung aufweisen.

Download Full-text

Differentiation of Receiver Operating Characteristics Based on Corpus Callosum Volume in a Visual-Short Term Memory Paradigm

PsycEXTRA Dataset ◽

10.1037/e524912015-192 ◽

2014 ◽

Author(s):

Shuo Qin ◽

Chandramalika Basak ◽

Nithya Ramakrishnan

Keyword(s):

Corpus Callosum ◽

Short Term Memory ◽

Receiver Operating Characteristics ◽

Operating Characteristics ◽

Short Term ◽

Term Memory ◽

Visual Short Term Memory ◽

Receiver Operating

Download Full-text

Comparison of Rheumatological Diagnoses by a Bayesian Program and by Physicians

Methods of Information in Medicine ◽

10.1055/s-0038-1634834 ◽

1991 ◽

Vol 30 (03) ◽

pp. 187-193 ◽

Cited By ~ 7

Author(s):

H. J. Moens ◽

J. K. van der Korst

Keyword(s):

Receiver Operating Characteristics ◽

Bayesian Decision ◽

Operating Characteristics ◽

Likelihood Ratios ◽

Improve Performance ◽

Mathematical Basis ◽

Human Expert ◽

Test Population ◽

Sensitivity Specificity ◽

Diagnostic Outcome

AbstractA Bayesian decision support system was developed for the diagnosis of rheumatic disorders. Knowledge in this system is represented as evidential weights of findings. Simple weights were calculated as the logarithm of likelihood ratios on the basis of 1,000 consecutive patients from a rheumatological clinic. The effect of various methods to improve performance of the system by modification of the weights was studied. Three methods had a mathematical basis; a fourth consisted of weights adapted by a human expert, which allowed inclusion of diagnostic rules such as defined in widely accepted criteria sets. The system’s performance was measured in a test population of 570 different cases from the same clinic and compared with predictions of diagnostic outcome made by rheumatologists. The weights from a human expert gave optimal results (sensitivity 65% and specificity 96%), that were close to the physicians’ predictions (sensitivity 64% and specificity 98%). The methods to measure the performance of the various models used in this study emphasize sensitivity, specificity and the use of receiver operating characteristics.

Download Full-text

Arterial Digital Pulse Photoplethysmography in Patients with Suspected Thoracic Outlet Syndrome: A Study of the “Ca+Pra” Maneuver

Diagnostics ◽

10.3390/diagnostics11061128 ◽

2021 ◽

Vol 11 (6) ◽

pp. 1128

Author(s):

Jeanne Hersant ◽

Pierre Ramondou ◽

Francine Thouveny ◽

Mickael Daligault ◽

Mathieu Feuilloy ◽

...

Keyword(s):

Thoracic Outlet Syndrome ◽

Pulse Amplitude ◽

Receiver Operating Characteristics ◽

Upper Limbs ◽

Operating Characteristics ◽

Digital Pulse ◽

Arterial Inflow ◽

Receiver Operating

The level of pulse amplitude (PA) change in arterial digital pulse plethysmography (A-PPG) that should be used to diagnose thoracic outlet syndrome (TOS) is debated. We hypothesized that a modification of the Roos test (by moving the arms forward, mimicking a prayer position (“Pra”)) releasing an eventual compression that occurs in the surrender/candlestick position (“Ca”) would facilitate interpretation of A-PPG results. In 52 subjects, we determined the optimal PA change from rest to predict compression at imaging (ultrasonography +/− angiography) with receiver operating characteristics (ROC). “Pra”-PA was set as 100%, and PA was expressed in normalized amplitude (NA) units. Imaging found arterial compression in 23 upper limbs. The area under ROC was 0.765 ± 0.065 (p < 0.0001), resulting in a 91.4% sensitivity and a 60.9% specificity for an increase of fewer than 3 NA from rest during “Ca”, while results were 17.4% and 98.8%, respectively, for the 75% PA decrease previously proposed in the literature. A-PPG during a “Ca+Pra” test provides demonstrable proof of inflow impairment and increases the sensitivity of A-PPG for the detection of arterial compression as determined by imaging. The absence of an increase in PA during the “Ca” phase of the “Ca+Pra” maneuver should be considered indicative of arterial inflow impairment.

Download Full-text