The Clinical Utility of Psychometric Tests

2021 ◽  
pp. 304-323
Author(s):  
Luis Anunciação ◽  
Marco A. Arruda ◽  
J. Landeira-Fernandez
Keyword(s):  
Clinical Utility ◽  
Historical Review ◽  
Real Data ◽  
Statistical Decision Theory ◽  
Likelihood Ratios ◽  
Statistical Decision ◽  
Predictive Values ◽  
Receiver Operating Characteristic Curves ◽  
Wide Range ◽  
Sensitivity Specificity

The clinical utility of a measure involves its ability to support a wide range of decisions that enhance its pragmatism and use. Although several statistics are part of this feature, one centerpiece of this concept is the ability of an instrument to provide cutoff scores that can accurately discriminate between groups that consist of patients and non-patients. This latter aspect leads to such concepts as sensitivity, specificity, positive and negative predictive values and likelihood ratios, accuracy, and receiver operating characteristic curves. This chapter addresses these topics from two perspectives. First, because these features of clinical utility are encompassed as a subfield of statistical decision theory, the authors provide a historical review that links null hypothesis significance testing (NHST), signal detection theory (SDT), and psychological testing. Second, a real-data approach is used to demonstrate these concepts. Additionally, a free software program was developed to present these concepts.

scholarly journals Value and Diagnostic Efficacy of Fetal Morphology Assessment Using Ultrasound in A Poor-Resource Setting

Diagnostics ◽  
2019 ◽  
Vol 9 (3) ◽  
pp. 109 ◽  
Author(s):  
Ukweh ◽  
Ugbem ◽  
Okeke ◽  
Ekpo
Keyword(s):  
Predictive Value ◽  
Performance Metrics ◽  
Area Under The Curve ◽  
Youden Index ◽  
Likelihood Ratios ◽  
Diagnostic Efficacy ◽  
Predictive Values ◽  
Resource Setting ◽  
Post Test ◽  
Sensitivity Specificity

Background: Ultrasound is operator-dependent, and its value and efficacy in fetal morphology assessment in a low-resource setting is poorly understood. We assessed the value and efficacy of fetal morphology ultrasound assessment in a Nigerian setting. Materials and Methods: We surveyed fetal morphology ultrasound performed across five facilities and followed-up each fetus to ascertain the outcome. Fetuses were surveyed in the second trimester (18th–22nd weeks) using the International Society of Ultrasound in Obstetrics and Gynecology (ISUOG) guideline. Clinical and surgical reports were used as references to assess the diagnostic efficacy of ultrasound in livebirths, and autopsy reports to confirm anomalies in terminated pregnancies, spontaneous abortions, intrauterine fetal deaths, and still births. We calculated sensitivity, specificity, positive and negative predictive values, Area under the curve (AUC), Youden index, likelihood ratios, and post-test probabilities. Results: In total, 6520 fetuses of women aged 15–46 years (mean = 31.7 years) were surveyed. The overall sensitivity, specificity, and AUC were 77.1 (95% CI: 68–84.6), 99.5 (95% CI: 99.3–99.7), and 88.3 (95% CI: 83.7–92.2), respectively. Other performance metrics were: positive predictive value, 72.4 (95% CI: 64.7–79.0), negative predictive value, 99.6 (95% CI: 99.5–99.7), and Youden index (77.1%). Abnormality prevalence was 1.67% (95% CI: 1.37–2.01), and the positive and negative likelihood ratios were 254 (95% CI: 107.7–221.4) and 0.23 (95% CI: 0.16–0.33), respectively. The post-test probability for positive test was 72% (95% CI: 65–79). Conclusion: Fetal morphology assessment is valuable in a poor economics setting, however, the variation in the diagnostic efficacy across facilities and the limitations associated with the detection of circulatory system anomalies need to be addressed.

scholarly journals Predictors and their prognostic value for no ROSC and mortality after a non-cardiac surgery intraoperative cardiac arrest: a retrospective cohort study

2019 ◽  
Vol 9 (1) ◽  
Author(s):  
Matheus F. Vane ◽  
Maria J. C. Carmona ◽  
Sergio M. Pereira ◽  
Karl B. Kern ◽  
Sérgio Timerman ◽  
...  
Keyword(s):  
Cardiac Arrest ◽  
Prognostic Value ◽  
Laboratory Data ◽  
Spontaneous Circulation ◽  
Likelihood Ratios ◽  
Receiver Operating Characteristic Curves ◽  
Post Test ◽  
Verbal Component ◽  
Sensitivity Specificity ◽  
Test Probability

Abstract Data on predictors of intraoperative cardiac arrest (ICA) outcomes are scarce in the literature. This study analysed predictors of poor outcome and their prognostic value after an ICA. Clinical and laboratory data before and 24 hours (h) after ICA were analysed as predictors for no return of spontaneous circulation (ROSC) and 24 h and 1-year mortality. Receiver operating characteristic curves for each predictor and sensitivity, specificity, positive and negative likelihood ratios, and post-test probability were calculated. A total of 167,574 anaesthetic procedures were performed, including 158 cases of ICAs. Based on the predictors for no ROSC, a threshold of 13 minutes of ICA yielded the highest area under curve (AUC) (0.867[0.80–0.93]), with a sensitivity and specificity of 78.4% [69.6–86.3%] and 89.3% [80.4–96.4%], respectively. For the 1-year mortality, the GCS without the verbal component 24 h after an ICA had the highest AUC (0.616 [0.792–0.956]), with a sensitivity of 79.3% [65.5–93.1%] and specificity of 86.1 [74.4–95.4]. ICA duration and GCS 24 h after the event had the best prognostic value for no ROSC and 1-year mortality. For 24 h mortality, no predictors had prognostic value.

Neutrophil CD64 Index as a superior biomarker for early diagnosis of infection in febrile patients in the hematology department

2017 ◽  
Vol 55 (1) ◽  
pp. 82-90 ◽  
Author(s):  
Shu-Dao Xiong ◽  
Lian-Fang Pu ◽  
Hui-Ping Wang ◽  
Lin-Hui Hu ◽  
Yang-Yang Ding ◽  
...  
Keyword(s):  
High Specificity ◽  
Likelihood Ratios ◽  
Predictive Values ◽  
Combined Analysis ◽  
Reactive Protein ◽  
Neutrophil Cd64 ◽  
Special Value ◽  
Diagnostic Values ◽  
Neutropenic Patients ◽  
Sensitivity Specificity

Abstract Background: In the hematology department, the availability of biomarkers for early detection of infection is difficult to obtain. The present study aimed to compare the diagnostic values of neutrophil CD64 Index, procalcitonin (PCT), interleukin-6 (IL-6) and C-reactive protein (CRP) and to determine whether the combined analysis of these biomarkers offer stronger predictive power in the diagnosis for the infection of febrile patients. Methods: Neutrophil CD64 Index, PCT, IL-6 and CRP levels were determined in 356 febrile patients in the hematology ward from May 2013 to May 2015. Sensitivity, specificity, positive and negative likelihood ratios, positive and negative predictive values, receiver operating characteristic (ROC) areas under the curve (AUC), and logistic regression analysis were determined to evaluate the diagnostic values of these biomarkers. Results: The levels of the four biomarkers were higher in the infection patients (p<0.001), and the PCT and IL-6 were higher in the patients with positive microbial blood culture (p<0.01). The neutrophil CD64 Index, PCT, IL-6, CRP had AUCs of 0.95, 0.83, 0.75 and 0.73, respectively. The best cut-off value of the neutrophil CD64 Index to detect infections was 5.06, with high specificity (87.5%) and sensitivity (88.4%). Furthermore, neutrophil CD64 Index, PCT and IL-6 offered the best combination of diagnosis with sensitivity of 93.9% and an AUC of 0.95. In addition, the neutrophil CD64 Index may have a special value to assist the physician to diagnose infection in the neutropenic patients with fever. Conclusions: The neutrophil CD64 Index is useful for early identification of infections in febrile patients in the hematology department. The combined analysis of the CD64 Index, PCT and IL-6 could further improve its sensitivity.

A unique antiphospholipid assay recognizing phospholipid mixture compared with criteria antiphospholipid immunoassays in lupus patients

Lupus ◽  
2016 ◽  
Vol 26 (6) ◽  
pp. 606-615 ◽  
Author(s):  
Y Zuo ◽  
R Willis ◽  
E Papalardo ◽  
M Petri ◽  
E N Harris ◽  
...  
Keyword(s):  
Lupus Erythematosus ◽  
Clinical Manifestations ◽  
Case Series ◽  
Likelihood Ratios ◽  
Serum Samples ◽  
Predictive Values ◽  
Pregnancy Morbidity ◽  
Commercial Kits ◽  
Sensitivity Specificity

Background While essential for the classification of antiphospholipid syndrome (APS), anticardiolipin (aCL) assays lack specificity and anti-β2glycoproteinI (anti-β2GPI) assays lack sensitivity in this regard. Our aim was to perform a comparative analysis of the APhL ELISA assay (IgG/IgM) and criteria antiphospholipid (aPL) immunoassays in identifying APS-related clinical manifestations in a large group of patients with systemic lupus erythematosus (SLE). Methods Serum samples from 1178 patients from the Hopkins ( n = 543), LUMINA ( n = 588) and Jamaican SLE cohorts ( n = 47) were examined for IgG/IgM positivity in aCL (in-house), anti-β2GPI (two commercial kits) and APhL (Louisville APL) ELISA assays. Correlation of assay positivity with clinical manifestations and sensitivity, specificity, positive and negative predictive values and likelihood ratios were evaluated. A case series analysis was also performed in patients for whom there was isolated positivity in the specific aPL assays. Results The prevalence of aCL positivity was 34.9%, anti-β2GPI kit A was 22.6%, APhL was 11.5% and anti-β2GPI kit B was 7.6% in the study population. Anti-β2GPI kit B, aCL and APhL assays were correlated with venous thrombosis, while only APhL was significantly correlated with arterial thrombosis and consistently correlated with pregnancy-related morbidity. No significant correlations were noted for anti-β2GPI kit A. Sensitivity was greatest for aCL assays followed by anti-β2GPI kit A, APhL and anti-β2GPI kit B, while specificity was greatest and equal for anti-β2GPI kit B and APhL assays. Conclusions Overall, APhL antibodies, especially IgG, represent a promising biomarker for the classification of APS patients in the context of autoimmunity and in risk assessment with regards to pregnancy morbidity and thrombotic manifestations.

Screening Adolescents for Depression and Parent-Teenager Conflict in an Ambulatory Medical Setting: A Preliminary Investigation

PEDIATRICS ◽  
1990 ◽  
Vol 85 (5) ◽  
pp. 813-818
Author(s):  
Howard Schubiner ◽  
Arthur Robin
Keyword(s):  
Predictive Value ◽  
Preliminary Investigation ◽  
Conflict Behavior ◽  
Medical Setting ◽  
Predictive Values ◽  
Screening Questionnaire ◽  
Receiver Operating Characteristic Curves ◽  
Sensitivity Specificity ◽  
Very High ◽  
Simple Screening

Because more than 60% of adolescent visits to physicians last less than 10 minutes, simple screening devices to identify persons at risk for social morbidity are needed. The result of a pilot screening instrument for depression and parent-teenager conflict is described. A seven-item questionnaire (three items for depression and four items for parent-teenager conflict), the Beck Depression Inventory (BDI), and the Conflict Behavior Questionnaire (CBQ), were administered to 226 adolescents at a high-risk adolescent health center. After the visit, the provider rated each patient on a five-point Likert scale for level of depression and parent-teenager conflict. The screening questionnaire and the provider ratings were compared with the BDI and CBQ for sensitivity, specificity, and predictive value. A majority of the subjects (61% for depression and 63% for parent-teenager conflict) had no positive screening responses and, of these, less than 3% were classified as being depressed or having parent-teenager conflict by the BDI or CBQ. There were three or four positive screening responses (three responses for the depression questions and 20 responses for the parent-teenager conflict questions) among 23 adolescents and all of these scored in the abnormal range on the BDI or CBQ. Approximately one third of the subjects had intermediate responses. With one positive response, the predictive value for depression and parent-teenager conflict was 17% and 24%, respectively, whereas with two positive responses the predictive values were 56% and 26%. It was shown by receiver operating characteristic curves that the combined sensitivity and specificity of the screening questionnaire was superior to that of the physician ratings. In this study it was demonstrated that a simple seven-item instrument for depression and parent-teenager conflict can quickly and accurately classify the two thirds of patients with very high or very low scores and provide likelihoods of these problems for the remaining one third of patients who score in the intermediate ranges. This instrument can aid providers in efficient assessment of social morbidity factors in their adolescent patients.

scholarly journals Interpreting Validity Indexes for Diagnostic Tests: An Illustration Using the Berg Balance Test

1999 ◽  
Vol 79 (10) ◽  
pp. 939-948 ◽  
Author(s):  
Daniel L Riddle ◽  
Paul W Stratford
Keyword(s):  
Physical Therapists ◽  
Likelihood Ratios ◽  
Optimal Method ◽  
Balance Test ◽  
Predictive Values ◽  
Prognostic Accuracy ◽  
Clinical Scenarios ◽  
Risk Of Falls ◽  
Prognostic Tests ◽  
Sensitivity Specificity

Abstract Physical therapists routinely make diagnostic and prognostic decisions in the course of patient care. The purpose of this clinical perspective is to illustrate what we believe is the optimal method for interpreting the results of studies that describe the diagnostic or prognostic accuracy of examination procedures. To illustrate our points, we chose the Berg Balance Test as an exemplar measure. We combined the data from 2 previously published research reports designed to determine the validity of the Berg Balance Test for predicting risk of falls among elderly people. We calculated the most common validity indexes, including sensitivity, specificity, predictive values, and likelihood ratios for the combined data. Clinical scenarios were used to demonstrate how we believe these validity indexes should be used to guide clinical decisions. We believe therapists should use validity indexes to decrease the uncertainty associated with diagnostic and prognostic decisions. More studies of the accuracy of diagnostic and prognostic tests used by physical therapists are urgently needed.

Evaluation of the EYTEX® System as a Screening Method for the Ocular Irritancy of Chemical Products

1994 ◽  
Vol 22 (1) ◽  
pp. 32-50
Author(s):  
Jean-François Régnier ◽  
Christophe Imbert ◽  
Jean-Charles Boutonnet
Keyword(s):  
False Negative ◽  
Screening Method ◽  
In Vitro Test ◽  
Predictive Values ◽  
Ocular Irritation ◽  
Wide Range ◽  
Dangerous Substances ◽  
Sensitivity Specificity ◽  
Interlaboratory Reproducibility

The EYTEX® method is an in vitro test used to predict ocular irritation, based on alterations in a protein matrix. We have evaluated this method with the aim of using it to screen chemicals. One hundred and forty-two products (commodities and specialities), having a wide range of chemical properties and ocular irritation potential in the rabbit, were tested with either the standard, MPA, AMA, KMA or UMA protocols. The results were compared with in vivo data obtained previously for each chemical and with the EEC labelling of eye irritation for dangerous substances. Intralaboratory repeatability and interlaboratory reproducibility were evaluated with seven other laboratories. Ninety-three per cent of the chemicals tested were qualified with the EYTEX method. The coefficients of the linear correlation between the EYTEX score and the maximal Draize score on the one hand and the maximal corneal score on the other hand, were 0.69 and 0.65, respectively. Compared to the EEC classification, the labelling of dangerous substances and for the prediction of severe irritants, sensitivity, specificity and equivalence were 94%, 89% and 78%, respectively. We observed nine false positives (22%) and two false negatives (2%) for the identification of R41-, R34- and R35-labelled products. The predictive values, for identifying R41, R34 and R35 products and non-irritants or R36 products were 78% and 97%, respectively. Repeatability (6.3) and reproducibility (8.9) were quite satisfactory. The EYTEX system exhibits the characteristics of a good screening method: compatibility with a large range of chemicals; a simple and rapid procedure; good intralaboratory and interlaboratory reproducibility; cost effectiveness; high sensitivity, specificity and predictive value; and a low incidence of false negative and false positive results. Based on these results, we consider the EYTEX method to be a valuable tool for the screening of eye irritancy.

Sign in / Sign up

Export Citation Format

Share Document