Average Weighted Accuracy: Pragmatic Analysis for a Rapid Diagnostics in Categorizing Acute Lung Infections (RADICAL) Study

Abstract Patient management relies on diagnostic information to identify appropriate treatment. Standard evaluations of diagnostic tests consist of estimating sensitivity, specificity, positive/negative predictive values, likelihood ratios, and accuracy. Although useful, these metrics do not convey the tests’ clinical value, which is critical to informing decision-making. Full appreciation of the clinical impact of a diagnostic test requires analyses that integrate sensitivity and specificity, account for the disease prevalence within the population of test application, and account for the relative importance of specificity vs sensitivity by considering the clinical implications of false-positive and false-negative results. We developed average weighted accuracy (AWA), representing a pragmatic metric of diagnostic yield or global utility of a diagnostic test. AWA can be used to compare test alternatives, even across different studies. We apply the AWA methodology to evaluate a new diagnostic test developed in the Rapid Diagnostics in Categorizing Acute Lung Infections (RADICAL) study.

Download Full-text

S235 – Diagnostic Test Validation: CHAMP in Endolymphatic Hydrops

Otolaryngology ◽

10.1016/j.otohns.2008.05.410 ◽

2008 ◽

Vol 139 (2_suppl) ◽

pp. P154-P154

Author(s):

Eveling Y Rojas ◽

Leonardo Ordoñez ◽

Victor Hernandez ◽

Jose E Guzman ◽

Juan Izquierdo ◽

...

Keyword(s):

Diagnostic Test ◽

Endolymphatic Hydrops ◽

False Negative ◽

High Specificity ◽

Diagnostic Value ◽

Clinical Value ◽

Negative Results ◽

False Negative Results ◽

The Relationship

Objectives The Cochlear Hydrops Analysis Masking Procedure (CHAMP) has been reported with a high sensibility and specificity for the diagnosis of endolymphatic hydrops (ELH). Nevertheless, a complete validation of CHAMP with precise methods has not been done. Our objectives: To determine whether CHAMP has a diagnostic value in patients with definite, probable/possible diagnosis of Meniere's disease or endolymphatic hydrops, based on the criteria of the AAO-HNS. Methods Prospective validation study for a diagnostic test: phase II. Latency of wave V was assessed using CHAMP. We classified subjects in 5 groups: 1-Definite ELH, 2-Probable ELH, 3-Possible ELH, 4-subjects with other otovestibular symptoms, 5-normal hearing subjects. We included cases (groups 1–4) with an episode of vertigo with duration equal or longer than 20 minutes, with or without other otovestibular symptoms. Results We included 67 cases (132 ears). 38 cases (76 ears) completed the follow up and their results are presented. In group 1, the sensibility of CHAMP was 43% and its specificity was 100%. In groups 2–3, the sensitivity was 20% and the specificity 100%, with 0% false positive and 80% false negative results. Other indicators of validity for this diagnostic test are presented. Conclusions CHAMP has a high specificity and low sensibility for the diagnosis of ELH when the delay in wave V latency is used. Future studies may assess the relationship between wave amplitudes to improve the clinical value of CHAMP.

Download Full-text

The role of ultrasonography in the diagnosis of talocalcaneal coalitions

Acta Radiologica ◽

10.1177/0284185120944911 ◽

2020 ◽

pp. 028418512094491

Author(s):

Tiezheng Wang ◽

Hengtao Qi ◽

Kai Rong ◽

Shuqian Zhang ◽

Shougang Bao ◽

...

Keyword(s):

False Negative ◽

Fibrous Tissue ◽

Joint Space ◽

Clinical Suspicion ◽

Youden Index ◽

Imaging Method ◽

Predictive Values ◽

Negative Results ◽

Reduced Space ◽

Sensitivity Specificity

Background Patients with talocalcaneal coalitions (TCC) often undergo computed tomography (CT). However, ultrasonography diagnosis of TCC has been seldom done according to the literature. Purpose To investigate the accuracy of ultrasonography in diagnosing TCC compared to CT. Material and Methods Ninety-seven consecutive patients with a clinical suspicion of TCC were included. Ultrasonography was used to assess the classification and complication of TCC. The main sonographic criteria for a positive diagnosis in cases of osseous coalition were the joint space between the medial surface of talar head and the underlying sustentaculum tali of calcaneus disappearing and being replaced by a continuous hyperechoic bony structure. In cases of fibrous coalition, ultrasonography revealed a reduced space of the joint associated with an irregular, angular appearance of its outline and hypoechoic fibrous tissue inside. These data were compared with CT findings. κ statistic was applied to determine the level of agreement. The sensitivity, specificity, positive and negative predictive values, accuracy, and Youden index of ultrasonography as a diagnostic method were assessed. Results Ultrasonography findings were positive in 20 of 97 patients with a clinical suspicion of TCC. The diagnosis was confirmed by CT in 21 patients. There were one false-positive result and two false-negative results by ultrasonography. The κ value was 0.907. The sensitivity, specificity, positive and negative predictive values, accuracy, and Youden index of ultrasonography were 90.5%, 98.7%, 95.0%, 97.4%, 96.9%, and 0.892, respectively. Conclusion Ultrasonography could be a reliable, accurate, and non-radioactive diagnostic imaging method in diagnosis of patients with suspected TCC.

Download Full-text

Exploring 'best practice' for nucleic acid detection of Neisseria gonorrhoeae

Sexual Health ◽

10.1071/sh07050 ◽

2008 ◽

Vol 5 (1) ◽

pp. 17 ◽

Cited By ~ 40

Author(s):

David M. Whiley ◽

Suzanne M. Garland ◽

Geoffrey Harnett ◽

Gary Lum ◽

David W. Smith ◽

...

Keyword(s):

Nucleic Acid ◽

Neisseria Gonorrhoeae ◽

Best Practice ◽

Current Knowledge ◽

False Negative ◽

High Sensitivity ◽

Nucleic Acid Detection ◽

Predictive Values ◽

Neisseria Species ◽

Negative Results

Nucleic acid detection tests (NADT) have considerable benefits for the detection of Neisseria gonorrhoeae (GC), including high sensitivity across a range of specimen types and use under widely differing settings and conditions. However, sexual health practitioners and others who use data generated by NADT for GC should be aware of some important limitations of these tests. False-positive results caused by cross reaction with commensal Neisseria species have been observed in many assays, and have lead to unacceptably low positive-predictive values in some patient populations. Further, false-negative results can be caused by GC sequence variation, with some gonococci lacking certain NADT target sequences. This review examines the issues associated with gonococcal NADT and considers best practice for use of these assays based on current knowledge. We emphasise the need for supplementary testing and extensive assay validation, and suggest appropriate strategies for these requirements irrespective of the setting in which they are used. Further, we highlight the need to maintain culture-based testing for certain specimen sites as well as for antimicrobial resistance surveillance.

Download Full-text

Diagnostic Performance of Hrp2 Based Diagnostic Test Kits and Frequency of Pfhrp2 Gene Deletion in Plasmodium falciparum Isolates of Osogbo, Southwestern Nigeria

Journal of Advances in Medicine and Medical Research ◽

10.9734/jammr/2021/v33i730870 ◽

2021 ◽

pp. 20-25

Author(s):

A. S. Nassar ◽

A. S. Bakarey ◽

A. A. Abdulazeez ◽

O. O. Fayemi

Keyword(s):

Plasmodium Falciparum ◽

Diagnostic Test ◽

Gene Deletion ◽

False Negative ◽

University Teaching ◽

Pcr Analysis ◽

Southwestern Nigeria ◽

Blood Samples ◽

Negative Results ◽

Test Kit

Introduction: The introduction of P. falciparum encoded HRP-2 based malaria Rapid Diagnostic Test (RDT) kits is widely accepted in Nigeria and worldwide as a simplified form of diagnosis and a cheaper alternative to the microscopy technique (gold standard). However, deletion of Pfhrp2 gene contributes to false negative results and large number of such deletions has been reported in advanced countries thereby highlighting the importance of surveillance to detect such deletions in our local environment. Methodology: Microscopy as well as RDT techniques (using Rapid malaria test kit: SD BIOLINE Malaria Ag P.f/Pv, South Korea) were carried out on the blood samples of three hundred and twenty-three (323) febrile subjects attending Ladoke Akintola University Teaching Hospital, Osogbo, Osun State Nigeria. PCR analysis was also conducted on 50 blood samples that were positive for microscopy but negative for RDT. Results: The results from the study revealed that microscopy had a sensitivity of 99% and specificity of 99.2%. The RDT however had a sensitivity of 100% and a specificity of 60.1%. Fifty (50) samples that were positive for microscopy but negative for RDT were subjected to further PCR examination to detect the possible deletion of the Pfhrp-2 gene and the result revealed that the gene was present in 39 (78%) of the blood samples while remaining 11 (22%) samples lacked the gene which could possibly be the reason for the negative results obtained using the RDT kits. Conclusion: This study provides evidence of low level of presence of Pfhrp-2 gene deletion of Plasmodium falciparum parasites in our healthcare facility setting in Osogbo, Nigeria.

Download Full-text

Evaluation of anti-SARS-CoV-2 antibody testing in asymptomatic or mild COVID-19 patients in outbreak on a cruise ship

10.1101/2021.03.10.21253064 ◽

2021 ◽

Author(s):

Norihito Kaku ◽

Fumitaka Nishimura ◽

Yui Shigeishi ◽

Rina Tachiki ◽

Hironori Sakai ◽

...

Keyword(s):

Genetic Test ◽

False Negative ◽

Enzyme Linked Immunosorbent Assay ◽

Cruise Ship ◽

Igg Antibodies ◽

Antibody Testing ◽

Predictive Values ◽

Blood Samples ◽

Negative Results ◽

Genetic Test Results

AbstractBackgroundA few studies on antibody testing have focused on asymptomatic or mild coronavirus disease 2019 (COVID-19) patients with low initial anti-severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) antibody responses. Anti-SARS-CoV-2 antibody-testing performance was evaluated using blood samples from asymptomatic or mild COVID-19 patients.MethodsBlood samples were collected from 143 COVID-19 patients during an outbreak on a cruise ship 3 weeks after diagnosis. Simultaneously, a second SARS-CoV-2 genetic test was performed. Samples stored before the COVID-19 pandemic were also used to evaluate the lateral flow immunochromatographic assay (LFA) and electrochemiluminescence immunoassay (ECLIA). Titers of anti-SARS-CoV-2 IgM and IgG antibodies against the nucleocapsid and spike proteins were measured using the enzyme-linked immunosorbent assay to compare false-negative- with positive-result samples.ResultsSensitivity, specificity, positive-predictive, and negative-predictive values of LFA-detected IgM antibodies were 0.231, 1.000, 1.000, and 0.613, respectively; those of LFA-detected IgG antibodies were 0.483, 0.989, 0.972, and 0.601, respectively; and those of ECLIA-detected total antibodies were 0.783, 1.000, 1.000, and 0.848, respectively. IgM-, IgG-, and total-antibody positivity rates in the patients with negative results from the second genetic testing were 22.9%, 47.6%, and 72.4%, respectively. All antibody titers, especially those of the IgG antibody against nucleocapsid protein, were significantly lower in blood samples with false-negative results than in those with positive results.ConclusionsThese findings suggest that anti-SARS-CoV-2 antibody testing has lower performance in asymptomatic or mild COVID-19 patients than required in the guidelines, and situations in which it is useful are limited.Key pointsAnti-SARS-CoV-2 antibody testing in asymptomatic or mild COVID-19 patients is lower than the required clinical sensitivity, although it may be useful in patients at 3–4 weeks after symptom onset but with negative SARS-CoV-2 genetic test results.

Download Full-text

Rapid diagnostics of orthopedic implant-associated infections using Unyvero ITI implant and tissue infection application is not optimal for Staphylococcus species identification

BMC Research Notes ◽

10.1186/s13104-019-4755-5 ◽

2019 ◽

Vol 12 (1) ◽

Cited By ~ 2

Author(s):

Hege Vangstein Aamot ◽

Bjørn Odd Johnsen ◽

Inge Skråmm

Keyword(s):

Antibiotic Resistance ◽

Antibiotic Treatment ◽

False Negative ◽

Treatment Recommendation ◽

Lower Sensitivity ◽

Orthopedic Implant ◽

Rapid Diagnostics ◽

Negative Results ◽

False Negative Results ◽

Time To Detection

Abstract Objectives This pilot study aimed to compare the commercial Unyvero ITI multiplex PCR application (U-ITI, Curetis GmbH) with conventional culturing concerning (a) detection of pathogens, (b) time to detection of pathogens and (c) time to and quality of antibiotic treatment recommendation in diagnostics of orthopedic implant-associated infections (OIAI). Results 72 tissue biopsies from 15 consecutive patients with deep OIAI infections were analyzed with conventional culturing including phenotypic antibiotic susceptibility testing and the U-ITI. U-ITI showed lower sensitivity than conventional culturing concerning detection of pathogens (73% vs 93%). 4/15 patients would have been given false negative results by U-ITI, all of which were culture-positive for Staphylococcus species. Median time to detection of pathogens was 47 h and antibiotic resistance 89 h by conventional methods compared to 13.5 h with the U-ITI. The U-ITI did not detect antibiotic resistance, whereas conventional culturing showed resistance to antibiotics covered by the U-ITI panel in 2 patients. Time to detection of pathogens was improved, but the detection limit for staphylococci was unsatisfactory. Although the time to antibiotic treatment recommendation was significantly reduced, the U-ITI would have resulted in incorrect antibiotic recommendation in 2 patients. Our data do not support use of this assay in diagnostics.

Download Full-text

Submucosal Endoscopic Sampling for Indefinite Gastric Linitis Plastica Infiltrating into the Submucosal Layer

Journal of Gastrointestinal and Liver Diseases ◽

10.15403/jgld.2014.1121.243.chy ◽

2015 ◽

Vol 24 (3) ◽

pp. 375-378 ◽

Cited By ~ 6

Author(s):

Taiga Chiyo ◽

Hideki Kobara ◽

Hirohito Mori ◽

Naomi Katsuki ◽

Reiji Haba ◽

...

Keyword(s):

Diagnostic Yield ◽

False Negative ◽

Linitis Plastica ◽

Needle Aspiration ◽

Endoscopic Biopsy ◽

Direct Vision ◽

Tissue Samples ◽

Submucosal Tumors ◽

Negative Results ◽

Multiple Biopsies

The diagnosis of diffuse-type gastric cancer, named linitis plastica (LP), is difficult because of its infiltration into the submucosa. Conventional endoscopic biopsy sampling may show false-negative results because the superficial mucosa is often normal. These macroscopic features do not often permit the distinction between benign and malignant lesions, and sampling methods have some limitations. Accordingly, a secure sampling method is required in order to increase the diagnostic yield. We have developed a submucosal tunneling technique for sampling submucosal tumors, which can visualize tumor surfaces and obtain tissue samples under direct vision. We report a rare case of indefinite gastric LP that could be diagnosed by this method. As multiple biopsies and endoscopic ultrasound (EUS)-guided fine needle aspiration (FNA) did not help us histologically diagnose the lesion, our new method of submucosal endoscopy which has advantages of visualizing tumor surfaces and obtaining tissue samples under direct vision in the submucosa was introduced. Histological examination of all acquired samples confirmed the presence of a poorly differentiated adenocarcinoma. The present case demonstrates that this method is a reasonable option for indefinite LP with features inflating into submucosa, providing an update on the contemporary concepts.

Download Full-text

Diagnosis of Fall Risk in Parkinson Disease: An Analysis of Individual and Collective Clinical Balance Test Interpretation

Physical Therapy ◽

10.2522/ptj.20070082 ◽

2008 ◽

Vol 88 (3) ◽

pp. 323-332 ◽

Cited By ~ 69

Author(s):

Leland E Dibble ◽

Jesse Christensen ◽

D James Ballard ◽

K Bo Foreman

Keyword(s):

Parkinson Disease ◽

Fall Risk ◽

False Negative ◽

Likelihood Ratios ◽

Balance Test ◽

Multiple Tests ◽

Negative Results ◽

Balance Tests ◽

History Analysis ◽

Validity Indices

Background and PurposeParkinson disease (PD) results in an increased frequency of falls relative to the frequency in neurologically healthy people. The purpose of this study was to compare the accuracy of PD fall risk diagnosis based on one test with that based on the collective interpretation of multiple tests.ParticipantsSeventy people with PD (mean age=73.91 years) participated in this study.MethodClinical balance tests were conducted during the initial examinations of people with PD. Validity indices were calculated for individual tests and compared with validity indices calculated for a combination of multiple tests.ResultsThirty-six participants reported a fall history. Analysis of individual tests revealed broad variations in validity indices, whereas the collective interpretation of multiple tests improved sensitivity and negative likelihood ratios.Discussion and ConclusionCollective interpretation of clinical balance tests resulted in fewer false-negative results and more substantial adjustments to the posttest probability of being a “faller” than the interpretation of one test alone. These results should be confirmed in a prospective examination of fall risk in PD.

Download Full-text

Utilization, Yield, and Accuracy of the FilmArray Meningitis/Encephalitis Panel with Diagnostic Stewardship and Testing Algorithm

Journal of Clinical Microbiology ◽

10.1128/jcm.00311-20 ◽

2020 ◽

Vol 58 (9) ◽

Cited By ~ 2

Author(s):

M. Jana Broadhurst ◽

Shefali Dujari ◽

Indre Budvytiene ◽

Benjamin A. Pinsky ◽

Carl A. Gold ◽

...

Keyword(s):

False Negative ◽

Utilization Rate ◽

Calcofluor White ◽

Predictive Values ◽

Negative Results ◽

Panel Testing ◽

Acyclovir Treatment ◽

Wbc Count ◽

Testing Algorithm ◽

The Impact

ABSTRACT The impact of diagnostic stewardship and testing algorithms on the utilization and performance of the FilmArray meningitis/encephalitis (ME) panel has received limited investigation. We performed a retrospective single-center cohort study assessing all individuals with suspected ME between February 2017 and April 2019 for whom the ME panel was ordered. Testing was restricted to patients with cerebrospinal fluid (CSF) pleocytosis. Positive ME panel results were confirmed before reporting through correlation with direct staining (Gram and calcofluor white) and CSF cryptococcal antigen or by repeat ME panel testing. Outcomes included the ME panel test utilization rate, negative predictive value of nonpleocytic CSF samples, test yield and false-positivity rate, and time to appropriate deescalation of acyclovir. Restricting testing to pleocytic CSF samples reduced ME panel utilization by 42.7% (263 versus 459 tests performed) and increased the test yield by 61.8% (18.6% versus 11.5% positivity rate; P < 0.01) with the application of criteria. The negative predictive values of a normal CSF white blood cell (WBC) count for ME panel targets were 100% (195/195) for nonviral targets and 98.0% (192/196) overall. All pathogens detected in nonpleocytic CSF samples were herpesviruses. The application of a selective testing algorithm based on repeat testing of nonviral targets avoided 75% (3/4) of false-positive results without generating false-negative results. The introduction of the ME panel reduced the duration of acyclovir treatment from an average of 66 h (standard deviation [SD], 43 h) to 46 h (SD, 36 h) (P = 0.03). The implementation of the ME panel with restriction criteria and a selective testing algorithm for nonviral targets optimizes its utilization, yield, and accuracy.

Download Full-text

Quantifying the Accuracy of a Diagnostic Test or Marker

Clinical Chemistry ◽

10.1373/clinchem.2012.182543 ◽

2012 ◽

Vol 58 (9) ◽

pp. 1292-1301 ◽

Cited By ~ 78

Author(s):

Kristian Linnet ◽

Patrick M M Bossuyt ◽

Karel G M Moons ◽

Johannes B Reitsma

Keyword(s):

Diagnostic Test ◽

Prospective Cohort ◽

Roc Curves ◽

Test Accuracy ◽

Index Test ◽

Reference Standard ◽

Single Test ◽

Likelihood Ratios ◽

Predictive Values ◽

Cohort Design

Abstract BACKGROUND In recent years, increasing focus has been directed to the methodology for evaluating (new) tests or biomarkers. A key step in the evaluation of a diagnostic test is the investigation into its accuracy. CONTENT We reviewed the literature on how to assess the accuracy of diagnostic tests. Accuracy refers to the amount of agreement between the results of the test under evaluation (index test) and the results of a reference standard or test. The generally recommended approach is to use a prospective cohort design in patients who are suspected of having the disease of interest, in which each individual undergoes the index and same reference standard tests. This approach presents several challenges, including the problems that can arise with the verification of the index test results by the preferred reference standard test, the choice of cutoff value in case of a continuous index test result, and the determination of how to translate accuracy results to recommendations for clinical use. This first in a series of 4 reports presents an overview of the designs of single-test accuracy studies and the concepts of specificity, sensitivity, posterior probabilities (i.e., predictive values) for the presence of target disease, ROC curves, and likelihood ratios, all illustrated with empirical data from a study on the diagnosis of suspected deep venous thrombosis. Limitations of the concept of the diagnostic accuracy for a single test are also highlighted. CONCLUSIONS The prospective cohort design in patients suspected of having the disease of interest is the optimal approach to estimate the accuracy of a diagnostic test. However, the accuracy of a diagnostic index test is not constant but varies across different clinical contexts, disease spectrums, and even patient subgroups.

Download Full-text