Clinical evaluation of the APAS® Independence: Automated imaging and interpretation of urine cultures using artificial intelligence with composite reference standard discrepant resolution

Abstract Background Tuberculosis (TB) is the leading cause of death as a single infectious agent worldwide. In 2017, there were an estimated 1.3 million TB deaths among HIV-negative people and 300,000 deaths among HIV-positive people. The rapid and accurate diagnosis of TB in lymphnode specimens remains a challenging task today. In 2016, World Health Organization endorsed a commercial molecular assay, the LoopAMP™ Mycobacterium tuberculosis complex (MTBC) detection kit (Eiken Chemical Company, Tokyo, Japan), which uses loop-mediated isothermal amplification (LAMP) for sputum samples only. No prospective studies on LAMP in diagnosis of Tubercular Lymphadenitis in adults have been done yet. Methods A prospective observational study with a total of 70 lymph-node aspirate specimens from suspected cases of Tubercular Lymphadenitis with age >18 years were selected and subjected to Ziehl–Neelsen staining, LAMP and culture in mycobacterial growth indicator tube (MGIT960). The immunochromatographic test was used to confirm MTB complex (MTBC) in culture positive samples and phenotypic drug susceptibility testing was done using MGIT-960. The composite reference standard (CRS) used in the study includes symptoms, radiological evidence and follow-up of 2 months. 2 × 2 tables were made and Sensitivity, Specificity, PPV, NPV of TB-LAMP were calculated with respect to AFB smear and composite reference standard (CRS). Results LAMP assay was able to detect MTBC in 34.3% (24/70) of lymph-node specimens. Sensitivity and specificity of the assay were 100% and 69.7%, respectively, considering smear as gold standard. On comparing with CRS, the assay showed 100% sensitivity and 100% specificity in the diagnosis of MTBC. Conclusion In our study, LAMP assay was found to be a promising tool for the diagnosis of Tubercular Lymphadenitis and could be used for rapid and cost-effective diagnosis of Tubercular Lymphadenitis in resource-limited settings. Disclosures All authors: No reported disclosures.

Download Full-text

Clinical Evaluation of Three Sample-to-Answer Platforms for Detection of SARS-CoV-2

Journal of Clinical Microbiology ◽

10.1128/jcm.00783-20 ◽

2020 ◽

Vol 58 (8) ◽

Cited By ~ 68

Author(s):

Wei Zhen ◽

Elizabeth Smith ◽

Ryhana Manji ◽

Deborah Schron ◽

Gregory J. Berry

Keyword(s):

Severe Acute Respiratory Syndrome ◽

Drug Administration ◽

Clinical Evaluation ◽

Limit Of Detection ◽

Clinical Performance ◽

Molecular Diagnostic ◽

Analytical Sensitivity ◽

Reference Standard ◽

Slight Improvement ◽

Percent Agreement

ABSTRACT Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has now spread across the globe. As part of the worldwide response, many molecular diagnostic platforms have been granted emergency use authorization (EUA) by the Food and Drug Administration (FDA) to identify SARS-CoV-2 positive patients. Our objective was to evaluate three sample-to-answer molecular diagnostic platforms (Cepheid Xpert Xpress SARS-CoV-2 [Xpert Xpress], Abbott ID NOW COVID-19 [ID NOW], and GenMark ePlex SARS-CoV-2 Test [ePlex]) to determine analytical sensitivity, clinical performance, and workflow for the detection of SARS-CoV-2 in nasopharyngeal swabs from 108 symptomatic patients. We found that Xpert Xpress had the lowest limit of detection (100% detection at 100 copies/ml), followed by ePlex (100% detection at 1,000 copies/ml), and ID NOW (20,000 copies/ml). Xpert Xpress also had highest positive percent agreement (PPA) compared to our reference standard (98.3%) followed by ePlex (91.4%) and ID NOW (87.7%). All three assays showed 100% negative percent agreement (NPA). In the workflow analysis, ID NOW produced the lowest time to result per specimen (∼17 min) compared to Xpert Xpress (∼46 min) and ePlex (∼1.5 h), but what ID NOW gained in rapid results, it lost in analytical and clinical performance. ePlex had the longest time to results and showed a slight improvement in PPA over ID NOW. Information about the clinical and analytical performance of these assays, as well as workflow, will be critical in making informed and timely decisions on testing platforms.

Download Full-text

Clinical validation of near-infrared light transillumination for early proximal caries detection using a composite reference standard

Journal of Dentistry: X ◽

10.1016/j.jjodo.2020.100025 ◽

2020 ◽

Vol 4 ◽

pp. 100025

Author(s):

Eirini Stratigaki ◽

Fabian N. Jost ◽

Jan Kühnisch ◽

Friederike Litzenburger ◽

Adrian Lussi ◽

...

Keyword(s):

Near Infrared ◽

Infrared Light ◽

Caries Detection ◽

Clinical Validation ◽

Reference Standard ◽

Near Infrared Light ◽

Proximal Caries ◽

Composite Reference Standard

Download Full-text

QuantiFERON-TB Gold Plus Is a More Sensitive Screening Tool than QuantiFERON-TB Gold In-Tube for Latent Tuberculosis Infection among Older Adults in Long-Term Care Facilities

Journal of Clinical Microbiology ◽

10.1128/jcm.00427-18 ◽

2018 ◽

Vol 56 (8) ◽

Cited By ~ 8

Author(s):

Jung-Yien Chien ◽

Hsiu-Tzy Chiang ◽

Min-Chi Lu ◽

Wen-Chien Ko ◽

Chong-Jen Yu ◽

...

Keyword(s):

Older Adults ◽

Long Term Care ◽

Latent Tuberculosis Infection ◽

Latent Tuberculosis ◽

Reference Standard ◽

Term Care ◽

Care Facilities ◽

Composite Reference Standard ◽

Higher Sensitivity

ABSTRACT We investigated the prevalence of latent tuberculosis infection (LTBI) among the residents in seven long-term care facilities (LTCFs) located in different regions of Taiwan and compared the performance of two interferon gamma release assays, i.e., QuantiFERON-TB Gold In-Tube (QFT-GIT) and QuantiFERON-TB Gold Plus (QFT-Plus) for screening LTBI. We also assessed the diagnostic performance against a composite reference standard (subjects with persistent-positive, transient-positive, and negative results from QFTs during reproducibility analysis were classified as definite, possible, and not LTBI, respectively). Two hundred forty-four residents were enrolled, and 229 subjects were included in the analysis. The median age was 80 years (range, 60 to 102 years old), and 117 (51.1%) were male. Among them, 66 (28.8%) and 74 (32.3%) subjects had positive results from QFT-GIT and QFT-Plus, respectively, and the results for 215 (93.9%) subjects showed agreement. Using the composite reference standard, 66 (28.8%), 11 (4.8%), and 152 (66.4%) were classified as definite, possible, and not LTBI, respectively. For definite LTBI, the sensitivity, specificity, positive predictive value, and negative predictive value of QFT-GIT were 89.4%, 95.7%, 89.4%, and 95.7%, respectively, and those for QFT-Plus were 100.0%, 95.1%, 89.2%, and 100.0%, respectively. The sensitivity of QFT-GIT decreased gradually with patient age. Compared to QFT-GIT, QFT-Plus displayed significantly higher sensitivity (100.0% versus 89.4%, P = 0.013) and similar specificity (95.1% versus 95.7%). In conclusion, a high prevalence of LTBI was found among elders in LTCFs in Taiwan. The new QFT-Plus test demonstrated a higher sensitivity than QFT-GIT in the older adults in LTCFs.

Download Full-text

Accuracy of Rapid Point-of-Care Antibody Test in patients with suspected or confirmed COVID-19

10.1101/2020.11.17.20233296 ◽

2020 ◽

Author(s):

Rama Vancheeswaran ◽

Merlin L Willcox ◽

Beth Stuart ◽

Matthew Knight ◽

Hala Kandil ◽

...

Keyword(s):

Point Of Care ◽

Antibody Test ◽

Reference Laboratory ◽

List Type ◽

Reference Standard ◽

Point Of Care Test ◽

History Of ◽

Antibody Tests ◽

Composite Reference Standard ◽

Rapid Antibody

AbstractObjectivesTo assess the real-world diagnostic accuracy of the Livzon point-of-care rapid test for antibodies to SARS-COV-2DesignProspective cohort studySettingDistrict general hospital in EnglandParticipants173 Patients and 224 hospital staff with a history of COVID-19 symptoms, and who underwent PCR and/or reference antibody testing for COVID-19.InterventionsThe Livzon point-of-care (POC) lateral flow immunoassay rapid antibody test (IgM and IgG) was conducted at least 7 days after onset of symptoms and compared to the composite reference standard of PCR for SARS-COV-2 plus reference laboratory testing for antibodies to SARS-COV-2. The SARS-CoV-2 RT-PCR was tested using the available molecular technology during the study time (PHE laboratories, GeneXpert® system Xpert, Xpress SARS-CoV-2 and Source bioscience laboratory). All molecular platforms/assays were PHE/NHSE approved. The reference antibody test was the Elecsys Anti-SARS-CoV-2 assay (Roche diagnostics GmBH).Main outcome measuresSensitivity and specificity of the rapid antibody testResultsThe reference antibody test was positive in 190/268 (70.9%) of participants with a history of symptoms suggestive of COVID-19; in the majority (n=312) the POC test was taken 35 days or more after onset of symptoms. The POC antibody test had an overall sensitivity of 90.1% (292/328, 95% CI 86.3 – 93.1) and specificity of 100% (68/68, 95% CI 94.7 - 100) for confirming prior SARS-CoV-2 infection when compared to the composite reference standard. Sensitivity was 97.8% (89/92, 95% CI 92.3% to 99.7%) in participants who had been admitted to hospital and 84.4% (124/147, 95% CI 77.5% to 89.8%) in those with milder illness who had never been seen in hospital.ConclusionsThe Livzon point-of-care antibody test had comparable sensitivity and specificity to the reference laboratory antibody test, so could be used in clinical settings to support decision-making about patients presenting with more than 10 days of symptoms of COVID-19.What is already known on this topic-Presence of IgG and IgM antibodies to SARS-COV-2 indicates that the person was infected at least 7 days previously and is usually no longer infectious.-Rapid point-of-care tests for antibodies to SARS-COV-2 are widely available, cheap and easy to use-Preliminary evaluations suggested that rapid antibody tests may have insufficient accuracy to be useful for testing individual patients.What this study adds-The rapid point-of-care test for antibodies to SARS-COV-2 was 90.1% sensitive and 100% specific compared to reference standards for prior infection with COVID-19.-This is comparable to reference antibody tests-The point-of-care test evaluated in this study could be used to support clinical decision-making in real time, for patients presenting with symptoms of possible COVID-19 with at least 10 days of symptoms.

Download Full-text

Artificial intelligence system can achieve comparable results to experts for bone age assessment of Chinese children with abnormal growth and development

PeerJ ◽

10.7717/peerj.8854 ◽

2020 ◽

Vol 8 ◽

pp. e8854

Author(s):

Fengdan Wang ◽

Xiao Gu ◽

Shi Chen ◽

Yongliang Liu ◽

Qing Shen ◽

...

Keyword(s):

Artificial Intelligence ◽

Growth And Development ◽

Bone Age ◽

Chinese Children ◽

Absolute Difference ◽

Reference Standard ◽

Limits Of Agreement ◽

Altman Plot ◽

Bland Altman Plot ◽

Abnormal Growth

Objective Bone age (BA) is a crucial indicator for revealing the growth and development of children. This study tested the performance of a fully automated artificial intelligence (AI) system for BA assessment of Chinese children with abnormal growth and development. Materials and Methods A fully automated AI system based on the Greulich and Pyle (GP) method was developed for Chinese children by using 8,000 BA radiographs from five medical centers nationwide in China. Then, a total of 745 cases (360 boys and 385 girls) with abnormal growth and development from another tertiary medical center of north China were consecutively collected between January and October 2018 to test the system. The reference standard was defined as the result interpreted by two experienced reviewers (a radiologist with 10 years and an endocrinologist with 15 years of experience in BA reading) through consensus using the GP atlas. BA accuracy within 1 year, root mean square error (RMSE), mean absolute difference (MAD), and 95% limits of agreement according to the Bland-Altman plot were statistically calculated. Results For Chinese pediatric patients with abnormal growth and development, the accuracy of this new automated AI system within 1 year was 84.60% as compared to the reference standard, with the highest percentage of 89.45% in the 12- to 18-year group. The RMSE, MAD, and 95% limits of agreement of the AI system were 0.76 years, 0.58 years, and −1.547 to 1.428, respectively, according to the Bland-Altman plot. The largest difference between the AI and experts’ BA result was noted for patients of short stature with bone deformities, severe osteomalacia, or different rates of maturation of the carpals and phalanges. Conclusions The developed automated AI system could achieve comparable BA results to experienced reviewers for Chinese children with abnormal growth and development.

Download Full-text

Development of a Method for Clinical Evaluation of Artificial Intelligence–Based Digital Wound Assessment Tools

JAMA Network Open ◽

10.1001/jamanetworkopen.2021.7234 ◽

2021 ◽

Vol 4 (5) ◽

pp. e217234

Author(s):

Raelina S. Howell ◽

Helen H. Liu ◽

Aziz A. Khan ◽

Jon S. Woods ◽

Lawrence J. Lin ◽

...

Keyword(s):

Artificial Intelligence ◽

Clinical Evaluation ◽

Assessment Tools ◽

Wound Assessment

Download Full-text

Diversity in Typhoid Diagnostic Protocols And Recommendation for Composite Reference Standard

10.20944/preprints202107.0515.v2 ◽

2021 ◽

Author(s):

Bernard Igiri ◽

Stanley Irobekhian Reuben Okoduwa ◽

Shaibu Ahmed Munirat ◽

Iquo Bassey Otu-Bassey ◽

Abdullahi Bashir ◽

...

Keyword(s):

Typhoid Fever ◽

Diagnostic Accuracy ◽

Predictive Value ◽

Reference Standard ◽

Widal Test ◽

Average Percentage ◽

Public Health Burden ◽

Significant Difference ◽

Prospective Study Design ◽

Composite Reference Standard

Typhoid fever is a major public health burden which causes substantial global morbidity and mortality due to lack of decisive diagnostic protocols. The capacity of commonly use diagnostic test to validate the absence of typhoid fever is controversial. This study explores to evaluate new techniques for typhoid diagnosis and proposed a harmonised suitable standardized composite reference to be adopted. Published peer-reviewed articles indexed in PubMed, MEDLINE and Google scholar were reviewed for hospital-based studies. This study reveals new typhoid diagnostic techniques such as proteomics, serology, Rapid Diagnostic tests (RDTs), transcriptomics, genomics, and metabolomics. 34.4% of the studies use prospective study design. The study result establishes that, Widal test has a moderate diagnostic accuracy with average percentage sensitivity (52.9%), specificity (54%), positive predictive value (PPV) (56.8%) as well as negative predictive value (NPV) (55.6%) when compared with 29.4%, 28%, 29.5%, and 27.8% of Typhidot respectively. The findings showed a statistically significant difference on the sensitivity between Widal and Typhidot t (40) = 2.639, p = 0.012 at p<0.05 using independent sample t-test. When there is no perfect reference standard that has an optimal diagnostic accuracy, the need for a harmonised suitable standardized composite reference is essential. Hence, this study recommends that, peripheral blood culture with established sensitivity of 60% and Widal test with average sensitivity of 52.9% be adopted as a consensus composite reference standard for typhoid fever diagnosis in other to improve confidence in prevalence estimates.

Download Full-text

Diagnostic accuracy of GeneXpert MTB/RIF assay for detection of tubercular pleural effusion

PLoS ONE ◽

10.1371/journal.pone.0251618 ◽

2021 ◽

Vol 16 (6) ◽

pp. e0251618

Author(s):

Jyoti Jain ◽

Pooja Jadhao ◽

Shashank Banait ◽

Preetam Salunkhe

Keyword(s):

Pleural Effusion ◽

Diagnostic Accuracy ◽

Tertiary Care ◽

Public Health Problem ◽

P Value ◽

Care Hospital ◽

Reference Standard ◽

Likelihood Ratios ◽

Predictive Values ◽

Composite Reference Standard

India has been engaged in tuberculosis (TB) control activities for over 50 years and yet TB continues to remain India’s important public health problem. The present study was conducted to compare the performance of GeneXpert MTB/RIF (GXpert) assay with composite reference standard in diagnosing cases of tubercular pleural effusion (TPE) and to evaluate the reliability of rifampicin resistance. A cross-sectional study was performed in a Department of Medicine of a rural teaching tertiary care hospital in central India. In all consecutive patients with pleural effusion on chest radiograph presenting to Department of Medicine, GXpert assay and composite reference standard was performed to evaluate the diagnostic accuracy of GXpert assay for detecting TPE in comparison to composite reference standard. Standard formulae were used to calculate the sensitivity, specificity, positive predictive values (PPV), negative predictive values (NPV), positive likelihood ratios (LR+) and negative likelihood ratios (LR-). Mc-Nemar’s test was applied to compare variables. All comparisons were two-tailed. We considered the difference to be statistically significant if the P value was less than 0.05. The sensitivity of the GXpert assay in diagnosing TPE was 16.6% among 158 study participants, the specificity was 100% and diagnostic accuracy was 52.5% which was statistically significant (p value < 0.05). It had a PPV of 100% (95%CI: 88.3% - 100%) and a NPV of 47.5% (95%CI: 39.3% - 55.7%). The LR+ and LR-were 23.5 (95%CI: 1.43–38.6) and 0.83 (95%CI: 0.76–0.91) respectively. GXpert assay has a very high specificity in diagnosing TPE but has a low sensitivity. In comparison to composite reference standard Thus its clinical utility is limited when used as a standalone test. A physician’s clinical acumen in combination with routine pleural fluid analysis should be the key factor in the diagnosis of TPE in clinically and radiologically suspected patients, especially in high TB burden countries.

Download Full-text