Comparing the Diagnostic Accuracy of Clinician Judgement to a Novel Host Response Diagnostic for Acute Respiratory Illness

Abstract Background Difficulty discriminating bacterial from viral infections drives antibacterial misuse. Host gene expression tests discriminate bacterial and viral etiologies, but their clinical utility has not been evaluated. Methods Host gene expression and procalcitonin levels were measured in 582 Emergency Department participants with suspected infection. We also recorded clinician diagnosis, and clinician-recommended treatment. These four diagnostic strategies were compared to clinical adjudication as the reference. To estimate the clinical impact of host gene expression, we calculated the change in overall net benefit (∆NB, the difference in net benefit comparing one diagnostic strategy to a reference) across a range of prevalence estimates while factoring in the clinical significance of false positive and negative errors. Results Gene expression correctly classified bacterial, viral, or non-infectious illness in 74.1% of subjects, similar to the other strategies. Clinical diagnosis and clinician-recommended treatment revealed a bias toward overdiagnosis of bacterial infection resulting in high sensitivity (92.6% and 94.5%, respectively), but poor specificity (67.2% and 58.8%, respectively) resulting in a 33.3% rate of inappropriate antibacterial use. Gene expression offered a more balanced sensitivity (79.0%) and specificity (80.7%), which corresponded to a statistically significant improvement in average weighted accuracy (79.9% vs. 71.5% for procalcitonin and 76.3% for clinician-recommended treatment, p<0.0001 for both). Consequently, host gene expression had greater net benefit in diagnosing bacterial infection than clinician-recommended treatment (∆NB=6.4%) and procalcitonin (∆NB=17.4%). Conclusions Host gene expression-based tests to distinguish bacterial and viral infection can facilitate appropriate treatment, improving patient outcomes and mitigating the antibacterial resistance crisis.

Download Full-text

1218. Comparing the Diagnostic Accuracy of Clinician Judgement to a Novel Host Response Diagnostic for Acute Respiratory Illness

Open Forum Infectious Diseases ◽

10.1093/ofid/ofaa439.1403 ◽

2020 ◽

Vol 7 (Supplement_1) ◽

pp. S630-S630

Author(s):

Ian S Jaffe ◽

Anja K Jaehne ◽

Eugenia Quackenbush ◽

Micah T McClain ◽

Geoffrey S Ginsburg ◽

...

Keyword(s):

Gene Expression ◽

Diagnostic Accuracy ◽

Bacterial Infections ◽

Viral Infections ◽

Treatment Plan ◽

Respiratory Illness ◽

Host Gene ◽

Host Gene Expression ◽

Net Benefit ◽

Material Support

Abstract Background Discriminating bacterial and viral infections remains clinically challenging. The resulting antibacterial misuse contributes to antimicrobial resistance. Host gene expression-based tests are a promising strategy to discriminate of bacterial and viral infections, but their potential clinical utility has not yet been evaluated. Methods A host gene expression biosignature was measured using either qRT-PCR or microarray in 683 ED subjects with suspected infection. Based on chart reviews, we recorded clinical diagnosis as defined both by the provider assessment and by the provider treatment plan. The biosignature, diagnosis, treatment plan, and procalcitonin were compared to clinical adjudication as the reference standard. With this as a baseline, we then calculated average weighted accuracy (AWA) and change in overall net benefit (∆NB), weighting bacterial false negatives four times more seriously than false positives. Results Gene expression correctly classified the three possible disease etiologies (bacterial, viral, or non-infectious) 76.1% of the time, outperforming provider diagnosis, provider treatment, and procalcitonin (Table 1). Overall accuracy was higher in subjects with bacterial infections (n=278, 83.8% accurate) compared to those with viral (n=234, 76.5%) and non-infectious (n=171, 63.2%) etiologies. Due to a strong sensitivity bias to treat bacterial infections at the expense of diagnostic accuracy and specificity, the provider diagnosis was overall more accurate than the corresponding treatment plan (71.4% accuracy vs. 68.1%), resulting in inappropriate antibiotic use in 41.0% of cases where antibiotics were prescribed. The gene expression test had significantly higher AWA for the diagnosis of bacterial infection than both procalcitonin and provider treatment (82.4% vs. 70.3% and 74.4%, respectively; p < 0.0001). Consequently, the host gene expression test had greater net benefit than provider treatment (∆NBbact = 9.9%), provider diagnosis (∆NBbact = 4.4%), and procalcitonin (∆NBbact = 27.1%). Table 1: Summary of provider, procalcitonin, and host gene expression test performance in a cohort of 683 subjects. Conclusion Host gene expression-based tests to distinguish bacterial and viral infection can facilitate more appropriate treatment, leading to improved patient outcomes and mitigating the antibiotic resistance crisis. Disclosures Geoffrey S. Ginsburg, MD PhD, Predigen, Inc (Shareholder, Other Financial or Material Support) Ephraim L. Tsalik, MD, MHS, PhD, Predigen (Shareholder, Other Financial or Material Support, Founder)

Download Full-text

1216. Benchmarking Published Gene Signatures for Robust Infection Classification

Open Forum Infectious Diseases ◽

10.1093/ofid/ofaa439.1401 ◽

2020 ◽

Vol 7 (Supplement_1) ◽

pp. S629-S630

Author(s):

Nicholas Bodkin ◽

Melissa H Ross ◽

Ricardo Henao ◽

Ephraim L Tsalik

Keyword(s):

Gene Expression ◽

Bacterial Infection ◽

Viral Infection ◽

Test Performance ◽

Host Gene ◽

Diagnostic Strategy ◽

Host Gene Expression ◽

Gene Signatures ◽

Bacterial Classification ◽

Clinical Populations

Abstract Background Host gene expression has emerged as a promising diagnostic strategy to discriminate bacterial and viral infection. Multiple gene signatures of varying size and complexity have been developed in various clinical populations. However, there has been no systematic comparison of these signatures. It is also unclear how these signatures apply to different clinical populations. This meta-analysis examined 19 published signatures, validated in 49 publicly available datasets for a total of 4750 patients. The objectives were to understand how the signatures compared to each other with respect to composition and performance, and to evaluate their performance in different patient subgroups. Methods Signatures were characterized with respect to size, platform, and discovery population. For each of the 19 signatures, we ran leave-one-out cross-validation to generate AUCs for bacterial classification and viral classification. We then applied dataset-specific thresholds to generate accuracies, pooling patients across datasets. Results Signature performance varied significantly with a median AUC across all validation datasets ranging from 0.55 to 0.94 for bacterial classification and 0.79 to 0.96 for viral classification. Signature size varied (1- 341 genes) with smaller signatures generally performing more poorly for both bacterial classification (P < .001) and for viral classification (P = 0.02). Viral infection was easier to diagnose than bacterial infection (85% vs. 80% overall accuracy, respectively; P < .001). Host gene expression classifiers performed more poorly in children < 12-years compared to those older than 12-years for both bacterial infection (77% vs. 83%, respectively; P < .001) and for viral infection (82% vs. 89%, respectively; P < .001). We did not observe differences based on illness severity as defined by ICU care for either bacterial or viral infections. Conclusion We observed significant differences among gene expression signatures for bacterial/viral discrimination, though these were not due to variations in the discovery methods or populations. Signature size directly correlated with test performance, which was generally better for the diagnosis of viral infection and in populations >12-years. Disclosures Ephraim L. Tsalik, MD, MHS, PhD, Predigen (Shareholder, Other Financial or Material Support, Founder)

Download Full-text

1226. Performance of a Host Response Test for Bacterial/Viral Discrimination in Immunocompromised Patients

Open Forum Infectious Diseases ◽

10.1093/ofid/ofaa439.1411 ◽

2020 ◽

Vol 7 (Supplement_1) ◽

pp. S633-S634

Author(s):

Rachael E Mahle ◽

Sunil Suchindran ◽

Ricardo Henao ◽

Julie M Steinbrink ◽

Thomas W Burke ◽

...

Keyword(s):

Gene Expression ◽

Bacterial Infection ◽

Viral Infection ◽

Host Response ◽

Host Gene ◽

Response Strategy ◽

Solid Organ ◽

Host Gene Expression ◽

Material Support ◽

Response Test

Abstract Background Difficulty distinguishing bacterial and viral infections contributes to excess antibiotic use. A host response strategy overcomes many limitations of pathogen-based tests, but depends on a functional immune system. This approach may therefore be limited in immunocompromised (IC) hosts. Here, we evaluated a host response test in IC subjects, which has not been extensively studied in this manner. Methods An 81-gene signature was measured using qRT-PCR in previously enrolled IC subjects (chemotherapy, solid organ transplant, immunomodulatory agents, AIDS) with confirmed bacterial infection, viral infection, or non-infectious illness (NI). A regularized logistic regression model estimated the likelihood of bacterial, viral, and noninfectious classes. Clinical adjudication was the reference standard. Results A host gene expression model trained in a cohort of 136 immunocompetent subjects (43 bacterial, 41 viral, and 52 NI) had an overall accuracy of 84.6% for the diagnosis of bacterial vs. non-bacterial infection and 80.8% for viral vs. non-viral infection. The model was validated in an independent cohort of 134 IC subjects (64 bacterial, 28 viral, 42 NI). The overall accuracy was 73.9% for bacterial infection (p=0.03 vs. training cohort) and 75.4% for viral infection (p=0.27). Test utility could be improved by reporting probability ranges. For example, results divided into probability quartiles would allow the highest quartile to be used to rule in infection and the lowest to rule out infection. For IC subjects in the lowest quartile, the test had 90.1% and 96.4% sensitivity for bacterial and viral infection, respectively. For the highest quartile, the test had 91.4% and 84.0% specificity for bacterial and viral infection, respectively. The type or number of immunocompromising conditions did not impact performance. Illness Etiology Probabilities Conclusion A host gene expression test discriminated bacterial, viral, and non-infectious etiologies at a lower overall accuracy in IC patients compared to immunocompetent patients, though this difference was only significant for bacterial vs non-bacterial disease. With modified interpretive criteria, a host response strategy may offer clinically useful and complementary diagnostic information for IC patients. Disclosures Thomas W. Burke, PhD, Predigen, Inc (Consultant) Geoffrey S. Ginsburg, MD PhD, Predigen, Inc (Shareholder, Other Financial or Material Support) Christopher W. Woods, MD, MPH, FIDSA, Predigen, Inc (Shareholder, Other Financial or Material Support) Ephraim L. Tsalik, MD, MHS, PhD, FIDSA, Predigen, Inc (Scientific Research Study Investigator, Shareholder, Other Financial or Material Support)

Download Full-text

Host gene expression classifiers diagnose acute respiratory illness etiology

Science Translational Medicine ◽

10.1126/scitranslmed.aad6873 ◽

2016 ◽

Vol 8 (322) ◽

pp. 322ra11-322ra11 ◽

Cited By ~ 114

Author(s):

Ephraim L. Tsalik ◽

Ricardo Henao ◽

Marshall Nichols ◽

Thomas Burke ◽

Emily R. Ko ◽

...

Keyword(s):

Gene Expression ◽

Respiratory Illness ◽

Host Gene ◽

Acute Respiratory Illness ◽

Host Gene Expression

Download Full-text

Previously Derived Host Gene Expression Classifiers Identify Bacterial and Viral Etiologies of Acute Febrile Respiratory Illness in a South Asian Population

Open Forum Infectious Diseases ◽

10.1093/ofid/ofaa194 ◽

2020 ◽

Vol 7 (6) ◽

Author(s):

L Gayani Tillekeratne ◽

Sunil Suchindran ◽

Emily R Ko ◽

Elizabeth A Petzold ◽

Champica K Bodinayake ◽

...

Keyword(s):

Gene Expression ◽

Bacterial Infection ◽

Sensitivity And Specificity ◽

South Asian ◽

Peripheral Blood ◽

Ribonucleic Acid ◽

Classification Accuracy ◽

Host Gene ◽

Host Gene Expression ◽

Atypical Pathogens

Abstract Background Pathogen-based diagnostics for acute respiratory infection (ARI) have limited ability to detect etiology of illness. We previously showed that peripheral blood-based host gene expression classifiers accurately identify bacterial and viral ARI in cohorts of European and African descent. We determined classifier performance in a South Asian cohort. Methods Patients ≥15 years with fever and respiratory symptoms were enrolled in Sri Lanka. Comprehensive pathogen-based testing was performed. Peripheral blood ribonucleic acid was sequenced and previously developed signatures were applied: a pan-viral classifier (viral vs nonviral) and an ARI classifier (bacterial vs viral vs noninfectious). Results Ribonucleic acid sequencing was performed in 79 subjects: 58 viral infections (36 influenza, 22 dengue) and 21 bacterial infections (10 leptospirosis, 11 scrub typhus). The pan-viral classifier had an overall classification accuracy of 95%. The ARI classifier had an overall classification accuracy of 94%, with sensitivity and specificity of 91% and 95%, respectively, for bacterial infection. The sensitivity and specificity of C-reactive protein (>10 mg/L) and procalcitonin (>0.25 ng/mL) for bacterial infection were 100% and 34%, and 100% and 41%, respectively. Conclusions Previously derived gene expression classifiers had high predictive accuracy at distinguishing viral and bacterial infection in South Asian patients with ARI caused by typical and atypical pathogens.

Download Full-text

Rapid, Sample-to-Answer Host Gene Expression Test to Diagnose Viral Infection

Open Forum Infectious Diseases ◽

10.1093/ofid/ofz466 ◽

2019 ◽

Cited By ~ 1

Author(s):

Ephraim L Tsalik ◽

AyeAye Khine ◽

Abdossamad Talebpour ◽

Alaleh Samiei ◽

Vilcy Parmar ◽

...

Keyword(s):

Gene Expression ◽

Viral Infection ◽

Whole Blood ◽

Host Response ◽

Human Subjects ◽

Host Gene ◽

Host Gene Expression ◽

Suspected Infection ◽

Gene Expression Signatures ◽

Infectious Illness

Abstract Background Distinguishing bacterial, viral, or other etiologies of acute illness is diagnostically challenging with significant implications for appropriate antimicrobial use. Host gene-expression offers a promising approach although no clinically useful tests have yet been developed to accomplish this. Here, Qvella’s FAST™ HR process was developed to quantify previously identified host gene-expression signatures in whole blood in <45 minutes. Methods Whole blood was collected from 128 human subjects (mean age 47, range 18-88) with clinically adjudicated, microbiologically confirmed viral infection, bacterial infection, non-infectious illness, or healthy controls. Stabilized mRNA was released from cleaned and stabilized RNA-surfactant complexes using e-lysisTM, an electrical process providing a qRT-PCR-ready sample. Threshold cycle values (CT) for 10 host response targets were normalized to HPRT1 expression, a control mRNA. The transcripts in the signature were specifically chosen to discriminate viral from non-viral infection (bacterial, non-infectious illness, or healthy). Classification accuracy was determined using cross-validated sparse logistic regression. Results Reproducibility of mRNA quantification was within 1 cycle as compared to the difference seen between subjects with viral vs. non-viral infection (up to 5.0 normalized CT difference). Classification of 128 subjects into viral or non-viral etiologies demonstrated 90.6% overall accuracy compared to 82.0% for procalcitonin (p=0.06). FASTTM HR achieved rapid and accurate measurement of the host response to viral infection in less than 45 minutes. Conclusions These results demonstrate the ability to translate host gene expression signatures to clinical platforms for use in patients with suspected infection.

Download Full-text

2018. Host Gene Expression Classifiers Distinguish Bacterial and Viral Infections in Sri Lankan Patients with Acute Febrile Respiratory Illness

Open Forum Infectious Diseases ◽

10.1093/ofid/ofy210.1674 ◽

2018 ◽

Vol 5 (suppl_1) ◽

pp. S588-S588

Author(s):

L Gayani Tillekeratne ◽

Sunil Suchindran ◽

Emily Ko ◽

Elizabeth Petzold ◽

Champica K Bodinayake ◽

...

Keyword(s):

Gene Expression ◽

Viral Infections ◽

Respiratory Illness ◽

Host Gene ◽

Sri Lankan ◽

Host Gene Expression ◽

Febrile Respiratory Illness

Download Full-text

Validation of a host gene expression test for bacterial/viral discrimination in immunocompromised hosts

Clinical Infectious Diseases ◽

10.1093/cid/ciab043 ◽

2021 ◽

Author(s):

Rachael E Mahle ◽

Sunil Suchindran ◽

Ricardo Henao ◽

Julie M Steinbrink ◽

Thomas W Burke ◽

...

Keyword(s):

Gene Expression ◽

Bacterial Infection ◽

Viral Infection ◽

Host Response ◽

Pathogen Detection ◽

Host Gene ◽

Response Strategy ◽

Solid Organ ◽

Host Gene Expression ◽

The Impact

Abstract Background Host gene expression has emerged as a complementary strategy to pathogen detection tests for the discrimination of bacterial and viral infection. The impact of immunocompromise on host response tests remains unknown. We evaluated a host response test discriminating bacterial, viral, and non-infectious conditions in immunocompromised subjects. Methods An 81-gene signature was measured using RT-PCR in subjects with immunocompromise (chemotherapy, solid organ transplant, immunomodulatory agents, AIDS) with bacterial infection, viral infection, or noninfectious illness. A regularized logistic regression model trained in immunocompetent subjects was used to estimate the likelihood of each class in immunocompromised subjects. Results Accuracy in the 136-subject immunocompetent training cohort was 84.6% for bacterial vs. non-bacterial discrimination and 80.8% for viral vs. non-viral discrimination. Model validation in 134 immunocompromised subjects showed overall accuracy of 73.9% for bacterial infection (p=0.04 relative to immunocompetent subjects) and 75.4% for viral infection (p=0.30). A scheme reporting results by quartile improved test utility. The highest probability quartile ruled-in bacterial and viral infection with 91.4% and 84.0% specificity, respectively. The lowest probability quartile ruled-out infection with 90.1% and 96.4% sensitivity for bacterial and viral infection, respectively. Performance was independent of the type or number of immunocompromising conditions. Conclusion A host gene expression test discriminated bacterial, viral, and non-infectious etiologies at a lower overall accuracy in immunocompromised patients compared to immunocompetent patients, though this difference was only significant for bacterial infection classification. With modified interpretive criteria, a host response strategy may offer clinically useful diagnostic information for patients with immunocompromise.

Download Full-text