How to best test suspected cases of COVID-19: an analysis of the diagnostic performance of RT-PCR and alternative molecular methods for the detection of SARS-CoV-2

AbstractAs COVID-19 testing is rolled out increasingly widely, the use of a range of alternative testing methods will be beneficial in ensuring testing systems are resilient and adaptable to different clinical and public health scenarios. Here, we compare and discuss the diagnostic performance of a range of different molecular assays designed to detect the presence of SARS-CoV-2 infection in people with suspected COVID-19. Using findings from a systematic review of 103 studies, we categorised COVID-19 molecular assays into 12 different test classes, covering point-of-care tests, various alternative RT-PCR protocols, and alternative methods such as isothermal amplification. We carried out meta-analyses to estimate the diagnostic accuracy and clinical utility of each test class. We also estimated the positive and negative predictive values of all diagnostic test classes across a range of prevalence rates. Using previously validated RT-PCR assays as a reference standard, 11 out of 12 classes showed a summary sensitivity estimate of at least 92% and a specificity estimate of at least 99%. Several diagnostic test classes were estimated to have positive predictive values of 100% throughout the investigated prevalence spectrum, whilst estimated negative predictive values were more variable and sensitive to disease prevalence. We also report the results of clinical utility models that can be used to determine the information gained from a positive and negative test result in each class, and whether each test is more suitable for confirmation or exclusion of disease. Our analysis suggests that several tests exist that are suitable alternatives to standard RT-PCR and we discuss scenarios in which these could be most beneficial, such as where time to test result is critical or, where resources are constrained. However, we also highlight methodological concerns with the design and conduct of many included studies, and also the existence of likely publication bias for some test classes. Our results should be interpreted with these shortcomings in mind. Furthermore, our conclusions on test performance are limited to their use in symptomatic populations: we did not identify sufficient suitable data to allow analysis of testing in asymptomatic populations.

Download Full-text

ASSESSMENT OF PERFORMANCE AND IMPLEMENTATION CHARACTERISTICS OF RAPID POINT OF CARE SARS-CoV-2 ANTIGEN TESTING IN KENYA

10.1101/2021.06.03.21258290 ◽

2021 ◽

Author(s):

Jesse Gitaka ◽

Eva Muthamia ◽

Samuel Mbugua ◽

Mary Mungai ◽

Gama Bandawe ◽

...

Keyword(s):

Confidence Interval ◽

Test Performance ◽

Gold Standard ◽

Point Of Care ◽

Turnaround Time ◽

Ease Of Use ◽

Control Measures ◽

Antigen Test ◽

Rt Pcr ◽

Predictive Values

Abstract Background: The COVID-19 pandemic has resulted in a need for rapid identification of infectious cases. Testing barriers have prohibited adequate screening for SARS COV2, resulting in significant delays in treatment provision and commencement of outbreak control measures. This study aimed to generate evidence on the performance and implementation characteristics of the BD Veritor rapid antigen test as compared to the gold standard test for diagnosis of SARS COV2 in Kenya. Methods: This was a field test performance evaluation in symptomatic and asymptomatic adults undergoing testing for SARS COV2. Recruited participants were classified as SARS-CoV2-positive based on the locally implemented gold standard reverse transcription polymerase chain reaction (RT-PCR) test performed on nasopharyngeal swabs. 272 antigen tests were performed with simultaneous gold standard testing, allowing us to estimate sensitivity, specificity, positive and negative predictive values for the BD Veritor rapid antigen test platform. Implementation characteristics were assessed using the Consolidated Framework for Implementation Research for feasibility, acceptability, turn-around time, and ease-of-use metrics. Results and Discussion: We enrolled 97 PCR negative symptomatic and 128 PCR negative asymptomatic, and 28 PCR positive symptomatic and 19 PCR positive asymptomatic participants. Compared to the gold standard, the sensitivity of the BD Veritor antigen test was 94% (95% confidence interval [CI] 86.6 to 100.0) while the specificity was 98% (95% confidence interval [CI] 96 to 100). The sensitivity of BD Veritor antigen test was higher among symptomatic (100%) compared to asymptomatic (84%) participants, although this difference was not statistically significant. There was also a lack of association between cycle threshold value and sensitivity of BD Veritor test. The BD Veritor test had quick turnaround time and minimal resource requirements, and laboratory personnel conducting testing felt that it was easier to use than the gold standard RT-PCR. Conclusion: The BD Veritor rapid antigen test exhibited excellent sensitivity and specificity when used to detect SARS-CoV-2 infection among both symptomatic and asymptomatic individuals in varied population settings in Kenya. It was feasible to implement and easy to use, with rapid turnaround time.

Download Full-text

Development and evaluation of a new IgM/IgG rapid diagnostic test for SARS-CoV-2

10.1101/2020.10.09.20209866 ◽

2020 ◽

Author(s):

P.J. Ducrest

Keyword(s):

Diagnostic Test ◽

Rapid Diagnostic Test ◽

Gold Standard ◽

Diagnostic Performance ◽

Point Of Care ◽

Negative Control ◽

Simple Point ◽

Rt Pcr ◽

High Diagnostic Accuracy ◽

Plasma Displays

AbstractThere is an urgent need in rapid diagnostic test (RDT) to detect antibodies against SARS-CoV-2. We have developed a rapid and simple point-of-care lateral flow immunoassay (LFIA) detecting IgM and IgG against SARS-CoV-2 in 10 minutes. The aim of this study is to evaluate the diagnostic performance of this RDT. RT-PCR positive plasma samples (n=35) for SARS-CoV-2 and 97 negative control samples were studied. Diagnostic performance of IgG/IgM RDT was assessed using both gold standard RT-PCR and Electro-chemiluminescence immunoassay (ECLIA) Elecsys® Anti-SARS-CoV-2 total Ig. Overall, RDT sensitivity was 100% (95% confidence interval [95%CI]: 88-100%) and specificity 93% (95% CI: 85-97%). This IgG/IgM RDT done in plasma displays a high diagnostic accuracy for SARS-CoV-2 IgG/IgM in high COVID-19 prevalence settings. Its use could be considered in the absence of routine diagnostic serology facilities for samples collected between 10 and 180 days after symptoms onset.

Download Full-text

Evaluation of binary diagnostic tests accuracy for medical researches

Turkish Journal of Biochemistry ◽

10.1515/tjb-2020-0337 ◽

2020 ◽

Vol 0 (0) ◽

Author(s):

Jale Karakaya

Keyword(s):

Diagnostic Test ◽

Performance Measures ◽

Test Performance ◽

Diagnostic Tests ◽

Sample Size Calculation ◽

Rapid Test ◽

Screening Tests ◽

Youden Index ◽

Predictive Values ◽

Test Result

AbstractObjectivesThe aim of this study is to introduce the features of diagnostic tests. In addition, it will be demonstrated which performance measures can be used for diagnostic tests with binary results, the properties of these measures and how to interpret them.Materials and MethodsThe evaluation of the diagnostic test performance measures may differ depending on whether the test result is numerical or binary. When the diagnostic test result is continuous numerical data, ROC analysis is often utilized. The performance of a diagnostic test with binary results are usually evaluated using the measures of sensitivity and specificity. However, there are some important measures other than these two measures for binary test results. These measures are predictive values, overall accuracy, diagnostic odds ratio, Youden index, and likelihood ratios.ResultsA hypothetical data has been produced based on the studies conducted on the performance of rapid tests (Specific IgM/IgG) according to the RT-PCR test for Covid 19 in the literature. An example of a diagnostic test (Specific IgM/IgG) with a binary result is given and all measurements and their confidence interval are obtained for this data. The performance of rapid test was examined and interpreted.ConclusionIt is important to design and evaluate the performance of diagnostic/screening tests for health care. In this review, some basic definitions, performance measures that can be used only in evaluating the diagnostic tests with binary results and their confidence intervals are mentioned. Having many different measures provides different interpretations in the evaluation of test performance. Accurately predicting the performance of a diagnostic test depends on many factors. These factors can be study design, criteria of participant selection, sample size calculation, test methods etc. There are guidelines that ensure that all information regarding the conditions under which the study was conducted is in report, in terms of such factors. Therefore, these guidelines are recommended for use of the checklist by many publishers.

Download Full-text

Performance evaluation of the Simtomax® CoronaCheck rapid diagnostic test

10.1101/2020.10.28.20219667 ◽

2020 ◽

Author(s):

P. J. Ducrest ◽

A. Freymond ◽

J.-M. Segura

Keyword(s):

Diagnostic Test ◽

Rapid Diagnostic Test ◽

Diagnostic Performance ◽

High Sensitivity ◽

High Specificity ◽

Negative Control ◽

List Type ◽

Plasma Samples ◽

Rt Pcr ◽

Two Samples

AbstractThe aim of this study was to evaluate the diagnostic performance of Simtomax® CoronaCheck, a serology rapid diagnostic test (RDT) for the detection of IgG and IgM against SARS-CoV-2. 48 plasma samples positive for SARS-CoV-2 based on RT-PCR and 98 negative control samples were studied. Diagnostic performance of the IgG/IgM RDT was assessed against RT-PCR and the electro-chemiluminescence immunoassay (ECLIA) Elecsys® Anti-SARS-CoV-2 total Ig. Overall, the RDT sensitivity was 92% (95% confidence interval [95%CI]: 79-97), specificity 97% (95% CI: 91-99%), PPV 94% (95% CI: 81-98) and the NPV 96% (95% CI: 89-99). When considering only samples collected ≥ 15 days post-symptoms (DPS), the sensitivity increased to 98% (95%CI: 86-100) and the specificity was 97% (95% CI: 91-99%). Two samples with 180 DPS were still positive for IgG. Globally, this IgG/IgM RDT displayed a high diagnostic accuracy for SARS-CoV-2 IgG/IgM detection in plasma samples in high COVID-19 prevalence settings. It could be effectively used, in absence of facilities for routine diagnostic serology, for samples with a DPS between 15 and 180 days.Highlights–The rapid diagnostic test Simtomax CoronaCheck displays a high sensitivity of 98% and a high specificity of 97% for SARS-CoV-2 IgG/IgM detection in plasma samples after 15 days post-symptoms.–The rapid diagnostic test Simtomax CoronaCheck can detect SARS-CoV-2 antibodies in plasma up to 180 days after symptom onset.–The rapid diagnostic test Simtomax CoronaCheck could be effectively used as an alternative to serological analysis using laboratory facilities.

Download Full-text

Closing the loop on test results to reduce communication failures: a rapid review of evidence, practice and patient perspectives

BMC Health Services Research ◽

10.1186/s12913-020-05737-x ◽

2020 ◽

Vol 20 (1) ◽

Author(s):

Breanna Wright ◽

Alyse Lennox ◽

Mark L. Graber ◽

Peter Bragge

Keyword(s):

Diagnostic Test ◽

Point Of Care ◽

Patient Perspectives ◽

Audit And Feedback ◽

Rapid Review ◽

Test Results ◽

Multiple Perspectives ◽

Communication Breakdown ◽

Closing The Loop ◽

Test Result

Abstract Background Communication failures involving test results contribute to issues of patient harm and sentinel events. This article aims to synthesise review evidence, practice insights and patient perspectives addressing problems encountered in the communication of diagnostic test results. Methods The rapid review identified ten systematic reviews and four narrative reviews. Five practitioner interviews identified insights into interventions and implementation, and a citizen panel with 15 participants explored the patient viewpoint. Results The rapid review provided support for the role of technology to ensure effective communication; behavioural interventions such as audit and feedback could be effective in changing clinician behaviour; and point-of-care tests (bedside testing) eliminate the communication breakdown problem altogether. The practice interviews highlighted transparency, and clarifying the lines of responsibility as central to improving test result communication. Enabling better information sharing, implementing adequate planning and utilising technology were also identified in the practice interviews as viable strategies to improve test result communication. The citizen panel highlighted technology as critical to improving communication of test results to both health professionals and patients. Patients also highlighted the importance of having different ways of accessing test results, which is particularly pertinent when ensuring suitability for vulnerable populations. Conclusions This paper draws together multiple perspectives on the problem of failures in diagnostic test results communication to inform appropriate interventions. Across the three studies, technology was identified as the most feasible option for closing the loop on test result communication. However, the importance of clear, consistent communication and more streamlined processes were also key elements that emerged. Review registration The protocol for the rapid review was registered with PROSPERO CRD42018093316.

Download Full-text

Mutual Information as an Index of Diagnostic Test Performance

Methods of Information in Medicine ◽

10.1055/s-0038-1634358 ◽

2003 ◽

Vol 42 (03) ◽

pp. 260-264 ◽

Cited By ~ 15

Author(s):

W. A. Benish

Keyword(s):

Mutual Information ◽

Diagnostic Test ◽

Test Performance ◽

Random Variable ◽

Disease State ◽

Necessary Condition ◽

Conditional Probabilities ◽

Amount Of Information ◽

Diagnostic Test Performance ◽

Test Result

Summary Objectives: This paper demonstrates that diagnostic test performance can be quantified as the average amount of information the test result (R) provides about the disease state (D). Methods: A fundamental concept of information theory, mutual information, is directly applicable to this problem. This statistic quantifies the amount of information that one random variable contains about another random variable. Prior to performing a diagnostic test, R and D are random variables. Hence, their mutual information, I(D;R), is the amount of information that R provides about D. Results: I(D;R) is a function of both 1) the pretest probabilities of the disease state and 2) the set of conditional probabilities relating each possible test result to each possible disease state. The area under the receiver operating characteristic curve (AUC) is a popular measure of diagnostic test performance which, in contrast to I(D;R), is independent of the pretest probabilities; it is a function of only the set of conditional probabilities. The AUC is not a measure of diagnostic information. Conclusions: Because I(D;R) is dependent upon pretest probabilities, knowledge of the setting in which a diagnostic test is employed is a necessary condition for quantifying the amount of information it provides. Advantages of I(D;R) over the AUC are that it can be calculated without invoking an arbitrary curve fitting routine, it is applicable to situations in which multiple diagnoses are under consideration, and it quantifies test performance in meaningful units (bits of information).

Download Full-text

Does exposure to simulated patient cases improve accuracy of clinicians’ predictive value estimates of diagnostic test results? A within-subjects experiment at St Michael’s Hospital, Toronto, Canada

BMJ Open ◽

10.1136/bmjopen-2017-019241 ◽

2018 ◽

Vol 8 (2) ◽

pp. e019241 ◽

Cited By ~ 3

Author(s):

Bonnie Armstrong ◽

Julia Spaniol ◽

Nav Persaud

Keyword(s):

Diagnostic Test ◽

Predictive Value ◽

Simulated Patient ◽

Estimation Accuracy ◽

Absolute Difference ◽

Accurate Estimation ◽

Estimation Errors ◽

Predictive Values ◽

Within Subjects ◽

Test Result

ObjectiveClinicians often overestimate the probability of a disease given a positive test result (positive predictive value; PPV) and the probability of no disease given a negative test result (negative predictive value; NPV). The purpose of this study was to investigate whether experiencing simulated patient cases (ie, an ‘experience format’) would promote more accurate PPV and NPV estimates compared with a numerical format.DesignParticipants were presented with information about three diagnostic tests for the same fictitious disease and were asked to estimate the PPV and NPV of each test. Tests varied with respect to sensitivity and specificity. Information about each test was presented once in the numerical format and once in the experience format. The study used a 2 (format: numerical vs experience) × 3 (diagnostic test: gold standard vs low sensitivity vs low specificity) within-subjects design.SettingThe study was completed online, via Qualtrics (Provo, Utah, USA).Participants50 physicians (12 clinicians and 38 residents) from the Department of Family and Community Medicine at St Michael’s Hospital in Toronto, Canada, completed the study. All participants had completed at least 1 year of residency.ResultsEstimation accuracy was quantified by the mean absolute error (MAE; absolute difference between estimate and true predictive value). PPV estimation errors were larger in the numerical format (MAE=32.6%, 95% CI 26.8% to 38.4%) compared with the experience format (MAE=15.9%, 95% CI 11.8% to 20.0%,d=0.697, P<0.001). Likewise, NPV estimation errors were larger in the numerical format (MAE=24.4%, 95% CI 14.5% to 34.3%) than in the experience format (MAE=11.0%, 95% CI 6.5% to 15.5%,d=0.303, P=0.015).ConclusionsExposure to simulated patient cases promotes accurate estimation of predictive values in clinicians. This finding carries implications for diagnostic training and practice.

Download Full-text

Decision-Analytic Modeling to Evaluate Benefits and Harms of Medical Tests: Uses and Limitations

Medical Decision Making ◽

10.1177/0272989x09345022 ◽

2009 ◽

Vol 29 (5) ◽

pp. E22-E29 ◽

Cited By ~ 56

Author(s):

Thomas A. Trikalinos ◽

Uwe Siebert ◽

Joseph Lau

Keyword(s):

Patient Outcomes ◽

Test Performance ◽

Comparative Evaluation ◽

Clinical Utility ◽

Decision Modeling ◽

Test And Treat ◽

Medical Tests ◽

Downstream Effects ◽

Alternate Test ◽

Test Result

The clinical utility of medical tests is measured by whether the information they provide affects patient-relevant outcomes. To a large extent, effects of medical tests are indirect in nature. In principle, a test result affects patient outcomes mainly by influencing treatment choices. This indirectness in the link between testing and its downstream effects poses practical challenges to comparing alternate test-and-treat strategies in clinical trials. Keeping in mind the broader audience of researchers who perform comparative effectiveness reviews and technology assessments, the authors summarize the rationale for and pitfalls of decision modeling in the comparative evaluation of medical tests by virtue of specific examples. Modeling facilitates the interpretation of test performance measures by connecting the link between testing and patient outcomes, accounting for uncertainties and explicating assumptions, and allowing the systematic study of tradeoffs and uncertainty. The authors discuss challenges encountered when modeling test-and-treat strategies, including but not limited to scarcity of data on important parameters, transferring estimates of test performance across studies, choosing modeling outcomes, and obtaining summary estimates for test performance data.

Download Full-text

The Use of Latent Class Analysis to Estimate the Sensitivities and Specificities of Diagnostic Tests for Squash vein yellowing virus in Cucurbit Species When There Is No Gold Standard

Phytopathology ◽

10.1094/phyto-03-13-0071-r ◽

2013 ◽

Vol 103 (12) ◽

pp. 1243-1251 ◽

Cited By ~ 6

Author(s):

William W. Turechek ◽

Craig G. Webster ◽

Jingyi Duan ◽

Pamela D. Roberts ◽

Chandrasekar S. Kousik ◽

...

Keyword(s):

Latent Class Analysis ◽

Diagnostic Test ◽

Test Performance ◽

Gold Standard ◽

Diagnostic Tests ◽

Latent Class ◽

Visual Assessment ◽

Class Analysis ◽

Rt Pcr ◽

Yellowing Virus

Squash vein yellowing virus (SqVYV) is the causal agent of viral watermelon vine decline, one of the most serious diseases in watermelon (Citrullus lanatus L.) production in the southeastern United States. At present, there is not a gold standard diagnostic test for determining the true status of SqVYV infection in plants. Current diagnostic methods for identification of SqVYV-infected plants or tissues are based on the reverse-transcription polymerase chain reaction (RT-PCR), tissue blot nucleic acid hybridization assays (TB), and expression of visual symptoms. A quantitative assessment of the performance of these diagnostic tests is lacking, which may lead to an incorrect interpretation of results. In this study, latent class analysis (LCA) was used to estimate the sensitivities and specificities of RT-PCR, TB, and visual assessment of symptoms as diagnostic tests for SqVYV. The LCA model assumes that the observed diagnostic test responses are linked to an underlying latent (nonobserved) disease status of the population, and can be used to estimate sensitivity and specificity of the individual tests, as well as to derive an estimate of the incidence of disease when a gold standard test does not exist. LCA can also be expanded to evaluate the effect of factors and was done here to determine whether diagnostic test performances varied among the type of plant tissue being tested (crown versus vine tissue), where plant samples were taken relative to the position of the crown (i.e., distance from the crown), host (i.e., genus), and habitat (field-grown versus greenhouse-grown plants). Results showed that RT-PCR had the highest sensitivity (0.94) and specificity (0.98) of the three tests. TB had better sensitivity than symptoms for detection of SqVYV infection (0.70 versus 0.32), while the visual assessment of symptoms was more specific than TB and, thus, a better indicator of noninfection (0.98 versus 0.65). With respect to the grouping variables, RT-PCR and TB had better sensitivity but poorer specificity for diagnosing SqVYV infection in crown tissue than it did in vine tissue, whereas symptoms had very poor sensitivity but excellent specificity in both tissues for all cucurbits analyzed in this study. Test performance also varied with habitat and genus but not with distance from the crown. The results given here provide quantitative measurements of test performance for a range of conditions and provide the information needed to interpret test results when tests are used in parallel or serial combination for a diagnosis.

Download Full-text

Field performance evaluation of the PanBio rapid SARS-CoV-2 antigen assay in an epidemic driven by 501Y.v2 (lineage B.1.351) in the Eastern Cape, South Africa

10.1101/2021.02.03.21251057 ◽

2021 ◽

Author(s):

Oluwakemi Laguda Akingba ◽

Kaitlin Sprong ◽

Diana Ruth Hardie

Keyword(s):

South Africa ◽

Test Performance ◽

Field Performance ◽

Evaluation Study ◽

Clinical Samples ◽

Eastern Cape ◽

Rt Pcr ◽

Assay Sensitivity ◽

Predictive Values ◽

Mobile Community

AbstractBackgroundSouth Africa was the African country most severely affected by the SARS-CoV-2 pandemic during 2020, experiencing 2 waves of infection. During the first wave, diagnostics were largely based on reverse transcription-linked PCR (RT-PCR). The Abbott PanBio antigen test was deployed during the 2nd wave which was driven by emergence of the 501Y.v2 variant. At the time of evaluation in mid-November 2020, 501Y.v2 was the dominant circulating virus in Nelson Mandela Bay, in the Eastern Cape Province.MethodsA prospective diagnostic evaluation study was undertaken, during a period of high community transmission, to evaluate the field performance of the PanBio antigen RTD. Testing was conducted at mobile community testing centres on 677 ambulant patients seeking SARS-CoV-2 testing. RT-PCR was performed on the original naso-pharyngeal antigen swabs to evaluate test performance.ResultsOf 146 RT-PCR positive individuals, 101 were RTD positive in the clinic. The antigen RTD had an overall sensitivity of 69.2% (95%CI 61.4, 75.8) and specificity of 99.0% (95%CI 98.8, 99.3) in this clinical context. Sensitivity was strongly dependent on the amount of virus in clinical samples, as reflected by the PCR cycle threshold (CT) value, with 100% detection in samples where the CT was <20, 96% with CT between 20-25, 89% with CT between 26-30 and 64% when CT was 31-35.ConclusionsThe assay reliably detected 501Y.v2 infections in ambulatory ill patients. Assay sensitivity was >90% in patients with high viral loads who are expected to be most infectious. Negative and positive predictive values were also >90%.

Download Full-text