scholarly journals Risk of bias and limits of reporting in diagnostic accuracy studies for commercial point-of-care tests for respiratory pathogens

2018 ◽  
Vol 146 (6) ◽  
pp. 747-756
Author(s):  
J.M. Hughes ◽  
C. Penney ◽  
S. Boyd ◽  
P. Daley

AbstractCommercial point-of-care (POC) diagnostic tests for Group A Streptococcus, Streptococcus pneumoniae, and influenza virus have large potential diagnostic and financial impact. Many published reports on test performance, often funded by diagnostics companies, are prone to bias. The Standards for Reporting of Diagnostic Accuracy (STARD 2015) are a protocol to encourage accurate, transparent reporting. The Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool evaluates risk of bias and transportability of results. We used these tools to evaluate diagnostic test accuracy studies of POC studies for three respiratory pathogens. For the 96 studies analysed, compliance was <25% for 14/34 STARD 2015 standards, and 3/7 QUADAS-2 domains showed a high risk of bias. All reports lacked reporting of at least one criterion. These biases should be considered in the interpretation of study results.

2020 ◽  
Vol 10 (1) ◽  
Author(s):  
Pakpoom Subsoontorn ◽  
Manupat Lohitnavy ◽  
Chuenjid Kongkaew

AbstractMany recent studies reported coronavirus point-of-care tests (POCTs) based on isothermal amplification. However, the performances of these tests have not been systematically evaluated. Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy was used as a guideline for conducting this systematic review. We searched peer-reviewed and preprint articles in PubMed, BioRxiv and MedRxiv up to 28 September 2020 to identify studies that provide data to calculate sensitivity, specificity and diagnostic odds ratio (DOR). Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) was applied for assessing quality of included studies and Preferred Reporting Items for a Systematic Review and Meta-analysis of Diagnostic Test Accuracy Studies (PRISMA-DTA) was followed for reporting. We included 81 studies from 65 research articles on POCTs of SARS, MERS and COVID-19. Most studies had high risk of patient selection and index test bias but low risk in other domains. Diagnostic specificities were high (> 0.95) for included studies while sensitivities varied depending on type of assays and sample used. Most studies (n = 51) used reverse transcription loop-mediated isothermal amplification (RT-LAMP) to diagnose coronaviruses. RT-LAMP of RNA purified from COVID-19 patient samples had pooled sensitivity at 0.94 (95% CI: 0.90–0.96). RT-LAMP of crude samples had substantially lower sensitivity at 0.78 (95% CI: 0.65–0.87). Abbott ID Now performance was similar to RT-LAMP of crude samples. Diagnostic performances by CRISPR and RT-LAMP on purified RNA were similar. Other diagnostic platforms including RT- recombinase assisted amplification (RT-RAA) and SAMBA-II also offered high sensitivity (> 0.95). Future studies should focus on the use of un-bias patient cohorts, double-blinded index test and detection assays that do not require RNA extraction.


2018 ◽  
Vol 57 (2) ◽  
Author(s):  
Leslie J. Donato ◽  
Nikki K. Myhre ◽  
Martha A. Murray ◽  
Margaret R. McDonah ◽  
Jane F. Myers ◽  
...  

ABSTRACT Although U.S. Food and Drug Administration-approved and CLIA-waived point-of-care (POC) molecular systems are being implemented in routine clinical practice, instrument reliability, test performance in the hands of end users, and the potential for environmental contamination resulting from use of POC molecular systems have not been extensively evaluated. We performed a prospective evaluation of the Roche cobas Liat group A streptococcus (GAS) assay compared to routine real-time PCR. We evaluated test accuracy, instrument failure rate, and monitored for environmental contamination when testing was performed by minimally trained end users in an Express Care Clinic environment. The overall concordance of the Liat GAS assay with routine testing was 97.2% (455/468). The average Liat failure rate across three analyzers was 6.6% (33/501) (range, 3.7 to 11.6%), and no environmental contamination was detected during the course of the study. The cobas Liat platform and GAS assay demonstrated reliable performance in the end user setting and may serve as a rapid, POC option for routine diagnostic testing for certain infectious diseases, including GAS.


2020 ◽  
Vol 24 (31) ◽  
pp. 1-232 ◽  
Author(s):  
Hannah Fraser ◽  
Daniel Gallacher ◽  
Felix Achana ◽  
Rachel Court ◽  
Sian Taylor-Phillips ◽  
...  

Background Sore throat is a common condition caused by an infection of the airway. Most cases are of a viral nature; however, a number of these infections may be caused by the group A Streptococcus bacterium. Most viral and bacterial sore throat infections resolve spontaneously within a few weeks. Point-of-care testing in primary care has been recognised as an emerging technology for aiding targeted antibiotic prescribing for sore throat in cases that do not spontaneously resolve. Objective Systematically review the evidence for 21 point-of-care tests for detecting group A Streptococcus bacteria and develop a de novo economic model to compare the cost-effectiveness of point-of-care tests alongside clinical scoring tools with the cost-effectiveness of clinical scoring tools alone for patients managed in primary care and hospital settings. Data sources Multiple electronic databases were searched from inception to March 2019. The following databases were searched in November and December 2018 and searches were updated in March 2019: MEDLINE [via OvidSP (Health First, Rockledge, FL, USA)], MEDLINE In-Process & Other Non-Indexed Citations (via OvidSP), MEDLINE Epub Ahead of Print (via OvidSP), MEDLINE Daily Update (via OvidSP), EMBASE (via OvidSP), Cochrane Database of Systematic Reviews [via Wiley Online Library (John Wiley & Sons, Inc., Hoboken, NJ, USA)], Cochrane Central Register of Controlled Trials (CENTRAL) (via Wiley Online Library), Database of Abstracts of Reviews of Effects (DARE) (via Centre for Reviews and Dissemination), Health Technology Assessment database (via the Centre for Reviews and Dissemination), Science Citation Index and Conference Proceedings [via the Web of Science™ (Clarivate Analytics, Philadelphia, PA, USA)] and the PROSPERO International Prospective Register of Systematic Reviews (via the Centre for Reviews and Dissemination). Review methods Eligible studies included those of people aged ≥ 5 years presenting with sore throat symptoms, studies comparing point-of-care testing with antibiotic-prescribing decisions, studies of test accuracy and studies of cost-effectiveness. Quality assessment of eligible studies was undertaken. Meta-analysis of sensitivity and specificity was carried out for tests with sufficient data. A decision tree model estimated costs and quality-adjusted life-years from an NHS and Personal Social Services perspective. Results The searches identified 38 studies of clinical effectiveness and three studies of cost-effectiveness. Twenty-six full-text articles and abstracts reported on the test accuracy of point-of-care tests and/or clinical scores with biological culture as a reference standard. In the population of interest (patients with Centor/McIsaac scores of ≥ 3 points or FeverPAIN scores of ≥ 4 points), point estimates were 0.829 to 0.946 for sensitivity and 0.849 to 0.991 for specificity. There was considerable heterogeneity, even for studies using the same point-of-care test, suggesting that is unlikely that any single study will have accurately captured a test’s true performance. There is some randomised controlled trial evidence to suggest that the use of rapid antigen detection tests may help to reduce antibiotic-prescribing rates. Sensitivity and specificity estimates for each test in each age group and care setting combination were obtained using meta-analyses where appropriate. Any apparent differences in test accuracy may not be attributable to the tests, and may have been caused by known differences in the studies, latent characteristics or chance. Fourteen of the 21 tests reviewed were included in the economic modelling, and these tests were not cost-effective within the current National Institute for Health and Care Excellence’s cost-effectiveness thresholds. Uncertainties in the cost-effectiveness estimates included model parameter inputs and assumptions that increase the cost of testing, and the penalty for antibiotic overprescriptions. Limitations No information was identified for the elderly population or pharmacy setting. It was not possible to identify which test is the most accurate owing to the paucity of evidence. Conclusions The systematic review and the cost-effectiveness models identified uncertainties around the adoption of point-of-care tests in primary and secondary care settings. Although sensitivity and specificity estimates are promising, we have little information to establish the most accurate point-of-care test. Further research is needed to understand the test accuracy of point-of-care tests in the proposed NHS pathway and in comparable settings and patient groups. Study registration The protocol of the review is registered as PROSPERO CRD42018118653. Funding This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 24, No. 31. See the NIHR Journals Library website for further project information.


Viruses ◽  
2020 ◽  
Vol 13 (1) ◽  
pp. 18
Author(s):  
Michèle Bergmann ◽  
Mike Holzheu ◽  
Yury Zablotski ◽  
Stephanie Speck ◽  
Uwe Truyen ◽  
...  

Measuring antibodies to evaluate dogs´ immunity against canine parvovirus (CPV) is useful to avoid unnecessary re-vaccinations. The study aimed to evaluate the quality and practicability of four point-of-care (POC) tests for detection of anti-CPV antibodies. The sera of 198 client-owned and 43 specific pathogen-free (SPF) dogs were included; virus neutralization was the reference method. Specificity, sensitivity, positive and negative predictive value (PPV and NPV), and overall accuracy (OA) were calculated. Specificity was considered to be the most important indicator for POC test performance. Differences between specificity and sensitivity of POC tests in the sera of all dogs were determined by McNemar, agreement by Cohen´s kappa. Prevalence of anti-CPV antibodies in all dogs was 80% (192/241); in the subgroup of client-owned dogs, it was 97% (192/198); and in the subgroup of SPF dogs, it was 0% (0/43). FASTest® and CanTiCheck® were easiest to perform. Specificity was highest in the CanTiCheck® (overall dogs, 98%; client-owned dogs, 83%; SPF dogs, 100%) and the TiterCHEK® (overall dogs, 96%; client-owned dogs, 67%; SPF dogs, 100%); no significant differences in specificity were observed between the ImmunoComb®, the TiterCHEK®, and the CanTiCheck®. Sensitivity was highest in the FASTest® (overall dogs, 95%; client-owned dogs, 95%) and the CanTiCheck® (overall dogs, 80%; client-owned dogs, 80%); sensitivity of the FASTest® was significantly higher compared to the one of the other three tests (McNemars p-value in each comparison: <0.001). CanTiCheck® would be the POC test of choice when considering specificity and practicability. However, differences in the number of false positive results between CanTiCheck®, TiterCHEK®, and ImmunoComb® were minimal.


BMJ ◽  
2020 ◽  
pp. m2516 ◽  
Author(s):  
Mayara Lisboa Bastos ◽  
Gamuchirai Tavaziva ◽  
Syed Kunal Abidi ◽  
Jonathon R Campbell ◽  
Louis-Patrick Haraoui ◽  
...  

AbstractObjectiveTo determine the diagnostic accuracy of serological tests for coronavirus disease-2019 (covid-19).DesignSystematic review and meta-analysis.Data sourcesMedline, bioRxiv, and medRxiv from 1 January to 30 April 2020, using subject headings or subheadings combined with text words for the concepts of covid-19 and serological tests for covid-19.Eligibility criteria and data analysisEligible studies measured sensitivity or specificity, or both of a covid-19 serological test compared with a reference standard of viral culture or reverse transcriptase polymerase chain reaction. Studies were excluded with fewer than five participants or samples. Risk of bias was assessed using quality assessment of diagnostic accuracy studies 2 (QUADAS-2). Pooled sensitivity and specificity were estimated using random effects bivariate meta-analyses.Main outcome measuresThe primary outcome was overall sensitivity and specificity, stratified by method of serological testing (enzyme linked immunosorbent assays (ELISAs), lateral flow immunoassays (LFIAs), or chemiluminescent immunoassays (CLIAs)) and immunoglobulin class (IgG, IgM, or both). Secondary outcomes were stratum specific sensitivity and specificity within subgroups defined by study or participant characteristics, including time since symptom onset.Results5016 references were identified and 40 studies included. 49 risk of bias assessments were carried out (one for each population and method evaluated). High risk of patient selection bias was found in 98% (48/49) of assessments and high or unclear risk of bias from performance or interpretation of the serological test in 73% (36/49). Only 10% (4/40) of studies included outpatients. Only two studies evaluated tests at the point of care. For each method of testing, pooled sensitivity and specificity were not associated with the immunoglobulin class measured. The pooled sensitivity of ELISAs measuring IgG or IgM was 84.3% (95% confidence interval 75.6% to 90.9%), of LFIAs was 66.0% (49.3% to 79.3%), and of CLIAs was 97.8% (46.2% to 100%). In all analyses, pooled sensitivity was lower for LFIAs, the potential point-of-care method. Pooled specificities ranged from 96.6% to 99.7%. Of the samples used for estimating specificity, 83% (10 465/12 547) were from populations tested before the epidemic or not suspected of having covid-19. Among LFIAs, pooled sensitivity of commercial kits (65.0%, 49.0% to 78.2%) was lower than that of non-commercial tests (88.2%, 83.6% to 91.3%). Heterogeneity was seen in all analyses. Sensitivity was higher at least three weeks after symptom onset (ranging from 69.9% to 98.9%) compared with within the first week (from 13.4% to 50.3%).ConclusionHigher quality clinical studies assessing the diagnostic accuracy of serological tests for covid-19 are urgently needed. Currently, available evidence does not support the continued use of existing point-of-care serological tests.Study registrationPROSPERO CRD42020179452.


2016 ◽  
Vol 130 ◽  
pp. 18-32 ◽  
Author(s):  
Elise H. Tatone ◽  
Jessica L. Gordon ◽  
Jessie Hubbs ◽  
Stephen J. LeBlanc ◽  
Trevor J. DeVries ◽  
...  

2021 ◽  
Vol Publish Ahead of Print ◽  
Author(s):  
Edith Angel-Müller ◽  
Carlos Fernando Grillo-Ardila ◽  
Jairo Amaya-Guio ◽  
Nicolas Torres-Montañez

2018 ◽  
Vol 23 (1) ◽  
pp. 13-16 ◽  
Author(s):  
Thomas R Fanshawe ◽  
Michael Power ◽  
Sara Graziadio ◽  
José M Ordóñez-Mena ◽  
John Simpson ◽  
...  

Information about the performance of diagnostic tests is typically presented in the form of measures of test accuracy such as sensitivity and specificity. These measures may be difficult to translate directly into decisions about patient treatment, for which information presented in the form of probabilities of disease after a positive or a negative test result may be more useful. These probabilities depend on the prevalence of the disease, which is likely to vary between populations. This article aims to clarify the relationship between pre-test (prevalence) and post-test probabilities of disease, and presents two free, online interactive tools to illustrate this relationship. These tools allow probabilities of disease to be compared with decision thresholds above and below which different treatment decisions may be indicated. They are intended to help those involved in communicating information about diagnostic test performance and are likely to be of benefit when teaching these concepts. A substantive example is presented using C reactive protein as a diagnostic marker for bacterial infection in the older adult population. The tools may also be useful for manufacturers of clinical tests in planning product development, for authors of test evaluation studies to improve reporting and for users of test evaluations to facilitate interpretation and application of the results.


2013 ◽  
Vol 137 (4) ◽  
pp. 566-575 ◽  
Author(s):  
Robert L. Schmidt ◽  
Rachel E. Factor ◽  
Benjamin L. Witt ◽  
Lester J. Layfield

Context.—The quality of diagnostic accuracy studies is determined by 2 key factors: risk of bias and comparability. Bias can distort accuracy estimates and poor reporting impairs comparability. While diagnostic accuracy studies for fine-needle aspiration cytology (FNAC) are frequently published, the methodologic issues associated with this body of literature have never been reviewed. Objective.—To assess the quality of design and reporting of diagnostic test accuracy studies in FNAC. Data Sources.—Diagnostic accuracy studies were identified by a Medline (US National Library of Medicine) search. Sixty-four FNAC diagnostic test accuracy studies were randomly selected for structured review with the Quality Assessment of Diagnostic Accuracy Studies (QUADAS) survey. Studies were divided between 2 time periods: 2000-2001 and 2009-2011. Conclusions.—Diagnostic test accuracy studies of FNAC suffer from numerous deficiencies in study design, which negatively affect the reliability of accuracy estimates.


Sign in / Sign up

Export Citation Format

Share Document