scholarly journals Performance Comparison of Different Analytic Methods in Proficiency Testing for Mutations in the BRAF, EGFR, and KRAS Genes: A Study of the College of American Pathologists Molecular Oncology Committee

2019 ◽  
Vol 143 (10) ◽  
pp. 1203-1211 ◽  
Author(s):  
Joel T. Moncur ◽  
Angela N. Bartley ◽  
Julia A. Bridge ◽  
Suzanne Kamel-Reid ◽  
Alexander J. Lazar ◽  
...  
Keyword(s):  
Next Generation Sequencing ◽  
Proficiency Testing ◽  
Critical Issue ◽  
Performance Comparison ◽  
Test Method ◽  
Next Generation ◽  
Molecular Oncology ◽  
Assay Methods ◽  
Commercial Kits ◽  
Generation Sequencing

Context.— The performance of laboratory testing has recently come under increased scrutiny as part of important and ongoing debates on regulation and reimbursement. To address this critical issue, this study compares the performance of assay methods, using either commercial kits or assays designed and implemented by single laboratories (“home brews”), including next-generation sequencing methods, on proficiency testing provided by the College of American Pathologists Molecular Oncology Committee. Objective.— To compare the performance of different assay methods on College of American Pathologists proficiency testing for variant analysis of 3 common oncology analytes: BRAF, EGFR, and KRAS. Design.— There were 6897 total responses across 35 different proficiency testing samples interrogating 13 different variants as well as wild-type sequences for BRAF, EGFR, and KRAS. Performance was analyzed by test method, kit manufacturer, variants tested, and preanalytic and postanalytic practices. Results.— Of 26 reported commercial kits, 23 achieved greater than 95% accuracy. Laboratory-developed tests with no kit specified demonstrated 96.8% or greater accuracy across all 3 analytes (1123 [96.8%] acceptable of 1160 total responses for BRAF; 848 [97.5%] acceptable of 870 total responses for EGFR; 942 [97.0%] acceptable of 971 total responses for KRAS). Next-generation sequencing platforms (summed across all analytes and 2 platforms) demonstrated 99.4% accuracy for these analytes (165 [99.4%] acceptable of 166 total next-generation sequencing responses). Slight differences in performance were noted among select commercial assays, dependent upon the particular design and specificity of the assay. Wide differences were noted in the lower limits of neoplastic cellularity laboratories accepted for testing. Conclusions.— These data demonstrate the high degree of accuracy and comparable performance across all laboratories, regardless of methodology. However, care must be taken in understanding the diagnostic specificity and reported analytic sensitivity of individual methods.

scholarly journals Next-Generation Sequencing (NGS) Methods Show Superior or Equivalent Performance to Non-NGS Methods on BRAF, EGFR, and KRAS Proficiency Testing Samples

2019 ◽  
Vol 143 (8) ◽  
pp. 980-984 ◽  
Author(s):  
Lea F. Surrey ◽  
Fredrick D. Oakley ◽  
Jason D. Merker ◽  
Thomas A. Long ◽  
Patricia Vasalos ◽  
...  
Keyword(s):  
Next Generation Sequencing ◽  
Proficiency Testing ◽  
Rapid Expansion ◽  
Next Generation ◽  
Limited Data ◽  
Comparative Performance ◽  
Panel Surveys ◽  
Molecular Oncology ◽  
Next Generation Sequencing Ngs ◽  
Generation Sequencing

Context.— There has been a rapid expansion of next-generation sequencing (NGS)–based assays for the detection of somatic variants in solid tumors. However, limited data are available regarding the comparative performance of NGS and non-NGS assays using standardized samples across a large number of laboratories. Objective.— To compare the performance of NGS and non-NGS assays using well-characterized proficiency testing samples provided by the College of American Pathologists (CAP) Molecular Oncology Committee. A secondary goal was to compare the use of preanalytic and postanalytic practices. Design.— A total of 17 343 responses were obtained from participants in the BRAF, EGFR, KRAS, and the Multigene Tumor Panel surveys across 84 different proficiency testing samples interrogating 16 variants and 3 wild-type sequences. Performance and preanalytic/postanalytic practices were analyzed by method. Results.— While both NGS and non-NGS achieved an acceptable response rate of greater than 95%, the overall performance of NGS methods was significantly better than that of non-NGS methods for the identification of variants in BRAF (overall 97.8% versus 95.6% acceptable responses, P = .001) and EGFR (overall 98.5% versus 97.3%, P = .01) and was similar for KRAS (overall 98.8% and 97.6%, P = .10). There were specific variant differences, but in all discrepant cases, NGS methods outperformed non-NGS methods. NGS laboratories also more consistently used preanalytic and postanalytic practices suggested by the CAP checklist requirements than non-NGS laboratories. Conclusions.— The overall analytic performance of both methods was excellent. For specific BRAF and EGFR variants, NGS outperformed non-NGS methods and NGS laboratories report superior adherence to suggested laboratory practices.

scholarly journals A Window Into Clinical Next-Generation Sequencing–Based Oncology Testing Practices

2017 ◽  
Vol 141 (12) ◽  
pp. 1679-1685 ◽  
Author(s):  
Rakesh Nagarajan ◽  
Angela N. Bartley ◽  
Julia A. Bridge ◽  
Lawrence J. Jennings ◽  
Suzanne Kamel-Reid ◽  
...  
Keyword(s):  
Next Generation Sequencing ◽  
Proficiency Testing ◽  
Clinical Laboratory ◽  
Precision Oncology ◽  
Next Generation ◽  
Single Nucleotide Variants ◽  
Molecular Oncology ◽  
Survey Results ◽  
Testing Practices ◽  
Generation Sequencing

Context.— Detection of acquired variants in cancer is a paradigm of precision medicine, yet little has been reported about clinical laboratory practices across a broad range of laboratories. Objective.— To use College of American Pathologists proficiency testing survey results to report on the results from surveys on next-generation sequencing–based oncology testing practices. Design.— College of American Pathologists proficiency testing survey results from more than 250 laboratories currently performing molecular oncology testing were used to determine laboratory trends in next-generation sequencing–based oncology testing. Results.— These presented data provide key information about the number of laboratories that currently offer or are planning to offer next-generation sequencing–based oncology testing. Furthermore, we present data from 60 laboratories performing next-generation sequencing–based oncology testing regarding specimen requirements and assay characteristics. The findings indicate that most laboratories are performing tumor-only targeted sequencing to detect single-nucleotide variants and small insertions and deletions, using desktop sequencers and predesigned commercial kits. Despite these trends, a diversity of approaches to testing exists. Conclusions.— This information should be useful to further inform a variety of topics, including national discussions involving clinical laboratory quality systems, regulation and oversight of next-generation sequencing–based oncology testing, and precision oncology efforts in a data-driven manner.

scholarly journals A Model Study of In Silico Proficiency Testing for Clinical Next-Generation Sequencing

2016 ◽  
Vol 140 (10) ◽  
pp. 1085-1091 ◽  
Author(s):  
Eric J. Duncavage ◽  
Haley J. Abel ◽  
Jason D. Merker ◽  
John B. Bodner ◽  
Qin Zhao ◽  
...  
Keyword(s):  
Next Generation Sequencing ◽  
Proficiency Testing ◽  
In Silico ◽  
Absolute Difference ◽  
Ion Torrent ◽  
Next Generation ◽  
Single Nucleotide Variants ◽  
Single Nucleotide ◽  
Clinical Laboratories ◽  
Generation Sequencing

Context.—Most current proficiency testing challenges for next-generation sequencing assays are methods-based proficiency testing surveys that use DNA from characterized reference samples to test both the wet-bench and bioinformatics/dry-bench aspects of the tests. Methods-based proficiency testing surveys are limited by the number and types of mutations that either are naturally present or can be introduced into a single DNA sample. Objective.—To address these limitations by exploring a model of in silico proficiency testing in which sequence data from a single well-characterized specimen are manipulated electronically. Design.—DNA from the College of American Pathologists reference genome was enriched using the Illumina TruSeq and Life Technologies AmpliSeq panels and sequenced on the MiSeq and Ion Torrent platforms, respectively. The resulting data were mutagenized in silico and 26 variants, including single-nucleotide variants, deletions, and dinucleotide substitutions, were added at variant allele fractions (VAFs) from 10% to 50%. Participating clinical laboratories downloaded these files and analyzed them using their clinical bioinformatics pipelines. Results.—Laboratories using the AmpliSeq/Ion Torrent and/or the TruSeq/MiSeq participated in the 2 surveys. On average, laboratories identified 24.6 of 26 variants (95%) overall and 21.4 of 22 variants (97%) with VAFs greater than 15%. No false-positive calls were reported. The most frequently missed variants were single-nucleotide variants with VAFs less than 15%. Across both challenges, reported VAF concordance was excellent, with less than 1% median absolute difference between the simulated VAF and mean reported VAF. Conclusions.—The results indicate that in silico proficiency testing is a feasible approach for methods-based proficiency testing, and demonstrate that the sensitivity and specificity of current next-generation sequencing bioinformatics across clinical laboratories are high.

scholarly journals Potentially Actionable Targets: Evidence Standards for Credible Next Generation Sequencing Technology Assessment Claims

2017 ◽  
Vol 8 (3) ◽  
pp. 9
Author(s):  
Paul C Langley
Keyword(s):  
Next Generation Sequencing ◽  
Technology Assessment ◽  
Critical Issue ◽  
Evidence Base ◽  
Cost Of Care ◽  
Next Generation ◽  
Next Generation Sequencing Technology ◽  
Next Generation Sequencing Ngs ◽  
Generation Sequencing

Despite considerable resources devoted to developing databases to support competitive credible claims for next generation sequencing (NGS) claims, we have yet to meet the standards required in health technology assessment to support such claims. The purpose of this commentary is to consider options open in establishing claims for NGS recommendations. Although NGS platforms offer potential promise in improving clinical outcomes, supporting cost-effectiveness and reducing the overall cost of care in target populations, this has yet to be demonstrated on a scale that is likely to satisfy reimbursers and health care decision makers. Issues addressed include (i) the importance of credible, evaluable and replicable claims from individual NGS platforms; (ii) the difficulties in moving beyond broad-brush claims for improved survival; (iii) the standards required for an NGS evidence base; (iv) protocol designs in establishing the independent contribution of NGS actionable therapy recommendations to outcomes claims; (v) the role of NGS registries; and (vi) protocols to support ongoing credible, evaluable and replicable claims in target patient populations. The critical issue is not analytical and clinical validity but clinical utility. This has yet to be demonstrated.   Type: Commentary

scholarly journals Proficiency Testing of Standardized Samples Shows Very High Interlaboratory Agreement for Clinical Next-Generation Sequencing–Based Oncology Assays

2018 ◽  
Vol 143 (4) ◽  
pp. 463-471 ◽  
Author(s):  
Jason D. Merker ◽  
Kelly Devereaux ◽  
A. John Iafrate ◽  
Suzanne Kamel-Reid ◽  
Annette S. Kim ◽  
...  
Keyword(s):  
Next Generation Sequencing ◽  
Proficiency Testing ◽  
False Negative ◽  
High Specificity ◽  
Variant Allele ◽  
Next Generation ◽  
Single Nucleotide Variants ◽  
Sequencing Platform ◽  
Negative Results ◽  
Generation Sequencing

Context.— Next-generation sequencing–based assays are being increasingly used in the clinical setting for the detection of somatic variants in solid tumors, but limited data are available regarding the interlaboratory performance of these assays. Objective.— To examine proficiency testing data from the initial College of American Pathologists (CAP) Next-Generation Sequencing Solid Tumor survey to report on laboratory performance. Design.— CAP proficiency testing results from 111 laboratories were analyzed for accuracy and associated assay performance characteristics. Results.— The overall accuracy observed for all variants was 98.3%. Rare false-negative results could not be attributed to sequencing platform, selection method, or other assay characteristics. The median and average of the variant allele fractions reported by the laboratories were within 10% of those orthogonally determined by digital polymerase chain reaction for each variant. The median coverage reported at the variant sites ranged from 1922 to 3297. Conclusions.— Laboratories demonstrated an overall accuracy of greater than 98% with high specificity when examining 10 clinically relevant somatic single-nucleotide variants with a variant allele fraction of 15% or greater. These initial data suggest excellent performance, but further ongoing studies are needed to evaluate the performance of lower variant allele fractions and additional variant types.

scholarly journals Proficiency Testing of Standardized Samples Shows High Interlaboratory Agreement for Clinical Next-Generation Sequencing–Based Hematologic Malignancy Assays With Survey Material–Specific Differences in Variant Frequencies

2020 ◽  
Vol 144 (8) ◽  
pp. 959-966 ◽  
Author(s):  
Alissa Keegan ◽  
Julia A. Bridge ◽  
Neal I. Lindeman ◽  
Thomas A. Long ◽  
Jason D. Merker ◽  
...  
Keyword(s):  
Next Generation Sequencing ◽  
Proficiency Testing ◽  
Hematologic Malignancy ◽  
False Negative ◽  
False Negative Rate ◽  
Hematologic Malignancies ◽  
Variant Allele ◽  
Next Generation ◽  
Single Nucleotide Variants ◽  
Generation Sequencing

Context.— As laboratories increasingly turn from single-analyte testing in hematologic malignancies to next-generation sequencing–based panel testing, there is a corresponding need for proficiency testing to ensure adequate performance of these next-generation sequencing assays for optimal patient care. Objective.— To report the performance of laboratories on proficiency testing from the first 4 College of American Pathologists Next-Generation Sequencing Hematologic Malignancy surveys. Design.— College of American Pathologists proficiency testing results for 36 different engineered variants and/or allele fractions as well as a sample with no pathogenic variants were analyzed for accuracy and associated assay performance characteristics. Results.— The overall sensitivity observed for all variants was 93.5% (2190 of 2341) with 99.8% specificity (22 800 of 22 840). The false-negative rate was 6.5% (151 of 2341), and the largest single cause of these errors was difficulty in identifying variants in the sequence of CEBPA that is rich in cytosines and guanines. False-positive results (0.18%; 40 of 22 840) were most likely the result of preanalytic or postanalytic errors. Interestingly, the variant allele fractions were almost uniformly lower than the engineered fraction (as measured by digital polymerase chain reaction). Extensive troubleshooting identified a multifactorial cause for the low variant allele fractions, a result of an interaction between the linearized nature of the plasmid and the Illumina TruSeq chemistry. Conclusions.— Laboratories demonstrated an overall accuracy of 99.2% (24 990 of 25 181) with 99.8% specificity and 93.5% sensitivity when examining 36 clinically relevant somatic single-nucleotide variants with a variant allele fraction of 10% or greater. The data also highlight an issue with artificial linearized plasmids as survey material for next-generation sequencing.

Sign in / Sign up

Export Citation Format

Share Document