scholarly journals Considerations for performance metrics of metagenomic next generation sequencing analyses

2020 ◽  
Author(s):  
Jason G. Kralj ◽  
Stephanie L. Servetas ◽  
Samuel P. Forry ◽  
Scott A. Jackson
Keyword(s):  
Performance Metrics ◽  
Limit Of Detection ◽  
Clinical Performance ◽  
Ground Truth ◽  
Negative Control ◽  
Fitness For Purpose ◽  
Performance Metric ◽  
The One ◽  
Sensitivity Specificity ◽  
Harmonic Means

AbstractEvaluating the performance of metagenomics analyses has proven a challenge, due in part to limited ground-truth standards, broad application space, and numerous evaluation methods and metrics. Application of traditional clinical performance metrics (i.e. sensitivity, specificity, etc.) using taxonomic classifiers do not fit the “one-bug-one-test” paradigm. Ultimately, users need methods that evaluate fitness-for-purpose and identify their analyses’ strengths and weaknesses. Within a defined cohort, reporting performance metrics by taxon, rather than by sample, will clarify this evaluation. An estimated limit of detection, positive and negative control samples, and true positive and negative true results are necessary criteria for all investigated taxa. Use of summary metrics should be restricted to comparing results of similar cohorts and data, and should employ harmonic means and continuous products for each performance metric rather than arithmetic mean. Such consideration will ensure meaningful comparisons and evaluation of fitness-for-purpose.

scholarly journals Considerations for Performance Metrics of Metagenomic Next Generation Sequencing Analyses

2021 ◽  
Author(s):  
Jason Kralj ◽  
Stephanie L. Servetas ◽  
Samuel P. Forry ◽  
Scott A. Jackson
Keyword(s):  
Performance Metrics ◽  
Clinical Performance ◽  
Ground Truth ◽  
Negative Control ◽  
Fitness For Purpose ◽  
Performance Metric ◽  
The One ◽  
Sensitivity Specificity ◽  
Harmonic Means ◽  
Generation Sequencing

Abstract Background: Evaluating the performance of metagenomics analyses has proven a challenge, due in part to limited ground-truth standards, broad application space, and numerous evaluation methods and metrics. Application of traditional clinical performance metrics (i.e. sensitivity, specificity, etc.) using taxonomic classifiers do not fit the “one-bug-one-test” paradigm. Ultimately, users need methods that evaluate fitness-for-purpose and identify their analyses’ strengths and weaknesses. Within a defined cohort, reporting performance metrics by taxon, rather than by sample, will clarify this evaluation.Results: For a complete assessment, estimated limits of detection, positive and negative control samples, and true positive and negative true results are necessary criteria for all investigated taxa. Use of summary metrics should be restricted to comparing results of similar, or ideally the same, cohorts and data, and should employ harmonic means and continuous products for each performance metric rather than arithmetic mean. Conclusions: Organism-centric analysis and reporting will enable clear performance assessment and meaningful comparisons between methods in evaluating fitness for purpose of metagenomic analyses with their intended applications.

scholarly journals One-Year Clinical Evaluation of the Bonding Effectiveness of a One-Step, Self-Etch Adhesive in Noncarious Cervical Lesion Therapy

2015 ◽  
Vol 2015 ◽  
pp. 1-5 ◽  
Author(s):  
Babacar Faye ◽  
Mouhamed Sarr ◽  
Khaly Bane ◽  
Adjaratou Wakha Aidara ◽  
Seydina Ousmane Niang ◽  
...  
Keyword(s):  
Clinical Performance ◽  
Retention Rate ◽  
Recall Rate ◽  
Acid Etching ◽  
Significant Difference ◽  
Usphs Criteria ◽  
One Step ◽  
One Year ◽  
The One ◽  
Self Etch

This study evaluated the one-year clinical performance of a one-step, self-etch adhesive (Optibond All-in-One, Kerr, CA, USA) combined with a composite (Herculite XRV Ultra, Kerr Hawe, CA, USA) to restore NCCLs with or without prior acid etching. Restorations performed by the same practitioner were evaluated at baseline and after 3, 6, and 12 months using modified USPHS criteria. At 6 months, the recall rate was 100%. The retention rate was 84.2% for restorations with prior acid etching, but statistically significant differences were observed between baseline and 6 months. Without acid etching, the retention rate was 77%, and no statistically significant difference was noted between 3 and 6 months. Marginal integrity (93.7% with and 87.7% without acid etching) and discoloration (95.3% with and 92.9% without acid etching) were scored as Alpha or Bravo, with better results after acid etching. After one year, the recall rate was 58.06%. Loss of pulp vitality, postoperative sensitivity, or secondary caries were not observed. After one year retention rate was of 90.6% and 76.9% with and without acid conditioning. Optibond All-in-One performs at a satisfactory clinical performance level for restoration of NCCLs after 12 months especially after acid etching.

scholarly journals Detection of Active BoNT/C and D by EndoPep-MS Using MALDI Biotyper Instrument and Comparison with the Mouse Test Bioassay

Toxins ◽  
2020 ◽  
Vol 13 (1) ◽  
pp. 10
Author(s):  
Ilenia Drigo ◽  
Elena Tonon ◽  
Simone Pascoletti ◽  
Fabrizio Anniballi ◽  
Suzanne R. Kalb ◽  
...  
Keyword(s):  
Limit Of Detection ◽  
Animal Health ◽  
Lethal Dose ◽  
Reference Method ◽  
Mouse Bioassay ◽  
Peptide Fragments ◽  
Mass Spectrometers ◽  
Detection And Identification ◽  
Maldi Biotyper ◽  
Sensitivity Specificity

Botulinum neurotoxins (BoNTs) are among the most poisonous known biological substances, and therefore the availability of reliable, easy-to use tools for BoNT detection are important goals for food safety and human and animal health. The reference method for toxin detection and identification is the mouse bioassay (MBA). An EndoPep-MS method for BoNT differentiation has been developed based on mass spectrometry. We have validated and implemented the EndoPep-MS method on a Bruker MALDI Biotyper for the detection of BoNT/C and D serotypes. The method was extensively validated using experimentally and naturally contaminated samples comparing the results with those obtained with the MBA. Overall, the limit of detection (LoD) for both C and D toxins were less than or equal to two mouse lethal dose 50 (mLD50) per 500 µL for all tested matrices with the exception of feces spiked with BoNT/C which showed signals not-related to specific peptide fragments. Diagnostic sensitivity, specificity and positive predictive value were 100% (95% CI: 87.66–100%), 96.08% (95% CI: 86.54–99.52%), and 93.33% (95% CI: 78.25–98.20%), respectively, and accuracy was 97.47% (95% CI: 91.15–99.69%). In conclusion, the tests carried out showed that the EndoPep-MS method, initially developed using more powerful mass spectrometers, can be applied to the Bruker MALDI Biotyper instrument with excellent results including for detection of the proteolytic activity of BoNT/C, BoNT/D, BoNT/CD, and BoNT/DC toxins.

Analytical assessment of ortho clinical diagnostics high-sensitivity cardiac troponin I assay

2020 ◽  
Vol 0 (0) ◽  
Author(s):  
Peter A. Kavsak ◽  
Tara Edge ◽  
Chantele Roy ◽  
Paul Malinowski ◽  
Karen Bamford ◽  
...  
Keyword(s):  
Cardiac Troponin ◽  
Troponin I ◽  
Limit Of Detection ◽  
Clinical Performance ◽  
Characteristic Curve ◽  
Area Under The Curve ◽  
High Sensitivity ◽  
Cardiac Troponin I ◽  
Clinical Diagnostics ◽  
Ortho Clinical Diagnostics

AbstractObjectivesTo analytically evaluate Ortho Clinical Diagnostics VITROS high-sensitivity cardiac troponin I (hs-cTnI) assay in specific matrices with comparison to other hs-cTn assays.MethodsThe limit of detection (LoD), imprecision, interference and stability testing for both serum and lithium heparin (Li-Hep) plasma for the VITROS hs-cTnI assay was determined. We performed Passing-Bablok regression analyses between sample types for the VITROS hs-cTnI assay and compared them to the Abbott ARCHITECT, Beckman Access and the Siemens ADVIA Centaur hs-cTnI assays. We also performed Receiver-operating characteristic curve analyses with the area under the curve (AUC) determined in an emergency department (ED)-study population (n=131) for myocardial infarction (MI).ResultsThe VITROS hs-cTnI LoD was 0.73 ng/L (serum) and 1.4 ng/L (Li-Hep). Stability up to five freeze-thaws was observed for the Ortho hs-cTnI assay, with the analyte stability at room temperature in serum superior to Li-Hep with gross hemolysis also affecting Li-Hep plasma hs-cTnI results. Comparison of Li-Hep to serum concentrations (n=202), yielded proportionally lower concentrations in plasma with the VITROS hs-cTnI assay (slope=0.85; 95% confidence interval [CI]:0.83–0.88). In serum, the VITROS hs-cTnI concentrations were proportionally lower compared to other hs-cTnI assays, with similar slopes observed between assays in samples frozen <−70 °C for 17 years (ED-study) or in 2020. In the ED-study, the VITROS hs-cTnI assay had an AUC of 0.974 (95%CI:0.929–0.994) for MI, similar to the AUCs of other hs-cTn assays.ConclusionsLack of standardization of hs-cTnI assays across manufacturers is evident. The VITROS hs-cTnI assay yields lower concentrations compared to other hs-cTnI assays. Important differences exist between Li-Hep plasma and serum, with evidence of stability and excellent clinical performance comparable to other hs-cTn assays.

Does ProCore fine needle biopsy really improve the clinical outcome of endoscopic ultrasound-guided sampling of pancreatic masses?

2021 ◽  
Author(s):  
Gabriele Delconte ◽  
Federica Cavalcoli ◽  
Andrea Magarotto ◽  
Giovanni Centonze ◽  
Cristina Bezzio ◽  
...  
Keyword(s):  
Adverse Events ◽  
Diagnostic Accuracy ◽  
Likelihood Ratio ◽  
Endoscopic Ultrasound ◽  
Needle Biopsy ◽  
Clinical Performance ◽  
Negative Likelihood Ratio ◽  
Fine Needle Biopsy ◽  
Fine Needle ◽  
Sensitivity Specificity

Introduction: Fine Needle Biopsy (FNB) has been suggested to provide better histological samples as compared to endoscopic ultrasound fine needle aspiration (EUS-FNA). However, studies comparing EUS-FNA and EUS-FNB for pancreatic lesions reported contrasting results. The aim of this study was to compare the clinical performance of EUS-FNA versus EUS-FNB with ProCore needle for the investigation of pancreatic lesions. Methods: We reviewed all patients undergoing EUS for the investigation of pancreatic lesions from August 2012 to September 2018. From August 2012 to January 2015 all procedures were performed with standard needles, whereas from February 2015 to September 2018 the use of ProCore needles had been introduced. Data on diagnostic accuracy, number of needle passes and/or adverse events were collected. Results: 324 patients were retrospectively evaluated: 190 (58.6%) underwent EUS-FNA and 134 (41.4%) EUS-FNB. Both EUS-FNA and EUS-FNB showed high diagnostic accuracy for malignancy [94% (CI 95%:89-97%) vs 94% (CI 95%:89-98%)]. Notable, there were no differences between EUS-FNA and EUS-FNB in terms of sensitivity, specificity, positive and negative likelihood ratio, histological core tissue retrieval, adverse events or number of needle passes. However, subgroup analysis noted a higher diagnostic accuracy for 25G EUS-FNB as compared to 25G EUS-FNA (85,7% vs 55,5%; *p=0.023). Conclusion: EUS-FNB with ProCore needle is safe and feasible in pancreatic lesions. ProCore needle did not provide any advantage in terms of diagnostic accuracy, sensitivity, specificity, positive and/or negative likelihood ratio, or acquisition of core specimen, therefore its routine application is not

scholarly journals The One-Year In Vivo Comparison of Lithium Disilicate and Zirconium Dioxide Inlays

Materials ◽  
2021 ◽  
Vol 14 (11) ◽  
pp. 3102
Author(s):  
Rini Behera ◽  
Lora Mishra ◽  
Darshan Devang Divakar ◽  
Abdulaziz A. Al-Kheraif ◽  
Naomi Ranjan Singh ◽  
...  
Keyword(s):  
Zirconium Dioxide ◽  
Clinical Performance ◽  
Lithium Disilicate ◽  
Study Groups ◽  
Resin Cement ◽  
Internal Surfaces ◽  
Colour Match ◽  
One Year ◽  
The One

The objective of the present study was to evaluate the one-year clinical performance of lithium disilicate (LD) and zirconium dioxide (ZrO2) class II inlay restorations. Thirty healthy individuals who met the inclusion criteria were enrolled for the study. The patients were randomly divided into two study groups (n = 15): LD (IPS e.max press) and ZrO2 (Dentcare Zirconia). In the ZrO2 group, the internal surfaces of the inlays were sandblasted and silanized with Monobond N (Ivoclar, Leichsteistein, Germany). In the LD group, the internal surfaces of the inlays were etched with 5% hydrofluoric acid. The ceramic inlays were cemented with self-cure resin cement (Multilink N). Clinical examinations were performed using modified United State Public Health Codes and Criteria (USPHS) after 2 weeks, 4 weeks, 6 months and 1 year. The one-year survival rate was evaluated. In total, one failure was observed in the ZrO2 group. The survival probability after 1 year for the ZrO2 inlays was 93%, and for the LD inlays was 100%, which was statistically insignificant. The differences between both groups for most USPHS criteria (except for colour match) were statistically insignificant. Within the imitations of the present study, the lithium disilicate- and zirconia dioxide-based inlays exhibited comparable clinical performances. However, the colour and translucency match was superior for the lithium disilicate restorations.

scholarly journals Value and Diagnostic Efficacy of Fetal Morphology Assessment Using Ultrasound in A Poor-Resource Setting

Diagnostics ◽  
2019 ◽  
Vol 9 (3) ◽  
pp. 109 ◽  
Author(s):  
Ukweh ◽  
Ugbem ◽  
Okeke ◽  
Ekpo
Keyword(s):  
Predictive Value ◽  
Performance Metrics ◽  
Area Under The Curve ◽  
Youden Index ◽  
Likelihood Ratios ◽  
Diagnostic Efficacy ◽  
Predictive Values ◽  
Resource Setting ◽  
Post Test ◽  
Sensitivity Specificity

Background: Ultrasound is operator-dependent, and its value and efficacy in fetal morphology assessment in a low-resource setting is poorly understood. We assessed the value and efficacy of fetal morphology ultrasound assessment in a Nigerian setting. Materials and Methods: We surveyed fetal morphology ultrasound performed across five facilities and followed-up each fetus to ascertain the outcome. Fetuses were surveyed in the second trimester (18th–22nd weeks) using the International Society of Ultrasound in Obstetrics and Gynecology (ISUOG) guideline. Clinical and surgical reports were used as references to assess the diagnostic efficacy of ultrasound in livebirths, and autopsy reports to confirm anomalies in terminated pregnancies, spontaneous abortions, intrauterine fetal deaths, and still births. We calculated sensitivity, specificity, positive and negative predictive values, Area under the curve (AUC), Youden index, likelihood ratios, and post-test probabilities. Results: In total, 6520 fetuses of women aged 15–46 years (mean = 31.7 years) were surveyed. The overall sensitivity, specificity, and AUC were 77.1 (95% CI: 68–84.6), 99.5 (95% CI: 99.3–99.7), and 88.3 (95% CI: 83.7–92.2), respectively. Other performance metrics were: positive predictive value, 72.4 (95% CI: 64.7–79.0), negative predictive value, 99.6 (95% CI: 99.5–99.7), and Youden index (77.1%). Abnormality prevalence was 1.67% (95% CI: 1.37–2.01), and the positive and negative likelihood ratios were 254 (95% CI: 107.7–221.4) and 0.23 (95% CI: 0.16–0.33), respectively. The post-test probability for positive test was 72% (95% CI: 65–79). Conclusion: Fetal morphology assessment is valuable in a poor economics setting, however, the variation in the diagnostic efficacy across facilities and the limitations associated with the detection of circulatory system anomalies need to be addressed.

scholarly journals UJI DAYA ANTIINFLAMASI EKSTRAK ETANOLIK KULIT TERONG BELANDA (Solanum betaceum, Cav.) PADA MENCIT JANTAN YANG DIINDUKSI KARAGENIN

2020 ◽  
Vol 16 (02) ◽  
pp. 86
Author(s):  
Margareta Retno Priamsari ◽  
Rina Ayu Krismonikawati
Keyword(s):  
Diclofenac Sodium ◽  
Ethanolic Extract ◽  
Positive Control ◽  
Negative Control ◽  
Test Material ◽  
Edema Formation ◽  
Flavonoid Compounds ◽  
The One ◽  
Post Hoc ◽  
Solanum Betaceum

ABSTRACT Inflammation is a normal response to injury due to release of chemical mediators such as prostaglandins. Solanum betaceum Cav. skin contains flavonoid compounds. This study aims to determine the antiinflammatory power and the effect of variations in the dose of ethanolic extract of the skin of Solanum betaceum Cav on Swiss male strain mice caragenin-induced. The ethanolic extract of Solanum betaceum Cav skin was obtained using remaseration method with 96% ethanol solvent. Antiinflammatory power test using edema formation method with induction of caragenin 1% intraplantar 30 minutes before administration of the test material. Tests were carried out on 25 mice records which were divided into 5 groups namely negative control (0.5% CMC Na suspension), positive control (diclofenac sodium suspension in 0.5% CMC Na dose 7 mg/kgBW) and given suspension respectively. Dutch eggplant skin ethanolic extract in CMC Na 0.5% dose 70 mg/kgBW; 140 mg/kgBW; 280 mg/kgBW orally. The parameters observed were edema thickness measured using calipers at 30 minute intervals for 360 minutes. The results of edema thickness difference are used to calculate the Area Under Curve (AUC) and the percentage of antiinflammatory power (% AIP). Dat a were analyzed statistically with the One Way Anova test and continued by the Post Hoc Test with a 95% confidence level. The results showed that the ethanolic extract of Solanum betaceum Cav skin contains flavonoid compounds and has antiinflammatory power. The greater the dose of ethanolic extract of Solanum betaceum Cav skin, the greater the anti-inflammatory power. Keywords: Solanum betaceum Cav., antinflammation, caragenin

scholarly journals Optimal Stiffness Design for an Exhaustive Parallel Compliance Matrix in Multiactuator Robotic Limbs

2018 ◽  
Vol 10 (3) ◽  
Author(s):  
Nathan M. Cahill ◽  
Thomas Sugar ◽  
Yi Ren ◽  
Kyle Schroeder
Keyword(s):  
Energy Efficient ◽  
Peak Power ◽  
Performance Metrics ◽  
Slow Growth ◽  
Legged Robots ◽  
Power Requirement ◽  
Compliance Matrix ◽  
Performance Metric ◽  
Stiffness Design ◽  
Optimal Stiffness

Comparatively slow growth in energy density of both power storage and generation technologies has placed added emphasis on the need for energy-efficient designs in legged robots. This paper explores the potential of parallel springs in robot limb design. We start by adding what we call the exhaustive parallel compliance matrix (EPCM) to the design. The EPCM is a set of parallel springs, which includes a parallel spring for each joint and a multijoint parallel spring for all possible combinations of the robot's joints. Then, we carefully formulate and compare two performance metrics, which improve various aspects of the system performance. Each performance metric is analyzed and compared, their strengths and weaknesses being rigorously presented. The performance benefits associated with this approach are dramatic. Implementing the spring matrix reduces the sum of square power (SSP) exerted by the actuators by up to 47%, the peak power requirement by almost 40%, the sum of squared current by 55%, and the peak current by 55%. These results were generated using a planar robot limb and a gait trajectory borrowed from biology. We use a fully dynamic model of the robotic system including inertial effects. We also test the design robustness using a perturbation study, which shows that the parallel springs are effective even in the presence of trajectory perturbation.

scholarly journals A Heuristic Ranking of Different Characteristic Mining Based Mathematical Formulae Retrieval Models

2019 ◽  
Vol 9 (1) ◽  
pp. 893-901
Keyword(s):  
Performance Metrics ◽  
Text File ◽  
Retrieval Models ◽  
Retrieval Time ◽  
Retrieval Algorithms ◽  
Ranking Model ◽  
Sensitivity Specificity

The significant difficulty in the present circumstances is how to classify the math related keywords from a given text file and group them in one math file. Through this article a heuristic ranking model was developed and was evaluated on different mathematical formulae retrieval algorithms based on Characteristic mining. Our proposed heuristic ranking model was developed using the performance metrics of exiting retrieval algorithms such as NMF clustering, Levenstein distance, Sequence matcher, Fuzzy-wuzzy and Tensorflow. Performance metrics such as sensitivity, specificity, efficiency, accuracy and retrieval time were used in developing our heuristic ranking model.

Sign in / Sign up

Export Citation Format

Share Document