scholarly journals Combining Analytical Approaches and Multiple Sources of Information to Improve Interpretation of Diagnostic Test Results for Tuberculosis in Wild Meerkats

Animals ◽  
2021 ◽  
Vol 11 (12) ◽  
pp. 3453
Author(s):  
Stuart J. Patterson ◽  
Charlene Clarke ◽  
Tim H. Clutton-Brock ◽  
Michele A. Miller ◽  
Sven D. C. Parsons ◽  
...  
Keyword(s):  
Test Performance ◽  
Diagnostic Tests ◽  
Credible Interval ◽  
Small Sample ◽  
Test Results ◽  
Sources Of Information ◽  
Sample Sizes ◽  
Multiple Sources ◽  
Suricata Suricatta ◽  
Analytical Approaches

Diagnostic tests are used to classify individual animals’ infection statuses. However, validating test performance in wild animals without gold standard tests is extremely challenging, and the issue is further complicated in chronic conditions where measured immune parameters vary over time. Here, we demonstrate the value of combining evidence from different diagnostic approaches to aid interpretation in the absence of gold standards, large sample sizes, and controlled environments. Over a two-year period, we sampled 268 free-living meerkats (Suricata suricatta) longitudinally for Mycobacterium suricattae (a causative agent of tuberculosis), using three ante-mortem diagnostic tests based on mycobacterial culture, and antigen-specific humoral and cell-mediated immune responses, interpreting results both independently and in combination. Post-mortem cultures confirmed M. suricattae infection in 22 animals, which had prior ante-mortem information, 59% (13/22) of which were test-positive on a parallel test interpretation (PTI) of the three ante-mortem diagnostic assays (95% confidence interval: 37–79%). A similar ability to detect infection, 65.7% (95% credible interval: 42.7–84.7%), was estimated using a Bayesian approach to examine PTI. Strong evidence was found for a near doubling of the hazard of death (Hazard Ratio 1.75, CI: 1.14–2.67, p = 0.01), associated with a positive PTI result, thus demonstrating that these test results are related to disease outcomes. For individual tests, small sample sizes led to wide confidence intervals, but replication of conclusions, using different methods, increased our confidence in these results. This study demonstrates that combining multiple methodologies to evaluate diagnostic tests in free-ranging wildlife populations can be a useful approach for exploiting such valuable datasets.

Methods to evaluate test results based on small sample sizes

2019 ◽  
Vol 1 (2) ◽  
pp. 74-84
Author(s):  
Steffen Marx ◽  
Jürgen Grünberg ◽  
Gregor Schacht
Keyword(s):  
Small Sample ◽  
Test Results ◽  
Sample Sizes ◽  
Small Sample Sizes

scholarly journals Predicting protein model correctness in Coot using machine learning

2020 ◽  
Vol 76 (8) ◽  
pp. 713-723 ◽  
Author(s):  
Paul S. Bond ◽  
Keith S. Wilson ◽  
Kevin D. Cowtan
Keyword(s):  
Neural Networks ◽  
Test Performance ◽  
Model Building ◽  
Main Chain ◽  
Side Chains ◽  
Sources Of Information ◽  
Multiple Sources ◽  
Protein Model ◽  
Density Values ◽  
Automated Model Building

Manually identifying and correcting errors in protein models can be a slow process, but improvements in validation tools and automated model-building software can contribute to reducing this burden. This article presents a new correctness score that is produced by combining multiple sources of information using a neural network. The residues in 639 automatically built models were marked as correct or incorrect by comparing them with the coordinates deposited in the PDB. A number of features were also calculated for each residue using Coot, including map-to-model correlation, density values, B factors, clashes, Ramachandran scores, rotamer scores and resolution. Two neural networks were created using these features as inputs: one to predict the correctness of main-chain atoms and the other for side chains. The 639 structures were split into 511 that were used to train the neural networks and 128 that were used to test performance. The predicted correctness scores could correctly categorize 92.3% of the main-chain atoms and 87.6% of the side chains. A Coot ML Correctness script was written to display the scores in a graphical user interface as well as for the automatic pruning of chains, residues and side chains with low scores. The automatic pruning function was added to the CCP4i2 Buccaneer automated model-building pipeline, leading to significant improvements, especially for high-resolution structures.

A comparison of confidence/credible interval methods for the area under the ROC curve for continuous diagnostic tests with small sample size

2015 ◽  
Vol 26 (6) ◽  
pp. 2603-2621 ◽  
Author(s):  
Dai Feng ◽  
Giuliana Cortese ◽  
Richard Baumgartner
Keyword(s):  
Sample Size ◽  
Roc Curve ◽  
Diagnostic Tests ◽  
Small Sample Size ◽  
Real Life ◽  
Credible Interval ◽  
Small Sample ◽  
Interval Methods ◽  
Unequal Variances ◽  
Non Parametric

The receiver operating characteristic (ROC) curve is frequently used as a measure of accuracy of continuous markers in diagnostic tests. The area under the ROC curve (AUC) is arguably the most widely used summary index for the ROC curve. Although the small sample size scenario is common in medical tests, a comprehensive study of small sample size properties of various methods for the construction of the confidence/credible interval (CI) for the AUC has been by and large missing in the literature. In this paper, we describe and compare 29 non-parametric and parametric methods for the construction of the CI for the AUC when the number of available observations is small. The methods considered include not only those that have been widely adopted, but also those that have been less frequently mentioned or, to our knowledge, never applied to the AUC context. To compare different methods, we carried out a simulation study with data generated from binormal models with equal and unequal variances and from exponential models with various parameters and with equal and unequal small sample sizes. We found that the larger the true AUC value and the smaller the sample size, the larger the discrepancy among the results of different approaches. When the model is correctly specified, the parametric approaches tend to outperform the non-parametric ones. Moreover, in the non-parametric domain, we found that a method based on the Mann–Whitney statistic is in general superior to the others. We further elucidate potential issues and provide possible solutions to along with general guidance on the CI construction for the AUC when the sample size is small. Finally, we illustrate the utility of different methods through real life examples.

Assessing diagnostic tests once an optimal cutoff point has been selected.

1986 ◽  
Vol 32 (7) ◽  
pp. 1341-1346 ◽  
Author(s):  
K Linnet ◽  
E Brandt
Keyword(s):  
Test Performance ◽  
Diagnostic Tests ◽  
Statistical Tests ◽  
Cutoff Point ◽  
Standard Errors ◽  
Sample Sizes ◽  
Power Functions ◽  
Optimal Cutoff ◽  
Specificity And Sensitivity ◽  
Optimal Cutoff Point

Abstract The specificity and sensitivity of a quantitative diagnostic test depends on the chosen cutoff point. The common practice of selecting a cutoff point that maximizes the specificity plus the sensitivity, as judged from the observed test results, is studied here by simulation. Test performance is on average assessed too optimistically by this procedure--a phenomenon of importance when sample sizes are small. For example, the average positive bias is up to 15% of the test performance for sample sizes of 25. Furthermore, binomial calculated standard errors of specificity and sensitivity estimates are incorrect. A Monte Carlo statistical method--the "bootstrap procedure"--is applied to correct for bias and to estimate standard errors, including the standard error of the optimal cutoff point. Independent and paired comparisons of two diagnostic tests are also considered when optimal cutoff points have been selected. For this purpose, binomial statistical tests behave satisfactorily. Examples of power functions are presented.

No Evidence that Experiencing Physical Warmth Promotes Interpersonal Warmth: Two Failures to Replicate Williams and Bargh (2008)

2018 ◽  
Author(s):  
Christopher Chabris ◽  
Patrick Ryan Heck ◽  
Jaclyn Mandart ◽  
Daniel Jacob Benjamin ◽  
Daniel J. Simons
Keyword(s):  
Null Hypothesis ◽  
Small Sample ◽  
Sample Sizes ◽  
Double Blind ◽  
Bayesian Analyses ◽  
Physical Warmth ◽  
Small Sample Sizes ◽  
Interpersonal Warmth

Williams and Bargh (2008) reported that holding a hot cup of coffee caused participants to judge a person’s personality as warmer, and that holding a therapeutic heat pad caused participants to choose rewards for other people rather than for themselves. These experiments featured large effects (r = .28 and .31), small sample sizes (41 and 53 participants), and barely statistically significant results. We attempted to replicate both experiments in field settings with more than triple the sample sizes (128 and 177) and double-blind procedures, but found near-zero effects (r = –.03 and .02). In both cases, Bayesian analyses suggest there is substantially more evidence for the null hypothesis of no effect than for the original physical warmth priming hypothesis.

Supporting the Development of Clinical Reasoning of Preprofessional Novices in Dysphagia Management

2019 ◽  
Vol 40 (03) ◽  
pp. 151-161 ◽  
Author(s):  
Sebastian Doeltgen ◽  
Stacie Attrill ◽  
Joanne Murray
Keyword(s):  
Clinical Reasoning ◽  
Clinical Decision Making ◽  
Management Practice ◽  
Clinical Decision ◽  
Assessment Process ◽  
Evidence Based ◽  
Sources Of Information ◽  
Multiple Sources ◽  
Implicit Processing ◽  
Reasoning Skills

AbstractProficient clinical reasoning is a critical skill in high-quality, evidence-based management of swallowing impairment (dysphagia). Clinical reasoning in this area of practice is a cognitively complex process, as it requires synthesis of multiple sources of information that are generated during a thorough, evidence-based assessment process and which are moderated by the patient's individual situations, including their social and demographic circumstances, comorbidities, or other health concerns. A growing body of health and medical literature demonstrates that clinical reasoning skills develop with increasing exposure to clinical cases and that the approaches to clinical reasoning differ between novices and experts. It appears that it is not the amount of knowledge held, but the way it is used, that distinguishes a novice from an experienced clinician. In this article, we review the roles of explicit and implicit processing as well as illness scripts in clinical decision making across the continuum of medical expertise and discuss how they relate to the clinical management of swallowing impairment. We also reflect on how this literature may inform educational curricula that support SLP students in developing preclinical reasoning skills that facilitate their transition to early clinical practice. Specifically, we discuss the role of case-based curricula to assist students to develop a meta-cognitive awareness of the different approaches to clinical reasoning, their own capabilities and preferences, and how and when to apply these in dysphagia management practice.

scholarly journals Diagnosis of tuberculosis in wildlife: a systematic review

2021 ◽  
Vol 52 (1) ◽  
Author(s):  
Jobin Thomas ◽  
Ana Balseiro ◽  
Christian Gortázar ◽  
María A. Risalde
Keyword(s):  
Systematic Review ◽  
Test Performance ◽  
Diagnostic Tests ◽  
Large Scale ◽  
Diagnostic Techniques ◽  
Review Literature ◽  
Meat Industry ◽  
Post Mortem ◽  
Serological Tests ◽  
Ongoing Research

AbstractAnimal tuberculosis (TB) is a multi-host disease caused by members of the Mycobacterium tuberculosis complex (MTC). Due to its impact on economy, sanitary standards of milk and meat industry, public health and conservation, TB control is an actively ongoing research subject. Several wildlife species are involved in the maintenance and transmission of TB, so that new approaches to wildlife TB diagnosis have gained relevance in recent years. Diagnosis is a paramount step for screening, epidemiological investigation, as well as for ensuring the success of control strategies such as vaccination trials. This is the first review that systematically addresses data available for the diagnosis of TB in wildlife following the Preferred Reporting Items of Systematic Reviews and Meta-Analyses (PRISMA) guidelines. The article also gives an overview of the factors related to host, environment, sampling, and diagnostic techniques which can affect test performance. After three screenings, 124 articles were considered for systematic review. Literature indicates that post-mortem examination and culture are useful methods for disease surveillance, but immunological diagnostic tests based on cellular and humoral immune response detection are gaining importance in wildlife TB diagnosis. Among them, serological tests are especially useful in wildlife because they are relatively inexpensive and easy to perform, facilitate large-scale surveillance and can be used both ante- and post-mortem. Currently available studies assessed test performance mostly in cervids, European badgers, wild suids and wild bovids. Research to improve diagnostic tests for wildlife TB diagnosis is still needed in order to reach accurate, rapid and cost-effective diagnostic techniques adequate to a broad range of target species and consistent over space and time to allow proper disease monitoring.

scholarly journals High heterogeneity undermines generalization of differential expression results in RNA-Seq analysis

2021 ◽  
Vol 15 (1) ◽  
Author(s):  
Weitong Cui ◽  
Huaru Xue ◽  
Lei Wei ◽  
Jinghua Jin ◽  
Xuewen Tian ◽  
...  
Keyword(s):  
Gene Expression ◽  
Differential Expression ◽  
Small Sample ◽  
Differentially Expressed ◽  
Cancer Type ◽  
Rna Seq ◽  
Sample Sizes ◽  
Large Sample ◽  
Expression Levels ◽  
Gene Expression Levels

Abstract Background RNA sequencing (RNA-Seq) has been widely applied in oncology for monitoring transcriptome changes. However, the emerging problem that high variation of gene expression levels caused by tumor heterogeneity may affect the reproducibility of differential expression (DE) results has rarely been studied. Here, we investigated the reproducibility of DE results for any given number of biological replicates between 3 and 24 and explored why a great many differentially expressed genes (DEGs) were not reproducible. Results Our findings demonstrate that poor reproducibility of DE results exists not only for small sample sizes, but also for relatively large sample sizes. Quite a few of the DEGs detected are specific to the samples in use, rather than genuinely differentially expressed under different conditions. Poor reproducibility of DE results is mainly caused by high variation of gene expression levels for the same gene in different samples. Even though biological variation may account for much of the high variation of gene expression levels, the effect of outlier count data also needs to be treated seriously, as outlier data severely interfere with DE analysis. Conclusions High heterogeneity exists not only in tumor tissue samples of each cancer type studied, but also in normal samples. High heterogeneity leads to poor reproducibility of DEGs, undermining generalization of differential expression results. Therefore, it is necessary to use large sample sizes (at least 10 if possible) in RNA-Seq experimental designs to reduce the impact of biological variability and DE results should be interpreted cautiously unless soundly validated.

Sign in / Sign up

Export Citation Format

Share Document