scholarly journals Comparison of Bayesian and frequentist methods for prevalence estimation under misclassification

2020 ◽  
Author(s):  
Matthias Flor ◽  
Michael Weiβ ◽  
Thomas Selhorst ◽  
Christine Müller-Graf ◽  
Matthias Greiner
Keyword(s):  
Confidence Interval ◽  
Confidence Intervals ◽  
Bayesian Method ◽  
Estimation Method ◽  
Simulated Data ◽  
Credible Interval ◽  
Interval Length ◽  
Point Estimate ◽  
Data Sets ◽  
Prevalence Estimation

Abstract Background: Various methods exist for statistical inference about a prevalence that consider misclassifications due to an imperfect diagnostic test. However, traditional methods are known to suffer from truncation of the prevalence estimate and the confidence intervals constructed around the point estimate, as well as from under-performance of the confidence intervals' coverage. Methods : In this study, we used simulated data sets to validate a Bayesian prevalence estimation method and compare its performance to frequentist methods, i.e. the Rogan-Gladen estimate for prevalence, RGE, in combination with several methods of confidence interval construction. Our performance measures are (i) error distribution of the point estimate against the simulated true prevalence and (ii) coverage and length of the confidence interval, or credible interval in the case of the Bayesian method.Results: Across all data sets, the Bayesian point estimate and the RGE produced similar error distributions with slight advanteges of the former over the latter. In addition, the Bayesian estimate did not suffer from the RGE's truncation problem at zero or unity. With respect to coverage performance of the confidence and credible intervals, all of the traditional frequentist methods exhibited strong under-coverage, whereas the Bayesian credible interval as well as a newly developed frequentist method by Lang and Reiczigel performed as desired, with the Bayesian method having a very slight advantage in terms of interval length. Conclusion: The Bayesian prevalence estimation method should be prefered over traditional frequentist methods. An acceptable alternative is to combine the Rogan-Gladen point estimate with the Lang-Reiczigel confidence interval.

scholarly journals Comparison of Bayesian and frequentist methods for prevalence estimation under misclassification

2020 ◽  
Author(s):  
Matthias Flor ◽  
Michael Weiβ ◽  
Thomas Selhorst ◽  
Christine Müller-Graf ◽  
Matthias Greiner
Keyword(s):  
Confidence Interval ◽  
Confidence Intervals ◽  
Bayesian Method ◽  
Estimation Method ◽  
Simulated Data ◽  
Credible Interval ◽  
Interval Length ◽  
Point Estimate ◽  
Data Sets ◽  
Prevalence Estimation

Abstract Background: Various methods exist for statistical inference about a prevalence that consider misclassifications due to an imperfect diagnostic test. However, traditional methods are known to suffer from truncation of the prevalence estimate and the confidence intervals constructed around the point estimate, as well as from under-performance of the confidence intervals' coverage. Methods: In this study, we used simulated data sets to validate a Bayesian prevalence estimation method and compare its performance to frequentist methods, i.e. the Rogan-Gladen estimate for prevalence, RGE, in combination with several methods of confidence interval construction. Our performance measures are (i) error distribution of the point estimate against the simulated true prevalence and (ii) coverage and length of the confidence interval, or credible interval in the case of the Bayesian method. Results: Across all data sets, the Bayesian point estimate and the RGE produced similar error distributions with slight advanteges of the former over the latter. In addition, the Bayesian estimate did not suffer from the RGE's truncation problem at zero or unity. With respect to coverage performance of the confidence and credible intervals, all of the traditional frequentist methods exhibited strong under-coverage, whereas the Bayesian credible interval as well as a newly developed frequentist method by Lang and Reiczigel performed as desired, with the Bayesian method having a very slight advantage in terms of interval length. Conclusion: The Bayesian prevalence estimation method should be prefered over traditional frequentist methods. An acceptable alternative is to combine the Rogan-Gladen point estimate with the Lang-Reiczigel confidence interval.

scholarly journals Validation of a generic Bayesian method for prevalence estimation under misclassification

2020 ◽  
Author(s):  
Matthias Flor ◽  
Michael Weiβ ◽  
Thomas Selhorst ◽  
Christine Müller-Graf ◽  
Matthias Greiner
Keyword(s):  
Confidence Interval ◽  
Confidence Intervals ◽  
Bayesian Method ◽  
Estimation Method ◽  
Simulated Data ◽  
Credible Interval ◽  
Interval Length ◽  
Point Estimate ◽  
Data Sets ◽  
Prevalence Estimation

Abstract Background: Various methods exist for statistical inference about a prevalence that consider misclassifications due to an imperfect diagnostic test. However, traditional methods are known to suffer from censoring of the prevalence estimate and the confidence intervals constructed around the point estimate, as well as from under-performance of the confidence intervals' coverage. Methods: In this study, we used simulated data sets to validate a Bayesian prevalence estimation method and compare its performance to frequentist methods, i.e. the Rogan-Gladen estimate for prevalence, RGE, in combination with several methods of confidence interval construction. Our performance measures are (i) bias of the point estimate against the simulated true prevalence and (ii) coverage and length of the confidence interval, or credible interval in the case of the Bayesian method. Results: Across all data sets, the Bayesian point estimate and the RGE produced similar bias distributions with slight advanteges of the former over the latter. In addition, the Bayesian estimate did not suffer from the RGE's censoring problem at zero or unity. With respect to coverage performance of the confidence and credible intervals, all of the traditional frequentist methods exhibited strong under-coverage, whereas the Bayesian credible interval as well as a newly developed frequentist method by Lang and Reiczigel performed as desired, with the Bayesian method having a very slight advantage in terms of interval length. Conclusion: The Bayesian prevalence estimation method should be prefered over traditional frequentist methods. An acceptable alternative is to combine the Rogan-Gladen point estimate with the Lang-Reiczigel confidence interval.

Credible Intervals for Precision and Recall Based on a K-Fold Cross-Validated Beta Distribution

2016 ◽  
Vol 28 (8) ◽  
pp. 1694-1722 ◽  
Author(s):  
Yu Wang ◽  
Jihong Li
Keyword(s):  
Confidence Intervals ◽  
Cross Validation ◽  
Real Data ◽  
Credible Interval ◽  
Interval Length ◽  
Data Sets ◽  
Credible Intervals ◽  
Degree Of Confidence ◽  
T Distribution ◽  
Fold Cross Validation

In typical machine learning applications such as information retrieval, precision and recall are two commonly used measures for assessing an algorithm's performance. Symmetrical confidence intervals based on K-fold cross-validated t distributions are widely used for the inference of precision and recall measures. As we confirmed through simulated experiments, however, these confidence intervals often exhibit lower degrees of confidence, which may easily lead to liberal inference results. Thus, it is crucial to construct faithful confidence (credible) intervals for precision and recall with a high degree of confidence and a short interval length. In this study, we propose two posterior credible intervals for precision and recall based on K-fold cross-validated beta distributions. The first credible interval for precision (or recall) is constructed based on the beta posterior distribution inferred by all K data sets corresponding to K confusion matrices from a K-fold cross-validation. Second, considering that each data set corresponding to a confusion matrix from a K-fold cross-validation can be used to infer a beta posterior distribution of precision (or recall), the second proposed credible interval for precision (or recall) is constructed based on the average of K beta posterior distributions. Experimental results on simulated and real data sets demonstrate that the first credible interval proposed in this study almost always resulted in degrees of confidence greater than 95%. With an acceptable degree of confidence, both of our two proposed credible intervals have shorter interval lengths than those based on a corrected K-fold cross-validated t distribution. Meanwhile, the average ranks of these two credible intervals are superior to that of the confidence interval based on a K-fold cross-validated t distribution for the degree of confidence and are superior to that of the confidence interval based on a corrected K-fold cross-validated t distribution for the interval length in all 27 cases of simulated and real data experiments. However, the confidence intervals based on the K-fold and corrected K-fold cross-validated t distributions are in the two extremes. Thus, when focusing on the reliability of the inference for precision and recall, the proposed methods are preferable, especially for the first credible interval.

scholarly journals Benchmarking Statistical Multiple Sequence Alignment

2018 ◽  
Author(s):  
Michael Nute ◽  
Ehsan Saleh ◽  
Tandy Warnow
Keyword(s):  
Sequence Alignment ◽  
Multiple Sequence Alignment ◽  
Structural Alignment ◽  
Estimation Method ◽  
Simulated Data ◽  
Protein Sequences ◽  
Data Sets ◽  
Sequence Alignments ◽  
Multiple Sequence ◽  
Simulated Data Sets

AbstractThe estimation of multiple sequence alignments of protein sequences is a basic step in many bioinformatics pipelines, including protein structure prediction, protein family identification, and phylogeny estimation. Statistical co-estimation of alignments and trees under stochastic models of sequence evolution has long been considered the most rigorous technique for estimating alignments and trees, but little is known about the accuracy of such methods on biological benchmarks. We report the results of an extensive study evaluating the most popular protein alignment methods as well as the statistical co-estimation method BAli-Phy on 1192 protein data sets from established benchmarks as well as on 120 simulated data sets. Our study (which used more than 230 CPU years for the BAli-Phy analyses alone) shows that BAli-Phy is dramatically more accurate than the other alignment methods on the simulated data sets, but is among the least accurate on the biological benchmarks. There are several potential causes for this discordance, including model misspecification, errors in the reference alignments, and conflicts between structural alignment and evolutionary alignments; future research is needed to understand the most likely explanation for our observations. multiple sequence alignment, BAli-Phy, protein sequences, structural alignment, homology

scholarly journals A Generalized Modification of the Kumaraswamy Distribution for Modeling and Analyzing Real-Life Data

2020 ◽  
Vol 8 (2) ◽  
pp. 521-548
Author(s):  
Rafid Alshkaki
Keyword(s):  
Real Life ◽  
Estimation Method ◽  
Simulated Data ◽  
Likelihood Estimation ◽  
Data Sets ◽  
Kumaraswamy Distribution ◽  
Life Data ◽  
Real Life Data ◽  
Beta Power ◽  
Special Cases

In this paper, a generalized modification of the Kumaraswamy distribution is proposed, and its distributional and characterizing properties are studied. This distribution is closed under scaling and exponentiation, and has some well-known distributions as special cases, such as the generalized uniform, triangular, beta, power function, Minimax, and some other Kumaraswamy related distributions. Moment generating function, Lorenz and Bonferroni curves, with its moments consisting of the mean, variance, moments about the origin, harmonic, incomplete, probability weighted, L, and trimmed L moments, are derived. The maximum likelihood estimation method is used for estimating its parameters and applied to six different simulated data sets of this distribution, in order to check the performance of the estimation method through the estimated parameters mean squares errors computed from the different simulated sample sizes. Finally, four real-life data sets are used to illustrate the usefulness and the flexibility of this distribution in application to real-life data.  

scholarly journals The Reliability of Multidimensional Scales: A Comparison of Confidence Intervals and a Bayesian Alternative

2021 ◽  
Author(s):  
Julius M. Pfadt ◽  
Don van den Bergh ◽  
morten moshagen
Keyword(s):  
Confidence Interval ◽  
Confidence Intervals ◽  
Standard Error ◽  
Structural Equation ◽  
Factor Model ◽  
Credible Interval ◽  
Equation Modeling ◽  
Uncertainty Estimates ◽  
Wide Range ◽  
Credible Intervals

The reliability of a multidimensional test instrument is commonly estimated using coefficients ωt (total) and ωh (hierarchical) based on a factor model approach. However, point estimates for the coefficients are rarely accompanied by uncertainty estimates. In this study we compare several methods to obtain confidence intervals for the two coefficients: bootstrap and normal-theory intervals. In addition, we adapt methodology from Bayesian structural equation modeling to develop Bayesian versions of coefficients ωt and ωh by sampling from a second-order factor model. Results from a comprehensive simulation study show that the bootstrap standard error confidence interval, the bootstrap standard error log-transformed confidence interval, the Wald confidence interval, and the Bayesian credible interval perform well across a wide range of conditions. This study provides researchers with more information about the ωt and ωh confidence intervals they wish to report in their research. Moreover, the study introduces ωt and ωh credible intervals that are easy to use and come with all the benefits of Bayesian parameter estimation.

scholarly journals Inferência bootstrap para intervalo de confiança de modelo de regressão quadrática

2020 ◽  
Vol 42 ◽  
pp. e56
Author(s):  
Nicásio Gouveia ◽  
Ana Lúcia Souza Silva Mateus ◽  
Augusto Maciel da Silva ◽  
Leandro Ferreira ◽  
Suelen Carpenedo Aimi
Keyword(s):  
Confidence Interval ◽  
Regression Model ◽  
Critical Point ◽  
Confidence Intervals ◽  
Regression Models ◽  
Parametric Bootstrap ◽  
Height Growth ◽  
Point Estimate ◽  
Growth Data ◽  
Bootstrap Methodology

This study was carried out with the purpose of proposing a construction of confidence intervals for the critical point of a second degree regression model using a parametric bootstrap methodology. To obtain the distribution of the critical point, height growth data of the plants were used. From the analysis, the theoretical variables for the error and the confidence intervals were constructed. In addition, we examined different variance expressions with the purpose of the bootstrap-t confidence interval. The point estimate of the critical point was 10.7423 g L-1 of fertilizer doses without growth of C. canjerana plants. It was verified that the confidence intervals that considered the expression of the variance with the covariance between the regression models, present more satisfactory results, that is, results with more precision.

A Monte Carlo Study of Confidence Interval Methods for Generalizability Coefficient

2021 ◽  
pp. 001316442110338
Author(s):  
Zhehan Jiang ◽  
Mark Raymond ◽  
Christine DiStefano ◽  
Dexin Shi ◽  
Ren Liu ◽  
...  
Keyword(s):  
Confidence Interval ◽  
Confidence Intervals ◽  
Generalizability Theory ◽  
Monte Carlo Study ◽  
Small Sample ◽  
Interval Methods ◽  
Point Estimate ◽  
Linear Mixed Effect Model ◽  
Mixed Effect ◽  
Measurement Design

Computing confidence intervals around generalizability coefficients has long been a challenging task in generalizability theory. This is a serious practical problem because generalizability coefficients are often computed from designs where some facets have small sample sizes, and researchers have little guide regarding the trustworthiness of the coefficients. As generalizability theory can be framed to a linear mixed-effect model (LMM), bootstrap and simulation techniques from LMM paradigm can be used to construct the confidence intervals. The purpose of this research is to examine four different LMM-based methods for computing the confidence intervals that have been proposed and to determine their accuracy under six simulated conditions based on the type of test scores (normal, dichotomous, and polytomous data) and data measurement design ( p× i× r and p× [ i:r]). A bootstrap technique called “parametric methods with spherical random effects” consistently produced more accurate confidence intervals than the three other LMM-based methods. Furthermore, the selected technique was compared with model-based approach to investigate the performance at the levels of variance components via the second simulation study, where the numbers of examines, raters, and items were varied. We conclude with the recommendation generalizability coefficients, the confidence interval should accompany the point estimate.

scholarly journals Debiasing FracMinHash and deriving confidence intervals for mutation rates across a wide range of evolutionary distances

2022 ◽  
Author(s):  
Mahmudur Rahman Hera ◽  
N Tessa Pierce-Ward ◽  
David Koslicki
Keyword(s):  
Confidence Interval ◽  
Confidence Intervals ◽  
Theoretical Perspective ◽  
Mutation Rates ◽  
Data Sets ◽  
Broad Application ◽  
Mutation Model ◽  
A Genome ◽  
Wide Range ◽  
Analyze Data

Sketching methods offer computational biologists scalable techniques to analyze data sets that continue to grow in size. MinHash is one such technique that has enjoyed recent broad application. However, traditional MinHash has previously been shown to perform poorly when applied to sets of very dissimilar sizes. FracMinHash was recently introduced as a modification of MinHash to compensate for this lack of performance when set sizes differ. While experimental evidence has been encouraging, FracMinHash has not yet been analyzed from a theoretical perspective. In this paper, we perform such an analysis and prove that while FracMinHash is not unbiased, this bias is easily corrected. Next, we detail how a simple mutation model interacts with FracMinHash and are able to derive confidence intervals for evolutionary mutation distances between pairs of sequences as well as hypothesis tests for FracMinHash. We find that FracMinHash estimates the containment of a genome in a large metagenome more accurately and more precisely when compared to traditional MinHash, and the confidence interval performs significantly better in estimating mutation distances. A python-based implementation of the theorems we derive is freely available at https://github.com/KoslickiLab/mutation-rate-ci-calculator. The results presented in this paper can be reproduced using the code at https://github.com/KoslickiLab/ScaledMinHash-reproducibles.

scholarly journals Assessing the Accuracy of Approximate Confidence Intervals Proposed for the Mean of Poisson Distribution

2020 ◽  
Vol 18 (1) ◽  
pp. 2-13
Author(s):  
Alireza Shirvani ◽  
Malek Fathizadeh
Keyword(s):  
Confidence Interval ◽  
Confidence Intervals ◽  
Poisson Distribution ◽  
Count Data ◽  
Coverage Probability ◽  
Discrete Distribution ◽  
Interval Length ◽  
Average Confidence ◽  
The Mean ◽  
Approximate Confidence Intervals

The Poisson distribution is applied as an appropriate standard model to analyze count data. Because this distribution is known as a discrete distribution, representation of accurate confidence intervals for its distribution mean is extremely difficult. Approximate confidence intervals were presented for the Poisson distribution mean. The purpose of this study is to simultaneously compare several confidence intervals presented, according to the average coverage probability and accurate confidence coefficient and the average confidence interval length criteria.

Sign in / Sign up

Export Citation Format

Share Document