scholarly journals Estimation of sensitivity and specificity and calculation of sample size for a validation study with stratified sampling

2020 ◽  
Author(s):  
Kiyoshi Kubota ◽  
Masao Iwagami ◽  
Takuhiro Yamaguchi
Keyword(s):  
Sample Size ◽  
Sensitivity And Specificity ◽  
Validation Study ◽  
Predictive Value ◽  
Validation Studies ◽  
Bootstrap Method ◽  
Stratified Sampling ◽  
Delta Method ◽  
Estimate Sample Size ◽  
The Bootstrap Method

Abstract Background:We propose and evaluate the approximation formulae for the 95% confidence intervals (CIs) of the sensitivity and specificity and a formula to estimate sample size in a validation study with stratified sampling where positive samples satisfying the outcome definition and negative samples that do not are selected with different extraction fractions. Methods:We used the delta method to derive the approximation formulae for estimating the sensitivity and specificity and their CIs. From those formulae, we derived the formula to estimate the size of negative samples required to achieve the intended precision and the formula to estimate the precision for a negative sample size arbitrarily selected by the investigator. We conducted simulation studies in a population where 4% were outcome definition positive, the positive predictive value (PPV)=0.8, and the negative predictive value (NPV)=0.96, 0.98 and 0.99. The size of negative samples, n0, was either selected to make the 95% CI fall within ± 0.1, 0.15 and 0.2 or set arbitrarily as 150, 300 and 600. We assumed a binomial distribution for the positive and negative samples. The coverage of the 95% CIs of the sensitivity and specificity was calculated as the proportion of CIs including the sensitivity and specificity in the population, respectively. For selected studies, the coverage was also estimated by the bootstrap method. The sample size was evaluated by examining whether the observed precision was within the pre-specified value.Results:For the sensitivity, the coverage of the approximated 95% CIs was larger than 0.95 in most studies but in 9 of 18 selected studies derived by the bootstrap method. For the specificity, the coverage of the approximated 95% CIs was approximately 0.93 in most studies, but the coverage was more than 0.95 in all 18 studies derived by the bootstrap method. The calculated size of negative samples yielded precisions within the pre-specified values in most of the studies.Conclusion:The approximation formulae for the 95% CIs of the sensitivity and specificity for stratified validation studies are presented. These formulae will help in conducting and analysing validation studies with stratified sampling.

scholarly journals Using Bootstrap Method to Evaluate the Power of Tests for Non-Linearity in Asymmetric Price Relationship

2013 ◽  
Vol 5 (4) ◽  
pp. 237-241
Author(s):  
Henry De-Graft Acquah
Keyword(s):  
Sample Size ◽  
Bootstrap Method ◽  
Complex Model ◽  
Model Complexity ◽  
Bootstrap Sample ◽  
Power Of The Test ◽  
Data Generating Process ◽  
Price Relationship ◽  
False Null Hypothesis ◽  
The Bootstrap Method

This paper introduces and applies the bootstrap method to compare the power of the test for asymmetry in the Granger and Lee (1989) and Von Cramon-Taubadel and Loy (1996) models. The results of the bootstrap simulations indicate that the power of the test for asymmetry depends on various conditions such as the bootstrap sample size, model complexity, difference in adjustment speeds and the amount of noise in the data generating process used in the application. The true model achieves greater power when compared with the complex model. With small bootstrap sample size or large noise, both models display low power in rejecting the (false) null hypothesis of symmetry.

scholarly journals Influence Function and Bootstrap Methods of Estimating the Standard Errors of the Estimators of Mixture Exponential Distribution Parameter

2021 ◽  
pp. 113-136
Author(s):  
J. I. Udobi ◽  
G. A. Osuji ◽  
S. I. Onyeagu ◽  
H. O. Obiora-Ilouno
Keyword(s):  
Sample Size ◽  
Standard Error ◽  
Influence Function ◽  
Bootstrap Method ◽  
Standard Errors ◽  
Sample Sizes ◽  
Trimmed Mean ◽  
Contamination Levels ◽  
M Estimate ◽  
The Bootstrap Method

This work estimated the standard error of the maximum likelihood estimator (MLE) and the robust estimators of the exponential mixture parameter (θ) using the influence function and the bootstrap approaches. Mixture exponential random samples of sizes 10, 15, 20, 25, 50, and 100 were generated using 3 mixture exponential models at 2%, 5% and 10% contamination levels. The selected estimators namely: mean, median, alpha-trimmed mean, Huber M-estimate and their standard errors (Tn ) were estimated using the two approaches at the indicated sample sizes and contamination levels. The results were compared using the coefficient of variation, confidence interval and the asymptotic relative efficiency of Tn in order to find out which approach yields the more reliable, precise and efficient estimate of Tn. The results of the analysis show that the two approaches do not equally perform at all conditions. From the results, the bootstrap method was found to be more reliable and efficient method of estimating the standard error of the arithmetic mean at all sample sizes and contamination levels. In estimating the standard error of the median, the influence function method was found to be more effective especially when the sample size is small and yet contamination is high. The influence function based approach yielded more reliable, precise and efficient estimates of the standard errors of the alpha-trimmed mean and the Huber M-estimate for all sample sizes and levels of contamination although the reliability of the bootstrap method improved better as sample size increased to 50 and above. All simulations and analysis were carried out in R programming language.

scholarly journals Effect of plot size and plant spatial arrangement on the efficiency of family selection in sugarcane

2021 ◽  
Vol 53 (2) ◽  
pp. 1-10
Author(s):  
Aparecido De Moraes ◽  
Matheus Henrique Silveira Mendes ◽  
Mauro Sérgio de Oliveira Leite ◽  
Regis De Castro Carvalho ◽  
Flávia Maria Avelar Gonçalves
Keyword(s):  
Sample Size ◽  
Bootstrap Method ◽  
Spatial Arrangement ◽  
Family Selection ◽  
Plot Size ◽  
Stalk Diameter ◽  
New Varieties ◽  
Selection Of ◽  
The Bootstrap Method ◽  
The Ideal

The purpose of this study was to identify the ideal sample size representing a family in its potential, to identify superior families and, in parallel, determine in which spatial arrangement they may have a better accuracy in the selection of new varieties of sugarcane. For such purpose, five families of full-sibs were evaluated, each with 360 individuals, in the randomized blocks design, with three replications in three different spacing among plants in the row (50 cm, 75 cm, and 100 cm) and 150 cm between the rows. To determine the ideal sample size, as well as the better spacing for evaluation, the bootstrap method was adopted. It was observed that 100 cm spacings provided the best average for the stalk numbers, stalk diameter and for estimated weight of stalks in the stool. The spacing of 75 cm between the plants allowed a better power of discrimination among the families for all characters evaluated. At this 75 cm spacing  was also possible to identify superior families with a sample of 30 plants each plot and 3 reps in the trial. Highlights The bootstrap method was efficient to determine the ideal sample size, as well as the best spacing for evaluation. The 75-cm spacing had the highest power of discrimination among families, indicating that this spacing is the most efficient in evaluating sugarcane families for selection purposes. From all the results and considering selective accuracy as the guiding parameter for decision making, the highest values obtained considering the number of stalks and weight of stalks in the stools were found at the 75-cm spacing.

Calculation of VaR with the Bootstrap Method Based on Experts Predict in Mechanical Engineering

2013 ◽  
Vol 681 ◽  
pp. 60-64
Author(s):  
Xia Shi
Keyword(s):  
Risk Management ◽  
Empirical Research ◽  
Predictive Value ◽  
Bootstrap Method ◽  
Composite Index ◽  
Financial Data ◽  
Short Term ◽  
Volatility Clustering ◽  
Auto Correlation ◽  
The Bootstrap Method

Based on GARCH model to catch the financial data of auto correlation and volatility clustering, while the use of expert predictive value and the short term newer data using the Bootstrap method for Vary estimation. Through the SSE Composite Index of empirical research, the results show that this method can avoid the old data invalid information, at the same time the financial experts forecast was introduced risk management, improve the accuracy of the computation

Asymptotic Standard Errors of Generalized Partial Credit Model True Score Equating Using Characteristic Curve Methods

2021 ◽  
pp. 014662162110131
Author(s):  
Zhonghua Zhang
Keyword(s):  
Bootstrap Method ◽  
Characteristic Curve ◽  
Delta Method ◽  
Standard Errors ◽  
Partial Credit Model ◽  
Generalized Partial Credit Model ◽  
True Score Equating ◽  
Generalized Partial Credit ◽  
Multiple Imputation Method ◽  
The Bootstrap Method

In this study, the delta method was applied to estimate the standard errors of the true score equating when using the characteristic curve methods with the generalized partial credit model in test equating under the context of the common-item nonequivalent groups equating design. Simulation studies were further conducted to compare the performance of the delta method with that of the bootstrap method and the multiple imputation method. The results indicated that the standard errors produced by the delta method were very close to the criterion empirical standard errors as well as those yielded by the bootstrap method and the multiple imputation method under all the manipulated conditions.

scholarly journals Comparison of the Bootstrap and Delta Method Variances of the Variance Estimator of the Bernoulli Distribution

2018 ◽  
pp. 1-10
Author(s):  
Ying- Ying Zhang ◽  
Teng- Zhong Rong ◽  
Man- Man Li
Keyword(s):  
Bootstrap Method ◽  
Delta Method ◽  
Variance Estimator ◽  
Bernoulli Distribution ◽  
Bernoulli Trials ◽  
The Absolute ◽  
Variance Estimates ◽  
Parameter Values ◽  
The Bootstrap Method

It is interesting to calculate the variance of the variance estimator of the Bernoulli distribution. Therefore, we compare the Bootstrap and Delta Method variances of the variance estimator of the Bernoulli distribution in this paper. Firstly, we provide the correct Bootstrap, Delta Method, and true variances of the variance estimator of the Bernoulli distribution for three parameter values in Table 2.1. Secondly, we obtain the estimates of the variance of the variance estimator of the Bernoulli distribution by the Delta Method (analytically), the true method (analytically), and the Bootstrap Method (algorithmically). Thirdly, we compare the Bootstrap and Delta Methodsin terms of the variance estimates, the errors, and the absolute errors in three gures for 101 parameter values in [0, 1], with the purpose to explain the di erences between the Bootstrap and Delta Methods. Finally, we give three examples of the Bernoulli trials to illustrate the three methods.

scholarly journals Inferring sensitivity and specificity of phenotyping algorithms using positive and negative predictive value in validation study in observational health data

2018 ◽  
Vol 3 (4) ◽  
Author(s):  
Mingkai Peng ◽  
Rosa Gini ◽  
Tyler Williamson
Keyword(s):  
Sensitivity And Specificity ◽  
Validation Study ◽  
Validation Studies ◽  
Indirect Method ◽  
Direct Method ◽  
Population Sample ◽  
Simulated Data ◽  
Health Data ◽  
Comparable Accuracy ◽  
Low Prevalence

IntroductionIn observational health data, phenotyping algorithms are needed to process raw information into clinically relevant features. Validation studies traditionally estimate sensitivity and specificity by comparing the phenotyping algorithm with a reference standard on a population sample. There are challenges to conduct validation studies for conditions with low prevalence. Objectives and ApproachWe propose a novel and efficient method for conducting validation studies to indirectly estimate the sensitivity and specificity. We simulated datasets with different levels of disease prevalence and phenotyping algorithms with different sensitivities and specificity. We applied both the traditional (direct) and new (indirect) method on simulated data to estimate the sensitivity and specificity and compare the performance of the two methods. We also designed a gate to exclude true negatives to improve study efficiency on conditions with low prevalence and sensitive analysis was conducted on the imperfect gate. ResultsThe new (indirect) method provided better or comparable accuracy in estimating both sensitivity and specificity compared to the traditional (direct) method. Applying a gate enabled us to conduct validation study in conditions with very low prevalence. An imperfect gate results in the overestimation of sensitivity but has minimal effect on specificity. Conclusion/ImplicationsThe new (indirect) method provides an alternative way to conduct validation studies in observational health data with improvement in estimating accuracy.

An investigation of new toxicity test method performance in validation studies: 3. sensitivity and specificity are not independent of prevalence or distribution of toxicity

2002 ◽  
Vol 21 (6) ◽  
pp. 325-334 ◽  
Author(s):  
L H Bruner ◽  
G J Carr ◽  
J W Harbell ◽  
R D Curren
Keyword(s):  
Positive Predictive Value ◽  
Sensitivity And Specificity ◽  
Test Performance ◽  
Predictive Value ◽  
Validation Studies ◽  
Toxicity Test ◽  
Test Method ◽  
Toxic Substances ◽  
Method Performance ◽  
Sensitivity Specificity

Often, the only measures of toxicity test performance provided in validation studies are the contingent probability statistics (CPS) sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). Sensitivity and specificity are generally used in preference to NPV and PPV since NPV and PPV are assumed to vary with changes in prevalence while sensitivity and specificity are assumed to be independent of changes in prevalence. The purpose of the studies reported here was to test whether or not sensitivity and specificity are actually independent of changes in prevalence. Results derived from these studies indicate that sensitivity and specificity vary significantly depending on the prevalence of toxic substances in the set of chemicals being tested. This means sensitivity and specificity should not always be considered constant indicators of toxicity test performance.

scholarly journals Applying the triads method in the validation of dietary intake using biomarkers

2010 ◽  
Vol 26 (11) ◽  
pp. 2027-2037 ◽  
Author(s):  
Renata Tiene de Carvalho Yokota ◽  
Edina Shizue Miyazaki ◽  
Marina Kiyomi Ito
Keyword(s):  
Dietary Intake ◽  
Confidence Intervals ◽  
Food Frequency Questionnaire ◽  
Validation Studies ◽  
Measurement Errors ◽  
Bootstrap Method ◽  
Reference Method ◽  
Food Frequency ◽  
Heywood Case ◽  
The Bootstrap Method

The triads method is applied in validation studies of dietary intake to evaluate the correlation between three measurements (food frequency questionnaire, reference method and biomarker) and the true intake using validity coefficients (Á). The main advantage of this technique is the inclusion of the biomarker, which presents independent errors compared with those of the traditional methods. The method assumes the linearity between the three measurements and the true intake and independence between the three measurement errors. Limitations of this technique include the occurrence of Á > 1, known as "Heywood case", and the existence of negative correlations, which do not allow the calculation of Á. The objective of this review is to present the concept of the method, describe its application and examine the validation studies of dietary intake that use the triads method. We also conceptualize the "bootstrap" method, used to estimate the confidence intervals of the validity coefficients.

scholarly journals A Weighted Residual Bootstrap Method for Multilevel Modeling with Sampling Weights

2021 ◽  
Author(s):  
Wen Luo ◽  
Hok Chio Lai
Keyword(s):  
Sample Size ◽  
Multilevel Modeling ◽  
Bootstrap Method ◽  
Intraclass Correlation ◽  
R Package ◽  
Sampling Weights ◽  
Multistage Sampling ◽  
The Impact ◽  
Level 2 ◽  
The Bootstrap Method

Multilevel modeling is often used to analyze survey data collected with a multistage sampling design. When the selection is informative, sampling weights need to be incorporated in the estimation. We propose a weighted residual bootstrap method as an alternative to the multilevel pseudo-maximum likelihood (MPML) estimators. In a Monte Carlo simulation using two-level linear mixed effects models, the bootstrap method showed advantages over MPML for the estimates and the statistical inferences of the intercept, the slope of the level-2 predictor, and the variance components at level-2. The impact of sample size, selection mechanism, intraclass correlation (ICC), and distributional assumptions on the performance of the methods were examined. The performance of MPML was suboptimal when sample size and ICC were small and when the normality assumption was violated. The bootstrap estimates performed generally well across all the simulation conditions, but had notably suboptimal performance in estimating the covariance component in a random slopes model when sample size and ICCs were large. As an illustration, the bootstrap method is applied to the American data of the OECD’s Program for International Students Assessment (PISA) survey on math achievement using the R package bootmlm.

Sign in / Sign up

Export Citation Format

Share Document