scholarly journals Randomized quantile residuals for diagnosing zero-inflated generalized linear mixed models with applications to microbiome count data

2021 ◽  
Vol 22 (1) ◽  
Author(s):  
Wei Bai ◽  
Mei Dong ◽  
Longhai Li ◽  
Cindy Feng ◽  
Wei Xu
Keyword(s):  
Count Data ◽  
Large Scale ◽  
Linear Models ◽  
Type I Error ◽  
Error Rates ◽  
Type I ◽  
Simulation Studies ◽  
Differential Abundance ◽  
Nominal Level ◽  
Differential Abundance Analysis

Abstract Background For differential abundance analysis, zero-inflated generalized linear models, typically zero-inflated NB models, have been increasingly used to model microbiome and other sequencing count data. A common assumption in estimating the false discovery rate is that the p values are uniformly distributed under the null hypothesis, which demands that the postulated model fit the count data adequately. Mis-specification of the distribution of the count data may lead to excess false discoveries. Therefore, model checking is critical to control the FDR at a nominal level in differential abundance analysis. Increasing studies show that the method of randomized quantile residual (RQR) performs well in diagnosing count regression models. However, the performance of RQR in diagnosing zero-inflated GLMMs for sequencing count data has not been extensively investigated in the literature. Results We conduct large-scale simulation studies to investigate the performance of the RQRs for zero-inflated GLMMs. The simulation studies show that the type I error rates of the GOF tests with RQRs are very close to the nominal level; in addition, the scatter-plots and Q–Q plots of RQRs are useful in discerning the good and bad models. We also apply the RQRs to diagnose six GLMMs to a real microbiome dataset. The results show that the OTU counts at the genus level of this dataset (after a truncation treatment) can be modelled well by zero-inflated and zero-modified NB models. Conclusion RQR is an excellent tool for diagnosing GLMMs for zero-inflated count data, particularly the sequencing count data arising in microbiome studies. In the supplementary materials, we provided two generic R functions, called and , for calculating the RQRs given fitting outputs of the R package .

scholarly journals On the Conditional and Unconditional Type I Error Rates and Power of Tests in Linear Models with Heteroscedastic Errors

2019 ◽  
Vol 17 (2) ◽  
Author(s):  
Patrick J. Rosopa ◽  
Alice M. Brawley ◽  
Theresa P. Atkinson ◽  
Stephen A. Robertson
Keyword(s):  
Linear Models ◽  
Type I Error ◽  
Weighted Least Squares ◽  
Error Rates ◽  
Type I ◽  
Least Squares Regression ◽  
Type I Error Rates ◽  
Power Of Tests ◽  
Wide Range ◽  
Heteroscedastic Errors

Preliminary tests for homoscedasticity may be unnecessary in general linear models. Based on Monte Carlo simulations, results suggest that when testing for differences between independent slopes, the unconditional use of weighted least squares regression and HC4 regression performed the best across a wide range of conditions.

Directional-sum test for nonparametric Behrens-Fisher problem with applications to the dietary intervention trial

2021 ◽  
pp. 096228022110028
Author(s):  
Zhen Meng ◽  
Qinglong Yang ◽  
Qizhai Li ◽  
Baoxue Zhang
Keyword(s):  
Dietary Intervention ◽  
Type I Error ◽  
Statistical Significance ◽  
Error Rates ◽  
Type I ◽  
Simulation Studies ◽  
Intervention Trial ◽  
Combination Strategy ◽  
Type I Error Rates ◽  
Dietary Intervention Trial

For a nonparametric Behrens-Fisher problem, a directional-sum test is proposed based on division-combination strategy. A one-layer wild bootstrap procedure is given to calculate its statistical significance. We conduct simulation studies with data generated from lognormal, t and Laplace distributions to show that the proposed test can control the type I error rates properly and is more powerful than the existing rank-sum and maximum-type tests under most of the considered scenarios. Applications to the dietary intervention trial further show the performance of the proposed test.

scholarly journals Statistically efficient association analysis of quantitative traits with haplotypes and untyped SNPs in family studies

BMC Genetics ◽  
2020 ◽  
Vol 21 (1) ◽  
Author(s):  
Guoqing Diao ◽  
Dan-yu Lin
Keyword(s):  
Association Analysis ◽  
Quantitative Traits ◽  
Type I Error ◽  
Asthma Management ◽  
Management Program ◽  
Error Rates ◽  
Family Data ◽  
Type I ◽  
Simulation Studies ◽  
Extensive Simulation

Abstract Background Associations between haplotypes and quantitative traits provide valuable information about the genetic basis of complex human diseases. Haplotypes also provide an effective way to deal with untyped SNPs. Two major challenges arise in haplotype-based association analysis of family data. First, haplotypes may not be inferred with certainty from genotype data. Second, the trait values within a family tend to be correlated because of common genetic and environmental factors. Results To address these challenges, we present an efficient likelihood-based approach to analyzing associations of quantitative traits with haplotypes or untyped SNPs. This approach properly accounts for within-family trait correlations and can handle general pedigrees with arbitrary patterns of missing genotypes. We characterize the genetic effects on the quantitative trait by a linear regression model with random effects and develop efficient likelihood-based inference procedures. Extensive simulation studies are conducted to examine the performance of the proposed methods. An application to family data from the Childhood Asthma Management Program Ancillary Genetic Study is provided. A computer program is freely available. Conclusions Results from extensive simulation studies show that the proposed methods for testing the haplotype effects on quantitative traits have correct type I error rates and are more powerful than some existing methods.

scholarly journals Moving beyond the classic difference-in-differences model: a simulation study comparing statistical methods for estimating effectiveness of state-level policies

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Beth Ann Griffin ◽  
Megan S. Schuler ◽  
Elizabeth A. Stuart ◽  
Stephen Patrick ◽  
Elizabeth McNeer ◽  
...  
Keyword(s):  
Null Hypothesis ◽  
Linear Models ◽  
Type I Error ◽  
Mean Squared Error ◽  
State Level ◽  
Error Rates ◽  
Type I ◽  
Directional Bias ◽  
Squared Error ◽  
Ar Models

Abstract Background Reliable evaluations of state-level policies are essential for identifying effective policies and informing policymakers’ decisions. State-level policy evaluations commonly use a difference-in-differences (DID) study design; yet within this framework, statistical model specification varies notably across studies. More guidance is needed about which set of statistical models perform best when estimating how state-level policies affect outcomes. Methods Motivated by applied state-level opioid policy evaluations, we implemented an extensive simulation study to compare the statistical performance of multiple variations of the two-way fixed effect models traditionally used for DID under a range of simulation conditions. We also explored the performance of autoregressive (AR) and GEE models. We simulated policy effects on annual state-level opioid mortality rates and assessed statistical performance using various metrics, including directional bias, magnitude bias, and root mean squared error. We also reported Type I error rates and the rate of correctly rejecting the null hypothesis (e.g., power), given the prevalence of frequentist null hypothesis significance testing in the applied literature. Results Most linear models resulted in minimal bias. However, non-linear models and population-weighted versions of classic linear two-way fixed effect and linear GEE models yielded considerable bias (60 to 160%). Further, root mean square error was minimized by linear AR models when we examined crude mortality rates and by negative binomial models when we examined raw death counts. In the context of frequentist hypothesis testing, many models yielded high Type I error rates and very low rates of correctly rejecting the null hypothesis (< 10%), raising concerns of spurious conclusions about policy effectiveness in the opioid literature. When considering performance across models, the linear AR models were optimal in terms of directional bias, root mean squared error, Type I error, and correct rejection rates. Conclusions The findings highlight notable limitations of commonly used statistical models for DID designs, which are widely used in opioid policy studies and in state policy evaluations more broadly. In contrast, the optimal model we identified--the AR model--is rarely used in state policy evaluation. We urge applied researchers to move beyond the classic DID paradigm and adopt use of AR models.

Type I Error Rates and Power Estimates for Selected Two-Sample Tests of Scale

1989 ◽  
Vol 14 (4) ◽  
pp. 373-384 ◽  
Author(s):  
James Algina ◽  
Stephen Olejnik ◽  
Romer Ocanto
Keyword(s):  
Type I Error ◽  
Error Rates ◽  
Type I ◽  
Sample Sizes ◽  
Nominal Level ◽  
Nonnormal Distributions ◽  
Group Design ◽  
Type I Error Rates

Estimated Type I error rates and power are reported for a modified Fligner-Killeen test, O’Brien’s test, O’Brien’s test using Welch’s modified ANOVA, the Brown-Forsythe test, and two tests developed by Tiku. Normal and nonnormal distributions and a two-group design were investigated. O’Brien’s test and the Brown-Forsythe test had estimated Type I error rates near the nominal level for all conditions investigated. To maximize power when the sample sizes are equal, O’Brien’s test should be used with platykurtic distributions and the Brown-Forsythe test with leptokurtic distributions. Either test can be used when the kurtosis is zero. When the sample sizes are unequal, O’Brien’s test should be used with platykurtic distributions and the Brown-Forsythe with symmetric-leptokurtic distributions. With other distributions, the tests have similar power.

scholarly journals Differences of Type I error rates for ANOVA and Multilevel-Linear-Models using SAS and SPSS for repeated measures designs

2019 ◽  
Vol 3 ◽  
Author(s):  
Nicolas Haverkamp ◽  
André Beauducel
Keyword(s):  
Repeated Measures ◽  
Linear Models ◽  
Type I Error ◽  
Error Rates ◽  
Small Sample ◽  
Small Samples ◽  
Type I ◽  
Sample Sizes ◽  
Type I Error Rates ◽  
Multilevel Linear Models

  To derive recommendations on how to analyze longitudinal data, we examined Type I error rates of Multilevel Linear Models (MLM) and repeated measures Analysis of Variance (rANOVA) using SAS and SPSS. We performed a simulation with the following specifications: To explore the effects of high numbers of measurement occasions and small sample sizes on Type I error, measurement occasions of m = 9 and 12 were investigated as well as sample sizes of n = 15, 20, 25 and 30. Effects of non-sphericity in the population on Type I error were also inspected: 5,000 random samples were drawn from two populations containing neither a within-subject nor a between-group effect. They were analyzed including the most common options to correct rANOVA and MLM-results: The Huynh-Feldt-correction for rANOVA (rANOVA-HF) and the Kenward-Roger-correction for MLM (MLM-KR), which could help to correct progressive bias of MLM with an unstructured covariance matrix (MLM-UN). Moreover, uncorrected rANOVA and MLM assuming a compound symmetry covariance structure (MLM-CS) were also taken into account. The results showed a progressive bias for MLM-UN for small samples which was stronger in SPSS than in SAS. Moreover, an appropriate bias correction for Type I error via rANOVA-HF and an insufficient correction by MLM-UN-KR for n < 30 were found. These findings suggest MLM-CS or rANOVA if sphericity holds and a correction of a violation via rANOVA-HF. If an analysis requires MLM, SPSS yields more accurate Type I error rates for MLM-CS and SAS yields more accurate Type I error rates for MLM-UN.

Evaluation of the Normality Assumption in Meta-Analyses

2019 ◽  
Vol 189 (3) ◽  
pp. 235-242 ◽  
Author(s):  
Chia-Chun Wang ◽  
Wen-Chung Lee
Keyword(s):  
Random Effects ◽  
Type I Error ◽  
Meta Analysis ◽  
Error Rates ◽  
Type I ◽  
Simulation Studies ◽  
Type I Error Rates ◽  
Normality Assumption ◽  
Normality Test ◽  
Meta Analyses

Abstract Random-effects meta-analysis is one of the mainstream methods for research synthesis. The heterogeneity in meta-analyses is usually assumed to follow a normal distribution. This is actually a strong assumption, but one that often receives little attention and is used without justification. Although methods for assessing the normality assumption are readily available, they cannot be used directly because the included studies have different within-study standard errors. Here we present a standardization framework for evaluation of the normality assumption and examine its performance in random-effects meta-analyses with simulation studies and real examples. We use both a formal statistical test and a quantile-quantile plot for visualization. Simulation studies show that our normality test has well-controlled type I error rates and reasonable power. We also illustrate the real-world significance of examining the normality assumption with examples. Investigating the normality assumption can provide valuable information for further analysis or clinical application. We recommend routine examination of the normality assumption with the proposed framework in future meta-analyses.

scholarly journals Unweighted regression models perform better than weighted regression techniques for respondent-driven sampling data: results from a simulation study

2019 ◽  
Vol 19 (1) ◽  
Author(s):  
Lisa Avery ◽  
Nooshin Rotondi ◽  
Constance McKnight ◽  
Michelle Firestone ◽  
Janet Smylie ◽  
...  
Keyword(s):  
Regression Analysis ◽  
Regression Models ◽  
Linear Models ◽  
Type I Error ◽  
Error Rates ◽  
Type I ◽  
Respondent Driven Sampling ◽  
Regression Techniques ◽  
Coverage Rates ◽  
Low Prevalence

Abstract Background It is unclear whether weighted or unweighted regression is preferred in the analysis of data derived from respondent driven sampling. Our objective was to evaluate the validity of various regression models, with and without weights and with various controls for clustering in the estimation of the risk of group membership from data collected using respondent-driven sampling (RDS). Methods Twelve networked populations, with varying levels of homophily and prevalence, based on a known distribution of a continuous predictor were simulated using 1000 RDS samples from each population. Weighted and unweighted binomial and Poisson general linear models, with and without various clustering controls and standard error adjustments were modelled for each sample and evaluated with respect to validity, bias and coverage rate. Population prevalence was also estimated. Results In the regression analysis, the unweighted log-link (Poisson) models maintained the nominal type-I error rate across all populations. Bias was substantial and type-I error rates unacceptably high for weighted binomial regression. Coverage rates for the estimation of prevalence were highest using RDS-weighted logistic regression, except at low prevalence (10%) where unweighted models are recommended. Conclusions Caution is warranted when undertaking regression analysis of RDS data. Even when reported degree is accurate, low reported degree can unduly influence regression estimates. Unweighted Poisson regression is therefore recommended.

scholarly journals The more the merrier? Multivariate approaches to genome-wide association analysis

2019 ◽  
Author(s):  
César-Reyer Vroom ◽  
Christiaan de Leeuw ◽  
Danielle Posthuma ◽  
Conor V. Dolan ◽  
Sophie van der Sluis
Keyword(s):  
Complex Traits ◽  
Large Scale ◽  
Type I Error ◽  
Error Rates ◽  
Genome Wide Association ◽  
Data Sets ◽  
Type I ◽  
Genome Wide ◽  
Statistical Background ◽  
Gwa Studies

AbstractThe vast majority of genome-wide association (GWA) studies analyze a single trait while large-scale multivariate data sets are available. As complex traits are highly polygenic, and pleiotropy seems ubiquitous, it is essential to determine when multivariate association tests (MATs) outperform univariate approaches in terms of power. We discuss the statistical background of 19 MATs and give an overview of their statistical properties. We address the Type I error rates of these MATs and demonstrate which factors can cause bias. Finally, we examine, compare, and discuss the power of these MATs, varying the number of traits, the correlational pattern between the traits, the number of affected traits, and the sign of the genetic effects. Our results demonstrate under which circumstances specific MATs perform most optimal. Through sharing of flexible simulation scripts, we facilitate a standard framework for comparing Type I error rate and power of new MATs to that of existing ones.

Sign in / Sign up

Export Citation Format

Share Document