scholarly journals Parallelized calculation of permutation tests

2020 ◽  
Author(s):  
Markus Ekvall ◽  
Michael Höhle ◽  
Lukas Käll
Keyword(s):  
Dynamic Programming ◽  
Sample Size ◽  
Permutation Test ◽  
Statistical Tests ◽  
Permutation Tests ◽  
Supplementary Information ◽  
Attractive Alternative ◽  
Test Statistic ◽  
Sample Distribution ◽  
Running Time

Abstract Motivation Permutation tests offer a straightforward framework to assess the significance of differences in sample statistics. A significant advantage of permutation tests are the relatively few assumptions about the distribution of the test statistic are needed, as they rely on the assumption of exchangeability of the group labels. They have great value, as they allow a sensitivity analysis to determine the extent to which the assumed broad sample distribution of the test statistic applies. However, in this situation, permutation tests are rarely applied because the running time of naïve implementations is too slow and grows exponentially with the sample size. Nevertheless, continued development in the 1980s introduced dynamic programming algorithms that compute exact permutation tests in polynomial time. Albeit this significant running time reduction, the exact test has not yet become one of the predominant statistical tests for medium sample size. Here, we propose a computational parallelization of one such dynamic programming-based permutation test, the Green algorithm, which makes the permutation test more attractive. Results Parallelization of the Green algorithm was found possible by non-trivial rearrangement of the structure of the algorithm. A speed-up—by orders of magnitude—is achievable by executing the parallelized algorithm on a GPU. We demonstrate that the execution time essentially becomes a non-issue for sample sizes, even as high as hundreds of samples. This improvement makes our method an attractive alternative to, e.g. the widely used asymptotic Mann-Whitney U-test. Availabilityand implementation In Python 3 code from the GitHub repository https://github.com/statisticalbiotechnology/parallelPermutationTest under an Apache 2.0 license. Supplementary information Supplementary data are available at Bioinformatics online.

scholarly journals Parallelized calculation of permutation tests

2020 ◽  
Author(s):  
Markus Ekvall ◽  
Michael Höhle ◽  
Lukas Käll
Keyword(s):  
Dynamic Programming ◽  
Sample Size ◽  
Permutation Test ◽  
Statistical Tests ◽  
Permutation Tests ◽  
Supplementary Information ◽  
Attractive Alternative ◽  
Test Statistic ◽  
Sample Distribution ◽  
Running Time

AbstractMotivationPermutation tests offer a straight forward framework to assess the significance of differences in sample statistics. A significant advantage of permutation tests are the relatively few assumptions about the distribution of the test statistic are needed, as they rely on the assumption of exchangeability of the group labels. They have great value, as they allow a sensitivity analysis to determine the extent to which the assumed broad sample distribution of the test statistic applies. However, in this situation, permutation tests are rarely applied because the running time of naive implementations is too slow and grows exponentially with the sample size. Nevertheless, continued development in the 1980s introduced dynamic programming algorithms that compute exact permutation tests in polynomial time. Albeit this significant running time reduction, the exact test has not yet become one of the predominant statistical tests for medium sample size. Here, we propose a computational parallelization of one such dynamic programming-based permutation test, the Green algorithm, which makes the permutation test more attractive.ResultsParallelization of the Green algorithm was found possible by nontrivial rearrangement of the structure of the algorithm. A speed-up – by orders of magnitude – is achievable by executing the parallelized algorithm on a GPU. We demonstrate that the execution time essentially becomes a non-issue for sample sizes, even as high as hundreds of samples. This improvement makes our method an attractive alternative to, e.g., the widely used asymptotic Mann-Whitney U-test.AvailabilityIn Python 3 code from the GitHub repository https://github.com/statisticalbiotechnology/parallelPermutationTest under an Apache 2.0 [email protected] informationSupplementary data are available at Bioinformatics online.

scholarly journals Generalised block bootstrap and its use in meteorology

2017 ◽  
Vol 3 (1) ◽  
pp. 55-66 ◽  
Author(s):  
László Varga ◽  
András Zempléni
Keyword(s):  
Sample Size ◽  
Statistical Tests ◽  
Block Size ◽  
Temperature Data ◽  
Effective Sample Size ◽  
Block Bootstrap ◽  
Test Statistic ◽  
Positive Real ◽  
Data Set ◽  
Bootstrap Methodology

Abstract. In an earlier paper, Rakonczai et al.(2014) emphasised the importance of investigating the effective sample size in case of autocorrelated data. The simulations were based on the block bootstrap methodology. However, the discreteness of the usual block size did not allow for exact calculations. In this paper we propose a new generalisation of the block bootstrap methodology, which allows for any positive real number as expected block size. We relate it to the existing optimisation procedures and apply it to a temperature data set. Our other focus is on statistical tests, where quite often the actual sample size plays an important role, even in the case of relatively large samples. This is especially the case for copulas. These are used for investigating the dependencies among data sets. As in quite a few real applications the time dependence cannot be neglected, we investigated the effect of this phenomenon on the used test statistic. The critical value can be computed by the proposed new block bootstrap simulation, where the block size is determined by fitting a VAR model to the observations. The results are illustrated for models of the used temperature data.

scholarly journals Enlarging the Scope of Randomization and Permutation Tests in Neuroimaging and Neuroscience

2019 ◽  
Author(s):  
Eric Maris
Keyword(s):  
Rate Control ◽  
Explanatory Variable ◽  
Statistical Tests ◽  
Permutation Tests ◽  
Informed Choice ◽  
Companion Paper ◽  
Nonparametric Tests ◽  
Test Statistic ◽  
Drastic Increase ◽  
Nonparametric Statistical

AbstractEspecially for the high-dimensional data collected in neuroscience, nonparametric statistical tests are an excellent alternative for parametric statistical tests. Because of the freedom to use any function of the data as a test statistic, nonparametric tests have the potential for a drastic increase in sensitivity by making a biologically-informed choice for a test statistic. In a companion paper (Geerligs & Maris, 2020), we demonstrate that such a drastic increase is actually possible. This increase in sensitivity is only useful if, at the same time, the false alarm (FA) rate can be controlled. However, for some study types (e.g., within-participant studies), nonparametric tests do not control the FA rate (see Eklund, Nichols, & Knutsson, 2016). In the present paper, we present a family of nonparametric randomization and permutation tests of which we prove exact FA rate control. Crucially, these proofs hold for a much larger family of study types than before, and they include both within-participant studies and studies in which the explanatory variable is not under experimental control. The crucial element of this statistical innovation is the adoption of a novel but highly relevant null hypothesis: statistical independence between the biological and the explanatory variable.

scholarly journals On the Monitoring Complex Multivariate Processes

2016 ◽  
Vol 3 (322) ◽  
Author(s):  
Angelina Rajda-Tasior ◽  
Grzegorz Kończak
Keyword(s):  
Computer Simulation ◽  
Permutation Test ◽  
Stable Process ◽  
Permutation Tests ◽  
Test Statistic ◽  
Multivariate Processes ◽  
The Matrix ◽  
Quality Status

This article presents a proposal of the method of monitoring complex multidimensional processes. The problem relates to monitoring the quality of production with some attribute variables when the production is performed by some operators. To describe the quality status we used the matrix in which elements are the numbers of defective units.The proposed method uses permutation tests. The "out-of-order" signal is obtained by comparing the matrix in period t to the matrix from stable process. The test statistic used in permutation test is based on a function of distance between matrices. The properties of the proposed method have been described using computer simulation.

scholarly journals TreeMerge: a new method for improving the scalability of species tree estimation methods

2019 ◽  
Vol 35 (14) ◽  
pp. i417-i426 ◽  
Author(s):  
Erin K Molloy ◽  
Tandy Warnow
Keyword(s):  
Large Scale ◽  
Species Tree ◽  
New Method ◽  
Divide And Conquer ◽  
Supplementary Information ◽  
Estimation Methods ◽  
Running Time ◽  
Tree Estimation ◽  
Computationally Intensive ◽  
A Minor

Abstract Motivation At RECOMB-CG 2018, we presented NJMerge and showed that it could be used within a divide-and-conquer framework to scale computationally intensive methods for species tree estimation to larger datasets. However, NJMerge has two significant limitations: it can fail to return a tree and, when used within the proposed divide-and-conquer framework, has O(n5) running time for datasets with n species. Results Here we present a new method called ‘TreeMerge’ that improves on NJMerge in two ways: it is guaranteed to return a tree and it has dramatically faster running time within the same divide-and-conquer framework—only O(n2) time. We use a simulation study to evaluate TreeMerge in the context of multi-locus species tree estimation with two leading methods, ASTRAL-III and RAxML. We find that the divide-and-conquer framework using TreeMerge has a minor impact on species tree accuracy, dramatically reduces running time, and enables both ASTRAL-III and RAxML to complete on datasets (that they would otherwise fail on), when given 64 GB of memory and 48 h maximum running time. Thus, TreeMerge is a step toward a larger vision of enabling researchers with limited computational resources to perform large-scale species tree estimation, which we call Phylogenomics for All. Availability and implementation TreeMerge is publicly available on Github (http://github.com/ekmolloy/treemerge). Supplementary information Supplementary data are available at Bioinformatics online.

scholarly journals Importance of Inoculum Size and Sampling Effect in Rapid Antigen Detection for Diagnosis of Streptococcus pyogenes Pharyngitis

2000 ◽  
Vol 38 (1) ◽  
pp. 279-281 ◽  
Author(s):  
Bradley Kurtz ◽  
Michael Kurtz ◽  
Martha Roe ◽  
James Todd
Keyword(s):  
Sample Size ◽  
Streptococcus Pyogenes ◽  
Disease Status ◽  
Inoculum Size ◽  
Antigen Detection ◽  
Detection Rates ◽  
Sample Distribution ◽  
Antigen Tests ◽  
The One ◽  
Antigen Testing

ABSTRACT Current recommendations suggest that negative rapid Streptococcus pyogenes antigen tests be backed up with a culture, reflecting evidence that culture may have a higher sensitivity and also that testing of a second swab may yield a different (i.e., a positive) result because of variation in sample size or distribution. If the latter is common, the sensitivities of current antigen detection tests might be improved by simply increasing the amount of sample tested. The present study assessed the effect of antigen testing of two swabs extracted together compared to independent testing of each swab extracted separately for children with clinical pharyngitis. S. pyogenes grew from one or both swabs for 198 (37%) of 537 children. The combined culture was significantly ( P < 0.05) more sensitive than culture of either swab alone. Compared to combined culture, antigen testing of two swabs extracted and tested together was significantly more sensitive than two single swab extractions (94.1 versus 80%; P = 0.03); however, the specificity was decreased (81.5 versus 89.8 to 92.7%; P < 0.05). This study suggests that sample size and/or uneven sample distribution may have influenced the apparent sensitivities of prior studies that compared antigen tests to a single plate culture. A strategy, such as the one used in the present study, that increases the sample size available for antigen testing (i.e., extraction of samples from both swabs) may improve detection rates to a level that will better approximate true disease status and obviate the need for backup cultures if specificity can be improved.

scholarly journals The Effects of Manifest Residual Variances, Indicator Communality, and Sample Size on the χ2-Test Statistic of the Metric Invariance Model

2019 ◽  
Author(s):  
Eric Klopp ◽  
Stefan Klößner
Keyword(s):  
Sample Size ◽  
Measurement Invariance ◽  
Signal To Noise Ratio ◽  
Practical Implication ◽  
Residual Variance ◽  
Test Statistic ◽  
Monte Carlo Studies ◽  
Key Factor ◽  
Metric Measurement ◽  
Invariance Model

In this contribution, we investigate the effects of manifest residual variance, indicator communality and sample size on the χ2-test statistic of the metric measurement invariance model, i.e. the model with equality constraints on all loadings. We demonstrate by means of Monte Carlo studies that the χ2-test statistic relates inversely to manifest residual variance, whereas sample size and χ2-test statistic show the well-known pro- portional relation. Moreover, we consider indicator communality as a key factor for the size of the χ2-test statistic. In this context, we introduce the concept of signal-to-noise ratio as a tool for studying the effects of manifest residual error and indicator commu- nality and demonstrate its use with some examples. Finally, we discuss the limitations of this contribution and its practical implication for the analysis of metric measurement invariance models.

scholarly journals Variations in optical properties of aerosols on monsoon seasonal change and estimation of aerosol optical depth using ground-based meteorological and air quality data

2014 ◽  
Vol 14 (13) ◽  
pp. 19747-19789
Author(s):  
F. Tan ◽  
H. S. Lim ◽  
K. Abdullah ◽  
T. L. Yoon ◽  
B. Holben
Keyword(s):  
Optical Properties ◽  
Aerosol Optical Depth ◽  
Optical Depth ◽  
Statistical Tests ◽  
Air Pollutant ◽  
Quality Data ◽  
Supplementary Information ◽  
Northeast Monsoon ◽  
Data Set ◽  
Proposed Model

Abstract. In this study, the optical properties of aerosols in Penang, Malaysia were analyzed for four monsoonal seasons (northeast monsoon, pre-monsoon, southwest monsoon, and post-monsoon) based on data from the AErosol RObotic NETwork (AERONET) from February 2012 to November 2013. The aerosol distribution patterns in Penang for each monsoonal period were quantitatively identified according to the scattering plots of the aerosol optical depth (AOD) against the Angstrom exponent. A modified algorithm based on the prototype model of Tan et al. (2014a) was proposed to predict the AOD data. Ground-based measurements (i.e., visibility and air pollutant index) were used in the model as predictor data to retrieve the missing AOD data from AERONET because of frequent cloud formation in the equatorial region. The model coefficients were determined through multiple regression analysis using selected data set from in situ data. The predicted AOD of the model was generated based on the coefficients and compared against the measured data through standard statistical tests. The predicted AOD in the proposed model yielded a coefficient of determination R2 of 0.68. The corresponding percent mean relative error was less than 0.33% compared with the real data. The results revealed that the proposed model efficiently predicted the AOD data. Validation tests were performed on the model against selected LIDAR data and yielded good correspondence. The predicted AOD can beneficially monitor short- and long-term AOD and provide supplementary information in atmospheric corrections.

scholarly journals Comparison of statistical methods for analysis of small sample sizes for detecting the differences in efficacy between treatments for knee osteoarthritis

2020 ◽  
Author(s):  
Chia-Lung Shih ◽  
Te-Yu Hung
Keyword(s):  
Knee Osteoarthritis ◽  
Sample Size ◽  
Statistical Methods ◽  
Statistical Power ◽  
Permutation Test ◽  
Small Sample Size ◽  
Small Sample ◽  
T Test ◽  
Sample Sizes ◽  
Knee Oa

Abstract Background A small sample size (n < 30 for each treatment group) is usually enrolled to investigate the differences in efficacy between treatments for knee osteoarthritis (OA). The objective of this study was to use simulation for comparing the power of four statistical methods for analysis of small sample size for detecting the differences in efficacy between two treatments for knee OA. Methods A total of 10,000 replicates of 5 sample sizes (n=10, 15, 20, 25, and 30 for each group) were generated based on the previous reported measures of treatment efficacy. Four statistical methods were used to compare the differences in efficacy between treatments, including the two-sample t-test (t-test), the Mann-Whitney U-test (M-W test), the Kolmogorov-Smirnov test (K-S test), and the permutation test (perm-test). Results The bias of simulated parameter means showed a decreased trend with sample size but the CV% of simulated parameter means varied with sample sizes for all parameters. For the largest sample size (n=30), the CV% could achieve a small level (<20%) for almost all parameters but the bias could not. Among the non-parametric tests for analysis of small sample size, the perm-test had the highest statistical power, and its false positive rate was not affected by sample size. However, the power of the perm-test could not achieve a high value (80%) even using the largest sample size (n=30). Conclusion The perm-test is suggested for analysis of small sample size to compare the differences in efficacy between two treatments for knee OA.

Sign in / Sign up

Export Citation Format

Share Document