scholarly journals Data Analysis and Power Simulations with General Linear Mixed Modelling for Psychophysical Data – A Practical, R-Based Guide

2021 ◽  
Author(s):  
Björn Jörges
Keyword(s):  
Data Analysis ◽  
Sample Size ◽  
Statistical Power ◽  
Statistical Tests ◽  
Forced Choice ◽  
Choice Data ◽  
Psychophysical Data ◽  
Statistical Validity ◽  
Sample Size Planning ◽  
Mixed Modelling

Sample size planning is not straight-forward for the complex designs that are usually employed in psychophysical (two-alternative forced-choice) experiments: characteristics such as binary response variables and nested data structures where responses may be correlated differently within participants and experimental sessions than across participants and experimental sessions make it harder to estimate the necessary number of participants and trials with traditional means. In this practical R-based guide, we first show in detail how we can simulate verisimilar psychophysical data. We then use these simulations to compare two different methods by which two-alternative forced-choice data can be analyzed: (1) the “two-step” approach, where first psychometric functions are fitted and then statistical tests are performed over the parameters of these fitted psychometric functions; (2) an approach based on Generalized Linear Mixed Modeling (GLMM) that does not require the intermediary step of fitting psychometric functions. We argue that the GLMM approach enhances statistical validity and show that it can increase statistical power. Finally, we provide a sample implementation of a simulation-based power analysis that can be used as-is for many simple designs, but is also easily adaptable for more complex designs. Overall, we show that a GLMM-based approach can be beneficial for data analysis and sample size planning for typical (two-alternative forced-choice) psychophysical designs.

The importance of considering effect size and statistical power in research

2015 ◽  
Author(s):  
David Clark-Carter
Keyword(s):  
Sample Size ◽  
Effect Size ◽  
Statistical Power ◽  
Statistical Tests ◽  
Clinical Psychologists ◽  
Future Research

This chapter explores why effect size needs to be taken into account when designing and reporting research. It gives an effect size for each of the standard statistical tests which health and clinical psychologists employ, and looks at the need to consider statistical power when choosing a sample size for a study and how statistical power can help to guide the advice which can be given when discussing future research.

SAMPLE SIZE PLANNING IN QUANTITATIVE L2 RESEARCH

2020 ◽  
Vol 42 (4) ◽  
pp. 849-870
Author(s):  
Reza Norouzian
Keyword(s):  
Sample Size ◽  
Statistical Power ◽  
Open Science ◽  
Effect Sizes ◽  
Size Estimation ◽  
Alternative Approach ◽  
Sample Size Planning ◽  
Effect Size Estimation ◽  
Definition Of ◽  
Actual Size

AbstractResearchers are traditionally advised to plan for their required sample size such that achieving a sufficient level of statistical power is ensured (Cohen, 1988). While this method helps distinguishing statistically significant effects from the nonsignificant ones, it does not help achieving the higher goal of accurately estimating the actual size of those effects in an intended study. Adopting an open-science approach, this article presents an alternative approach, accuracy in effect size estimation (AESE), to sample size planning that ensures that researchers obtain adequately narrow confidence intervals (CI) for their effect sizes of interest thereby ensuring accuracy in estimating the actual size of those effects. Specifically, I (a) compare the underpinnings of power-analytic and AESE methods, (b) provide a practical definition of narrow CIs, (c) apply the AESE method to various research studies from L2 literature, and (d) offer several flexible R programs to implement the methods discussed in this article.

scholarly journals TAXONOMIC LEVEL NEEDED TO DETECT SIGNIFICANT CHANGES IN POLYCHAETE COMMUNITIES OVER DIFFERENT MACROPHYTE ASSEMBLAGES ON ROCKY INTERTIDAL SHORES

2017 ◽  
Vol 42 (2) ◽  
pp. 93-108
Author(s):  
Hadiyanto
Keyword(s):  
Data Analysis ◽  
Sample Size ◽  
Statistical Power ◽  
Linear Models ◽  
Ecological Monitoring ◽  
Rocky Intertidal ◽  
Global Changes ◽  
Taxonomic Level ◽  
Significant Relationships ◽  
Family Based

Environmental degradation has more significant impacts on rocky intertidal communities after global changes increase progressively. Thus, ecological monitoring should be conducted properly to analyse potential drivers and their impacts. However, most of the ecological monitoring in rocky intertidal shores is more interested in macroalgae. Polychaetes associated with macrophyte assemblages should be also involved in the monitoring because they are important in determining coastal health and productivity. A successful ecological monitoring should consider three factors: taxonomic level, statistical power, and sample size. In this study, those factors were analysed in the relationships between polychaetes and macrophytes. Four taxonomic levels of polychaetes (i.e. order, family, genus, species) were tested based on 25 samples collected from rocky intertidal shores of Gunung Kidul, Yogyakarta, Indonesia. Relationships between each of taxonomic richness of polychaetes and each of macrophytes variables (i.e. species richness, biomass, species composition) were analysed using a Generalised Linear Models fitted by Poisson Distribution and log link. The statistical power of those relationships and the sample size needed to obtain a strong statistical power (>0.8) were also recorded. Relationships between each of taxonomic composition of polychaetes and each of macrophyte variables were analysed using a distance-based Redundancy Analysis based on Bray-Curtis dissimilarity on log(x+1) transformed abundance data with 999 permutations. Results showed that family-based data analysis was sufficient to detect significant relationships between polychaetes and macrophytes. However, the statistical power of most relationships was relatively weak (<0.8). Hence, the family-based data analysis should select a 44-sample size to gain significant relationships with a strong statistical power. 

Sample-Size Planning for More Accurate Statistical Power: A Method Adjusting Sample Effect Sizes for Publication Bias and Uncertainty

2017 ◽  
Vol 28 (11) ◽  
pp. 1547-1562 ◽  
Author(s):  
Samantha F. Anderson ◽  
Ken Kelley ◽  
Scott E. Maxwell
Keyword(s):  
Sample Size ◽  
Publication Bias ◽  
Effect Size ◽  
Statistical Power ◽  
Web Applications ◽  
R Package ◽  
Effect Sizes ◽  
Alternative Approach ◽  
Sample Size Planning ◽  
User Friendly

The sample size necessary to obtain a desired level of statistical power depends in part on the population value of the effect size, which is, by definition, unknown. A common approach to sample-size planning uses the sample effect size from a prior study as an estimate of the population value of the effect to be detected in the future study. Although this strategy is intuitively appealing, effect-size estimates, taken at face value, are typically not accurate estimates of the population effect size because of publication bias and uncertainty. We show that the use of this approach often results in underpowered studies, sometimes to an alarming degree. We present an alternative approach that adjusts sample effect sizes for bias and uncertainty, and we demonstrate its effectiveness for several experimental designs. Furthermore, we discuss an open-source R package, BUCSS, and user-friendly Web applications that we have made available to researchers so that they can easily implement our suggested methods.

scholarly journals Required sample size for comparing means in two paired samples

2021 ◽  
Vol 6 (4) ◽  
pp. 82-88
Author(s):  
A. M. Grjibovski ◽  
M. A. Gorbatova ◽  
A. N. Narkevich ◽  
K. A. Vinogradov
Keyword(s):  
Sample Size ◽  
Biomedical Research ◽  
Statistical Power ◽  
Statistical Tests ◽  
Simple Algorithm ◽  
Statistical Test ◽  
Minimal Sample ◽  
Paired Samples ◽  
Student’S T ◽  
Paired T Test

This paper continues our series of articles for beginners on required sample size for the most common basic statistical tests used in biomedical research. The most common statistical test for comparing means in paired samples is Student’s paired t-test. In this paper we present a simple algorithm for calculating required sample size for comparing two means in paired samples. As in our earlier papers we demonstrate how to perform calculations using WinPepi and Stata software. Moreover, we have created a table with calculated minimal sample sizes required for using Student’s t-tests for different scenarios with the confidence level of 95% and statistical power of 80%.

scholarly journals REQUIRED SAMPLE SIZE FOR COMPARING PROPORTIONS IN TWO INDEPENDENT SAMPLES

2020 ◽  
Vol 6 (3) ◽  
pp. 76-83
Author(s):  
A. M. Grjibovski ◽  
M. A. Gorbatova ◽  
A. N. Narkevich ◽  
K. A. Vinogradov
Keyword(s):  
Sample Size ◽  
Statistical Power ◽  
Statistical Tests ◽  
Simple Algorithm ◽  
Statistical Test ◽  
Type Ii ◽  
Sample Size Calculations ◽  
Chi Squared ◽  
Type Ii Errors ◽  
False Null Hypothesis

This paper continues our series of articles on required sample size for the most common basic statistical tests used in biomedical research. Sample size calculations are rarely performed in research planning in Russia often resulting in Type II errors, i.e. on acceptance on false null hypothesis due to insufficient sample size. The most common statistical test for analyzing proportions in independent samples is Pearson’s chi-squared test. In this paper we present a simple algorithm for calculating required sample size for comparing two independent proportions. In addition to manual calculations we present a step-by-step guide on how to use WinPepi and Stata software for calculating sample size for independent proportions. In addition, we present a table for junior researchers with already calculated sample sizes for comparing proportions from 0,1 to 0,9 by 0,1 with 95% confidence level and 80% statistical power.

scholarly journals Finding domain-general metacognitive mechanisms requires using appropriate tasks

2017 ◽  
Author(s):  
Eugene Ruby ◽  
Nathan Giles ◽  
Hakwan Lau
Keyword(s):  
Visual Perception ◽  
Sample Size ◽  
False Positive ◽  
Statistical Power ◽  
Forced Choice ◽  
The Other ◽  
Online Experiments ◽  
Positive Correlation ◽  
Domain Specific ◽  
Unresolved Question

AbstractAn important yet unresolved question is whether or not metacognition consists of domain-general or domain-specific mechanisms. While most studies on this topic suggest a dissociation between metacognitive abilities at the neural level, there are conflicting reports at the behavioral level. Specifically, while McCurdy et al. (2013) found a positive correlation between metacognitive efficiency for visual perception and memory, Baird et al. (2013) didn’t find this correlation. One possible explanation for this disparity is that the former included two-alternative-forced choice (2AFC) judgments in both their visual and memory tasks, whereas the latter used 2AFC for one task and yes/no (YN) judgments for the other. In support of this hypothesis, we ran two online experiments meant to mirror McCurdy et al. (2013) and Baird et al. (2013) with considerable statistical power (n=100), and replicated the main findings of both studies. This suggests the finding of McCurdy et al (2013) was not a ‘fluke’ (i.e. false positive). In a third experiment with the same sample size, which included YN judgments for both tasks, we did not find a correlation between metacognitive functions, suggesting that the conflict between McCurdy et al. (2013) and Baird et al. (2013) stemmed from the use of YN judgments in the latter study. Our results underscore the need to avoid conflating 2AFC and YN judgments, which is a common problem.

scholarly journals Sample Size and Statistical Power in [15O]H2O Studies of Human Cognition

1996 ◽  
Vol 16 (5) ◽  
pp. 804-816 ◽  
Author(s):  
Nancy C. Andreasen ◽  
Stephan Arndt ◽  
Ted Cizadlo ◽  
Daniel S. O'Leary ◽  
G. Leonard Watkins ◽  
...  
Keyword(s):  
Data Analysis ◽  
Sample Size ◽  
Statistical Power ◽  
Functional Dependence ◽  
Human Memory ◽  
Human Cognition ◽  
Positron Emission ◽  
Large N ◽  
Crucial Component ◽  
Small N

Determining the appropriate sample size is a crucial component of positron emission tomography (PET) studies. Power calculations, the traditional method for determining sample size, were developed for hypothesis-testing approaches to data analysis. This method for determining sample size is challenged by the complexities of PET data analysis: use of exploratory analysis strategies, search for multiple correlated nodes on interlinked networks, and analysis of large numbers of pixels that may have correlated values due to both anatomical and functional dependence. We examine the effects of variable sample size in a study of human memory, comparing large (n = 33), medium (n = 16,17), small (n = 11, 11, 11), and very small (n = 6, 6, 7, 7, 7) samples. Results from the large sample are assumed to be the “gold standard.” The primary criterion for assessing sample size is replicability. This is evaluated using a hierarchically ordered group of parameters: pattern of peaks, location of peaks, number of peaks, size (volume) of peaks, and intensity of the associated t (or z) statistic. As sample size decreases, false negatives begin to appear, with some loss of pattern and peak detection; there is no corresponding increase in false positives. The results suggest that good replicability occurs with a sample size of 10–20 subjects in studies of human cognition that use paired subtraction comparisons of single experimental/baseline conditions with blood flow differences ranging from 4 to 13%.

The Power of Replicated Measures to Increase Statistical Power

2019 ◽  
Vol 2 (3) ◽  
pp. 199-213 ◽  
Author(s):  
Marc-André Goulet ◽  
Denis Cousineau
Keyword(s):  
Sample Size ◽  
Null Hypothesis ◽  
Statistical Power ◽  
Statistical Tests ◽  
Cognitive Tasks ◽  
Type Ii ◽  
Type Ii Error ◽  
Multiple Measures ◽  
Multiple Measurements ◽  
Sufficient Statistical Power

When running statistical tests, researchers can commit a Type II error, that is, fail to reject the null hypothesis when it is false. To diminish the probability of committing a Type II error (β), statistical power must be augmented. Typically, this is done by increasing sample size, as more participants provide more power. When the estimated effect size is small, however, the sample size required to achieve sufficient statistical power can be prohibitive. To alleviate this lack of power, a common practice is to measure participants multiple times under the same condition. Here, we show how to estimate statistical power by taking into account the benefit of such replicated measures. To that end, two additional parameters are required: the correlation between the multiple measures within a given condition and the number of times the measure is replicated. An analysis of a sample of 15 studies (total of 298 participants and 38,404 measurements) suggests that in simple cognitive tasks, the correlation between multiple measures is approximately .14. Although multiple measurements increase statistical power, this effect is not linear, but reaches a plateau past 20 to 50 replications (depending on the correlation). Hence, multiple measurements do not replace the added population representativeness provided by additional participants.

Sign in / Sign up

Export Citation Format

Share Document