Underpowered and Overworked: Problems With Data Analysis in Motor Learning Studies

2016 ◽  
Vol 4 (1) ◽  
pp. 37-58 ◽  
Author(s):  
Keith Lohse ◽  
Taylor Buchanan ◽  
Matthew Miller
Keyword(s):  
Motor Learning ◽  
Statistical Tests ◽  
A Priori ◽  
Small Sample ◽  
Effect Sizes ◽  
Null Results ◽  
Editorial Boards ◽  
The Past ◽  
Dependent Variables ◽  
Small Sample Sizes

Appropriate statistical analysis is essential for accurate and reliable research. Statistical practices have an immediate impact on the perceived results of a single study but also remote effects on the dissemination of information among scientists and the cumulative nature of research. To accurately quantify potential problems facing the field of motor learning, we systematically reviewed publications from seven journals over the past 2 years to find experiments that tested the effects of different training conditions on delayed retention and transfer tests (i.e., classic motor learning paradigms). Eighteen studies were included. These studies had small sample sizes (Mdn n/group = 11.00, interquartile range [IQR]= 9.6–15.5), multiple dependent variables (Mdn = 2, IQR = 2–4), and many statistical tests per article (Mdn = 83.5, IQR = 55.8–112.5). The observed effect sizes were large (d = 0.71, IQR = 0.49, 1.11). However, the distribution of effect sizes was biased, t(16) = 3.48, p < .01. These metadata indicate problems with the way motor learning research is conducted (or at least published). We recommend several potential solutions to address these issues: a priori power calculations, prespecified analyses, data sharing, and dissemination of null results. Furthermore, we hope these data will spark serious action from all stakeholders (researchers, editorial boards, and publishers) in the field.

scholarly journals JMASM 52: Extremely Efficient Permutation and Bootstrap Hypothesis Tests Using R

2020 ◽  
Vol 18 (2) ◽  
pp. 2-16
Author(s):  
Christina Chatzipantsiou ◽  
Marios Dimitriadis ◽  
Manos Papadakis ◽  
Michail Tsagris
Keyword(s):  
Efficient Method ◽  
Statistical Tests ◽  
Pearson Correlation ◽  
Small Sample ◽  
Pearson Correlation Coefficient ◽  
Computationally Efficient ◽  
Sample Mean ◽  
P Values ◽  
Mean Vectors ◽  
Small Sample Sizes

Re-sampling based statistical tests are known to be computationally heavy, but reliable when small sample sizes are available. Despite their nice theoretical properties not much effort has been put to make them efficient. Computationally efficient method for calculating permutation-based p-values for the Pearson correlation coefficient and two independent samples t-test are proposed. The method is general and can be applied to other similar two sample mean or two mean vectors cases.

scholarly journals statSuma: automated selection and performance of statistical comparisons for microbiome studies

2021 ◽  
Author(s):  
Robert J. Leigh ◽  
Richard A. Murphy ◽  
Fiona Walsh
Keyword(s):  
Statistical Tests ◽  
Data Distribution ◽  
Small Sample ◽  
Statistical Procedure ◽  
T Test ◽  
Desktop Computer ◽  
And Performance ◽  
Student’S T ◽  
Small Sample Sizes ◽  
Student’S T Test

There is a reproducibility crisis in scientific studies. Some of these crises arise from incorrect application of statistical tests to data that follow inappropriate distributions, have inconsistent equivariance, or have very small sample sizes. As determining which test is most appropriate for all data in a multicategorical study (such as comparing taxa between sites in microbiome studies), we present statsSuma, an interactive Python notebook (which can be run from any desktop computer using the Google Colaboratory web service) and does not require a user to have any programming experience. This software assesses underlying data structures in a given dataset to advise what pairwise or listwise statistical procedure would be best suited for all data. As some users may be interested in further mining specific trends, statSuma performs 5 different two-tailed pairwise tests (Student's t-test, Welch's t-test, Mann-Whitney U-test, Brunner-Munzel test, and a pairwise Kruskal-Wallis H-test) and advises the best test for each comparison. This software also advises whether ANOVA or a multicategorical Kruskal-Wallis H-test is most appropriate for a given dataset and performs both procedures. A data distribution-vs-Gaussian distribution plot is produced for each taxon at each site and a variance plot between all combinations of 2 taxa at each site are produced so Gaussian tests and variance tests can be visually confirmed alongside associated statistical determinants.

scholarly journals A Meta-Analysis Shows That Screen Bottom Boards Can Significantly Reduce Varroa destructor Population

Insects ◽  
2020 ◽  
Vol 11 (9) ◽  
pp. 624
Author(s):  
Fang Liu ◽  
Xinjian Xu ◽  
Yuan Zhang ◽  
Hongxia Zhao ◽  
Zachary Y. Huang
Keyword(s):  
Apis Mellifera ◽  
Honey Bee ◽  
Varroa Destructor ◽  
Negative Impact ◽  
Meta Analysis ◽  
Small Sample ◽  
Effect Sizes ◽  
Negative Results ◽  
Mite Control ◽  
Small Sample Sizes

Varroa destructor is by far the most serious threat to the western honey bee, Apis mellifera. A screen bottom board, a cultural method for mite control, is a modified bottom board with a screen that allows mites to fall onto a sticky board, or the grass or soil below the screen. Whether or not a screen bottom board can reduce varroa significantly has been controversial. Most studies show a trend of lower varroa populations in colonies with these boards, but the results are usually not statistically significant. To understand whether the negative results have been due to small sample sizes, or because the board is actually ineffective, we conducted a meta-analysis with seven published studies with a total of 145 colonies. Meta-analysis showed that the confidence intervals of the combined effect sizes were negative with a Hedges’ g of −1.09 (SE 0.39, 95% CI −2.0 to −0.19, p < 0.01), which suggests that the varroa population in colonies with screen bottom boards is significantly lower compared to those with traditional wooden floors. We thus conclude that the screen bottom board does have a significantly negative impact on the varroa population and can be part of tool kits for mite control.

A comparative geochemical study of Mars and Earth basalt petrogenesis

2013 ◽  
Vol 50 (1) ◽  
pp. 78-93 ◽  
Author(s):  
John D. Greenough ◽  
Avee Ya’acoby
Keyword(s):  
Incompatible Element ◽  
A Priori ◽  
Correlation Coefficients ◽  
Small Sample ◽  
Geochemical Data ◽  
Data Set ◽  
Element Ratios ◽  
Highly Correlated ◽  
Small Sample Sizes ◽  
Priori Information

Geochemical data, from the Mars Meteorite Compendium web site, for 13 basaltic meteorites, possibly from only four localities on Mars, are used to study Martian petrogenetic processes. To achieve this goal, an exploratory data analysis technique, multidimensional scaling (MDS), is used to quantitatively assess the relative behavior (measured with correlation coefficients) of 160 incompatible element ratios involving 25 “trace” elements. The ratios behave as in Earth basalts, suggesting that relative element incompatibility is similar in both planets. Because mineralogy controls incompatibility, the mineralogy of Earth and Mars mantles appears similar. In addition, results suggest that ratios involving elements with highly different incompatibility (e.g., La/Yb) are dominantly controlled by % melting. Plots of SiO2 (pressure proxy; decreases with increasing pressure) versus La/Yb and Nb/Y (decrease as melting increases) imply that Mars basalts, like Earth tholeiites, reflect high percentages of melting, but opposite to Earth, % melting appears to increase with increasing pressure. The moderately correlated, positive, SiO2–La/Yb Mars relationship parallels highly correlated Lunar KREEP data and contrasts with Earth’s negative correlation. The positive relationships may reflect restricted mantle convection in some (Mars and the Moon are smaller) planetary bodies. Using similarly incompatible element ratios that are sensitive to source composition, to compare Mars and Earth with MDS, Mars sources most resemble depleted Earth mantle. Additionally, these ratios group Mars sources into enriched, depleted, and intermediate types. The groupings are the same as those suggested by isotopes, and we conclude that trace element data support the hypothesis that chemical variation in Mars may reflect crystallization of a Mars magma ocean. The natural patterns in ratios and samples revealed using MDS, which has no a priori information about relationships, support integrity of the geochemical data set, despite potential shortcomings such as small sample sizes, alteration, and weathering. However, whether the meteorites are representative of Mars as a whole is unknown.

The Relationship Between Sample Sizes and Effect Sizes in Systematic Reviews in Education

2009 ◽  
Vol 31 (4) ◽  
pp. 500-506 ◽  
Author(s):  
Robert Slavin ◽  
Dewi Smith
Keyword(s):  
Sample Size ◽  
Effect Size ◽  
Secondary Mathematics ◽  
Significant Negative Correlation ◽  
Small Sample ◽  
Effect Sizes ◽  
Sample Sizes ◽  
Large Samples ◽  
Small Sample Sizes ◽  
The Relationship

Research in fields other than education has found that studies with small sample sizes tend to have larger effect sizes than those with large samples. This article examines the relationship between sample size and effect size in education. It analyzes data from 185 studies of elementary and secondary mathematics programs that met the standards of the Best Evidence Encyclopedia. As predicted, there was a significant negative correlation between sample size and effect size. The differences in effect sizes between small and large experiments were much greater than those between randomized and matched experiments. Explanations for the effects of sample size on effect size are discussed.

Unsuccessful Treatments of “Writer's Block”: A Meta-Analysis

2014 ◽  
Vol 115 (1) ◽  
pp. 276-278 ◽  
Author(s):  
Derrick C. McLean ◽  
Benjamin R. Thomas
Keyword(s):  
Meta Analysis ◽  
Small Sample ◽  
The Body ◽  
Effect Sizes ◽  
Sample Sizes ◽  
Writer's Block ◽  
Unsuccessful Treatment ◽  
Group Treatments ◽  
Writer’S Block ◽  
Small Sample Sizes

A wide literature of the unsuccessful treatment of writer's block has emerged since the early 1970's. Findings within this literature seem to confer generalizability of this procedure; however, small sample sizes may limit this interpretation. This meta-analysis independently analyzed effect sizes for “self-treatments” and “group-treatments” using number of words in the body of the publication as indication of a failure to treat writer's block. Results of the reported findings suggest that group-treatments tend to be slightly more unsuccessful than self-treatments.

Empirical Studies of Transferability of Helsinki Metropolitan Area Travel Forecasting Models

1997 ◽  
Vol 1607 (1) ◽  
pp. 38-44 ◽  
Author(s):  
Nina Karasmaa ◽  
Matti Pursula
Keyword(s):  
Metropolitan Area ◽  
Model Updating ◽  
Statistical Tests ◽  
Empirical Studies ◽  
Estimation Procedure ◽  
Small Sample ◽  
Distribution Models ◽  
Travel Forecasting ◽  
The Difference ◽  
Small Sample Sizes

The temporal transferability of mode choice and trip distribution models was studied by using the data based on traffic surveys in the Helsinki, Finland, metropolitan area in 1981 and 1988. The updating procedures examined were the Bayesian updating, combined transfer estimation, transfer scaling, and joint context estimation procedures. The results of model updating indicated that finding the correct method and sample size for each case is not an unambiguous task. The best method depends on the difference in model coefficients between the initial and the final stages as well as the quality of the data. According to the statistical tests, no differences could be discerned between the models at all. However, the sample enumeration test proved that the models’ ability to predict changes in behavior can vary greatly according to the method used. On the basis of this research the transfer scaling seems to be the method best suited for simple models. In particular, the method is quite useful if the transfer bias is large. The combined transfer estimation procedure performs best when there is a great number of observations and the transfer bias is small. With small sample sizes the Bayesian approach and the joint context estimation give the best results.

scholarly journals What Do We Know about The Use of Virtual Reality in the Rehabilitation Field? A Brief Overview

Electronics ◽  
2021 ◽  
Vol 10 (9) ◽  
pp. 1042
Author(s):  
Antonino Naro ◽  
Rocco Salvatore Calabrò
Keyword(s):  
Virtual Reality ◽  
Low Cost ◽  
Assessment Method ◽  
Small Sample ◽  
Cognitive Domains ◽  
Rehabilitation Outcomes ◽  
The Past ◽  
Pros And Cons ◽  
Small Sample Sizes ◽  
Near Future

Over the past two decades, virtual reality technology (VRT)-based rehabilitation has been increasingly examined and applied to assist patient recovery in the physical and cognitive domains. The advantages of the use of VRT in the neurorehabilitation field consist of the possibility of training an impaired function as a way to stimulate neuron reorganization (to maximize motor learning and neuroplasticity) and restoring and regaining functions and abilities by interacting with a safe and nonthreatening yet realistic virtual reality environment (VRE). Furthermore, VREs can be tailored to patient needs and provide personalized feedback on performance. VREs may also support cognitive training and increases patient motivation and enjoyment. Despite these potential advantages, there are inconclusive data about the usefulness of VRT in neurorehabilitation settings, and some issues on feasibility and safety remain to be ascertained for some neurological populations. The present brief overview aims to summarize the available literature on VRT applications in neurorehabilitation settings, along with discussing the pros and cons of VR and introducing the practical issues for research. The available studies on VRT for rehabilitation purposes over the past two decades have been mostly preliminary and feature small sample sizes. Furthermore, the studies dealing with VRT as an assessment method are more numerous than those harnessing VRT as a training method; however, the reviewed studies show the great potential of VRT in rehabilitation. A broad application of VRT is foreseeable in the near future due to the increasing availability of low-cost VR devices and the possibility of personalizing VR settings and the use of VR at home, thus actively contributing to reducing healthcare costs and improving rehabilitation outcomes.

scholarly journals A systematic review of the efficacy of intermittent theta burst stimulation (iTBS) on cognitive enhancement

2021 ◽  
Author(s):  
Alexandria Pabst ◽  
Shannon Proksch ◽  
Butovens Médé ◽  
Daniel Comstock ◽  
Jessica M Ross ◽  
...  
Keyword(s):  
Meta Analysis ◽  
Small Sample ◽  
Cognitive Domain ◽  
Theta Burst Stimulation ◽  
Burst Stimulation ◽  
The Past ◽  
Cognitive Faculties ◽  
Study Heterogeneity ◽  
Small Sample Sizes ◽  
Theta Burst

Intermittent theta-burst stimulation (iTBS) has been used to focally regulate excitability of neural cortex over the past decade – however there is little consensus on the generalizability of effects reported in individual studies. Many studies use small sample sizes (n &lt; 30), and there is a considerable amount of methodological heterogeneity in application of the stimulation itself. This systematic meta-analysis aims to consolidate the extant literature and determine if up-regulatory theta-burst stimulation reliably enhances cognition through measurable behavior. Results show that iTBS – when compared to suitable control conditions — may enhance cognition when outlier studies are removed, and there is a significant amount of heterogeneity across studies. Significant contributors to between-study heterogeneity include location of stimulation and method of navigation to the stimulation site. Surprisingly, the type of cognitive domain investigated was not a significant contributor of heterogeneity. The findings of this meta-analysis demonstrate that standardization of iTBS is urgent and necessary to determine if neuroenhancement of particular cognitive faculties are reliable and robust, and measurable through observable behavior.

Sign in / Sign up

Export Citation Format

Share Document