Statistics for Small Groups: The Power of the Pretest

1988 ◽  
Vol 13 (3) ◽  
pp. 142-146 ◽  
Author(s):  
David A. Cole
Keyword(s):  
Statistical Power ◽  
Change Score ◽  
Small Sample ◽  
Analysis Of Covariance ◽  
Small Samples ◽  
Group Differences ◽  
Control Group ◽  
Small Sample Sizes ◽  
Almost All ◽  
Treatment Control Group

In the area of severe-profound retardation, researchers are faced with small sample sizes. The question of statistical power is critical. In this article, three commonly used tests for treatment-control group differences are compared with respect to their relative power: the posttest-only approach, the change-score approach, and an analysis of covariance (ANCOVA) approach. In almost all cases, the ANCOVA approach is the more powerful than the other two, even when very small samples are involved. Finally, a fourth approach involving ANCOVA plus alternate rank assignments is examined and found to be superior even to the ANCOVA approach, especially in small sample cases. Use of slightly more sophisticated statistics in small sample research is recommended.

scholarly journals G-computation and machine learning for estimating the causal effects of binary exposure statuses on binary outcomes

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Florent Le Borgne ◽  
Arthur Chatton ◽  
Maxime Léger ◽  
Rémi Lenain ◽  
Yohann Foucher
Keyword(s):  
Machine Learning ◽  
Support Vector Machine ◽  
Statistical Power ◽  
Small Sample ◽  
Causal Effects ◽  
Small Samples ◽  
Support Vector ◽  
Sample Sizes ◽  
Super Learner ◽  
Small Sample Sizes

AbstractIn clinical research, there is a growing interest in the use of propensity score-based methods to estimate causal effects. G-computation is an alternative because of its high statistical power. Machine learning is also increasingly used because of its possible robustness to model misspecification. In this paper, we aimed to propose an approach that combines machine learning and G-computation when both the outcome and the exposure status are binary and is able to deal with small samples. We evaluated the performances of several methods, including penalized logistic regressions, a neural network, a support vector machine, boosted classification and regression trees, and a super learner through simulations. We proposed six different scenarios characterised by various sample sizes, numbers of covariates and relationships between covariates, exposure statuses, and outcomes. We have also illustrated the application of these methods, in which they were used to estimate the efficacy of barbiturates prescribed during the first 24 h of an episode of intracranial hypertension. In the context of GC, for estimating the individual outcome probabilities in two counterfactual worlds, we reported that the super learner tended to outperform the other approaches in terms of both bias and variance, especially for small sample sizes. The support vector machine performed well, but its mean bias was slightly higher than that of the super learner. In the investigated scenarios, G-computation associated with the super learner was a performant method for drawing causal inferences, even from small sample sizes.

Impaired List Learning Is Not a General Property of Frontal Lesions

2009 ◽  
Vol 21 (7) ◽  
pp. 1422-1434 ◽  
Author(s):  
Michael P. Alexander ◽  
Donald Stuss ◽  
Susan Gillingham
Keyword(s):  
General Property ◽  
Memory Performance ◽  
Small Sample ◽  
Group Differences ◽  
Task Structure ◽  
Significant Group ◽  
List Learning ◽  
Small Sample Sizes ◽  
Almost All ◽  
Frontal Lesions

Background: List-learning tasks are frequently used to provide measures of “executive functions” that are believed necessary for successful memory performance. Small sample sizes, confounding anomia, and incomplete representation of all frontal regions have prevented consistent demonstration of distinct regional frontal effects on this task. Objective: To confirm specific effects of lesions in different frontal regions. Subjects: Forty-one patients with chronic focal frontal lesions and 38 control subjects. There were no group differences in naming scores. Methods: Two word lists were presented, one with unblocked words from related categories and one in a preblocked format. Standard measures of learning, recall, recognition, and strategies were obtained, first for the frontal group as a whole and then for large but defined frontal regions. For all measures with significant group differences, a lesion “hotspotting” method identified possible specific regional injury effects. Results: The frontal group was impaired on almost all measures, but impairments on most measures were particularly identified with lesions in the left superior frontal lobe (approximately area 9s) and some deficits in learning processes were surprisingly more prominent on the blocked list. Conclusion: Difficulty with list learning is not a general property of all frontal lesions. Lesions in different frontal regions impair list learning through specific mechanisms, and these effects may be modified by manipulations of the task structure.

Implications of Small Samples for Generalization: Adjustments and Rules of Thumb

2016 ◽  
Vol 41 (5) ◽  
pp. 472-505 ◽  
Author(s):  
Elizabeth Tipton ◽  
Kelly Hallberg ◽  
Larry V. Hedges ◽  
Wendy Chan
Keyword(s):  
Observational Studies ◽  
Small Sample ◽  
Average Treatment Effect ◽  
Small Samples ◽  
Sample Sizes ◽  
Random Samples ◽  
Rules Of Thumb ◽  
Large Populations ◽  
Small Sample Sizes ◽  
Combine Information

Background: Policy makers and researchers are frequently interested in understanding how effective a particular intervention may be for a specific population. One approach is to assess the degree of similarity between the sample in an experiment and the population. Another approach is to combine information from the experiment and the population to estimate the population average treatment effect (PATE). Method: Several methods for assessing the similarity between a sample and population currently exist as well as methods estimating the PATE. In this article, we investigate properties of six of these methods and statistics in the small sample sizes common in education research (i.e., 10–70 sites), evaluating the utility of rules of thumb developed from observational studies in the generalization case. Result: In small random samples, large differences between the sample and population can arise simply by chance and many of the statistics commonly used in generalization are a function of both sample size and the number of covariates being compared. The rules of thumb developed in observational studies (which are commonly applied in generalization) are much too conservative given the small sample sizes found in generalization. Conclusion: This article implies that sharp inferences to large populations from small experiments are difficult even with probability sampling. Features of random samples should be kept in mind when evaluating the extent to which results from experiments conducted on nonrandom samples might generalize.

scholarly journals Impact of Mindfulness Training on Spanish Police Officers’ Mental and Emotional Health: a Non-Randomized Pilot Study

Mindfulness ◽  
2022 ◽  
Author(s):  
Jaime Navarrete ◽  
Miguel Ángel García-Salvador ◽  
Ausiàs Cebolla ◽  
Rosa Baños
Keyword(s):  
Police Officers ◽  
Repeated Measures ◽  
Emotional Health ◽  
Analysis Of Covariance ◽  
Controlled Study ◽  
Group Differences ◽  
Control Group ◽  
Depression Symptoms ◽  
Control Group Design ◽  
Paired Samples

Abstract Objectives The purpose of this exploratory non-randomized controlled study was to determine the acceptance and effectiveness of an 8-week mindfulness-based intervention (MBI) co-designed by a police officer. Methods A pretest-posttest control group design was followed. Participants (MBI group = 20; control group = 18) answered baseline and post-training self-reported measures. In addition, the weekly emotional state of the MBI group was collected. Paired-samples t-test and analysis of covariance were performed for pre-post within-group and between-group differences, respectively, as well as linear mixed effects analysis of repeated measures for week-by-week data. Results High acceptance and attendance rates, as well as significant pre-post within-group differences in the MBI group in mindfulness (η2 = 0.43), self-compassion (η2 = 0.43), depression (η2 = 0.54), anxiety (η2 = 0.46), stress (η2 = 0.51), difficulties in emotion regulation, sleep quality (η2 = 0.57), and burnout (η2 = 0.31–0.47), were identified. Moreover, police officers who underwent the MBI experienced a week by week decrease of anger, disgust, anxiety, sadness, and desire. Finally, after adjusting for pre-test scores, significant between-group differences were found in the way of attending to internal and external experiences (observing mindfulness facet; ηp2 = 0.21), depression symptoms (ηp2 = 0.23), general distress (ηp2 = 0.24), and the degree of physical and psychological exhaustion (personal burnout; ηp2 = 0.20). Conclusions The preliminary effectiveness of this MBI on psychopathology and quality of life outcomes in Spanish police officers was discussed. Previous evidence regarding the promising use of MBIs in this population was supported.

Distinguishing autism from co-existing conditions: a behavioural profiling investigation

2016 ◽  
Vol 2 (1) ◽  
pp. 41-54
Author(s):  
Ashleigh Saunders ◽  
Karen E. Waldie
Keyword(s):  
Statistical Power ◽  
Psychiatric Comorbidity ◽  
Adaptive Functioning ◽  
Autism Spectrum ◽  
Small Sample ◽  
Comorbid Conditions ◽  
Sample Sizes ◽  
Content Type ◽  
Small Sample Sizes ◽  
Comorbid Adhd

Purpose – Autism spectrum disorder (ASD) is a lifelong neurodevelopmental condition for which there is no known cure. The rate of psychiatric comorbidity in autism is extremely high, which raises questions about the nature of the co-occurring symptoms. It is unclear whether these additional conditions are true comorbid conditions, or can simply be accounted for through the ASD diagnosis. The paper aims to discuss this issue. Design/methodology/approach – A number of questionnaires and a computer-based task were used in the current study. The authors asked the participants about symptoms of ASD, attention deficit hyperactivity disorder (ADHD) and anxiety, as well as overall adaptive functioning. Findings – The results demonstrate that each condition, in its pure form, can be clearly differentiated from one another (and from neurotypical controls). Further analyses revealed that when ASD occurs together with anxiety, anxiety appears to be a separate condition. In contrast, there is no clear behavioural profile for when ASD and ADHD co-occur. Research limitations/implications – First, due to small sample sizes, some analyses performed were targeted to specific groups (i.e. comparing ADHD, ASD to comorbid ADHD+ASD). Larger sample sizes would have given the statistical power to perform a full scale comparative analysis of all experimental groups when split by their comorbid conditions. Second, males were over-represented in the ASD group and females were over-represented in the anxiety group, due to the uneven gender balance in the prevalence of these conditions. Lastly, the main profiling techniques used were questionnaires. Clinical interviews would have been preferable, as they give a more objective account of behavioural difficulties. Practical implications – The rate of psychiatric comorbidity in autism is extremely high, which raises questions about the nature of the co-occurring symptoms. It is unclear whether these additional conditions are true comorbid conditions, or can simply be accounted for through the ASD diagnosis. Social implications – This information will be important, not only to healthcare practitioners when administering a diagnosis, but also to therapists who need to apply evidence-based treatment to comorbid and stand-alone conditions. Originality/value – This study is the first to investigate the nature of co-existing conditions in ASD in a New Zealand population.

scholarly journals Nothing wrong about change: the adequate choice of the dependent variable and design in prediction of cognitive training success

2020 ◽  
Vol 20 (1) ◽  
Author(s):  
André Mattes ◽  
Mandy Roheger
Keyword(s):  
Regression Model ◽  
Cognitive Training ◽  
Test Score ◽  
Statistical Power ◽  
Individualized Medicine ◽  
Intervention Group ◽  
Change Score ◽  
Control Group ◽  
Absolute Change ◽  
True Model

Abstract Background Even though investigating predictors of intervention success (e.g Cognitive Training, CT) is gaining more and more interest in the light of an individualized medicine, results on specific predictors of intervention success in the overall field are mixed and inconsistent due to different and sometimes inappropriate statistical methods used. Therefore, the present paper gives a guidance on the appropriate use of multiple regression analyses to identify predictors of CT and similar non-pharmacological interventions. Methods We simulated data based on a predefined true model and ran a series of different analyses to evaluate their performance in retrieving the true model coefficients. The true model consisted of a 2 (between: experimental vs. control group) × 2 (within: pre- vs. post-treatment) design with two continuous predictors, one of which predicted the success in the intervention group and the other did not. In analyzing the data, we considered four commonly used dependent variables (post-test score, absolute change score, relative change score, residual score), five regression models, eight sample sizes, and four levels of reliability. Results Our results indicated that a regression model including the investigated predictor, Group (experimental vs. control), pre-test score, and the interaction between the investigated predictor and the Group as predictors, and the absolute change score as the dependent variable seemed most convenient for the given experimental design. Although the pre-test score should be included as a predictor in the regression model for reasons of statistical power, its coefficient should not be interpreted because even if there is no true relationship, a negative and statistically significant regression coefficient commonly emerges. Conclusion Employing simulation methods, theoretical reasoning, and mathematical derivations, we were able to derive recommendations regarding the analysis of data in one of the most prevalent experimental designs in research on CT and external predictors of CT success. These insights can contribute to the application of considered data analyses in future studies and facilitate cumulative knowledge gain.

Statistical Power for Analyses of Changes in Randomized Controlled Trials

2005 ◽  
Vol 84 (3) ◽  
pp. 283-287 ◽  
Author(s):  
Y.-K. Tu ◽  
A. Blance ◽  
V. Clerehugh ◽  
M.S. Gilthorpe
Keyword(s):  
Randomized Controlled Trials ◽  
Statistical Methods ◽  
Statistical Power ◽  
Change Score ◽  
Error Rates ◽  
Analysis Of Covariance ◽  
Controlled Trials ◽  
Multivariate Statistical ◽  
Randomized Controlled ◽  
Study Power

Randomized controlled trials (RCTs) are widely recommended as the most useful study design to generate reliable evidence and guidance to daily practices in medicine and dentistry. However, it is not well-known in dental research that different statistical methods of data analysis can yield substantial differences in study power. In this study, computer simulations are used to explore how using different univariate and multivariate statistical methods of analyzing change in continuous outcome variables affects study power, and the sample size required for RCTs. Results show that, in general, analysis of covariance (ANCOVA) yields greater power than other statistical methods in testing the superiority of one treatment over another, or in testing the equivalence between two treatments. Therefore, ANCOVA should be used in preference to change score or percentage change score to reduce type II error rates.

Taphonomy and ecology of modern avifaunal remains from Amboseli Park, Kenya

Paleobiology ◽  
2003 ◽  
Vol 29 (1) ◽  
pp. 52-70 ◽  
Author(s):  
Anna K. Behrensmeyer ◽  
C. Tristan Stayton ◽  
Ralph E. Chapman
Keyword(s):  
Habitat Preferences ◽  
Large Body ◽  
Bird Community ◽  
Skeletal Remains ◽  
Small Sample ◽  
Small Samples ◽  
Scaling Analysis ◽  
Ecological Data ◽  
Small Sample Sizes ◽  
Grassland Habitats

Avian skeletal remains occur in many fossil assemblages, and in spite of small sample sizes and incomplete preservation, they may be a source of valuable paleoecological information. In this paper, we examine the taphonomy of a modern avian bone assemblage and test the relationship between ecological data based on avifaunal skeletal remains and known ecological attributes of a living bird community. A total of 54 modern skeletal occurrences and a sample of 126 identifiable bones from Amboseli Park, Kenya, were analyzed for weathering features and skeletal part preservation in order to characterize preservation features and taphonomic biases. Avian remains, with the exception of ostrich, decay more rapidly than adult mammal bones and rarely reach advanced stages of weathering. Breakage and the percentage of anterior limb elements serve as indicators of taphonomic overprinting that may affect paleoecological signals. Using ecomorphic categories including body weight, diet, and habitat, we compared species in the bone assemblage with the living Amboseli avifauna. The documented bone sample is biased toward large body size, representation of open grassland habitats, and grazing or scavenging diets. In spite of this, multidimensional scaling analysis shows that the small faunal sample (16 out of 364 species) in the pre-fossil bone assemblage accurately represents general features of avian ecospace in Amboseli. This provides a measure of the potential fidelity of paleoecological reconstructions based on small samples of avian remains. In the Cenozoic, the utility of avian fossils is enhanced because bird ecomorphology is relatively well known and conservative through time, allowing back-extrapolations of habitat preferences, diet, etc. based on modern taxa.

scholarly journals Nothing wrong about change: The adequate choice of the dependent variable and design in prediction of cognitive training success

2020 ◽  
Author(s):  
André Mattes ◽  
Mandy Roheger
Keyword(s):  
Regression Model ◽  
Cognitive Training ◽  
Test Score ◽  
Statistical Power ◽  
Individualized Medicine ◽  
Intervention Group ◽  
Change Score ◽  
Control Group ◽  
Absolute Change ◽  
True Model

Abstract Background Even though investigating predictors of intervention success (e.g Cognitive Training, CT) is gaining more and more interest in the light of an individualized medicine, results on specific predictors of intervention success in the overall field are mixed and inconsistent due to different and sometimes inappropriate statistical methods used. Therefore, the present paper gives a guidance on the appropriate use of multiple regression analyses to identify predictors of CT and similar non-pharmacological interventions.Methods We simulated data based on a predefined true model and ran a series of different analyses to evaluate their performance in retrieving the true model coefficients. The true model consisted of a 2 (between: experimental vs. control group) x 2 (within: pre- vs. post-treatment) design with two continuous predictors, one of which predicted the success in the intervention group and the other did not. In analyzing the data, we considered four commonly used dependent variables (post-test score, absolute change score, relative change score, residual score), five regression models, eight sample sizes, and four levels of reliability.Results Our results indicated that a regression model including the investigated predictor, Group (experimental vs. control), pre-test score, and the interaction between the investigated predictor and the Group as predictors, and the absolute change score as the dependent variable seemed most convenient for the given experimental design. Although the pre-test score should be included as a predictor in the regression model for reasons of statistical power, its coefficient should not be interpreted because even if there is no true relationship, a negative and statistically significant regression coefficient commonly emerges.Conclusion Employing simulation methods, theoretical reasoning, and mathematical derivations, we were able to derive recommendations regarding the analysis of data in one of the most prevalent experimental designs in research on CT and external predictors of CT success. These insights can contribute to the application of considered data analyses in future studies and facilitate cumulative knowledge gain.

Brief Report on the Psychophysiological Effects of a Yoga Intervention for Chronic Stress

2017 ◽  
Vol 31 (1) ◽  
pp. 38-48 ◽  
Author(s):  
Kaitlin N. Harkess ◽  
Paul Delfabbro ◽  
Jane Mortimer ◽  
Zara Hannaford ◽  
Sarah Cohen-Woods
Keyword(s):  
Allostatic Load ◽  
Small Sample Size ◽  
Well Being ◽  
Small Sample ◽  
Group Differences ◽  
Control Group ◽  
Wait List Control ◽  
Yoga Class ◽  
Yoga Group ◽  
Potential Benefits

Abstract. This paper evaluates the results of a longitudinal investigation of the potential benefits of yoga in a nonclinical sample of chronically stressed women (N = 116). Women undertook a twice weekly, hour-long yoga class for a period of 2 months, measuring psychological and physical indicators of health periodically. Changes in both areas were compared against a wait-list control group. The reported energy expenditure between groups was estimated to be similar, which suggests that the control group engaged in physical activities other than yoga. Of the six psychological outcomes measured, we found improvements in three. Specifically, those in the practicing yoga group experienced increases in positive affect, decreases in levels of distress and stress, as well as a decrease in waist circumference and increased flexibility. No between-group differences were found in mindfulness, well-being, and negative affect. These findings are generally consistent with an emerging literature, suggesting that yoga may provide both psychological and physiological effects that extend beyond its more obvious physical benefits, and are discussed in terms of the body’s allostatic load. These results should be considered in light of this study’s limitations, which include its small sample size, lack of an “active” control group, and female-only participants.

Sign in / Sign up

Export Citation Format

Share Document