scholarly journals Interpreting Average Effect Sizes: Never a Center Without a Spread

2019 ◽  
Vol 103 (4) ◽  
pp. 273-280
Author(s):  
Thomas R. Guskey
Keyword(s):  
Effect Size ◽  
School Leaders ◽  
Effect Sizes ◽  
Average Effect ◽  
Average Effect Size ◽  
Education Leaders ◽  
Meta Analyses

School leaders today are making important decisions regarding education innovations based on published average effect sizes, even though few understand exactly how effect sizes are calculated or what they mean. This article explains how average effect sizes are determined in meta-analyses and the importance of including measures of variability with any average effect size. By considering the variation in effect sizes among studies of the same innovation, education leaders can make better decisions about innovations and greatly increase the likelihood of achieving optimal results from implementation.

scholarly journals Effect Sizes, Power, and Biases in Intelligence Research: A Meta-Meta-Analysis

2018 ◽  
Author(s):  
Michele B. Nuijten ◽  
Marcel A. L. M. van Assen ◽  
Hilde Augusteijn ◽  
Elise Anne Victoire Crompvoets ◽  
Jelte M. Wicherts
Keyword(s):  
Publication Bias ◽  
Effect Size ◽  
Meta Analysis ◽  
Convincing Evidence ◽  
Small Study ◽  
Effect Sizes ◽  
Medium Effect ◽  
Average Effect ◽  
Average Effect Size ◽  
Meta Analyses

In this meta-study, we analyzed 2,442 effect sizes from 131 meta-analyses in intelligence research, published from 1984 to 2014, to estimate the average effect size, median power, and evidence for bias. We found that the average effect size in intelligence research was a Pearson’s correlation of .26, and the median sample size was 60. Furthermore, across primary studies, we found a median power of 11.9% to detect a small effect, 54.5% to detect a medium effect, and 93.9% to detect a large effect. We documented differences in average effect size and median estimated power between different types of in intelligence studies (correlational studies, studies of group differences, experiments, toxicology, and behavior genetics). On average, across all meta-analyses (but not in every meta-analysis), we found evidence for small study effects, potentially indicating publication bias and overestimated effects. We found no differences in small study effects between different study types. We also found no convincing evidence for the decline effect, US effect, or citation bias across meta-analyses. We conclude that intelligence research does show signs of low power and publication bias, but that these problems seem less severe than in many other scientific fields.

scholarly journals Effect Sizes, Power, and Biases in Intelligence Research: A Meta-Meta-Analysis

2020 ◽  
Vol 8 (4) ◽  
pp. 36
Author(s):  
Michèle B. Nuijten ◽  
Marcel A. L. M. van Assen ◽  
Hilde E. M. Augusteijn ◽  
Elise A. V. Crompvoets ◽  
Jelte M. Wicherts
Keyword(s):  
Publication Bias ◽  
Effect Size ◽  
Meta Analysis ◽  
Convincing Evidence ◽  
Small Study ◽  
Effect Sizes ◽  
Medium Effect ◽  
Average Effect ◽  
Average Effect Size ◽  
Meta Analyses

In this meta-study, we analyzed 2442 effect sizes from 131 meta-analyses in intelligence research, published from 1984 to 2014, to estimate the average effect size, median power, and evidence for bias. We found that the average effect size in intelligence research was a Pearson’s correlation of 0.26, and the median sample size was 60. Furthermore, across primary studies, we found a median power of 11.9% to detect a small effect, 54.5% to detect a medium effect, and 93.9% to detect a large effect. We documented differences in average effect size and median estimated power between different types of intelligence studies (correlational studies, studies of group differences, experiments, toxicology, and behavior genetics). On average, across all meta-analyses (but not in every meta-analysis), we found evidence for small-study effects, potentially indicating publication bias and overestimated effects. We found no differences in small-study effects between different study types. We also found no convincing evidence for the decline effect, US effect, or citation bias across meta-analyses. We concluded that intelligence research does show signs of low power and publication bias, but that these problems seem less severe than in many other scientific fields.

scholarly journals A Meta–Analytic and Conceptual Update on the Associations between Procrastination and Multidimensional Perfectionism

2017 ◽  
Vol 31 (2) ◽  
pp. 137-159 ◽  
Author(s):  
Fuschia M. Sirois ◽  
Danielle S. Molnar ◽  
Jameson K. Hirsch ◽  
Mitja Back
Keyword(s):  
Effect Size ◽  
Meta Analysis ◽  
Self Regulation ◽  
Average Effect ◽  
Personality Psychology ◽  
Average Effect Size ◽  
Partial Correlations ◽  
Unidimensional Construct ◽  
Meta Analyses ◽  
Behavioural Dynamics

The equivocal and debated findings from a 2007 meta–analysis, which viewed perfectionism as a unidimensional construct, suggested that perfectionism was unrelated to procrastination. The present meta–analysis aimed to provide a conceptual update and reanalysis of the procrastination–perfectionism association guided by both a multidimensional view of perfectionism and self–regulation theory. The random–effects meta–analyses revealed a small to medium positive average effect size ( r = .23; k = 43, N = 10 000; 95% confidence interval (95% CI) [0.19, 0.27]) for trait procrastination and perfectionistic concerns and a small to medium negative average effect size ( r = −.22; k = 38, N = 9544; 95% CI [−0.26, −0.18]) for procrastination and perfectionistic strivings. The average correlations remained significant after statistically accounting for the joint variance between the two perfectionism dimensions via semi–partial correlations. For perfectionistic concerns, but not perfectionistic strivings, the effects depended on the perfectionism measure used. All effects did not vary by the trait procrastination measure used or the respondent's sex. Our findings confirm that from a multidimensional perspective, trait procrastination is both positively and negatively associated with higher–order perfectionism dimensions and further highlights the value of a self–regulation perspective for understanding the cognitive, affective and behavioural dynamics that characterise these traits. Copyright © 2017 European Association of Personality Psychology

scholarly journals The Association of Cognitive Ability with Right–Wing Ideological Attitudes and Prejudice: A Meta–Analytic Review

2015 ◽  
Vol 29 (6) ◽  
pp. 599-621 ◽  
Author(s):  
Emma Onraet ◽  
Alain Van Hiel ◽  
Kristof Dhont ◽  
Gordon Hodson ◽  
Mark Schittekatte ◽  
...  
Keyword(s):  
Cognitive Ability ◽  
Effect Size ◽  
Model Building ◽  
Effect Sizes ◽  
Right Wing ◽  
Average Effect ◽  
Average Effect Size ◽  
Analytic Review ◽  
Ideological Attitudes ◽  
The Relationship

The cognitive functioning of individuals with stronger endorsement of right–wing and prejudiced attitudes has elicited much scholarly interest. Whereas many studies investigated cognitive styles, less attention has been directed towards cognitive ability. Studies investigating the latter topic generally reveal lower cognitive ability to be associated with stronger endorsement of right–wing ideological attitudes and greater prejudice. However, this relationship has remained widely unrecognized in literature. The present meta–analyses revealed an average effect size of r = −. 20 [95% confidence interval (95% CI) [−0.23, −0.17]; based on 67 studies, N = 84 017] for the relationship between cognitive ability and right–wing ideological attitudes and an average effect size of r = −.19 (95% CI [−0.23, −0.16]; based on 23 studies, N = 27 011) for the relationship between cognitive ability and prejudice. Effect sizes did not vary significantly across different cognitive abilities and sample characteristics. The effect strongly depended on the measure used for ideological attitudes and prejudice, with the strongest effect sizes for authoritarianism and ethnocentrism. We conclude that cognitive ability is an important factor in the genesis of ideological attitudes and prejudice and thus should become more central in theorizing and model building. Copyright © 2015 European Association of Personality Psychology

scholarly journals Postprint - Heterogeneity in direct replications in psychology and its association with effect size

2020 ◽  
Author(s):  
Anton Olsson-Collentine ◽  
Marcel A. L. M. van Assen ◽  
Jelte M. Wicherts
Keyword(s):  
Cognitive Psychology ◽  
Effect Size ◽  
Effect Sizes ◽  
Sample Population ◽  
Significant Heterogeneity ◽  
Average Effect ◽  
True Effect Size ◽  
Replication Studies ◽  
True Effect ◽  
Meta Analyses

We examined the evidence for heterogeneity (of effect sizes) when only minor changes to sample population and settings were made between studies and explored the association between heterogeneity and average effect size in a sample of 68 meta-analyses from thirteen pre-registered multi-lab direct replication projects in social and cognitive psychology. Amongst the many examined effects, examples include the Stroop effect, the “verbal overshadowing” effect, and various priming effects such as “anchoring” effects. We found limited heterogeneity; 48/68 (71%) meta-analyses had non-significant heterogeneity, and most (49/68; 72%) were most likely to have zero to small heterogeneity. Power to detect small heterogeneity (as defined by Higgins, 2003) was low for all projects (mean 43%), but good to excellent for medium and large heterogeneity. Our findings thus show little evidence of widespread heterogeneity in direct replication studies in social and cognitive psychology, suggesting that minor changes in sample population and settings are unlikely to affect research outcomes in these fields of psychology. We also found strong correlations between observed average effect sizes (standardized mean differences and log odds ratios) and heterogeneity in our sample. Our results suggest that heterogeneity and moderation of effects is unlikely for a zero average true effect size, but increasingly likely for larger average true effect size.

The perceived conditions for living well: Positive perceptions of primary goods linked with basic psychological needs and wellness

2019 ◽  
Author(s):  
Emma L Bradshaw ◽  
Cody R. DeHaan ◽  
Philip Parker ◽  
Randall Curren ◽  
Jasper Duineveld ◽  
...  
Keyword(s):  
Effect Size ◽  
Well Being ◽  
The United States ◽  
Psychological Needs ◽  
The Philippines ◽  
Need Satisfaction ◽  
Basic Psychological Needs ◽  
Average Effect ◽  
Average Effect Size ◽  
Primary Goods

We integrate Rawls’ (1971/2009, 1993, 2001) concept of primary goods with self-determination theory (SDT; Ryan & Deci, 2017), to examine the link between people’s perceptions of primary goods (i.e., views of society as just and fair), basic psychological need satisfaction, and well-being. In Study 1 (N=762, countries = Australia, the United States, South Africa, India, and the Philippines) and Study 2 (N=1479, groups = ethnic minority, sexual minority, political group, religious group), we used partial least squares structural equation modelling (PLS-SEM) to assess associations between perceptions of primary goods and wellness, and the intermediary role of basic psychological needs. Perceptions of primary goods linked positively to well-being (average effect size = 0.48), and negatively to ill-being (average effect size = -0.46). These associations were strongly mediated by basic psychological needs (average percentage mediated: 53% Study 1 and 68% Study 2). Results signify the importance of primary goods’ perceptions to wellness.

scholarly journals Cross-linguistic influence in simultaneous and early sequential bilingual children: a meta-analysis

2021 ◽  
pp. 1-33
Author(s):  
Chantal VAN DIJK ◽  
Elise VAN WONDEREN ◽  
Elly KOUTAMANIS ◽  
Gerrit Jan KOOTSTRA ◽  
Ton DIJKSTRA ◽  
...  
Keyword(s):  
Effect Size ◽  
Meta Analysis ◽  
Experimental Studies ◽  
Uniform Method ◽  
Bilingual Children ◽  
Average Effect ◽  
Language Dominance ◽  
Average Effect Size ◽  
Comprehensive Theory ◽  
Bilingual Development

Abstract Although cross-linguistic influence at the level of morphosyntax is one of the most intensively studied topics in child bilingualism, the circumstances under which it occurs remain unclear. In this meta-analysis, we measured the effect size of cross-linguistic influence and systematically assessed its predictors in 750 simultaneous and early sequential bilingual children in 17 unique language combinations across 26 experimental studies. We found a significant small to moderate average effect size of cross-linguistic influence, indicating that cross-linguistic influence is part and parcel of bilingual development. Language dominance, operationalized as societal language, was a significant predictor of cross-linguistic influence, whereas surface overlap, language domain and age were not. Perhaps an even more important finding was that definitions and operationalisations of cross-linguistic influence and its predictors varied considerably between studies. This could explain the absence of a comprehensive theory in the field. To solve this issue, we argue for a more uniform method of studying cross-linguistic influence.

scholarly journals A Statistical Method for Synthesizing Meta-Analyses

2013 ◽  
Vol 2013 ◽  
pp. 1-9 ◽  
Author(s):  
Liansheng Larry Tang ◽  
Michael Caudy ◽  
Faye Taxman
Keyword(s):  
Statistical Method ◽  
Effect Size ◽  
Meta Analysis ◽  
Effect Sizes ◽  
Weighted Mean ◽  
Summary Effect ◽  
Different Types ◽  
Meta Analyses ◽  
Size Estimates ◽  
Similar Search

Multiple meta-analyses may use similar search criteria and focus on the same topic of interest, but they may yield different or sometimes discordant results. The lack of statistical methods for synthesizing these findings makes it challenging to properly interpret the results from multiple meta-analyses, especially when their results are conflicting. In this paper, we first introduce a method to synthesize the meta-analytic results when multiple meta-analyses use the same type of summary effect estimates. When meta-analyses use different types of effect sizes, the meta-analysis results cannot be directly combined. We propose a two-step frequentist procedure to first convert the effect size estimates to the same metric and then summarize them with a weighted mean estimate. Our proposed method offers several advantages over existing methods by Hemming et al. (2012). First, different types of summary effect sizes are considered. Second, our method provides the same overall effect size as conducting a meta-analysis on all individual studies from multiple meta-analyses. We illustrate the application of the proposed methods in two examples and discuss their implications for the field of meta-analysis.

scholarly journals Morpho-textural changes of the patellar tendon in professional volleyball players: a six month longitudinal study

2019 ◽  
Vol 02 (02) ◽  
pp. 126-126
Author(s):  
de Groot Ferrando A.
Keyword(s):  
Patellar Tendon ◽  
Effect Size ◽  
Repeated Measures ◽  
Structural Changes ◽  
Patellar Tendinopathy ◽  
Average Effect ◽  
Average Effect Size ◽  
Significant Difference ◽  
Volleyball Players ◽  
Textural Changes

Abstract Introduction Tendinopathy is a term used to describe a clinical presentation of pain and dysfunction, which, at times, is accompanied by pathologic structural changes in the tendon matrix. Jumper's knee has been studied for some time, however the etiology is still unknown. The deep fibers of the proximal region are the most affected area in tendinopathies affecting the patellar tendon. Ultrasound (US) scanning may have the greatest sensitivity for confirming clinically diagnosed patellar tendinopathy compared with other imaging techniques such as magnetic resonance. Some studies have found that a minimum of six months is required to observe significant structural changes, although other studies reveal that structural changes can be visible in a shorter period of time. Thus, sonography has been used to evaluate quantitative variables such as thickness, echogenicity, texture measures and the internal tendon matrix. Objective The purpose of this study was to evaluate the sonographic behavior of morpho-textural changes of the patellar tendon in professional male volleyball players over a one year season. Material and Methods This is an observational, longitudinal and analytical study. A sample of 66 patellar tendons from 33 volleyball players of the Spanish super league was recruited, with an average of: 28 years, 86kg and 192cm. An ultrasound scan was performed using the Sonosite Titan ultrasound system, L38, 5–10 MHz, with a cross section of 5mm from the lower pole of the patella. Two depths were analyzed: a.39mm: echogenicity, echogenicity variation, circularity, width and thickness. b.24mm: entropy and contrast. Statistical analysis and analysis of variance (ANOVA) was performed for repeated measures and, in addition, the effect size was calculated with Cohen's d. The confidence interval was 95%. SPSS Statistics software was used. Results For textural variables: contrast increased significantly with an average effect size, and entropy showed no significant difference. Circularity was significantly lower. Width was shown to be significantly greater, with an average effect size, while thickness was reduced significantly. Regarding echogenicity, no significant changes were detected and variation in echogenicity did not show significant changes. Conclusions Imaging analysis could be a potential tool to detect changes in the pattern of patellar tendons. Some morpho-textural changes were found in male volleyball players: the textural contrast variable increases significantly throughout the annual season, meaning that the tendon becomes slightly flatter and loses circularity in its transverse section. The entropy and echogenic parameters obtained from the gray-scale histogram did not reveal any significant changes.

scholarly journals Effects of age and speed on the ankle–foot system’s power during walking

2020 ◽  
Vol 10 (1) ◽  
Author(s):  
Lucas Santana da Silva ◽  
Reginaldo Kisho Fukuchi ◽  
Renato Naville Watanabe ◽  
Claudiane Arakaki Fukuchi ◽  
Marcos Duarte
Keyword(s):  
Effect Size ◽  
Gait Speed ◽  
Confounding Factor ◽  
Mechanical Power ◽  
Older Individuals ◽  
Average Effect ◽  
Average Effect Size ◽  
Negative Work ◽  
Age Related ◽  
Ankle And Foot

Abstract Structural and functional changes in the foot have been associated with age-related changes in gait mechanics, but walking speed may be a confounding factor in this relationship. The aim of this study was to investigate the effect of aging and speed on the ankle–foot power output during level walking. The effects of speed and aging on features of the mechanical power and work of the ankle and foot were quantified with a gait analysis of 24 young and 16 older individuals walking at different speeds. We observed gait speed having a significant effect on all the investigated features: peak power and positive and negative work of the ankle, foot, and sum of the ankle and foot (average effect size: 0.64 ± 0.22, from 0.26 to 0.87). We observed age having no effect on these same features (average effect size: 0.23 ± 0.12, from 0.03 to 0.39), with the exception of age’s effect when combined with speed on the negative work of the foot. We performed additional analysis to illustrate how the speed can become a confounding factor to the understanding of the age effect on the gait biomechanics. Based on the influence of gait speed on the mechanical power of the ankle–foot system, it is essential that studies control for the effect of gait speed if there is interest in understanding age-related effects, particularly when studying frail older individuals.

Sign in / Sign up

Export Citation Format

Share Document