A Comparison of Published Effect Sizes in Communication Research: Evaluation Against Population Effects and Cohen’s Conventions
Interpretations of effect size are typically made either through comparison against previous studies or established benchmarks. This study examines the distribution of published effects among studies with and without preregistration in a set of 22 communication journals. Building from previous research in psychological science, 440 effects were randomly drawn from past publications without preregistration and compared against 35 effects from preregistered studies, and against Cohen’s conventions for effect size. Reported effects from studies without preregistration (median r = .33) were larger compared to those with a preregistration plan (median r = 0.24). The magnitude of effects from studies without preregistration was greater across conventions for “small,” and “large” effects. Differences were also found based on communication subdiscipline. These findings suggest that studies without preregistration may overestimate population effects, and that global conventions may not be applicable in communication science.