Effect Size Standard Errors for the Non-Normal Non-Identically Distributed Case

1986 ◽  
Vol 11 (4) ◽  
pp. 293 ◽  
Author(s):  
Hoben Thomas
Keyword(s):  
Effect Size ◽  
Standard Errors ◽  
Size Standard

Effect Size Standard Errors for the Non-Normal Non-Identically Distributed Case

1986 ◽  
Vol 11 (4) ◽  
pp. 293-303 ◽  
Author(s):  
Hoben Thomas
Keyword(s):  
Effect Size ◽  
Standard Error ◽  
Standard Errors ◽  
Control Group ◽  
Control Groups ◽  
Minimal Sample ◽  
Sample Statistic ◽  
Size Standard ◽  
Mean Differences ◽  
And Control

Suppose there are k independent studies and for each study the experimental and control groups have been sampled from independent but essentially arbitrary populations. The problem is to construct a plausible standard error of the effect size mean (effect sizes are standardized experimental-control group mean differences) when given only minimal sample statistic information. Standard errors based on the sample standard error, or bootstrap, will typically be much too large and have very large variance. A normal theory estimator may prove practically useful in more general settings. Asymptotic distribution-free estimators are provided for two cases.

scholarly journals The effects of misspecifying the random part of multilevel models

Methodology ◽  
2020 ◽  
Vol 16 (3) ◽  
pp. 224-240
Author(s):  
David M. LaHuis ◽  
Daniel R. Jenkins ◽  
Michael J. Hartman ◽  
Shotaro Hakoyama ◽  
Patrick C. Clark
Keyword(s):  
Effect Size ◽  
Fixed Effects ◽  
Multilevel Model ◽  
Multilevel Models ◽  
The Other ◽  
Standard Errors ◽  
Moderate Effect Size ◽  
Moderate Effect ◽  
Level 1 ◽  
Random Part

This paper examined the amount bias in standard errors for fixed effects when the random part of a multilevel model is misspecified. Study 1 examined the effects of misspecification for a model with one Level 1 predictor. Results indicated that misspecifying random slope variance as fixed had a moderate effect size on the standard errors of the fixed effects and had a greater effect than misspecifying fixed slopes as random. In Study 2, a second Level 1 predictor was added and allowed for the examination of the effects of misspecifying the slope variance of one predictor on the standard errors for the fixed effects of the other predictor. Results indicated that only the standard errors of coefficient relevant to that predictor were impacted and that the effect size for the bias could be considered moderate to large. These results suggest that researchers can use a piecemeal approach to testing multilevel models with random effects.

Meta-Analysis: Literature Synthesis or Effect-Size Surface Estimation?

1992 ◽  
Vol 17 (4) ◽  
pp. 363-374 ◽  
Author(s):  
Donald B. Rubin
Keyword(s):  
Effect Size ◽  
Meta Analysis ◽  
Effect Sizes ◽  
Standard Errors ◽  
Statistical Techniques ◽  
Average Effect ◽  
Statistical Tools ◽  
Surface Estimation ◽  
Literature Synthesis ◽  
Relevant Factors

A traditional meta-analysis can be thought of as a literature synthesis, in which a collection of observed studies is analyzed to obtain summary judgments about overall significance and size of effects. Many aspects of the current set of statistical tools for meta-analysis are highly useful—for example, the development of clear and concise effect-size indicators with associated standard errors. I am less happy, however, with more esoteric statistical techniques and their implied objects of estimation (i.e., their estimands) which are tied to the conceptualization of average effect sizes, weighted or otherwise, in a population of studies. In contrast to these average effect sizes of literature synthesis, I believe that the proper estimand is an effect-size surface, which is a function only of scientifically relevant factors, and which can only be estimated by extrapolating a response surface of observed effect sizes to a region of ideal studies. This effect-size surface perspective is presented and contrasted with the literature synthesis perspective. The presentation is entirely conceptual. Moreover, it is designed to be provocative, thereby prodding researchers to rethink traditional meta-analysis and ideally stimulating meta-analysts to attempt effect-surface estimations.

Evaluation von Gesundheitsförderungsprogrammen - Methodische Stolpersteine und pragmatische Empfehlungen

2000 ◽  
Vol 8 (1) ◽  
pp. 18-24 ◽  
Author(s):  
Gert Kaluza ◽  
Hans-Henning Schulze
Keyword(s):  
Effect Size ◽  
Power Analysis ◽  
Prävention Und Gesundheitsförderung ◽  
Size Display ◽  
Methodische Probleme

Zusammenfassung. Die Evaluation von Interventionen zur Prävention und Gesundheitsförderung stellt ein zentrales Aufgabenfeld der gesundheitspsychologischen Forschung dar. Häufige methodische Probleme entsprechender Evaluationsstudien betreffen 1. Ausgangswert-Unterschiede bei nicht randomisierten Studiendesigns, 2. Abhängigkeit von Beobachtungen bei Gruppeninterventionsstudien, 3. Kapitalisierung von Irrtumswahrscheinlichkeiten aufgrund einer Vielzahl von abhängigen Variablen und 4. Beurteilung der praktischen Relevanz statistisch signifikanter Interventionseffekte. Zu deren pragmatischer Lösung werden u.a. 1. die Anwendung kovarianzanalytischer Auswertungsstrategien, 2. die Berechnung von Intraclass-Korrelationen und ggf. eine Datenauswertung auf der Ebene der Gruppenmittelwerte, 3. eine Reduktion der Anzahl der abhängigen Variablen mittels Hauptkomponentenanalyse sowie eine Alpha-Adjustierung unter Berücksichtigung der Teststärke (“compromise power analysis”) und 4. die Umrechnung gängiger Effektstärken in prozentuale Erfolgsraten (“binomial effect size display”) empfohlen.

The Level of Reaction Time Determines the ERP Correlates of Auditory Negative Priming

2006 ◽  
Vol 20 (3) ◽  
pp. 186-194 ◽  
Author(s):  
Susanne Mayr ◽  
Michael Niedeggen ◽  
Axel Buchner ◽  
Guido Orgs
Keyword(s):  
Reaction Time ◽  
Negative Priming ◽  
Effect Size ◽  
Priming Effect ◽  
Reaction Times ◽  
Negative Priming Effect ◽  
Episodic Retrieval ◽  
Time Level ◽  
Fast Reactions ◽  
Priming Condition

Responding to a stimulus that had to be ignored previously is usually slowed-down (negative priming effect). This study investigates the reaction time and ERP effects of the negative priming phenomenon in the auditory domain. Thirty participants had to categorize sounds as musical instruments or animal voices. Reaction times were slowed-down in the negative priming condition relative to two control conditions. This effect was stronger for slow reactions (above intraindividual median) than for fast reactions (below intraindividual median). ERP analysis revealed a parietally located negativity of the negative priming condition compared to the control conditions between 550-730 ms poststimulus. This replicates the findings of Mayr, Niedeggen, Buchner, and Pietrowsky (2003) . The ERP correlate was more pronounced for slow trials (above intraindividual median) than for fast trials (below intraindividual median). The dependency of the negative priming effect size on the reaction time level found in the reaction time analysis as well as in the ERP analysis is consistent with both the inhibition as well as the episodic retrieval account of negative priming. A methodological artifact explanation of this effect-size dependency is discussed and discarded.

Clinically Meaningful Change

Methodology ◽  
2019 ◽  
Vol 15 (3) ◽  
pp. 97-105
Author(s):  
Rodrigo Ferrer ◽  
Antonio Pardo
Keyword(s):  
Effect Size ◽  
False Negative ◽  
False Negative Rate ◽  
Point Of View ◽  
Skewed Distribution ◽  
Effect Sizes ◽  
False Negatives ◽  
Large Size ◽  
Before And After ◽  
Post Test

Abstract. In a recent paper, Ferrer and Pardo (2014) tested several distribution-based methods designed to assess when test scores obtained before and after an intervention reflect a statistically reliable change. However, we still do not know how these methods perform from the point of view of false negatives. For this purpose, we have simulated change scenarios (different effect sizes in a pre-post-test design) with distributions of different shapes and with different sample sizes. For each simulated scenario, we generated 1,000 samples. In each sample, we recorded the false-negative rate of the five distribution-based methods with the best performance from the point of view of the false positives. Our results have revealed unacceptable rates of false negatives even with effects of very large size, starting from 31.8% in an optimistic scenario (effect size of 2.0 and a normal distribution) to 99.9% in the worst scenario (effect size of 0.2 and a highly skewed distribution). Therefore, our results suggest that the widely used distribution-based methods must be applied with caution in a clinical context, because they need huge effect sizes to detect a true change. However, we made some considerations regarding the effect size and the cut-off points commonly used which allow us to be more precise in our estimates.

How to Detect Publication Bias in Psychological Research

2019 ◽  
Vol 227 (4) ◽  
pp. 261-279 ◽  
Author(s):  
Frank Renkewitz ◽  
Melanie Keiner
Keyword(s):  
Publication Bias ◽  
Effect Size ◽  
Statistical Power ◽  
Type I Error ◽  
Psychological Research ◽  
Type I ◽  
True Effect Size ◽  
Questionable Research Practices ◽  
True Effect ◽  
Meta Analyses

Abstract. Publication biases and questionable research practices are assumed to be two of the main causes of low replication rates. Both of these problems lead to severely inflated effect size estimates in meta-analyses. Methodologists have proposed a number of statistical tools to detect such bias in meta-analytic results. We present an evaluation of the performance of six of these tools. To assess the Type I error rate and the statistical power of these methods, we simulated a large variety of literatures that differed with regard to true effect size, heterogeneity, number of available primary studies, and sample sizes of these primary studies; furthermore, simulated studies were subjected to different degrees of publication bias. Our results show that across all simulated conditions, no method consistently outperformed the others. Additionally, all methods performed poorly when true effect sizes were heterogeneous or primary studies had a small chance of being published, irrespective of their results. This suggests that in many actual meta-analyses in psychology, bias will remain undiscovered no matter which detection method is used.

The Relationship Between Immorality and Cleansing

2018 ◽  
Vol 49 (5) ◽  
pp. 303-309 ◽  
Author(s):  
Jedidiah Siev ◽  
Shelby E. Zuckerman ◽  
Joseph J. Siev
Keyword(s):  
Effect Size ◽  
Meta Analysis ◽  
Weighted Mean ◽  
The Relationship

Abstract. In a widely publicized set of studies, participants who were primed to consider unethical events preferred cleansing products more than did those primed with ethical events ( Zhong & Liljenquist, 2006 ). This tendency to respond to moral threat with physical cleansing is known as the Macbeth Effect. Several subsequent efforts, however, did not replicate this relationship. The present manuscript reports the results of a meta-analysis of 15 studies testing this relationship. The weighted mean effect size was small across all studies (g = 0.17, 95% CI [0.04, 0.31]), and nonsignificant across studies conducted in independent laboratories (g = 0.07, 95% CI [−0.04, 0.19]). We conclude that there is little evidence for an overall Macbeth Effect; however, there may be a Macbeth Effect under certain conditions.

Sign in / Sign up

Export Citation Format

Share Document