Effect size: a statistical basis for clinical practice

OBJECTIVE: Effect size (ES) is the statistical measure which quantifies the strength of a phenomenon and is commonly applied to observational and interventional studies. The aim of this review was to describe the conceptual basis of this measure, including its application, calculation and interpretation.RESULTS: As well as being used to detect the magnitude of the difference between groups, to verify the strength of association between predictor and outcome variables, to calculate sample size and power, ES is also used in meta-analysis. ES formulas can be divided into these categories: I – Difference between groups, II – Strength of association, III – Risk estimation, and IV – Multivariate data. The d value was originally considered small (0.20 > d ≤ 0.49), medium (0.50 > d≤ 0.79) or large (d ≥ 0.80); however, these cut-off limits are not consensual and could be contextualized according to a specific field of knowledge. In general, a larger score implies that a larger difference was detected.CONCLUSION: The ES report, in conjunction with the confidence interval and P value, aims to strengthen interpretation and prevent the misinterpretation of data, and thus leads to clinical decisions being based on scientific evidence studies.

Download Full-text

Use of Effect Size Measures along with p-Value in Scientific Publications

Borneo Epidemiology Journal ◽

10.51200/bej.v2i2.3629 ◽

2021 ◽

Vol 2 (2) ◽

pp. 89-97

Author(s):

Sandheep Sugathan ◽

Lilli Jacob

Keyword(s):

Effect Size ◽

Meta Analysis ◽

Research Articles ◽

Outcome Variable ◽

P Value ◽

Continuous Variables ◽

Statistical Concept ◽

Scientific Publications ◽

Actual Difference ◽

The Difference

Background: To describe various measures for estimation of effect size, how it can be calculated and the scenarios in which each measures of effect size can be applied. Methods: The researchers can display the effect size measures in research articles which evaluate the difference between the means of continuous variables in different groups or the difference in proportions of outcomes in different groups of individuals. When p-value alone is displayed in a research article, without mentioning the effect size, reader may not get the correct pictures regarding the effect or role of independent variable on the outcome variable. Results: Effect size is a statistical concept that measures the actual difference between the groups or the strength of the relationship between two variables on a numeric scale. Conclusion: Effect size measures in scientific publications can communicate the actual difference between groups or the estimate of association between the variables, not just if the association or difference is statistically significant. The researchers can make their findings more interpretable, by displaying a suitable measure of effect size. Effect size measure can help the researchers to do meta-analysis by combining the data from multiple research articles.

Download Full-text

Effect Size

Psychology ◽

10.1093/obo/9780199828340-0247 ◽

2019 ◽

Author(s):

David B. Flora

Keyword(s):

Effect Size ◽

Null Hypothesis ◽

Power Analysis ◽

Meta Analysis ◽

Significance Test ◽

Effect Sizes ◽

Research Reporting ◽

The Difference ◽

Strength Of Association ◽

True Population

Simply put, effect size (ES) is the magnitude or strength of association between or among variables. Effect sizes (ESs) are commonly represented numerically (i.e., as parameters for population ESs and statistics for sample estimates of population ESs) but also may be communicated graphically. Although the word “effect” may imply that an ES quantifies the strength of a causal association (“cause and effect”), ESs are used more broadly to represent any empirical association between variables. Effect sizes serve three general purposes: research results reporting, power analysis, and meta-analysis. Even under the same research design, an ES that is appropriate for one of these purposes may not be ideal for another. Effect size can be conveyed graphically or numerically using either unstandardized metrics, which are interpreted relative to the original scales of the variables involved (e.g., the difference between two means or an unstandardized regression slope), or standardized metrics, which are interpreted in relative terms (e.g., Cohen’s d or multiple R2). Whereas unstandardized ESs and graphs illustrating ES are typically most effective for research reporting, that is, communicating the original findings of an empirical study, many standardized ES measures have been developed for use in power analysis and especially meta-analysis. Although the concept of ES is clearly fundamental to data analysis, ES reporting has been advocated as an essential complement to null hypothesis significance testing (NHST), or even as a replacement for NHST. A null hypothesis significance test involves making a dichotomous judgment about whether to reject a hypothesis that a true population effect equals zero. Even in the context of a traditional NHST paradigm, ES is a critical concept because of its central role in power analysis.

Download Full-text

Effectiveness of a Pharmacogenetic Tool at Improving Treatment Efficacy in Major Depressive Disorder: A Meta-Analysis of Three Clinical Studies

Pharmaceutics ◽

10.3390/pharmaceutics11090453 ◽

2019 ◽

Vol 11 (9) ◽

pp. 453 ◽

Cited By ~ 2

Author(s):

Vilches ◽

Tuson ◽

Vieta ◽

Álvarez ◽

Espadaler

Keyword(s):

Random Effects ◽

Effect Size ◽

Clinical Studies ◽

Clinical Utility ◽

Meta Analysis ◽

Severe Depression ◽

P Value ◽

Drug Selection ◽

Mild Depression ◽

Depressive Patients

Several pharmacogenetic tests to support drug selection in psychiatric patients have recently become available. The current meta-analysis aimed to assess the clinical utility of a commercial pharmacogenetic-based tool for psychiatry (Neuropharmagen®) in the treatment management of depressive patients. Random-effects meta-analysis of clinical studies that had examined the effect of this tool on the improvement of depressive patients was performed. Effects were summarized as standardized differences between treatment groups. A total of 450 eligible subjects from three clinical studies were examined. The random effects model estimated a statistically significant effect size for the pharmacogenetic-guided prescription (d = 0.34, 95% CI = 0.11–0.56, p-value = 0.004), which corresponded to approximately a 1.8-fold increase in the odds of clinical response for pharmacogenetic-guided vs. unguided drug selection. After exclusion of patients with mild depression, the pooled estimated effect size increased to 0.42 (95% CI = 0.19–0.65, p-value = 0.004, n = 287), corresponding to an OR = 2.14 (95% CI = 1.40–3.27). These results support the clinical utility of this pharmacogenetic-based tool in the improvement of health outcomes in patients with depression, especially those with moderate–severe depression. Additional pragmatic RCTs are warranted to consolidate these findings in other patient populations.

Download Full-text

Do animated triangles reveal a marked difficulty among autistic people with reading minds?

Autism ◽

10.1177/1362361321989152 ◽

2021 ◽

pp. 136236132198915

Author(s):

Alexander C Wilson

Keyword(s):

Effect Size ◽

Meta Analysis ◽

General Information ◽

Medium Effect ◽

Group Differences ◽

Social Challenges ◽

Autistic People ◽

The Social ◽

Small Effect Size ◽

The Difference

This meta-analysis tested whether autistic people show a marked, isolated difficulty with mentalising when assessed using the Frith -Happé Animations, an advanced test of mentalising (or ‘theory of mind’). Effect sizes were aggregated in multivariate meta-analysis from 33 papers reporting data for over 3000 autistic and non-autistic people. Relative to non-autistic individuals, autistic people underperformed, with a small effect size on the non-mentalising control conditions and a medium effect size on the mentalising condition. This indicates that studies have reliably found mentalising to be an area of challenge for autistic people, although the group differences were not large. It remains to be seen how important mentalising difficulties are in accounting for the social difficulties diagnostic of autism. As autistic people underperformed on the control conditions as well as the mentalising condition, it is likely that group differences on the test are partly due to domain-general information processing differences. Finally, there was evidence of publication bias, suggesting that true effects on the Frith -Happé Animations may be somewhat smaller than reported in the literature. Lay abstract Autistic people are thought to have difficulty with mentalising (our drive to track and understand the minds of other people). Mentalising is often measured by the Frith -Happé Animations task, where individuals need to interpret the interactions of abstract shapes. This review article collated results from over 3000 people to assess how autistic people performed on the task. Analysis showed that autistic people tended to underperform compared to non-autistic people on the task, although the scale of the difference was moderate rather than large. Also, autistic people showed some difficulty with the non-mentalising as well as mentalising aspects of the task. These results raise questions about the scale and specificity of mentalising difficulties in autism. It also remains unclear how well mentalising difficulties account for the social challenges diagnostic of autism.

Download Full-text

Effects of stimulants and atomoxetine on emotional lability in adults: A systematic review and meta-analysis

European Psychiatry ◽

10.1016/j.eurpsy.2017.05.021 ◽

2017 ◽

Vol 44 ◽

pp. 198-207 ◽

Cited By ~ 17

Author(s):

T.R. Moukhtarian ◽

R.E. Cooper ◽

E. Vassos ◽

P. Moran ◽

P. Asherson

Keyword(s):

Systematic Review ◽

Effect Size ◽

Meta Analysis ◽

Emotional Lability ◽

Adhd Symptoms ◽

Double Blind ◽

Hyperactivity Disorder ◽

Ovid Medline ◽

Weighted Effect ◽

The Difference

AbstractBackground:Emotional lability (EL) is an associated feature of attention-deficit/hyperactivity disorder (ADHD) in adults, contributing to functional impairment. Yet the effect of pharmacological treatments for ADHD on EL symptoms is unknown. We conducted a systematic review and meta-analysis to examine the effects of stimulants and atomoxetine on symptoms of EL and compare these with the effects on core ADHD symptoms.Methods:A systematic search was conducted on the databases Embase, PsychInfo, and Ovid Medline®and the clinicaltrials.gov website. We included randomised, double-blind, placebo-controlled trials of stimulants and atomoxetine in adults aged 18–60 years, with any mental health diagnosis characterised by emotional or mood instability, with at least one outcome measure of EL. All identified trials were on adults with ADHD. A random-effects meta-analysis with standardised mean difference and 95% confidence intervals was used to investigate the effect size on EL and compare this to the effect on core ADHD symptoms.Results:Of the 3,864 publications identified, nine trials met the inclusion criteria for the meta-analysis. Stimulants and atomoxetine led to large mean weighted effect-sizes for on ADHD symptoms (n= 9, SMD = −0.8, 95% CI:−1.07 to −0.53). EL outcomes showed more moderate but definite effects (n= 9, SMD = −0.41, 95% CI:−0.57 to −0.25).Conclusions:In this meta-analysis, stimulants and atomoxetine were moderately effective for EL symptoms, while effect size on core ADHD symptoms was twice as large. Methodological issues may partially explain the difference in effect size. Reduced average effect size could also reflect heterogeneity of EL with ADHD pharmacotherapy responsive and non-responsive sub-types. Our findings indicate that EL may be less responsive than ADHD symptoms overall, perhaps indicating the need for adjunctive psychotherapy in some cases. To clarify these questions, our findings need replication in studies selecting subjects for high EL and targeting EL as the primary outcome.

Download Full-text

The Association between rs2292239 Polymorphism in ERBB3 Gene and Type 1 Diabetes: A Meta-Analysis

BioMed Research International ◽

10.1155/2019/7689642 ◽

2019 ◽

Vol 2019 ◽

pp. 1-7 ◽

Cited By ~ 2

Author(s):

Dingjian Wang ◽

Guixia Pan

Keyword(s):

Type 1 Diabetes ◽

Large Scale ◽

Web Of Science ◽

Meta Analysis ◽

Cochrane Library ◽

P Value ◽

Case Control Studies ◽

Heterogeneity Test ◽

Strength Of Association

Objectives. The purpose of this study was to explore the association between rs2292239 polymorphism in ERBB3 gene and type 1 diabetes (T1D). Methods. A systematic search of studies on the association of rs2292239 polymorphism in ERBB3 gene with T1D susceptibility was conducted in PubMed, Web of science, Elsevier Science Direct, and Cochrane Library. Eventually, 9 published studies were included. The strength of association between rs2292239 polymorphism and T1D susceptibility was assessed by odds ratios (ORs) with its 95% confidence intervals (CIs). Results. A total of 9 case-control studies, consisting of 5369 T1D patients and 6920 controls, were included in the meta-analysis. This meta-analysis showed significant association between ERBB3 rs2292239 polymorphism and T1D susceptibility in overall population (A vs. C, OR: 1.292, 95% CI= 1.224-1.364, PH=0.450, PH is P value for the heterogeneity test). Similar results were found in subgroup analysis by ethnicity. Conclusions. ERBB3 rs2292239 polymorphism is associated with T1D susceptibility and rs2292239-A allele is a risk factor for T1D. However, more large-scale studies are warranted to replicate our findings.

Download Full-text

Revisiting and expanding the meta-analysis of variation: The log coefficient of variation ratio, lnCVR

10.1101/2020.01.06.896522 ◽

2020 ◽

Cited By ~ 1

Author(s):

Alistair M. Senior ◽

Wolfgang Viechtbauer ◽

Shinichi Nakagawa

Keyword(s):

Effect Size ◽

Meta Analysis ◽

Experimental Treatment ◽

Worked Examples ◽

Sampling Variance ◽

Post Test ◽

Total Variability ◽

The Difference ◽

Meta Analyses ◽

Analysis Of Variation

AbstractMeta-analyses are frequently used to quantify the difference in the average values of two groups (e.g., control and experimental treatment groups), but examine the difference in the variability (variance) of two groups. For such comparisons, the two relatively new effect size statistics, namely the log-transformed ‘variability ratio’ (the ratio of two standard deviations; lnVR) and the log-transformed ‘CV ratio’ (the ratio of two coefficients of variation; lnCVR) are useful. In practice, lnCVR may be of most use because a treatment may affect the mean and the variance simultaneously. We review current, and propose new, estimators for lnCVR and lnVR. We also present methods for use when the two groups are dependent (e.g., for cross-over and pre-test-post-test designs). A simulation study evaluated the performance of these estimators and we make recommendations about which estimators one should use to minimise bias. We also present two worked examples that illustrate the importance of accounting for the dependence of the two groups. We found that the degree to which dependence is accounted for in the sampling variance estimates can impact heterogeneity parameters such as τ2 (i.e., the between-study variance) and I2 (i.e., the proportion of the total variability due to between-study variance), and even the overall effect, and in turn qualitative interpretations. Meta-analytic comparison of the variability between two groups enables us to ask completely new questions and to gain fresh insights from existing datasets. We encourage researchers to take advantage of these convenient new effect size measures for the meta-analysis of variation.

Download Full-text

Different Spectral Analysis Methods for the Theta/Beta Ratio Calculate Different Ratios But Do Not Distinguish ADHD from Controls

Applied Psychophysiology and Biofeedback ◽

10.1007/s10484-020-09471-2 ◽

2020 ◽

Vol 45 (3) ◽

pp. 165-173

Author(s):

Hanneke van Dijk ◽

Roger deBeus ◽

Cynthia Kerson ◽

Michelle E. Roley-Roberts ◽

Vincent J. Monastra ◽

...

Keyword(s):

Signal Processing ◽

Effect Size ◽

Meta Analysis ◽

Healthy Controls ◽

Eeg Signal ◽

Ongoing Research ◽

Beta Power ◽

Signal Processing Algorithms ◽

The Difference ◽

Eeg Signal Processing

Abstract There has been ongoing research on the ratio of theta to beta power (Theta/Beta Ratio, TBR) as an EEG-based test in the diagnosis of ADHD. Earlier studies reported significant TBR differences between patients with ADHD and controls. However, a recent meta-analysis revealed a marked decline of effect size for the difference in TBR between ADHD and controls for studies published in the past decade. Here, we test if differences in EEG processing explain the heterogeneity of findings. We analyzed EEG data from two multi-center clinical studies. Five different EEG signal processing algorithms were applied to calculate the TBR. Differences between resulting TBRs were subsequently assessed for clinical usability in the iSPOT-A dataset. Although there were significant differences in the resulting TBRs, none distinguished between children with and without ADHD, and no consistent associations with ADHD symptoms arose. Different methods for EEG signal processing result in significantly different TBRs. However, none of the methods significantly distinguished between ADHD and healthy controls in our sample. The secular effect size decline for the TBR is most likely explained by factors other than differences in EEG signal processing, e.g. fewer hours of sleep in participants and differences in inclusion criteria for healthy controls.

Download Full-text

Prevalence of Olfactory Dysfunction Among COVID-19 Patients with Self-Reported Smell Loss Versus Objective Olfactory Tests: A Systematic Review and Meta-Analysis

Philippine Journal of Otolaryngology Head and Neck Surgery ◽

10.32412/pjohns.v36i1.1649 ◽

2021 ◽

Vol 36 (1) ◽

pp. 6

Author(s):

Joyce Anne Regalado ◽

Mariel Mae Tayam ◽

Romiena Santos ◽

January Gelera

Keyword(s):

Meta Analysis ◽

Small Sample ◽

Olfactory Dysfunction ◽

Contact Tracing ◽

P Value ◽

Objective Testing ◽

Significant Difference ◽

Objective Tests ◽

The Difference ◽

Smell Loss

ABSTRACTBackground: Olfactory dysfunction (OD) in COVID-19 presents as a sudden onset smell loss commonly seen in mild symptomatic cases with or without rhinitis but can occur as an isolated symptom. The reported prevalence of OD among COVID-19 patients ranged from 5% to 98%. Although numerous studies have been conducted about their association, these were mainly based on self-reported cases and subjective questionnaires. Objective: This study investigates whether there is a significant difference in the prevalence of olfactory dysfunction between self-reported and objective testing using validated objective olfactory tests among RT-PCR confirmed COVID-19 patients. Methods: PubMed (MEDLINE), Cochrane, Web of Science, and Google Scholar were searched for studies investigating the prevalence of OD by using objective olfactory tests among patients who self-reported OD (November 1, 2019 to July 31, 2020). All studies were assessed for quality and bias using the Cochrane bias tool. Patient demographics, type of objective olfactory test, and results of self-reported OD and objective testing were reported. Results: Nine studies encompassing 673 patients met the inclusion criteria. Validated objective olfactory tests used in the included studies were CCCRC, SST and SIT. Overall prevalence of OD among patients who self-reported was higher after objective testing (71% versus 81%). This was also seen in when we performed subgroup analysis based on the objective tests that were used. However, meta-analysis using random effects model showed no significant difference in the overall prevalence of OD (p value=.479, 95% CI 56.6 to 84.0 versus 71.2 to 89.8) as well as in the subgroups. Conclusion: To the best of our knowledge, this is the first meta-analysis that statistically reviewed articles that evaluated the difference between self-reported and objective tests done on the same patients. Results showing that self-reporting OD approximates the results of the objective tests among COVID-19 positive patients may imply that self-reporting can be sufficient in contact tracing and triggering swabbing and self quarantine during the time of COVID-19 and objective tests can be used as an adjunct in the diagnosis particularly in research. However, this study was limited by small sample size and articles done in European countries hence, interpretation and application of the results of this study must be approached with care. Further studies documenting the difference between self-reporting and objective test in large scale setting involving different countries may be helpful in establishing a definitive consensus.

Download Full-text

The Impact of Social Networking Sites on Body Dissatisfaction among Men and Women: a Meta-Analysis

10.31237/osf.io/vts75 ◽

2020 ◽

Author(s):

Fernando Castaneda

Keyword(s):

Body Dissatisfaction ◽

Social Networking ◽

Effect Size ◽

Social Networking Sites ◽

Body Satisfaction ◽

Meta Analysis ◽

Men And Women ◽

Gender And Age ◽

The Difference ◽

The Impact

The purpose of this study was to determine the difference in impact between men and women on the effects that social networking sites (SNS) have on body dissatisfaction. A total of eight studies (with 48 effect sizes) involving participants being assessed on SNS use frequency and body dissatisfaction in which some correlation was determined were used for this meta-analysis. The current study also chose to evaluate three different moderators: gender, age, and measurement type. Correlations from each study were collected in order to compute a single pooled effect size. The proportion of men and mean age were also collected from each study in order to assess the gender and age moderators. Types of measurements were coded either as 0 (study used a measurement specifically designed to assess body satisfaction/dissatisfaction) or 1 (study used a subscale from a larger measurement that assessed body satisfaction/dissatisfaction). The pooled effect size showed significance in the overall association between SNS use frequency and body dissatisfaction which supports the findings of previous research. However, neither of the moderators were found to be significant, ultimately rejecting the hypothesis of the current study. This finding may be due to the major limitation of the lack of research available surrounding this topic.

Download Full-text