A non-parametric effect size measure capturing changes in central tendency and shape of data distributions more flexibly than Cohen’s d

2020 ◽  
Author(s):  
Jörn Lötsch ◽  
Alfred Ultsch
Keyword(s):  
Effect Size ◽  
Data Science ◽  
Heterogeneous Data ◽  
Machine Learning Algorithms ◽  
Central Tendency ◽  
Effect Size Measure ◽  
Specific Data ◽  
Cohen’S D ◽  
Size Measure ◽  
Cohen's D

Abstract Calculating the magnitude of treatment effects or of differences between two groups is a common task in quantitative science. Standard effect size measures based on differences, such as the commonly used Cohen's, fail to capture the treatment-related effects on the data if the effects were not reflected by the central tendency. "Impact” is a novel nonparametric measure of effect size obtained as the sum of two separate components and includes (i) the change in the central tendency of the group-specific data, normalized to the overall variability, and (ii) the difference in the probability density of the group-specific data. Results obtained on artificial data and empirical biomedical data showed that impact outperforms Cohen's d by this additional component. It is shown that in a multivariate setting, while standard statistical analyses and Cohen’s d are not able to identify effects that lead to changes in the form of data distribution, “Impact” correctly captures them. The proposed effect size measure shares the ability to observe such an effect with machine learning algorithms. It is numerically stable even for degenerate distributions consisting of singular values. Therefore, the proposed effect size measure is particularly well suited for data science and artificial intelligence-based knowledge discovery from (big) and heterogeneous data.

scholarly journals Cliff´s Delta Calculator: A non-parametric effect size program for two groups of observations

2010 ◽  
Vol 10 (2) ◽  
pp. 545-555 ◽  
Author(s):  
Guillermo Macbeth ◽  
Eugenia Razumiejczyk ◽  
Rubén Daniel Ledesma
Keyword(s):  
Effect Size ◽  
Worked Examples ◽  
Visual Interpretation ◽  
Effect Size Measure ◽  
P Values ◽  
Size Measure ◽  
Parametric Case ◽  
Algorithmic Approaches ◽  
Complementary Analysis ◽  
Non Parametric

The Cliff´s Delta statistic is an effect size measure that quantifies the amount of difference between two non-parametric variables beyond p-values interpretation. This measure can be understood as a useful complementary analysis for the corresponding hypothesis testing. During the last two decades the use of effect size measures has been strongly encouraged by methodologists and leading institutions of behavioral sciences. The aim of this contribution is to introduce the Cliff´s Delta Calculator software that performs such analysis and offers some interpretation tips. Differences and similarities with the parametric case are analysed and illustrated. The implementation of this free program is fully described and compared with other calculators. Alternative algorithmic approaches are mathematically analysed and a basic linear algebra proof of its equivalence is formally presented. Two worked examples in cognitive psychology are commented. A visual interpretation of Cliff´s Delta is suggested. Availability, installation and applications of the program are presented and discussed.

scholarly journals 0636 Using AI To Predict Future CPAP Adherence and the Impact of Behavioral and Technical Interventions

SLEEP ◽  
2020 ◽  
Vol 43 (Supplement_1) ◽  
pp. A243-A243
Author(s):  
W Hevener ◽  
B Beine ◽  
J Woodruff ◽  
D Munafo ◽  
C Fernandez ◽  
...  
Keyword(s):  
Effect Size ◽  
Statistical Significance ◽  
Post Intervention ◽  
Cpap Adherence ◽  
Cohen’S D ◽  
Usage Patterns ◽  
Patient Outreach ◽  
Cohen's D ◽  
The Impact ◽  
Usage Data

Abstract Introduction Clinical management of CPAP adherence remains an ongoing challenge. Behavioral and technical interventions such as patient outreach, coaching, troubleshooting, and resupply may be deployed to positively impact adherence. Previous authors have described adherence phenotypes that retrospectively categorize patients by discrete usage patterns. We design an AI model that predictively categorizes patients into previously studied adherence phenotypes and analyzes the statistical significance and effect size of several types of interventions on subsequent CPAP adherence. Methods We collected a cross-sectional cohort of subjects (N = 13,917) with 455 days of daily CPAP usage data acquired. Patient outreach notes and resupply data were temporally synchronized with daily CPAP usage. Each 30-days of usage was categorized into one of four adherence phenotypes as defined by Aloia et al. (2008) including Good Users, Variable Users, Occasional Attempters, and Non-Users. Cross-validation was used to train and evaluate a Recurrent Neural Network model for predicting future adherence phenotypes based on the dynamics of prior usage patterns. Two-sided 95% bootstrap confidence intervals and Cohen’s d statistic were used to analyze the significance and effect size of changes in usage behavior 30-days before and after administration of several resupply interventions. Results The AI model predicted the next 30-day adherence phenotype with an average of 90% sensitivity, 96% specificity, 95% accuracy, and 0.83 Cohen’s Kappa. The AI model predicted the number of days of CPAP non-use, use under 4-hours, and use over 4-hours for the next 30-days with OLS Regression R-squared values of 0.94, 0.88, and 0.95 compared to ground truth. Ten resupply interventions were associated with statistically significant increases in adherence, and ranked by adherence effect size using Cohen’s d. The most impactful were new cushions or masks, with a mean post-intervention CPAP adherence increase of 7-14% observed in Variable User, Occasional Attempter, and Non-User groups. Conclusion The AI model applied past CPAP usage data to predict future adherence phenotypes and usage with high sensitivity and specificity. We identified resupply interventions that were associated with significant increases in adherence for struggling patients. This work demonstrates a novel application for AI to aid clinicians in maintaining CPAP adherence. Support  

An effect size measure and Bayesian analysis of single-case designs

2014 ◽  
Vol 52 (2) ◽  
pp. 213-230 ◽  
Author(s):  
Hariharan Swaminathan ◽  
H. Jane Rogers ◽  
Robert H. Horner
Keyword(s):  
Bayesian Analysis ◽  
Effect Size ◽  
Single Case ◽  
Effect Size Measure ◽  
Size Measure ◽  
Single Case Designs

scholarly journals The Unforgettables: a chorus for people with dementia with their family members and friends

2018 ◽  
Vol 30 (6) ◽  
pp. 779-789 ◽  
Author(s):  
Mary Sherman Mittelman ◽  
Panayiota Maria Papayannopoulou
Keyword(s):  
Quality Of Life ◽  
Social Support ◽  
Pilot Study ◽  
Effect Size ◽  
Family Members ◽  
Positive Effects ◽  
Cohen’S D ◽  
People With Dementia ◽  
Cohen's D

Summary/AbstractOur experience evaluating a museum program for people with dementia together with their family members demonstrated benefits for all participants. We hypothesized that participation in a chorus would also have positive effects, giving them an opportunity to share a stimulating and social activity that could improve their quality of life. We inaugurated a chorus for people with dementia and their family caregivers in 2011, which rehearses and performs regularly. Each person with dementia must be accompanied by a friend or family member and must commit to attending all rehearsals and the concert that ensues. A pilot study included a structured assessment, take home questionnaires and focus groups. Analyses of pre-post scores were conducted; effect size was quantified using Cohen's d. Results showed that quality of life and communication with the other member of the dyad improved (Effect size: Cohen's d between 0.32 and 0.72) for people with dementia; quality of life, social support, communication and self-esteem improved (d between 0.29 and 0.68) for caregivers. Most participants stated that benefits included belonging to a group, having a normal activity together and learning new skills. Participants attended rehearsals in spite of harsh weather conditions. The chorus has been rehearsing and performing together for more than 6 years and contributing to its costs. Results of this pilot study suggest that people in the early to middle stage of dementia and their family members and friends can enjoy and learn from rehearsing and performing in concerts that also engage the wider community. It is essential to conduct additional larger studies of the benefits of participating in a chorus, which may include improved quality of life and social support for all, and reduced cognitive decline among people with dementia.

scholarly journals Effect Size Guidelines, Sample Size Calculations, and Statistical Power in Gerontology

2019 ◽  
Vol 3 (4) ◽  
Author(s):  
Christopher R Brydges
Keyword(s):  
Individual Differences ◽  
Sample Size ◽  
Effect Size ◽  
Effect Sizes ◽  
Group Differences ◽  
Cohen’S D ◽  
Pearson's R ◽  
Sample Size Calculations ◽  
Cohen's D ◽  
Pearson’S R

Abstract Background and Objectives Researchers typically use Cohen’s guidelines of Pearson’s r = .10, .30, and .50, and Cohen’s d = 0.20, 0.50, and 0.80 to interpret observed effect sizes as small, medium, or large, respectively. However, these guidelines were not based on quantitative estimates and are only recommended if field-specific estimates are unknown. This study investigated the distribution of effect sizes in both individual differences research and group differences research in gerontology to provide estimates of effect sizes in the field. Research Design and Methods Effect sizes (Pearson’s r, Cohen’s d, and Hedges’ g) were extracted from meta-analyses published in 10 top-ranked gerontology journals. The 25th, 50th, and 75th percentile ranks were calculated for Pearson’s r (individual differences) and Cohen’s d or Hedges’ g (group differences) values as indicators of small, medium, and large effects. A priori power analyses were conducted for sample size calculations given the observed effect size estimates. Results Effect sizes of Pearson’s r = .12, .20, and .32 for individual differences research and Hedges’ g = 0.16, 0.38, and 0.76 for group differences research were interpreted as small, medium, and large effects in gerontology. Discussion and Implications Cohen’s guidelines appear to overestimate effect sizes in gerontology. Researchers are encouraged to use Pearson’s r = .10, .20, and .30, and Cohen’s d or Hedges’ g = 0.15, 0.40, and 0.75 to interpret small, medium, and large effects in gerontology, and recruit larger samples.

Comparing Normalized Gain g and Effect Size Cohen’s d to Measure the Improvement of Student’s Scientific Literacy

2019 ◽  
Author(s):  
Adib Rifqi Setiawan
Keyword(s):  
Education Research ◽  
Student Learning ◽  
Effect Size ◽  
Physics Education ◽  
Scientific Literacy ◽  
Biology Education ◽  
Physics Education Research ◽  
Cohen’S D ◽  
Cohen's D ◽  
Normalized Gain

In this work I investigate about my curiousity. My investigation focused on the implications on claims about student learning that result from choosing between one of two metrics. The metrics are normalized gain g, which is the most common method used in Physics Education Research (PER), and effect size Cohen’s d, which is broadly used in Discipline-Based Education Research (DBER) including Biology Education Research (BER). Data for the analyses came from the research about scientific literacy on Physics and Biology Education from courses at institutions across Indonesia. This work reveals that the bias in normalized gaing can harm efforts to improve student’s scientific literacy by misrepresenting the efficacy of teaching practices across populations of students and across institutions. This work, also, recommends use effect size Cohen’s d for measuring student learning, based on reliability statistical method for calculating student learning.

Cohen’s d and physicians’ opinion on effect size: a questionnaire on anemia treatment

2021 ◽  
pp. jim-2021-002031
Author(s):  
Kemal Hakan Gülkesen ◽  
Feyza Bora ◽  
Nevruz Ilhanli ◽  
Esin Avsar ◽  
Nese Zayim
Keyword(s):  
Effect Size ◽  
Clinical Importance ◽  
Deficiency Anemia ◽  
Duration Of Treatment ◽  
Cohen’S D ◽  
D Values ◽  
A Minor ◽  
Cohen's D ◽  
Anemia Treatment

A well-known effect size (ES) indicator is Cohen’s d. Cohen defined d measures of small, medium, and large ES as 0.2, 0.5, and 0.8, respectively. This approach has been criticized because practical and clinical importance depends on the context of research. The aim of the study was to examine physicians’ perception of ES using iron deficiency anemia treatment as an example and observing the effects of pretreatment level and duration of treatment on the magnitude of ES. We prepared a questionnaire describing four different clinical studies: (1) 1 month of treatment of anemia in a group of patients with a mean hemoglobin (Hb) of 10 g/dL; (2) 3 months of treatment at an Hb level of 10 g/dL; (3) 1 month of treatment at an Hb level of 8 g/dL; and (4) 3 months of treatment at an Hb level of 8 g/dL. In each scenario, respondents were required to evaluate six various levels of Hb improvement as being very small, small, medium, large, or very large effect: 0.1 g/dL, 0.3 g/dL, 0.7 g/dL, 1.1 g/dL, 1.7 g/dL, and 2.8 g/dL. The responses of 35 physicians were evaluated. For 10 mg/dL, the Cohen's d for small, medium, and large ES was 0.5, 0.8, and 1.2 respectively, for 1 month of treatment. In terms of 3 months of treatment, the Cohen's d was 0.8, 1.2, and 2, respectively. Two separate pretreatment Hb levels (8 g/dL and 10 g/dL) demonstrated a minor difference. Determination of ES during the planning phase of studies requires thorough evaluation of specific clinical cases. Our results are divergent from the classic Cohen’s d values. Additionally, duration of treatment affects ES perception.

Sign in / Sign up

Export Citation Format

Share Document