scholarly journals The Other Half of the Story: Effect Size Analysis in Quantitative Research

2013 ◽  
Vol 12 (3) ◽  
pp. 345-351 ◽  
Author(s):  
Jessica Middlemis Maher ◽  
Jonathan C. Markey ◽  
Diane Ebert-May
Keyword(s):  
Educational Research ◽  
Effect Size ◽  
Quantitative Research ◽  
Statistical Significance ◽  
The Other ◽  
Practical Significance ◽  
Significance Testing ◽  
Size Analysis ◽  
Significance Tests ◽  
Statistical Significance Testing

Statistical significance testing is the cornerstone of quantitative research, but studies that fail to report measures of effect size are potentially missing a robust part of the analysis. We provide a rationale for why effect size measures should be included in quantitative discipline-based education research. Examples from both biological and educational research demonstrate the utility of effect size for evaluating practical significance. We also provide details about some effect size indices that are paired with common statistical significance tests used in educational research and offer general suggestions for interpreting effect size measures. Finally, we discuss some inherent limitations of effect size measures and provide further recommendations about reporting confidence intervals.

scholarly journals Damaging Real Lives through Obstinacy: Re-Emphasising Why Significance Testing is Wrong

2016 ◽  
Vol 21 (1) ◽  
pp. 102-115 ◽  
Author(s):  
Stephen Gorard
Keyword(s):  
Statistical Significance ◽  
Poor Quality ◽  
Significance Testing ◽  
Significance Tests ◽  
Statistical Significance Testing ◽  
Wrong Answer ◽  
Repeated Calls ◽  
Numeric Data ◽  
Quality Of Research

This paper reminds readers of the absurdity of statistical significance testing, despite its continued widespread use as a supposed method for analysing numeric data. There have been complaints about the poor quality of research employing significance tests for a hundred years, and repeated calls for researchers to stop using and reporting them. There have even been attempted bans. Many thousands of papers have now been written, in all areas of research, explaining why significance tests do not work. There are too many for all to be cited here. This paper summarises the logical problems as described in over 100 of these prior pieces. It then presents a series of demonstrations showing that significance tests do not work in practice. In fact, they are more likely to produce the wrong answer than a right one. The confused use of significance testing has practical and damaging consequences for people's lives. Ending the use of significance tests is a pressing ethical issue for research. Anyone knowing the problems, as described over one hundred years, who continues to teach, use or publish significance tests is acting unethically, and knowingly risking the damage that ensues.

scholarly journals The use of effect size indices to determine practical significance

2006 ◽  
Vol 25 (3) ◽  
Author(s):  
H. S. Styn ◽  
S. M. Ellis
Keyword(s):  
Effect Size ◽  
Statistical Significance ◽  
Empirical Studies ◽  
Research Literature ◽  
Effect Sizes ◽  
Practical Significance ◽  
Significance Tests ◽  
Statistical Application ◽  
Significant Difference

The determination of significance of differences in means and of relationships between variables is of importance in many empirical studies. Usually only statistical significance is reported, which does not necessarily indicate an important (practically significant) difference or relationship. With studies based on probability samples, effect size indices should be reported in addition to statistical significance tests in order to comment on practical significance. Where complete populations or convenience samples are worked with, the determination of statistical significance is strictly speaking no longer relevant, while the effect size indices can be used as a basis to judge significance. In this article attention is paid to the use of effect size indices in order to establish practical significance. It is also shown how these indices are utilized in a few fields of statistical application and how it receives attention in statistical literature and computer packages. The use of effect sizes is illustrated by a few examples from the research literature.

If you've got an effect, test its significance; if you've got a weak effect, do a meta-analysis

1998 ◽  
Vol 21 (2) ◽  
pp. 205-206 ◽  
Author(s):  
John F. Kihlstrom
Keyword(s):  
Meta Analysis ◽  
Statistical Significance ◽  
Significance Testing ◽  
Weak Effect ◽  
Significance Tests ◽  
Statistical Significance Testing ◽  
Effect Test

Statistical significance testing has its problems, but so do the alternatives that are proposed; and the alternatives may be both more cumbersome and less informative. Significance tests remain legitimate aspects of the rhetoric of scientific persuasion.

scholarly journals The Case Against Statistical Significance Testing

1978 ◽  
Vol 48 (3) ◽  
pp. 378-399 ◽  
Author(s):  
Ronald Carver
Keyword(s):  
Educational Research ◽  
Statistical Methods ◽  
Scientific Method ◽  
Statistical Significance ◽  
Significance Testing ◽  
Statistical Significance Testing ◽  
Research Results

In recent years the use of traditional statistical methods in educational research has increasingly come under attack. In this article, Ronald P. Carver exposes the fantasies often entertained by researchers about the meaning of statistical significance. The author recommends abandoning all statistical significance testing and suggests other ways of evaluating research results. Carver concludes that we should return to the scientific method of examining data and replicating results rather than relying on statistical significance testing to provide equivalent information.

Effect Size and Effect Uncertainty in Organizational Research Methods

2021 ◽  
Author(s):  
Scott B. Morris ◽  
Arash Shokri
Keyword(s):  
Confidence Intervals ◽  
Effect Size ◽  
Sampling Error ◽  
Statistical Significance ◽  
Scientific Progress ◽  
Effect Sizes ◽  
Practical Significance ◽  
Significance Tests ◽  
Wide Range ◽  
Research Findings

To understand and communicate research findings, it is important for researchers to consider two types of information provided by research results: the magnitude of the effect and the degree of uncertainty in the outcome. Statistical significance tests have long served as the mainstream method for statistical inferences. However, the widespread misinterpretation and misuse of significance tests has led critics to question their usefulness in evaluating research findings and to raise concerns about the far-reaching effects of this practice on scientific progress. An alternative approach involves reporting and interpreting measures of effect size along with confidence intervals. An effect size is an indicator of magnitude and direction of a statistical observation. Effect size statistics have been developed to represent a wide range of research questions, including indicators of the mean difference between groups, the relative odds of an event, or the degree of correlation among variables. Effect sizes play a key role in evaluating practical significance, conducting power analysis, and conducting meta-analysis. While effect sizes summarize the magnitude of an effect, the confidence intervals represent the degree of uncertainty in the result. By presenting a range of plausible alternate values that might have occurred due to sampling error, confidence intervals provide an intuitive indicator of how strongly researchers should rely on the results from a single study.

The Significance of Statistical Significance Tests in Marketing Research

1983 ◽  
Vol 20 (2) ◽  
pp. 122-133 ◽  
Author(s):  
Alan G. Sawyer ◽  
J. Paul Peter
Keyword(s):  
Hypothesis Testing ◽  
Meta Analysis ◽  
Statistical Significance ◽  
Marketing Research ◽  
Significance Testing ◽  
Primary Method ◽  
Significance Tests ◽  
Statistical Significance Testing ◽  
Bayesian Hypothesis Testing ◽  
Strong Inference

Classical statistical significance testing is the primary method by which marketing researchers empirically test hypotheses and draw inferences about theories. The authors discuss the interpretation and value of classical statistical significance tests and suggest that classical inferential statistics may be misinterpreted and overvalued by marketing researchers in judging research results. Replication, Bayesian hypothesis testing, meta-analysis, and strong inference are examined as approaches for augmenting conventional statistical analyses.

scholarly journals EXPRESS: Efficacy and Safety of Tadalafil in a Pediatric Population with Pulmonary Arterial Hypertension: Phase 3 randomized, Double Blind Placebo-Controlled Study

2021 ◽  
pp. 204589402110249
Author(s):  
David D Ivy ◽  
Damien Bonnet ◽  
Rolf MF Berger ◽  
Gisela Meyer ◽  
Simin Baygani ◽  
...  
Keyword(s):  
Pediatric Patients ◽  
Statistical Significance ◽  
Significance Testing ◽  
Efficacy And Safety ◽  
Phase 3 ◽  
Statistical Significance Testing ◽  
Double Blind ◽  
Double Blind Placebo ◽  
Treatment Groups ◽  
Pulmonary Arterial

Objective: This study evaluated the efficacy and safety of tadalafil in pediatric patients with pulmonary arterial hypertension (PAH). Methods: This phase-3, international, randomized, multicenter (24 weeks double-blind placebo controlled period; 2-year, open-labelled extension period), add-on (patient’s current endothelin receptor antagonist therapy) study included pediatric patients aged <18 years with PAH. Patients received tadalafil 20 mg or 40 mg based on their weight (Heavy-weight: ≥40 kg; Middle-weight: ≥25—<40 kg) or placebo orally QD for 24 weeks. Primary endpoint was change from baseline in 6-minute walk (6MW) distance in patients aged ≥6 years at Week 24. Sample size was amended from 134 to ≥34 patients, due to serious recruitment challenges. Therefore, statistical significance testing was not performed between treatment groups. Results: Patient demographics and baseline characteristics (N=35; tadalafil=17; placebo=18) were comparable between treatment groups; median age was 14.2 years (6.2 to 17.9 years) and majority (71.4%, n=25) of patients were in HW cohort. Least square mean (SE) changes from baseline in 6MW distance at Week 24 was numerically greater with tadalafil versus placebo (60.48 [20.41] vs 36.60 [20.78] meters; placebo-adjusted mean difference [SD] 23.88 [29.11]). Safety of tadalafil treatment was as expected without any new safety concerns. During study period 1, two patients (1 in each group) discontinued due to investigator’s reported clinical worsening, and no deaths were reported. Conclusions: The statistical significance testing was not performed between the treatment groups due to low sample size, however, the study results show positive trend in improvement in non invasive measurements, commonly utilized by clinicians to evaluate the disease status for children with PAH. Safety of tadalafil treatment was as expected without any new safety signals.

Sign in / Sign up

Export Citation Format

Share Document