Summary Plots With Adjusted Error Bars: The superb Framework With an Implementation in R

Plotting the data of an experiment allows researchers to illustrate the main results of a study, show effect sizes, compare conditions, and guide interpretations. To achieve all this, it is necessary to show point estimates of the results and their precision using error bars. Often, and potentially unbeknownst to them, researchers use a type of error bars—the confidence intervals—that convey limited information. For instance, confidence intervals do not allow comparing results (a) between groups, (b) between repeated measures, (c) when participants are sampled in clusters, and (d) when the population size is finite. The use of such stand-alone error bars can lead to discrepancies between the plot’s display and the conclusions derived from statistical tests. To overcome this problem, we propose to generalize the precision of the results (the confidence intervals) by adjusting them so that they take into account the experimental design and the sampling methodology. Unfortunately, most software dedicated to statistical analyses do not offer options to adjust error bars. As a solution, we developed an open-access, open-source library for R— superb—that allows users to create summary plots with easily adjusted error bars.

Download Full-text

Diamond Plots: a tutorial to introduce a visualisation tool that facilitates interpretation and comparison of multiple sample estimates while respecting their inaccuracy

10.31234/osf.io/fzh6c ◽

2017 ◽

Cited By ~ 4

Author(s):

Gjalt - Jorn Ygram Peters

Keyword(s):

Confidence Intervals ◽

Open Source Software ◽

Null Hypothesis ◽

Current Paper ◽

Effect Sizes ◽

Full Disclosure ◽

Point Estimates ◽

Multiple Sample ◽

Minimal Effort ◽

Visualisation Technique

Although a shift from a focus on null hypothesis significance testing to reporting effect sizes and confidence intervals has been advocated for decades, researchers have been slow to implement this shift. One of the reasons may be that working with confidence intervals is interpreted as inconvenient. Diamond plots are a visualisation technique to ameliorate this disadvantage. The current paper introduces an implementation of diamond plots in the free and open source software R. This implementation is flexible and designed to also be accessible to researchers that are not used to working with R. The current paper also includes a tutorial to enable researchers to start producing diamond plots themselves with minimal effort. Combining a shift from reporting point estimates and confidence intervals in tables to using diamond plots with full disclosure enables presenting reports in a readable manner without loss of detail.

Download Full-text

Covid-19 Vaccine Efficacy: Accuracy, Uncertainty and Projection of Cases

10.1101/2020.12.16.20248359 ◽

2020 ◽

Author(s):

Wenjiang Fu ◽

Jieni Li ◽

Paul Scheet

Keyword(s):

Confidence Intervals ◽

Vaccine Efficacy ◽

Accuracy Assessment ◽

Point Estimate ◽

Societal Impact ◽

Limited Information ◽

Interim Report ◽

Vaccine Candidates ◽

Vaccine Trials ◽

Point Estimates

ABSTRACTBackgroundTwo vaccine candidates for coronavirus disease 2019 (Covid-19) have been announced by Pfizer-BioNTech and Moderna with above 90% efficacy. The efficacy of each vaccine changes between reports with no accuracy assessment.MethodsWe examined data in both vaccine trials, provided 95% confidence intervals, and projected the cases that would be prevented in communities of multi-million population.ResultsThe 95% confidence intervals reveal that the true vaccine efficacy could be as low as 86% for stated efficacy of 94.4% in an interim report, indicating the inaccuracy and uncertainty of efficacy point estimate. Both vaccines achieve an efficacy above 89% by the 95% confidence interval in updated reports. The Moderna vaccine would prevent more than 50,260 cases in communities of 1 million people with 1 year exposure.ConclusionsPoint estimates of vaccine efficacy transmit limited information. Corresponding statements of uncertainty, such as confidence intervals, should be provided and included in discussions of societal impact. The Covid-19 vaccines announced to date would prevent a substantial number of cases even at lower ends of the intervals.

Download Full-text

Publication Bias against Null Results

Psychological Reports ◽

10.2466/pr0.1997.80.1.337 ◽

1997 ◽

Vol 80 (1) ◽

pp. 337-338 ◽

Cited By ~ 29

Author(s):

Raymond Hubbard ◽

J. Scott Armstrong

Keyword(s):

Confidence Intervals ◽

Publication Bias ◽

Statistical Tests ◽

Effect Sizes ◽

Null Results ◽

Replication Studies

Studies suggest a bias against the publication of null ( p >.05) results. Instead of significance, we advocate reporting effect sizes and confidence intervals, and using replication studies. If statistical tests are used, power tests should accompany them.

Download Full-text

The Orchard Plot: Cultivating a Forest Plot for Use in Ecology, Evolution and Beyond

10.32942/osf.io/epqa7 ◽

2019 ◽

Author(s):

Shinichi Nakagawa ◽

Malgorzata Lagisz ◽

Rose E O'Dea ◽

Joanna Rutkowska ◽

Yefeng Yang ◽

...

Keyword(s):

Meta Analysis ◽

R Package ◽

Effect Sizes ◽

Forest Plot ◽

Point Estimates ◽

Aggregate Effect ◽

The Individual ◽

Meta Analyses ◽

Heterogeneous Effect ◽

Intuitive Interpretation

‘Classic’ forest plots show the effect sizes from individual studies and the aggregate effect from a meta-analysis. However, in ecology and evolution meta-analyses routinely contain over 100 effect sizes, making the classic forest plot of limited use. We surveyed 102 meta-analyses in ecology and evolution, finding that only 11% use the classic forest plot. Instead, most used a ‘forest-like plot’, showing point estimates (with 95% confidence intervals; CIs) from a series of subgroups or categories in a meta-regression. We propose a modification of the forest-like plot, which we name the ‘orchard plot’. Orchard plots, in addition to showing overall mean effects and CIs from meta-analyses/regressions, also includes 95% prediction intervals (PIs), and the individual effect sizes scaled by their precision. The PI allows the user and reader to see the range in which an effect size from a future study may be expected to fall. The PI, therefore, provides an intuitive interpretation of any heterogeneity in the data. Supplementing the PI, the inclusion of underlying effect sizes also allows the user to see any influential or outlying effect sizes. We showcase the orchard plot with example datasets from ecology and evolution, using the R package, orchard, including several functions for visualizing meta-analytic data using forest-plot derivatives. We consider the orchard plot as a variant on the classic forest plot, cultivated to the needs of meta-analysts in ecology and evolution. Hopefully, the orchard plot will prove fruitful for visualizing large collections of heterogeneous effect sizes regardless of the field of study.

Download Full-text

Experimental Design as a Retrieval Method for Assessing the Phenomenological Characteristics of Episodic Memories, Episodic Future Thoughts, and Episodic Counterfactual Thoughts

10.31234/osf.io/h56a9 ◽

2020 ◽

Author(s):

Jared Branch

Keyword(s):

Experimental Design ◽

Effect Sizes ◽

Retrieval Method ◽

Future Studies ◽

Within Subjects ◽

Demand Effects ◽

Episodic Memories ◽

Theoretical Considerations

Studies assessing the phenomenological characteristics of episodic memories, episodic future thoughts, and episodic counterfactual thoughts normally utilize a within-subjects design. As such, there are concerns that the observed similarities in phenomenological characteristics are the result of demand effects or other related matters, rather than theoretical considerations. In this study, a within-subjects experimental design was directly compared with a between-subjects experimental design. In both conditions, participants responded to existing questionnaires used to assess phenomenological characteristics of episodic memories, episodic future thoughts, and episodic counterfactual thoughts. The within-subjects design resulted more often in significant findings and larger effect sizes compared to the between-subjects design. The implications for experimental design in future studies is discussed.

Download Full-text

Assessing variability in surgical decision making among attending neurosurgeons at an academic center

Journal of Neurosurgery ◽

10.3171/2019.2.jns182658 ◽

2020 ◽

Vol 132 (6) ◽

pp. 1970-1976

Author(s):

Ashwin G. Ramayya ◽

H. Isaac Chen ◽

Paul J. Marcotte ◽

Steven Brem ◽

Eric L. Zager ◽

...

Keyword(s):

Spine Surgery ◽

Repeated Measures ◽

Elective Surgery ◽

Statistical Tests ◽

Neurosurgical Procedures ◽

P Values ◽

Surgical Interventions ◽

Radiology Reports ◽

Operative Notes ◽

First Time

OBJECTIVEAlthough it is known that intersurgeon variability in offering elective surgery can have major consequences for patient morbidity and healthcare spending, data addressing variability within neurosurgery are scarce. The authors performed a prospective peer review study of randomly selected neurosurgery cases in order to assess the extent of consensus regarding the decision to offer elective surgery among attending neurosurgeons across one large academic institution.METHODSAll consecutive patients who had undergone standard inpatient surgical interventions of 1 of 4 types (craniotomy for tumor [CFT], nonacute redo CFT, first-time spine surgery with/without instrumentation, and nonacute redo spine surgery with/without instrumentation) during the period 2015–2017 were retrospectively enrolled (n = 9156 patient surgeries, n = 80 randomly selected individual cases, n = 20 index cases of each type randomly selected for review). The selected cases were scored by attending neurosurgeons using a need for surgery (NFS) score based on clinical data (patient demographics, preoperative notes, radiology reports, and operative notes; n = 616 independent case reviews). Attending neurosurgeon reviewers were blinded as to performing provider and surgical outcome. Aggregate NFS scores across various categories were measured. The authors employed a repeated-measures mixed ANOVA model with autoregressive variance structure to compute omnibus statistical tests across the various surgery types. Interrater reliability (IRR) was measured using Cohen’s kappa based on binary NFS scores.RESULTSOverall, the authors found that most of the neurosurgical procedures studied were rated as “indicated” by blinded attending neurosurgeons (mean NFS = 88.3, all p values < 0.001) with greater agreement among neurosurgeon raters than expected by chance (IRR = 81.78%, p = 0.016). Redo surgery had lower NFS scores and IRR scores than first-time surgery, both for craniotomy and spine surgery (ANOVA, all p values < 0.01). Spine surgeries with fusion had lower NFS scores than spine surgeries without fusion procedures (p < 0.01).CONCLUSIONSThere was general agreement among neurosurgeons in terms of indication for surgery; however, revision surgery of all types and spine surgery with fusion procedures had the lowest amount of decision consensus. These results should guide efforts aimed at reducing unnecessary variability in surgical practice with the goal of effective allocation of healthcare resources to advance the value paradigm in neurosurgery.

Download Full-text