Meta-Analysis in Single-Subject Evaluation Research

2021 ◽  
pp. 143-154
Author(s):  
Charles Auerbach
Keyword(s):  
Meta Analysis ◽  
Evaluation Research ◽  
Effect Sizes ◽  
Single Subject ◽  
Building Research ◽  
Single Subject Research ◽  
Research Designs ◽  
The Individual ◽  
Meta Analyses ◽  
Intervening Variables

Meta-analytic techniques can be used to aggregate evaluation results across studies. In the case of single-subject research designs, we could combine findings from evaluations with 5, 10 or 20 clients to determine, on average, how effective an intervention is. This is a more complex and sophisticated way of understanding differences across studies than reporting those changes qualitatively or simply reporting the individual effect sizes for each study. In this chapter, the authors discuss why meta-analysis is important to consider in single-subject research, particularly in the context of building research evidence. They then demonstrate how to do this using SSD for R functions. Building upon effect sizes, introduced in Chapter 4, the authors illustrate the conditions under which it is appropriate to use traditional effect sizes to conduct meta-analyses, how to introduce intervening variables, and how to evaluate statistical output. Additionally, the authors discuss and illustrate the computation and interpretation of a mean Non-Overlap of All Pairs in situations which traditional effect sizes cannot be used.

The Orchard Plot: Cultivating a Forest Plot for Use in Ecology, Evolution and Beyond

2019 ◽  
Author(s):  
Shinichi Nakagawa ◽  
Malgorzata Lagisz ◽  
Rose E O'Dea ◽  
Joanna Rutkowska ◽  
Yefeng Yang ◽  
...  
Keyword(s):  
Meta Analysis ◽  
R Package ◽  
Effect Sizes ◽  
Forest Plot ◽  
Point Estimates ◽  
Aggregate Effect ◽  
The Individual ◽  
Meta Analyses ◽  
Heterogeneous Effect ◽  
Intuitive Interpretation

‘Classic’ forest plots show the effect sizes from individual studies and the aggregate effect from a meta-analysis. However, in ecology and evolution meta-analyses routinely contain over 100 effect sizes, making the classic forest plot of limited use. We surveyed 102 meta-analyses in ecology and evolution, finding that only 11% use the classic forest plot. Instead, most used a ‘forest-like plot’, showing point estimates (with 95% confidence intervals; CIs) from a series of subgroups or categories in a meta-regression. We propose a modification of the forest-like plot, which we name the ‘orchard plot’. Orchard plots, in addition to showing overall mean effects and CIs from meta-analyses/regressions, also includes 95% prediction intervals (PIs), and the individual effect sizes scaled by their precision. The PI allows the user and reader to see the range in which an effect size from a future study may be expected to fall. The PI, therefore, provides an intuitive interpretation of any heterogeneity in the data. Supplementing the PI, the inclusion of underlying effect sizes also allows the user to see any influential or outlying effect sizes. We showcase the orchard plot with example datasets from ecology and evolution, using the R package, orchard, including several functions for visualizing meta-analytic data using forest-plot derivatives. We consider the orchard plot as a variant on the classic forest plot, cultivated to the needs of meta-analysts in ecology and evolution. Hopefully, the orchard plot will prove fruitful for visualizing large collections of heterogeneous effect sizes regardless of the field of study.

N = 1 Research Methodology in Counselling: Focusing upon the Individual Client

2003 ◽  
Vol 13 (2) ◽  
pp. 133-144 ◽  
Author(s):  
Christopher F. Sharpley
Keyword(s):  
Data Analysis ◽  
Large Scale ◽  
Single Subject ◽  
Health Practitioner ◽  
Design Data ◽  
Single Subject Research ◽  
Research Designs ◽  
Research Findings ◽  
Individual Client ◽  
The Individual

Although the last 20years have seen a focus upon evidence-based therapies, there are arguments that much of the so-called “evidence” produced is, in fact, irrelevant to the mental health practitioner in the field, principally because of the use of large-scale group designs in clinical controlled studies of the effectiveness of one therapy over another. By contrast, and with particular relevance to the practitioner who is both scientist and therapist, single subject research designs and methodologies for data analysis can be applied in ways that allow for generalisation to everyday practice. To inform the readership, the rationale underlying n = 1 studies is described, with some explanation of the major designs and their application to typical cases in guidance and counselling. Issues of inferential deductions from data, variations of design, data analysis via visual and statistical procedures, and replication are discussed. Finally, a case is argued for the introduction of n = 1 reports within the Australian Journal of Guidance and Counselling to better inform the readership about clinical research findings relevant to their practices.

The Effects of Purchasing Skill Instruction for Individuals with Developmental Disabilities: A Meta-Analysis

2005 ◽  
Vol 71 (4) ◽  
pp. 379-400 ◽  
Author(s):  
Yan Ping Xin ◽  
Edward Grasso ◽  
Caroline M. Dipipi-Hoy ◽  
Asha Jitendra
Keyword(s):  
Intervention Studies ◽  
Treatment Effectiveness ◽  
Meta Analysis ◽  
Single Subject ◽  
Correction Procedure ◽  
Instructional Setting ◽  
Single Subject Research ◽  
Research Designs ◽  
Nonparametric Procedure ◽  
Positive Effect

This meta-analysis examines the effectiveness of functional mathematics instruction, specifically purchasing skill instruction, for individuals with disabilities. Twenty-eight intervention studies were identified and reviewed. Because all studies employed single-subject research designs, a nonparametric procedure, the percentage of nonoverlapping data (PND), was used to estimate treatment effects. Results indicated a moderately positive effect for purchasing skill instruction (median PND = 87%). Maintenance (median PND = 100%) and generalization effects (median PND = 86%) revealed large and moderate effects, respectively. Further categorical comparisons indicated that variables such as participants' entry skills, money skill adaptations, type of purchase, error correction procedure, and instructional setting were related to the treatment effectiveness.

Group Designs in Clinical Research

1987 ◽  
Vol 52 (3) ◽  
pp. 194-199 ◽  
Author(s):  
Gerald M. Siegel ◽  
Martin A. Young
Keyword(s):  
Clinical Research ◽  
Research Question ◽  
Research Strategy ◽  
Single Subject ◽  
Research Strategies ◽  
Individual Subject ◽  
Single Subject Research ◽  
Group Research ◽  
Research Designs ◽  
The Individual

Single-subject research designs, with their concentration on the individual subject over extended time durations, are similar in form to the design of therapy and have been represented as the best, if not the only, appropriate method for carrying out clinical research. Despite the similarity between single-subject research sessions and clinical sessions, it is argued that such designs are not intrinsically more appropriate than group designs for clinical research. Single-subject and group research strategies are alternative and often competing approaches to the same research question, and the choice resides as much in the predilections of the researcher as in any intrinsic advantage in one or the other research strategy.

Identification of and Correction for Publication Bias: Comment

2019 ◽  
Author(s):  
Amanda Kvarven ◽  
Eirik Strømland ◽  
Magnus Johannesson
Keyword(s):  
Publication Bias ◽  
False Positive ◽  
Large Scale ◽  
Meta Analysis ◽  
False Positive Rate ◽  
Effect Sizes ◽  
Replication Studies ◽  
Moderate Reduction ◽  
Positive Rate ◽  
Meta Analyses

Andrews & Kasy (2019) propose an approach for adjusting effect sizes in meta-analysis for publication bias. We use the Andrews-Kasy estimator to adjust the result of 15 meta-analyses and compare the adjusted results to 15 large-scale multiple labs replication studies estimating the same effects. The pre-registered replications provide precisely estimated effect sizes, which do not suffer from publication bias. The Andrews-Kasy approach leads to a moderate reduction of the inflated effect sizes in the meta-analyses. However, the approach still overestimates effect sizes by a factor of about two or more and has an estimated false positive rate of between 57% and 100%.

scholarly journals A protocol for the systematic review and meta-analysis of thigmotactic behaviour in the open field test in rodent models associated with persistent pain

2021 ◽  
Vol 5 (1) ◽  
pp. e100135
Author(s):  
Xue Ying Zhang ◽  
Jan Vollert ◽  
Emily S Sena ◽  
Andrew SC Rice ◽  
Nadia Soliman
Keyword(s):  
Systematic Review ◽  
Outcome Measure ◽  
Meta Analysis ◽  
Persistent Pain ◽  
Effect Sizes ◽  
Bias Assessment ◽  
Animal Pain ◽  
Trim And Fill ◽  
Meta Analyses ◽  
The Impact

ObjectiveThigmotaxis is an innate predator avoidance behaviour of rodents and is enhanced when animals are under stress. It is characterised by the preference of a rodent to seek shelter, rather than expose itself to the aversive open area. The behaviour has been proposed to be a measurable construct that can address the impact of pain on rodent behaviour. This systematic review will assess whether thigmotaxis can be influenced by experimental persistent pain and attenuated by pharmacological interventions in rodents.Search strategyWe will conduct search on three electronic databases to identify studies in which thigmotaxis was used as an outcome measure contextualised to a rodent model associated with persistent pain. All studies published until the date of the search will be considered.Screening and annotationTwo independent reviewers will screen studies based on the order of (1) titles and abstracts, and (2) full texts.Data management and reportingFor meta-analysis, we will extract thigmotactic behavioural data and calculate effect sizes. Effect sizes will be combined using a random-effects model. We will assess heterogeneity and identify sources of heterogeneity. A risk-of-bias assessment will be conducted to evaluate study quality. Publication bias will be assessed using funnel plots, Egger’s regression and trim-and-fill analysis. We will also extract stimulus-evoked limb withdrawal data to assess its correlation with thigmotaxis in the same animals. The evidence obtained will provide a comprehensive understanding of the strengths and limitations of using thigmotactic outcome measure in animal pain research so that future experimental designs can be optimised. We will follow the Preferred Reporting Items for Systematic Reviews and Meta-Analyses reporting guidelines and disseminate the review findings through publication and conference presentation.

Sign in / Sign up

Export Citation Format

Share Document