The Effectiveness of Direct Instruction Curricula: A Meta-Analysis of a Half Century of Research

2018 ◽  
Vol 88 (4) ◽  
pp. 479-507 ◽  
Author(s):  
Jean Stockard ◽  
Timothy W. Wood ◽  
Cristy Coughlin ◽  
Caitlin Rasplica Khoury
Keyword(s):  
Direct Instruction ◽  
Mixed Models ◽  
Meta Analysis ◽  
Effect Sizes ◽  
Single Subject ◽  
Affective Outcomes ◽  
Half Century ◽  
Study Designs ◽  
Single Subject Designs

Quantitative mixed models were used to examine literature published from 1966 through 2016 on the effectiveness of Direct Instruction. Analyses were based on 328 studies involving 413 study designs and almost 4,000 effects. Results are reported for the total set and subareas regarding reading, math, language, spelling, and multiple or other academic subjects; ability measures; affective outcomes; teacher and parent views; and single-subject designs. All of the estimated effects were positive and all were statistically significant except results from metaregressions involving affective outcomes. Characteristics of the publications, methodology, and sample were not systematically related to effect estimates. Effects showed little decline during maintenance, and effects for academic subjects were greater when students had more exposure to the programs. Estimated effects were educationally significant, moderate to large when using the traditional psychological benchmarks, and similar in magnitude to effect sizes that reflect performance gaps between more and less advantaged students.

scholarly journals The Efficacy of Recasts in Language Intervention: A Systematic Review and Meta-Analysis

2015 ◽  
Vol 24 (2) ◽  
pp. 237-255 ◽  
Author(s):  
Patricia L. Cleave ◽  
Stephanie D. Becker ◽  
Maura K. Curran ◽  
Amanda J. Owen Van Horne ◽  
Marc E. Fey
Keyword(s):  
Systematic Review ◽  
Research Evidence ◽  
Meta Analysis ◽  
Language Intervention ◽  
Effect Sizes ◽  
Study Quality ◽  
Average Effect ◽  
Grammatical Development ◽  
Meta Analyses ◽  
Study Designs

Purpose This systematic review and meta-analysis critically evaluated the research evidence on the effectiveness of conversational recasts in grammatical development for children with language impairments. Method Two different but complementary reviews were conducted and then integrated. Systematic searches of the literature resulted in 35 articles for the systematic review. Studies that employed a wide variety of study designs were involved, but all examined interventions where recasts were the key component. The meta-analysis only included studies that allowed the calculation of effect sizes, but it did include package interventions in which recasts were a major part. Fourteen studies were included, 7 of which were also in the systematic review. Studies were grouped according to research phase and were rated for quality. Results Study quality and thus strength of evidence varied substantially. Nevertheless, across all phases, the vast majority of studies provided support for the use of recasts. Meta-analyses found average effect sizes of .96 for proximal measures and .76 for distal measures, reflecting a positive benefit of about 0.75 to 1.00 standard deviation. Conclusion The available evidence is limited, but it is supportive of the use of recasts in grammatical intervention. Critical features of recasts in grammatical interventions are discussed.

Meta-Analysis in Single-Subject Evaluation Research

2021 ◽  
pp. 143-154
Author(s):  
Charles Auerbach
Keyword(s):  
Meta Analysis ◽  
Evaluation Research ◽  
Effect Sizes ◽  
Single Subject ◽  
Building Research ◽  
Single Subject Research ◽  
Research Designs ◽  
The Individual ◽  
Meta Analyses ◽  
Intervening Variables

Meta-analytic techniques can be used to aggregate evaluation results across studies. In the case of single-subject research designs, we could combine findings from evaluations with 5, 10 or 20 clients to determine, on average, how effective an intervention is. This is a more complex and sophisticated way of understanding differences across studies than reporting those changes qualitatively or simply reporting the individual effect sizes for each study. In this chapter, the authors discuss why meta-analysis is important to consider in single-subject research, particularly in the context of building research evidence. They then demonstrate how to do this using SSD for R functions. Building upon effect sizes, introduced in Chapter 4, the authors illustrate the conditions under which it is appropriate to use traditional effect sizes to conduct meta-analyses, how to introduce intervening variables, and how to evaluate statistical output. Additionally, the authors discuss and illustrate the computation and interpretation of a mean Non-Overlap of All Pairs in situations which traditional effect sizes cannot be used.

Toward Optimized Effectiveness of Employee Training Programs

2021 ◽  
Author(s):  
Dilek Uslu ◽  
Justin Marcus ◽  
Yasemin Kisbu-Sakarya
Keyword(s):  
Training Programs ◽  
Meta Analysis ◽  
Employee Training ◽  
Effect Sizes ◽  
Independent Effect ◽  
Workplace Training ◽  
Affective Outcomes ◽  
Training Outcomes ◽  
Training Interventions ◽  
Training Delivery

Abstract. Although organizations invest heavily on employee training, the effectiveness of employee training programs has not been well-established. In the current study, we examine the training delivery features of employee training programs to derive a better understanding of features that may be of best benefit in the improvement of employee affective outcomes. Specifically, and via the use of meta-analysis ( k = 79 studies totaling 107 independent effect sizes), we focus on two broad classes of affective employee training outcomes including attitudinal and motivational outcomes. Results evidence support for the effectiveness of employee workplace training interventions and indicate that employee training programs associated with attitudinal versus motivational outcomes require different features while being delivered to reach optimal effectiveness.

scholarly journals The impacts of hydropower on freshwater macroinvertebrate richness: A global meta-analysis

2021 ◽  
Author(s):  
Gabrielle Trottier ◽  
Katrine Turgeon ◽  
Daniel Boisclair ◽  
Cécile Bulle ◽  
Manuele Margni
Keyword(s):  
Variance Estimation ◽  
Meta Analysis ◽  
Effect Sizes ◽  
Hydroelectric Dams ◽  
Freshwater Biodiversity ◽  
Mixed Effect ◽  
Large Variability ◽  
Effect Model ◽  
Hydropower Dams ◽  
Study Designs

AbstractHydroelectric dams and their reservoirs have been suggested to affect freshwater biodiversity. However, studies investigating the consequences of hydroelectric dams and reservoirs on macroinvertebrate richness have reached opposite conclusions. We carried out a meta-analysis devised to elucidate the effects of hydropower dams and their reservoirs on macroinvertebrates richness while accounting for the potential role played by moderators such as biomes, impact types, study designs, sampling seasons and gears. We used a random and mixed effect model, combined with robust variance estimation, to conduct the meta-analysis on 72 pairs of observations (i.e., impacted versus reference) extracted from 17 studies (more than one observation per study). We observed a large range of effect sizes, from very negative to very positive impacts of hydropower. However, according to this meta-analysis, hydropower dams and their reservoirs did not have an overall clear, directional and statistically significant effect on macroinvertebrate richness. We tried to account for the large variability in effect sizes using moderators, but none of the moderators included in the meta-analysis had statistically significant effect. This suggests that some other moderators, which were unavailable for the 17 studies included in this meta-analysis, might be important (e.g., temperature, granulometry, wave disturbance and macrophytes) and that macroinvertebrate richness may be driven by local, smaller scale processes. As new studies become available, it would be interesting to keep enriching this meta-analysis, as well as collecting local habitat variables, to see if we could finally draw statistically significant conclusions about the impacts of hydropower on macroinvertebrate richness.

The Use of Single-Subject Design in Rehabilitation Counseling Research

2019 ◽  
Vol 33 (3) ◽  
pp. 168-179
Author(s):  
Michael Gerald ◽  
Noel Estrada-Hernández ◽  
John Wadsworth
Keyword(s):  
Rehabilitation Counseling ◽  
Single Subject ◽  
Single Subject Design ◽  
Evidence Based Practices ◽  
Rehabilitation Professionals ◽  
Single Subject Research ◽  
Subject Design ◽  
Data Collection And Analysis ◽  
Study Designs ◽  
Single Subject Designs

BackgroundThe utility of single-subject designs as a practical methodology for rehabilitation professionals working in the field and one that allows researchers the flexibility to identify or change treatments in the event their participant or client is not responding has been established for nearly a century (Dixon, 2002). However, a content analysis of the presence of single-subject research in rehabilitation research has been absent from the rehabilitation literature.PurposeThe purpose of this article is to describe the usability of single study designs as a research tool that contributes to the identification of Evidence-Based Practices (EBPs) in rehabilitation counseling.MethodsThrough manual and electronic searches of the literature, the authors identified journals and subsequently articles that clearly stated the use of single-subject design as their method of data collection and analysis.ResultsBetween 2001 to 2016, there have been a total of eight articles published that have either utilized single-subject experimental designs or provided an overview of the methods.ConclusionsA critical review of the rehabilitation literature indicated that single-subject methodologies have been recognized as contributing to the identification of EBPs, yet studies utilizing a single-subject methodology continue to be rare. The paucity of single-subject design may be the result of a lack of training opportunities, lack of models in the rehabilitation literature, and assumptions that single-subject design utility is limited to behavioral research.

scholarly journals Optimism, pessimism and judgement bias in animals: a systematic review and meta-analysis

2020 ◽  
Author(s):  
Malgorzata Lagisz ◽  
Josefina Zidar ◽  
Shinichi Nakagawa ◽  
Vikki Neville ◽  
Enrico Sorato ◽  
...  
Keyword(s):  
Systematic Review ◽  
Meta Analysis ◽  
Task Type ◽  
Effect Sizes ◽  
Affective States ◽  
Human Species ◽  
Treatment Groups ◽  
Active Choice ◽  
Judgement Bias ◽  
Study Designs

Just as happy people see the proverbial glass as half-full, ‘optimistic’ or ‘pessimistic’ responses to ambiguity might also reflect affective states in animals. Judgement bias tests, designed to measure these responses, are an increasingly popular way of assessing animal affect and there is now a substantial, but heterogeneous, literature on their use across different species, affect manipulations, and study designs. By conducting a systematic review and meta-analysis of 459 effect sizes from 71 studies of non-pharmacological affect manipulations on 22 non-human species, we show that animals in relatively better conditions, assumed to generate more positive affect, show more ‘optimistic’ judgements of ambiguity than those in relatively worse conditions. Overall effects are small when considering responses to all cues, but become more pronounced when non-ambiguous training cues are excluded from analyses or when focusing only on the most divergent responses between treatment groups. Task type (go/no-go; go/go active choice), training cue reinforcement (reward-punishment; reward-null; reward-reward) and sex of animals emerge as potential moderators of effect sizes in judgement bias tests.

scholarly journals Most oxytocin administration studies are statistically underpowered to reliably detect (or reject) a wide range of effect sizes

2020 ◽  
Author(s):  
Daniel S Quintana
Keyword(s):  
Statistical Power ◽  
Meta Analysis ◽  
Effect Sizes ◽  
Evidential Value ◽  
Statistical Equivalence ◽  
Wide Range ◽  
Noticeable Improvement ◽  
Meta Analyses ◽  
Study Designs ◽  
Analytical Approaches

The neuropeptide oxytocin has attracted substantial research interest for its role in behaviour and cognition; however, the evidence for its effects have been mixed. Meta-analysis is viewed as the gold-standard for synthesizing evidence, but the evidential value of a meta-analysis is dependent on the evidential value of the studies it synthesizes, and the analytical approaches used to derive conclusions. To assess the evidential value of oxytocin administration meta-analyses, this study calculated the statistical power of 107 studies from 35 meta-analyses and assessed the statistical equivalence of reported results. The mean statistical power across all studies was 12.2% and there has been no noticeable improvement in power over an eight-year period. None of the 26 non-significant meta-analyses were statistically equivalent, assuming a smallest effect size of interest of 0.1. Altogether, most oxytocin treatment study designs are statistically underpowered to either detect or reject a wide range of effect sizes that scholars may find worthwhile.

scholarly journals Paths to the Light and Dark Sides of Human Nature: A Meta-analysis of the Prosocial Benefits of Autonomy and the Antisocial Costs of Control

2021 ◽  
Author(s):  
James Donald ◽  
Emma L Bradshaw ◽  
James Haddon Conigrave ◽  
Philip David Parker ◽  
Lauren L. Byatt ◽  
...  
Keyword(s):  
Structural Equation ◽  
Meta Analysis ◽  
Self Determination Theory ◽  
Self Determination ◽  
Effect Sizes ◽  
Future Research ◽  
Equation Modelling ◽  
Study Designs ◽  
And Control ◽  
Practical Implications

Self-determination theory (SDT) posits that experiences of autonomy lead people to be more prosocial, whereas experiences of control lead to antisocial actions. In this meta-analysis, we tested the links between autonomy and prosociality, and control and antisociality, across 139 reports (167 studies) with 1,189 effect sizes (N = 75,546 participants). We used two-stage structural equation modelling including both correlational and longitudinal study designs. We found support for the hypothesized direct links between autonomy and prosociality, and between control and antisociality, with cross-paths between these constructs being weaker. In line with SDT’s claims that the salutary effects of autonomy are universal, results also showed that the hypothesized links were consistent across cultures, genders, and age categories. We also reviewed emerging experimental research on the effect of autonomy-priming interventions on prosociality. To conclude, we discuss the theoretical and practical implications of these findings and lay out an agenda for future research.

scholarly journals The Effects of Data-based Instruction (DBI) for Students with Learning Difficulties in Korea: A Single-subject Meta-analysis

PLoS ONE ◽  
2021 ◽  
Vol 16 (12) ◽  
pp. e0261120
Author(s):  
Dongil Kim ◽  
Seohyeon Choi
Keyword(s):  
Meta Analysis ◽  
Learning Difficulties ◽  
Effect Sizes ◽  
Single Subject ◽  
Struggling Learners ◽  
Educational Approach ◽  
Level Of Evidence ◽  
Intensive Intervention ◽  
Based Instruction

Data-based instruction (DBI) is an ongoing process to utilize students’ data for determining when and how to intensify intervention. It is an educational approach that is suggested as effective to enhance achievements of struggling learners, particularly for those who did not respond to intensive intervention in usual ways. In Korea, DBI was introduced and applied for students with learning difficulties especially since 2000 when the first Korea curriculum-based measurement (CBM) was developed as the name of Basic Academic Skills Assessment. Despite a number of studies accumulated since then, there has been a lack of research that examined the level of evidence-based practice (EBP) of DBI research. Thus, the present study sought to synthesize the DBI research so far in Korea by analyzing the effectiveness of DBI for school-aged students with learning difficulties via meta-analysis and evaluating the quality of the research. In this study, a total of 32 single-subject design studies were used. Multilevel meta-analysis revealed that the mean effect size of DBI was statistically significant (B = 1.34) and there was significant variance across participants in effect sizes. The results from the conditional model showed that exceptionality type, the number of sessions, and the length of each session were significantly accountable for the variability of effect sizes. In addition, the results of the qualitative analysis revealed the acceptable quality of the overall DBI research with some limitations. Based on these findings, implications and study limitations were discussed.

Sign in / Sign up

Export Citation Format

Share Document