An Algorithm to Evaluate Methodological Rigor and Risk of Bias in Single-Case Studies

2019 ◽  
pp. 014544551986303 ◽  
Author(s):  
Michael Perdices ◽  
Robyn L. Tate ◽  
Ulrike Rosenkoetter
Keyword(s):  
Single Case ◽  
Strong Association ◽  
Internal Validity ◽  
Risk Of Bias ◽  
Classification Systems ◽  
What Works ◽  
Methodological Rigor ◽  
Differential Weighting ◽  
What Works Clearinghouse ◽  
Single Case Designs

Critical appraisal scales play an important role in evaluating methodological rigor (MR) of between-groups and single-case designs (SCDs). For intervention research this forms an essential basis for ascertaining the strength of evidence. Yet, few such scales provide classifications that take into account the differential weighting of items contributing to internal validity. This study aimed to develop an algorithm derived from the Risk of Bias in N-of-1 Trials (RoBiNT) Scale to classify MR and risk of bias magnitude in SCDs. The algorithm was applied to 46 SCD experiments. Two experiments (4%) were classified as Very High MR, 14 (30%) as High, 5 (11%) as Moderate, 2 (4%) as Fair, 2 (4%) as Low, and 21 (46%) as Very Low. These proportions were comparable to the What Works Clearinghouse classifications: 13 (28%) met standards, 8 (17%) met standards with reservations, and 25 (54%) did not meet standards. There was strong association between the two classification systems.

Estimating the Impact of Design Standards on the Rigor of a Subset of Single-Case Research

2020 ◽  
pp. 104420732093404 ◽  
Author(s):  
Collin Shepley ◽  
Kathleen N. Zimmerman ◽  
Kevin M. Ayres
Keyword(s):  
Single Case ◽  
Single Case Design ◽  
Design Standards ◽  
Case Research ◽  
What Works ◽  
What Works Clearinghouse ◽  
Single Case Research ◽  
The Impact ◽  
Case Design

The implementation of research-based practices by teachers in public school classrooms is required under federal law as expressed in the Individuals With Disabilities Education Improvement Act of 2004. To aid teachers in identifying such practices, researchers conduct systematic reviews of the educational literature. Although recent attention has been given to changes in the quality of these reviews, there has been minimal discussion about changes in the quality of the studies that comprise them. Specifically, to what extent have educational policies leading to the creation of experimental design standards resulted in a change in the rigor of educational research? Using a subset of the single-case literature commonly published in special education journals, we estimate the impact of What Works Clearinghouse single-case design standards on the trend in the rigor of single-case studies using a comparative interrupted time series framework. Within this subset of single-case studies, our estimation strategy did not detect a change in the trend of the rigor of single-case research following the establishment of What Works Clearinghouse single-case design standards. Implications are discussed for practitioners and researchers. Study data, syntax, and supplemental materials are available for public use at https://osf.io/xp7wv/.

The Use of Computer-Assisted Instruction to Improve the Reading Comprehension of Students With Learning Disabilities: An Evaluation of the Evidence Base According to the What Works Clearinghouse Standards

2017 ◽  
Vol 38 (4) ◽  
pp. 233-245 ◽  
Author(s):  
Min Kyung Kim ◽  
John William McKenna ◽  
Yujeong Park
Keyword(s):  
Learning Disabilities ◽  
Comparison Group ◽  
Single Case ◽  
Evidence Base ◽  
Computer Assisted Instruction ◽  
Computer Assisted ◽  
Students With Learning Disabilities ◽  
What Works ◽  
What Works Clearinghouse ◽  
Assisted Instruction

The purpose of this study was to investigate the evidence base for using computer-assisted instruction (CAI) to improve the reading comprehension of students with learning disabilities (LD). Twelve peer-reviewed studies (seven comparison group studies, five single-case studies) met selection criteria and were evaluated according to the relevant What Works Clearinghouse (WWC) procedures and standards. Results showed that seven studies (five comparison group and two single-case studies) met WWC standards with or without reservations. Key instructional features employed in CAI studies meeting the WWC standards without reservations included practice opportunities, self-correction and immediate corrective feedback, teacher-directed instruction, and contingencies for enhancing student motivation and engagement. Implications for future research and suggestions for using quality indicators to improve the rigor of future CAI investigations are discussed.

A Meta-Analysis of Single-Case Research on Applied Behavior Analytic Interventions for People With Down Syndrome

2021 ◽  
Vol 126 (2) ◽  
pp. 114-141
Author(s):  
Nicole Neil ◽  
Ashley Amicarelli ◽  
Brianna M. Anderson ◽  
Kailee Liesemer
Keyword(s):  
Down Syndrome ◽  
Challenging Behavior ◽  
Single Case ◽  
Meta Analysis ◽  
Challenging Behaviors ◽  
Future Research ◽  
Case Research ◽  
What Works ◽  
What Works Clearinghouse ◽  
Single Case Research

Abstract This systematic review evaluates single-case research design studies investigating applied behavior analytic (ABA) interventions for people with Down syndrome (DS). One hundred twenty-five studies examining the efficacy of ABA interventions on increasing skills and/or decreasing challenging behaviors met inclusion criteria. The What Works Clearinghouse standards and Risk of Bias in N-of-1 Trials scale were used to analyze methodological characteristics, and Tau-U effect sizes were calculated. Results suggest the use of ABA-based interventions are promising for behavior change in people with DS. Thirty-six high-quality studies were identified and demonstrated a medium overall effect. A range of outcomes was targeted, primarily involving communication and challenging behavior. These outcomes will guide future research on ABA interventions and DS.

The What Works Clearinghouse Single-Case Design Pilot Standards

2014 ◽  
Vol 35 (3) ◽  
pp. 145-152 ◽  
Author(s):  
John H. Hitchcock ◽  
Robert H. Horner ◽  
Thomas R. Kratochwill ◽  
Joel R. Levin ◽  
Samuel L. Odom ◽  
...  
Keyword(s):  
Single Case ◽  
Single Case Design ◽  
What Works ◽  
What Works Clearinghouse ◽  
Case Design

Addressing Feeding Disorders Using High-Probability Sequencing for Children and Adolescents with Developmental Disabilities

2019 ◽  
Vol 44 (3) ◽  
pp. 169-185 ◽  
Author(s):  
Seth King ◽  
Hayley E. Johnson ◽  
Taneal Burch ◽  
Argnue Chitiyo
Keyword(s):  
Developmental Disabilities ◽  
Children And Adolescents ◽  
High Probability ◽  
Single Case ◽  
Experimental Studies ◽  
Autism Spectrum ◽  
Feeding Disorders ◽  
What Works ◽  
History Of ◽  
What Works Clearinghouse

Feeding disorders exhibited by children with developmental disabilities, which include limiting food intake or refusing to consume solid foods, often result in poor health consequences. Interventions for feeding disorders vary in terms of their acceptability to children with disabilities and their families. One specific procedure, the high-probability sequence, is a nonrestrictive approach where requests with which a child has a history of compliance are presented prior to requests associated with noncompliance. This article reviewed research concerning the application of high-probability sequencing for feeding disorders among children and adolescents with developmental disabilities. Experimental studies published between 1970 and 2018 in English as either peer-reviewed articles or dissertations were eligible for descriptive review. Identified articles ( N = 15) evaluated high-probability sequencing using various single-case experimental designs. Participants ( N = 21) were generally preschool-aged children (mean age = 5.7 years) diagnosed with autism spectrum disorder (ASD; n = 9) or other developmental disabilities ( n = 12). Subsequent analysis of articles that met quality indicators of the What Works Clearinghouse ( n = 12) suggests that research does not support the effectiveness of the procedure for increasing food acceptance. Implications for practice and future studies involving high-probability sequencing follow a discussion of findings.

What Works Clearinghouse Standards and Generalization of Single-Case Design Evidence

2015 ◽  
Vol 24 (4) ◽  
pp. 459-469 ◽  
Author(s):  
John H. Hitchcock ◽  
Thomas R. Kratochwill ◽  
Laura C. Chezan
Keyword(s):  
Single Case ◽  
Single Case Design ◽  
What Works ◽  
What Works Clearinghouse ◽  
Case Design

Analysis of Differences Across Two Unaffiliated Systematic Reviews Using What Works Clearinghouse Single-Case Design Standards

2020 ◽  
Vol 45 (3) ◽  
pp. 143-160
Author(s):  
Collin Shepley ◽  
Justin D. Lane ◽  
Melinda J. Ault
Keyword(s):  
Special Education ◽  
Systematic Reviews ◽  
Single Case ◽  
Single Case Design ◽  
Design Standards ◽  
Topic Area ◽  
System Of Least Prompts ◽  
What Works ◽  
What Works Clearinghouse ◽  
Case Design

The system of least prompts response prompting procedure has a rich history in special education research and practice. Recently, two independent systematic reviews were conducted to determine if the system of least prompts met criteria to be classified as an evidence-based practice. Both reviews used single-case design standards developed by What Works Clearinghouse to evaluate the rigor and effects of studies; however, findings and implications varied significantly across reviews. We examined the data supporting each review and discuss how two reviews on the same topic area using the same standards for evaluating studies could arrive at different conclusions. Results indicate that varying search parameters, visual analysis protocols, and the flexibility allotted by the design standards may have contributed to differences. We discuss the importance of multiple literature reviews on the same topic area with regard to replication research in special education. In addition, we highlight the necessity of open data in such reviews. Finally, we recommend how practitioners and researchers should collectively interpret the differing findings and conclusions from the reviews examining the system of least prompts.

Using data collection apps and single-case designs to research transformative learning in adults

2017 ◽  
Vol 23 (2) ◽  
pp. 206-225 ◽  
Author(s):  
Kevin M Roessger ◽  
Arie Greenleaf ◽  
Chad Hoggan
Keyword(s):  
Data Collection ◽  
Transformative Learning ◽  
Single Case ◽  
Internal Validity ◽  
Research Approach ◽  
Case Research ◽  
Research Designs ◽  
Single Case Research ◽  
Using Data ◽  
Single Case Designs

To overcome situational hurdles when researching transformative learning in adults, we outline a research approach using single-case research designs and smartphone data collection apps. This approach allows researchers to better understand learners’ current lived experiences and determine the effects of transformative learning interventions on demonstrable outcomes. We first discuss data collection apps and their features. We then describe how they can be integrated into single-case research designs to make causal inferences about a learning intervention’s effects when limited by researcher access and learner retrospective reporting. Design controls for internal validity threats and visual and statistical data analysis are then discussed. Throughout, we highlight applications to transformative learning and conclude by discussing the approach’s potential limitations.

Geometry Interventions for Students With Learning Disabilities: A Research Synthesis

2019 ◽  
pp. 073194871989202
Author(s):  
Meijia Liu ◽  
Diane Pedrotty Bryant ◽  
Elly Kiru ◽  
Maryam Nozari
Keyword(s):  
Learning Disabilities ◽  
Intervention Studies ◽  
Single Case ◽  
Technical Information ◽  
Student Outcome ◽  
Students With Learning Disabilities ◽  
Intervention Treatment ◽  
Methodological Rigor ◽  
Single Case Designs ◽  
Skill Modeling

The purpose of this study was to examine geometry interventions for students with learning disabilities. We synthesized nine intervention studies by focusing on the geometry concepts and skills taught to students with learning disabilities, intervention types used, instructional components embedded, and the methodological rigor of the studies. Intervention studies were mainly single-case designs. The geometry topics included angle recognition, and perimeter, area, and volume problems. The findings of this synthesis contribute to the current literature by showing that geometry interventions for students with learning disabilities incorporated proven effective instructional components (e.g., multiple representations, skill modeling). Regarding the methodological rigor, the results showed several issues, including the lack of description of interventionist training, intervention treatment fidelity, and adequate technical information for student outcome measures. Limitations of the studies and suggestions for future intervention research are discussed.

Sign in / Sign up

Export Citation Format

Share Document