Rating the methodological quality of single-subject designs andn-of-1 trials: Introducing the Single-Case Experimental Design (SCED) Scale

AbstractIn the context of evidence-based clinical practice (EBCP), the reliability of empirical data is largely determined by the methodological quality of research design. PsycBITE™ (Psychological Database of Brain Impairment Treatment Efficacy) is a web-based database listing all published, empirical reports on the effectiveness of nonpharmacological interventions for the psychological consequences of acquired brain impairment (ABI). The aim of this study was to survey the listings of PsycBITE™ and examine the methodological quality of the reports it contains. Reports listed in PsycBITE™ include systematic reviews (SRs), randomised controlled trials (RCTs), non-RCTs, case series (CSs) and single-subject designs (SSDs). They are indexed according to research design, neurological group, patient age group, target area and intervention type. The PEDro Scale is used to rate the methodological quality of RCTs, nonRCTs and CSs, with maximum obtainable methodological quality rating (MQR) of 10/10, 8/10 and 2/10 respectively. A search identified 1298 reports indexed in PsycBITE™. The largest proportion was SSDs (39%), followed by CSs (22%), RCTs (21%), non-RCTs (11%) and SRs (7%). The majority of reports was concerned with stroke (41%), traumatic brain injury (29%) and Alzheimer's and related dementias (22%). The most frequently investigated deficits were communication/language/speech disorders (24%); independent/self-care activities (19%); behaviour problems (17%); memory impairments (17%); anxiety, depression, stress, adjustment (15%). Approximately half of the RCTs, non-RCTs and CSs were rated for methodological quality. Mean MQR scores for RCTs, non-RCTs and CSs were 4.49, 2.85 and 1.15 respectively. While some PEDro criteria were met by a high proportion of RCTs and non-RCTs (≥ 70%), other criteria were only met by a small proportion of reports (as low as 1.6%). There was no significant difference in MQR scores between RCTs focusing on different neurological groups or target areas. Furthermore, there was no discernible improvement in MQR score for RCTs published over the last three decades. The methodological quality of studies investigating the efficacy of rehabilitation interventions in ABI has been consistently modest over several decades. This is largely attributable to poor adherence to fundamental tenets of research design, and requires urgent remediation. RCTs (and to a lesser extent, non-RCTs) are research methodologies which can potentially yield a high level of evidence, but only if they are adequately designed. PsycBITE™ has the facility to raise awareness of these issues and be instrumental in promoting EBCP in the field of ABI.

Download Full-text

Methodological quality of meta-analyses of single-case experimental studies

Research in Developmental Disabilities ◽

10.1016/j.ridd.2017.12.016 ◽

2018 ◽

Vol 79 ◽

pp. 97-115 ◽

Cited By ~ 12

Author(s):

Laleh Jamshidi ◽

Mieke Heyvaert ◽

Lies Declercq ◽

Belén Fernández-Castilla ◽

John M. Ferron ◽

...

Keyword(s):

Methodological Quality ◽

Single Case ◽

Experimental Studies ◽

Meta Analyses

Download Full-text

Outcome for the Single Case: Random Control Index, Single Subject Experimental Design, and Goal Attainment Scale

Encyclopedia of Behavioral Medicine ◽

10.1007/978-3-030-39903-0_906 ◽

2020 ◽

pp. 1587-1593

Author(s):

Kenneth J. Ottenbacher ◽

Margaret E. Ottenbacher

Keyword(s):

Experimental Design ◽

Goal Attainment ◽

Single Case ◽

Single Subject ◽

Goal Attainment Scale ◽

Random Control

Download Full-text

Assessing Choice Making of a Person with Profound Disabilities: A Preliminary Analysis

Journal of the Association for Persons with Severe Handicaps ◽

10.1177/154079699502000304 ◽

1995 ◽

Vol 20 (3) ◽

pp. 196-201 ◽

Cited By ~ 16

Author(s):

Kazuko Nozaki ◽

Akira Mochizuki

Keyword(s):

Preliminary Analysis ◽

Leisure Activity ◽

Single Subject ◽

Residential Setting ◽

Choice Making ◽

Musical Preference ◽

Single Subject Designs ◽

Choice Option

The purpose of this study was to examine two issues in choice-making research through a case study assessing the leisure-activity preferences of a woman with profound mental retardation in a residential setting. The issues include (a) providing the participant with the option of expressing “no” for given choice options and (b) analyzing environmental variables that might control choice-making behavior. The investigation consisted of two experiments; single-subject designs were used in each. The results of the two experiments suggested that (a) the participant could express distinct musical preference when given choice-making opportunities; (b) adding as a choice option a notebook that signaled the termination of a session allowed the participant to express a preference for ending the session when only less preferred options were presented; and (c) preference patterns varied with the activity partner and/or the locale. Results are discussed in terms of choice making and autonomy leading to an improved quality of life.

Download Full-text

Effects of Dog-Assisted Therapy on Social Behaviors and Emotional Expressions: A Single-Case Experimental Design in 3 People With Dementia

Journal of Geriatric Psychiatry and Neurology ◽

10.1177/0891988719868306 ◽

2019 ◽

Vol 33 (2) ◽

pp. 109-119 ◽

Cited By ~ 2

Author(s):

Enrique Pérez-Sáez ◽

Elisa Pérez-Redondo ◽

Elena González-Ingelmo

Keyword(s):

Quality Of Life ◽

Experimental Design ◽

Single Case ◽

Social Behaviors ◽

Prosocial Behaviors ◽

Emotional Expressions ◽

Improve Quality ◽

People With Dementia ◽

Therapy Sessions

This study examined the effects of dog-assisted therapy (DAT) on social behaviors, emotional manifestations, and experience during the activity of 3 people with dementia residing in a specialized dementia unit. The study used an A-B-A-B withdrawal single-case experimental design with two 5-session phases, baseline and DAT, replicating the same activities in each phase. The sessions were recorded and 2 independent coders quantified the frequency of social behaviors along with an assessment of the emotional manifestations and experience during the activity. Comparing with baseline sessions, DAT sessions showed an increase in prosocial behaviors (leans, looks, and verbalizations) and a significant impact on emotional manifestations with heightened pleasure. Dog-assisted therapy sessions also led to a better experience, with higher participation, pleasure, and relationship with others, together with lower rejection and displeasure than in the baseline sessions. Dog-assisted therapy seems to be a nonpharmacological therapy with potential to improve quality of life of people with dementia through promoting social behaviors and positive emotional manifestations.

Download Full-text

A Case for Increased Rigor in AAC Research: A Methodological Quality Review

10.35542/osf.io/9qnfp ◽

2021 ◽

Author(s):

Jay Ganz ◽

James E Pustejovsky ◽

Joe Reichle ◽

Kimberly Vannest ◽

Margaret Foster ◽

...

Keyword(s):

Quality Indicators ◽

Challenging Behavior ◽

Augmentative And Alternative Communication ◽

Methodological Quality ◽

Single Case ◽

Quality Criteria ◽

Future Research ◽

Communication Needs ◽

Review Reports

This comprehensive review reports on methodological quality of 162 single-case studies on augmentative and alternative communication for communication and challenging behavior in individuals diagnosed with autism or intellectual disabilities and with complex communication needs. Following review for inclusion criteria, documents were excluded if they failed to meet basic methodological standards. Each remaining study was evaluated for 10 detailed quality criteria. No studies met all standards without reservations. Only three of the included studies met all of the standards with reservations and the remainder met some but not all standards, with or without reservations. The included studies reported adequate detail for half of the quality indicators, but insufficient details for participant, setting, maintenance, and generalization, and social validity descriptions. An increased quantity and quality of research were found in over four decades. More recent studies have adequately reported half of the criteria investigated, including describing the materials, defining the outcome variables, describing baseline and intervention procedures, and evaluating procedural integrity. After identifying quality features, the authors report in more detail on low rated quality indicators particularly relevant to studies addressing social-communication interventions. The literature infrequently reported race, ethnicity, or home language. Future research should report characteristics of participants to ensure that research becomes representative of the population.

Download Full-text

Can They Do Anything? Ten Single-Subject Studies of the Engagement Level of Hospitalized Demented Patients

British Journal of Occupational Therapy ◽

10.1177/030802268805100409 ◽

1988 ◽

Vol 51 (4) ◽

pp. 129-132 ◽

Cited By ~ 9

Author(s):

M C Conroy ◽

Felicity Fincham ◽

Catherine Agard-Evans

Keyword(s):

Experimental Design ◽

Sampling Method ◽

Single Case ◽

The Elderly ◽

Single Subject ◽

Assessment Procedure ◽

Time Sampling ◽

The Individual

The authors used a time-sampling method and a single-case experimental design to see if 10 hospitalized demented patients were more engaged during activity sessions than when they were on the wards. They all participated more fully in the activity sessions. There was some improvement on the Clifton Assessment Procedure for the Elderly scores during phase B when the patients were attending the activity sessions, but it was small and might be attributed to other factors. The individual patients varied greatly in their participation and choice of activities, despite their infirmities.

Download Full-text