Supervisory alliance as a moderator of the effects of behavioral rehearsal on TF-CBT fidelity: results from a randomized trial of supervision strategies

The present study examined relations between supervisory alliance and fidelity to the trauma narrative component of Trauma-Focused Cognitive Behavioral Therapy, as well as how supervisory alliance might moderate the effect of behavioral rehearsals (i.e., role plays) on fidelity. Forty-two supervisors and 124clinicians from 28 different community-based mental health offices across Washington State participated.Clinicians were randomized to receive one of two supervision conditions—symptom and fidelity monitoring or symptom and fidelity monitoring with behavioral rehearsal. Supervisory alliance alone did not predict adherence or extensiveness of the trauma narrative. One aspect of alliance, client focus, significantly altered the effect of supervision condition on adherence (p=0.05); however, this effect was only seen in 43.5% of clinicians. A second aspect of alliance, rapport, altered the effect of condition on trauma narrative extensiveness with moderate significance (p=0.09). Future research should investigate strategies to improve supervisory alliance or match supervision strategies to specific supervisor-clinician dyads.

Implementation of a Remote Fidelity Oversight Program for a Multi-Site Exercise Intervention

The STEP-HI exercise protocol is a supervised, 2 phase, multimodal, high-intensity exercise program that emphasizes resistance training. Exercise sessions are conducted at an exercise facility and occur on two non-consecutive days/week for 6 months. During specified exercises, the exercise interventionist targets the participant’s eight-repetition maximum (8-RM), defined as the greatest resistance that can be moved 8 times through full range of motion with good form. A rigorous, remote fidelity monitoring program maximizes consistency of the intervention across sites. This fidelity oversight program is a model for future exercise studies because of its unique remote, hierarchical structure. All exercise interventionists are initially certified by written examination and direct observations. Some exercise sessions are also video recorded and reviewed using fidelity checklists. After initial certification, repeated direct observation and video-based verification of fidelity are repeated at prescribed intervals for each interventionist to ensure sustained consistency of implementation across sites.

Comparing fidelity monitoring methods in an evidence-based parenting intervention

Purpose Two common methods in community settings of assessing program fidelity, a critical implementation component for program effectiveness, are video and audio recordings of sessions. This paper aims to examine how these two methods compared when used for a home-based behavioral parenting-training model (SafeCare®). Design/methodology/approach Twenty-five SafeCare video-recorded sessions between home visitors and parents were scored by trained raters either using the video or audio-only portions of recordings. Sessions were coded using fidelity checklists, with items (n = 33) classified as one of two fidelity aspects, content [delivery of program components (n = 15)], or process [communication and rapport building (n = 11)]. Seven items were considered to overlap between constructs. Items were coded as having been done or not done appropriately. Coders rated items as “technological limitation” when scoring methods hindered coding. Analyses compared percent agreement and disagreement between audio and video coders. Findings Overall agreement between coders was 72.12%. Levels of agreement were higher for content items (M = 80.89%, SD = 19.68) than process items (58.54%, SD = 34.41). Disagreements due to technology limitations among audio coders were noted among 15 items; particularly, higher levels of disagreement were seen among process items (42.42%) than content items (9.64%). Originality/value Compared to video, fidelity monitoring via audio recordings was associated with some loss of process-related fidelity. However, audio recordings could be sufficient with supplements such as participant surveys, to better capture process items. Research should also examine how content and process fidelity relate to changes in family behavior to further inform optimal fidelity monitoring methods for program use.

The Necessary Thread of Mindfulness Intervention Fidelity Assurance: Enabling an Organizational Strategy to Promote Health Care Professional Well-Being

Purpose A growing waitlist for Mindfulness in Motion (MIM), an evidence-based worksite mindfulness-based intervention, necessitated a training system with built in fidelity assurance to meet program demand. MIM was delivered as part of an organizational strategy in a large academic health center to enhance Health Care Professional (HCP) well-being. In order to ensure that the intervention was being delivered the same way to each cohort, a process to ensure intervention fidelity was developed for MIM. Method The core components of MIM informed the development of a detailed fidelity monitoring system to ensure consistent intervention delivery. Each MIM cohort was conducted with both trained facilitators and trained intervention fidelity monitors. Results Across 11 cohort offerings of MIM, each 8 weeks in length, there was a mean adherence rate of 0.9886, SD = 0.0012. Conclusion The fidelity monitoring system allowed for a reliable expansion of MIM offerings to HCPs and for a seamless pivot to fully virtual MIM delivery, necessitated by COVID-19.

A consultation with feedback approach to supporting fidelity to a peer-delivered intervention for posttraumatic stress disorder

Background: Despite promising findings regarding the safety, fidelity, and effectiveness of peer-delivered behavioral health programs, there are training-related challenges to the integration of peers on health care teams. Specifically, there is a need to understand the elements of training and consultation that may be unique to peer-delivered interventions. Methods: As part of a pilot effectiveness-implementation study of an abbreviated version of Skills Training in Affective and Interpersonal Regulation (STAIR) for posttraumatic stress disorder (PTSD), we conducted a mixed-methods process evaluation utilizing multiple data sources (questionnaires and field notes) to characterize our approach to consultation and explore relations between fidelity, treatment outcome, and client satisfaction. Results: Peer interventionists exhibited high fidelity, defined by adherence ( M = 93.7%, SD = 12.3%) and competence ( M = 3.7 “competent,” SD = 0.5). Adherence, β = .69, t(1) = 3.69, p < .01, and competence, β = .585, t(1) = 2.88, p < .05, were each associated with trial participant’s satisfaction, but not associated with clinical outcomes. Our synthesis of fidelity-monitoring data and consultation field notes suggests that peer interventionists possess strengths in interpersonal effectiveness, such as rapport building, empathy, and appropriate self-disclosure. Peer interventionists evidenced minor challenges with key features of directive approaches, such as pacing, time efficiency, and providing strong theoretical rationale for homework and tracking. Conclusion: Due to promise of peers in expanding the behavioral health workforce and engaging individuals otherwise missed by the medical model, the current study aimed to characterize unique aspects of training and consultation. We found peer interventionists demonstrated high fidelity, supported through dynamic training and consultation with feedback. Research is needed to examine the impact of consultation approach on implementation and treatment outcomes. Plain Language Summary: Peers—paraprofessionals who use their lived experiences to engage and support the populations they serve—have been increasingly integrated into health care settings in the United States. Training peers to deliver interventions may provide cost savings by way of improving efficient utilization of professional services. Despite promising findings in regard to safety, intervention fidelity, and effectiveness of peer delivery, there are important challenges that need to be addressed if peers are to be more broadly integrated into the health care system as interventionists. These include challenges associated with highly variable training, inadequate supervision, and poor delineation of peer’s roles within the broader spectrum of care. Thus, there is a need to understand the unique components of training and consultation for peers. We report key findings from an evaluation of a pilot study of an abbreviated version of Skills Training in Affective and Interpersonal Regulation (STAIR) for posttraumatic stress disorder (PTSD), adapted for peer delivery. We characterize our approach to consultation with feedback and explore relations between fidelity, treatment outcome, and client satisfaction. Our study extends the small yet growing literature on training and consultation approaches to support fidelity (adherence and competence) among peer interventionists. Organizations hoping to integrate peers on health care teams could utilize our fidelity-monitoring approach to set benchmarks to ensure peer-delivered interventions are safe and effective.

Feasibility of Automating Fidelity Monitoring in a Dementia Care Intervention

Careful fidelity monitoring is critical to implementing evidence-based interventions in dementia care settings to ensure that the intervention is delivered consistently and as intended. Most approaches to fidelity monitoring rely on human coding of content that has been covered during a session or of stylistic aspects of the intervention, including rapport, empathy, enthusiasm and are unrealistic to implement on a large scale in real world settings. Technological advances in automatic speech recognition and language and speech processing offers potential solutions to overcome these barriers. We compare three commercial automatic speech recognition tools on spoken content drawn from dementia care interactions to determine the accuracy of recognition and the guarantees for privacy offered by each provider. Data were obtained from recorded sessions of the Dementia Behavior Study intervention trial (NCT01892579). We find that despite their impressive performance in general applications, automatic speech recognition systems work less well for older adults and people of color. We outline a plan for automating fidelity in interaction style and content which would be integrated in an online program for training dementia care providers.