The case for prioritizing implementation strategy fidelity measurement: benefits and challenges

Abstract Implementation strategies are systematic approaches to improve the uptake and sustainability of evidence-based interventions. They frequently focus on changing provider behavior through the provision of interventions such as training, coaching, and audit-and-feedback. Implementation strategies often impact intermediate behavioral outcomes like provider guideline adherence, in turn improving patient outcomes. Fidelity of implementation strategy delivery is defined as the extent to which an implementation strategy is carried out as it was designed. Implementation strategy fidelity measurement is under-developed and under-reported, with the quality of reporting decreasing over time. Benefits of fidelity measurement include the exploration of the extent to which observed effects are moderated by fidelity, and critical information about Type-III research errors, or the likelihood that null findings result from implementation strategy fidelity failure. Reviews of implementation strategy efficacy often report wide variation across studies, commonly calling for increased implementation strategy fidelity measurement to help explain variations. Despite the methodological benefits of rigorous fidelity measurement, implementation researchers face multi-level challenges and complexities. Challenges include the measurement of a complex variable, multiple data collection modalities with varying precision and costs, and the need for fidelity measurement to change in-step with adaptations. In this position paper, we weigh these costs and benefits and ultimately contend that implementation strategy fidelity measurement and reporting should be improved in trials of implementation strategies. We offer pragmatic solutions for researchers to make immediate improvements like the use of mixed methods or innovative data collection and analysis techniques, the inclusion of implementation strategy fidelity assessment in reporting guidelines, and the staged development of fidelity tools across the evolution of an implementation strategy. We also call for additional research into the barriers and facilitators of implementation strategy fidelity measurement to further clarify the best path forward.

Download Full-text

Engaging Stakeholders to Retrospectively Discern Implementation Strategies in Support of Program Evaluation

10.21203/rs.3.rs-573512/v1 ◽

2021 ◽

Author(s):

Jacob T. Painter ◽

Rebecca A. Raciborski ◽

Monica M. Matthieu ◽

Ciara Oliver ◽

David A. Adkins ◽

...

Keyword(s):

Program Evaluation ◽

Quality Improvement ◽

Data Collection ◽

Implementation Strategy ◽

Implementation Strategies ◽

Successful Implementation ◽

Evidence Based ◽

Main Body ◽

Strategy Selection ◽

Evidence Based Practices

Abstract Background: Successful implementation of evidence-based practices is key to healthcare quality improvement. However, it depends on appropriate selection of implementation strategies, the techniques that improve practice adoption or sustainment. When studying implementation of an evidence-based practice as part of a program evaluation, implementation scientists confront a challenge: the timing of strategy selection rarely aligns with the establishment of data collection protocols. Indeed, the exact implementation strategies used by an organization during a quality improvement initiative may be determined during implementation. Nevertheless, discernment of strategies is necessary to accurately estimate implementation effect and cost because this information can support decision making for sustainment, guide replication efforts, and inform the choice of implementation strategies for other evidence-based practices. Main body: We propose an iterative, stakeholder engaged process to discern implementation strategies when strategy choice was not made before data collection began. Stakeholders are centered in the process, providing a list of current and potential implementation activities. These activities are then mapped by an implementation science expert to an established taxonomy of implementation strategies. The mapping is then presented back to stakeholders for member checking and refinement. The final list can be used to survey those engaged in implementation activities in a language they are familiar with. A case study using this process is provided. Conclusion: It is challenging to estimate implementation effort when implementation strategy selection is disconnected from the data collection process. In these cases, a stakeholder-informed process to retrospectively identify implementation strategies by classifying activities performed using an established implementation strategy taxonomy provides the necessary information.

Download Full-text

Measuring Implementation Strategy Fidelity in HealthyHearts NYC: A Complex Intervention Using Practice Facilitation in Primary Care

American Journal of Medical Quality ◽

10.1177/1062860620959450 ◽

2020 ◽

pp. 106286062095945

Author(s):

Carolyn A. Berry ◽

Ann M. Nguyen ◽

Allison M. Cuthel ◽

Charles M. Cleland ◽

Nina Siman ◽

...

Keyword(s):

Primary Care ◽

Conceptual Framework ◽

Implementation Strategy ◽

Tracking System ◽

A Priori ◽

Implementation Strategies ◽

Implementation Fidelity ◽

Practice Facilitation ◽

Task List ◽

Fidelity Assessment

Few studies have assessed the fidelity of practice facilitation (PF) as an implementation strategy, and none have used an a priori definition or conceptual framework of fidelity to guide fidelity assessment. The authors adapted the Conceptual Framework for Implementation Fidelity to guide fidelity assessment in HealthyHearts NYC, an intervention that used PF to improve adoption of cardiovascular disease evidence-based guidelines in primary care practices. Data from a web-based tracking system of 257 practices measured fidelity using 4 categories: frequency, duration, content, and coverage. Almost all (94.2%) practices received at least the required 13 PF visits. Facilitators spent on average 26.3 hours at each site. Most practices (95.7%) completed all Task List items, and 71.2% were educated on all Chronic Care Model strategies. The majority (65.8%) received full coverage. This study provides a model that practice managers and implementers can use to evaluate fidelity of PF, and potentially other implementation strategies.

Download Full-text

The Balanced Opioid Initiative: Protocol for a clustered, sequential, multiple-assignment randomized trial to construct an adaptive implementation strategy to improve guideline-concordant opioid prescribing in primary care

10.21203/rs.3.rs-21109/v1 ◽

2020 ◽

Author(s):

Andrew Quanbeck ◽

Daniel Almirall ◽

Nora Jacobson ◽

Randall T. Brown ◽

Jillian K. Landeck ◽

...

Keyword(s):

Primary Care ◽

Randomized Trial ◽

Implementation Strategy ◽

Implementation Strategies ◽

Audit And Feedback ◽

Practice Facilitation ◽

Prescribing Practices ◽

Opioid Prescribing ◽

Multiple Assignment ◽

Educational Engagement

Abstract Background Rates of opioid prescribing tripled in the United States between 1999 and 2015 and were associated with significant increases in opioid misuse and overdose death. Roughly half of all opioids are prescribed in primary care. Although clinical guidelines describe recommended opioid prescribing practices, implementing these guidelines in a way that balances safety and effectiveness vs. risk remains a challenge. The literature offers little help about which implementation strategies work best in different clinical settings or how strategies could be tailored to optimize their effectiveness in different contexts. Systems consultation consists of: (1) educational/engagement meetings with audit and feedback reports, (2) practice facilitation, and (3) prescriber peer consulting. The study is designed to discover the most cost-effective sequence and combination of strategies for improving opioid prescribing practices in diverse primary care clinics.Methods/Design The study is a hybrid type 3 clustered, sequential, multiple-assignment randomized trial (SMART) that randomizes clinics from two health systems at two points, months 3 and 9, of a 21-month intervention. Clinics are provided one of four sequences of implementation strategies: a condition consisting of educational/engagement meetings and audit and feedback alone (EM/AF), EM/AF plus practice facilitation (PF), EM/AF + prescriber peer consulting (PPC), and EM/AF + PF + PPC. The study’s primary outcome is morphine-milligram equivalent (MME) dose by prescribing clinicians within clinics. The study’s primary aim is the comparison of EM/AF + PF + PPC versus EM/AF alone on change in MME from month 3 to month 21. The secondary aim is to derive cost estimates for each of the four sequences and compare them. The exploratory aim is to examine four tailoring variables that can be used to construct an adaptive implementation strategy to meet the needs of different primary care clinics.Discussion Systems consultation is a practical blend of implementation strategies used in this case to improve opioid prescribing practices in primary care. The blend offers a range of strategies in sequences from minimally to substantially intensive. The results of this study promise to help us understand how to cost effectively improve the implementation of evidence-based practices.Trial registration NCT 04044521 (ClinicalTrial.gov). Registered 05 August 2019.

Download Full-text

Integration of statistical and administrative agricultural data from Namibia

Statistical Journal of the IAOS ◽

10.3233/sji-200634 ◽

2021 ◽

pp. 1-22

Author(s):

Emily Berg ◽

Johgho Im ◽

Zhengyuan Zhu ◽

Colin Lewis-Beck ◽

Jie Li

Keyword(s):

Measurement Error ◽

Data Collection ◽

Administrative Data ◽

Statistical Data ◽

Data Sources ◽

Extension Programs ◽

Multiple Data ◽

Administrative Agencies ◽

Crop Area ◽

Using Data

Statistical and administrative agencies often collect information on related parameters. Discrepancies between estimates from distinct data sources can arise due to differences in definitions, reference periods, and data collection protocols. Integrating statistical data with administrative data is appealing for saving data collection costs, reducing respondent burden, and improving the coherence of estimates produced by statistical and administrative agencies. Model based techniques, such as small area estimation and measurement error models, for combining multiple data sources have benefits of transparency, reproducibility, and the ability to provide an estimated uncertainty. Issues associated with integrating statistical data with administrative data are discussed in the context of data from Namibia. The national statistical agency in Namibia produces estimates of crop area using data from probability samples. Simultaneously, the Namibia Ministry of Agriculture, Water, and Forestry obtains crop area estimates through extension programs. We illustrate the use of a structural measurement error model for the purpose of synthesizing the administrative and survey data to form a unified estimate of crop area. Limitations on the available data preclude us from conducting a genuine, thorough application. Nonetheless, our illustration of methodology holds potential use for a general practitioner.

Download Full-text

Improving mobility and participation of older people with vertigo, dizziness and balance disorders in primary care using a care pathway: feasibility study and process evaluation

BMC Family Practice ◽

10.1186/s12875-021-01410-2 ◽

2021 ◽

Vol 22 (1) ◽

Author(s):

Eva Seckler ◽

Verena Regauer ◽

Melanie Krüger ◽

Anna Gabriel ◽

Joachim Hermsdörfer ◽

...

Keyword(s):

Primary Care ◽

Data Collection ◽

Older People ◽

Process Evaluation ◽

Feasibility Study ◽

Health Professionals ◽

Implementation Strategy ◽

Care Pathway ◽

Main Trial ◽

Telephone Helpline

Abstract Background Community-dwelling older people are frequently affected by vertigo, dizziness and balance disorders (VDB). We previously developed a care pathway (CPW) to improve their mobility and participation by offering standardized approaches for general practitioners (GPs) and physical therapists (PTs). We aimed to assess the feasibility of the intervention, its implementation strategy and the study procedures in preparation for the subsequent main trial. Methods This 12-week prospective cohort feasibility study was accompanied by a process evaluation designed according to the UK Medical Research Council’s Guidance for developing and evaluating complex interventions. Patients with VDB (≥65 years), GPs and PTs in primary care were included. The intervention consisted of a diagnostic screening checklist for GPs and a guide for PTs. The implementation strategy included specific educational trainings and a telephone helpline. Data for mixed-method process evaluation were collected via standardized questionnaires, field notes and qualitative interviews. Quantitative data were analysed using descriptive statistics, qualitative data using content analysis. Results A total of five GP practices (seven single GPs), 10 PT practices and 22 patients were included in the study. The recruitment of GPs and patients was challenging (response rates: GP practices: 28%, PT practices: 39%). Ninety-one percent of the patients and all health professionals completed the study. The health professionals responded well to the educational trainings; the utilization of the telephone helpline was low (one call each from GPs and PTs). Familiarisation with the routine of application of the intervention and positive attitudes were emphasized as facilitators of the implementation of the intervention, whereas a lack of time was mentioned as a barrier. Despite difficulties in the GPs’ adherence to the intervention protocol, the GPs, PTs and patients saw benefit in the intervention. The patients’ treatment adherence to physical therapy was good. There were minor issues in data collection, but no unintended consequences. Conclusion Although the process evaluation provided good support for the feasibility of study procedures, the intervention and its implementation strategy, we identified a need for improvement in recruitment of participants, the GP intervention part and the data collection procedures. The findings will inform the main trial to test the interventions effectiveness in a cluster RCT. Trial registration Projektdatenbank Versorgungsforschung Deutschland (German registry Health Services Research) VfD_MobilE-PHY_17_003910, date of registration: 30.11.2017; Deutsches Register Klinischer Studien (German Clinical Trials Register) DRKS00022918, date of registration: 03.09.2020 (retrospectively registered).

Download Full-text

Acceptability and effectiveness of antimicrobial stewardship implementation strategies on fluoroquinolone prescribing

Infection Control and Hospital Epidemiology ◽

10.1017/ice.2021.10 ◽

2021 ◽

pp. 1-8

Author(s):

Katie J. Suda ◽

Gosia S. Clore ◽

Charlesnika T. Evans ◽

Heather Schacht Reisinger ◽

Ibuola Kale ◽

...

Keyword(s):

Antimicrobial Stewardship ◽

Clinical Pathways ◽

Statistical Significance ◽

Implementation Strategies ◽

Antibiotic Use ◽

Audit And Feedback ◽

Prior Approval ◽

Days Of Therapy ◽

Prospective Audit And Feedback ◽

Advanced Generation

Abstract Objective: To assess the effectiveness and acceptability of antimicrobial stewardship-focused implementation strategies on inpatient fluoroquinolones. Methods: Stewardship champions at 15 hospitals were surveyed regarding the use and acceptability of strategies to improve fluoroquinolone prescribing. Antibiotic days of therapy (DOT) per 1,000 days present (DP) for sites with and without prospective audit and feedback (PAF) and/or prior approval were compared. Results: Among all of the sites, 60% had PAF or prior approval implemented for fluoroquinolones. Compared to sites using neither strategy (64.2 ± 34.4 DOT/DP), fluoroquinolone prescribing rates were lower for sites that employed PAF and/or prior approval (35.5 ± 9.8; P = .03) and decreased from 2017 to 2018 (P < .001). This decrease occurred without an increase in advanced-generation cephalosporins. Total antibiotic rates were 13% lower for sites with PAF and/or prior approval, but this difference did not reach statistical significance (P = .20). Sites reporting that PAF and/or prior approval were “completely” accepted had lower fluoroquinolone rates than sites where it was “moderately” accepted (34.2 ± 5.7 vs 48.7 ± 4.5; P < .01). Sites reported that clinical pathways and/or local guidelines (93%), prior approval (93%), and order forms (80%) “would” or “may” be effective in improving fluoroquinolone use. Although most sites (73%) indicated that requiring infectious disease consults would or may be effective in improving fluoroquinolones, 87% perceived implementation to be difficult. Conclusions: PAF and prior approval implementation strategies focused on fluoroquinolones were associated with significantly lower fluoroquinolone prescribing rates and nonsignificant decreases in total antibiotic use, suggesting limited evidence for class substitution. The association of acceptability of strategies with lower rates highlights the importance of culture. These results may indicate increased acceptability of implementation strategies and/or sensitivity to FDA warnings.

Download Full-text

Developing implementation strategies to adopt Enhanced Recovery After Surgery (ERAS®) guidelines

BJS Open ◽

10.1093/bjsopen/zraa011 ◽

2020 ◽

Vol 5 (2) ◽

Author(s):

Jennifer Y Lam ◽

Alexandra Howlett ◽

Duncan McLuckie ◽

Lori M Stephen ◽

Scott D N Else ◽

...

Keyword(s):

Implementation Strategy ◽

Enhanced Recovery ◽

Enhanced Recovery After Surgery ◽

Healthcare Providers ◽

Implementation Strategies ◽

Multidisciplinary Teams ◽

Successful Implementation ◽

Care Pathways ◽

Audit System ◽

Local Contexts

Abstract Background Strong implementation strategies are critical to the success of Enhanced Recovery after Surgery (ERAS®) guidelines, though little documentation exists on effective strategies, especially in complex clinical situations and unfamiliar contexts. This study outlines the process taken to adopt a novel neonatal ERAS® guideline. Methods The implementation strategy was approached in a multi-pronged, concurrent but asynchronous fashion. Between September 2019 and January 2020, healthcare providers from various disciplines and different specialties as well as parents participated in the strategy. Multidisciplinary teams were created to consider existing literature and local contexts including potential facilitators and/or barriers. Task forces worked collaboratively to develop new care pathways. An audit system was developed to record outcomes and elicit feedback for revision. Results 32 healthcare providers representing 9 disciplines and 5 specialties as well as 8 parents participated. Care pathways and resources were created. Elements recommended for a successful implementation strategy included identification of champions, multidisciplinary stakeholder involvement, consideration of local contexts and insights, patient/family engagement, education, and creation of an audit system. Conclusion A multidisciplinary and structured process following principles of implementation science was used to develop an effective implementation strategy for initiating ERAS® guidelines.

Download Full-text

Results of the CHlorhexidine Gluconate Bathing implementation intervention to improve evidence-based nursing practices for prevention of central line associated bloodstream infections Study (CHanGing BathS): a stepped wedge cluster randomized trial

Implementation Science ◽

10.1186/s13012-021-01112-4 ◽

2021 ◽

Vol 16 (1) ◽

Author(s):

Staci S. Reynolds ◽

Patricia Woltz ◽

Edward Keating ◽

Janice Neff ◽

Jennifer Elliott ◽

...

Keyword(s):

Implementation Strategies ◽

Bloodstream Infections ◽

Central Line ◽

Audit And Feedback ◽

Educational Outreach ◽

Evidence Based ◽

Intervention Effect ◽

Cluster Randomized ◽

Outreach Visits ◽

Educational Outreach Visits

Abstract Background Central line-associated bloodstream infections (CLABSIs) result in approximately 28,000 deaths and approximately $2.3 billion in added costs to the U.S. healthcare system each year, and yet, many of these infections are preventable. At two large health systems in the southeast United States, CLABSIs continue to be an area of opportunity. Despite strong evidence for interventions to prevent CLABSI and reduce associated patient harm, such as use of chlorhexidine gluconate (CHG) bathing, the adoption of these interventions in practice is poor. The primary objective of this study was to assess the effect of a tailored, multifaceted implementation program on nursing staff’s compliance with the CHG bathing process and electronic health record (EHR) documentation in critically ill patients. The secondary objectives were to examine the (1) moderating effect of unit characteristics and cultural context, (2) intervention effect on nursing staff’s knowledge and perceptions of CHG bathing, and (3) intervention effect on CLABSI rates. Methods A stepped wedged cluster-randomized design was used with units clustered into 4 sequences; each sequence consecutively began the intervention over the course of 4 months. The Grol and Wensing Model of Implementation helped guide selection of the implementation strategies, which included educational outreach visits and audit and feedback. Compliance with the appropriate CHG bathing process and daily CHG bathing documentation were assessed. Outcomes were assessed 12 months after the intervention to assess for sustainability. Results Among the 14 clinical units participating, 8 were in a university hospital setting and 6 were in community hospital settings. CHG bathing process compliance and nursing staff’s knowledge and perceptions of CHG bathing significantly improved after the intervention (p = .009, p = .002, and p = .01, respectively). CHG bathing documentation compliance and CLABSI rates did not significantly improve; however, there was a clinically significant 27.4% decrease in CLABSI rates. Conclusions Using educational outreach visits and audit and feedback implementation strategies can improve adoption of evidence-based CHG bathing practices. Trial registration ClinicalTrials.gov, NCT03898115, Registered 28 March 2019.

Download Full-text

Effectiveness and meaningful use of paediatric surgical safety checklists and their implementation strategies: a systematic review with narrative synthesis

BMJ Open ◽

10.1136/bmjopen-2017-016298 ◽

2017 ◽

Vol 7 (10) ◽

pp. e016298 ◽

Cited By ~ 10

Author(s):

Janaka Lagoo ◽

Steven R Lopushinsky ◽

Alex B Haynes ◽

Paul Bain ◽

Helene Flageole ◽

...

Keyword(s):

Systematic Review ◽

High Risk ◽

Implementation Strategy ◽

Citation Index ◽

Improve Patient Safety ◽

Implementation Strategies ◽

Meaningful Use ◽

Narrative Synthesis ◽

Surgical Safety ◽

Full Text Review

ObjectiveTo examine the effectiveness and meaningful use of paediatric surgical safety checklists (SSCs) and their implementation strategies through a systematic review with narrative synthesis.Summary background dataSince the launch of the WHO SSC, checklists have been integrated into surgical systems worldwide. Information is sparse on how SSCs have been integrated into the paediatric surgical environment.MethodsA broad search strategy was created using Pubmed, Embase, CINAHL, Cochrane Central, Web of Science, Science Citation Index and Conference Proceedings Citation Index. Abstracts and full texts were screened independently, in duplicate for inclusion. Extracted study characteristic and outcomes generated themes explored through subgroup analyses and idea webbing.Results1826 of 1921 studies were excluded after title and abstract review (kappa 0.77) and 47 after full-text review (kappa 0.86). 20 studies were of sufficient quality for narrative synthesis. Clinical outcomes were not affected by SSC introduction in studies without implementation strategies. A comprehensive SSC implementation strategy in developing countries demonstrated improved outcomes in high-risk surgeries. Narrative synthesis suggests that meaningful compliance is inconsistently measured and rarely achieved. Strategies involving feedback improved compliance. Stakeholder-developed implementation strategies, including team-based education, achieved greater acceptance. Three studies suggest that parental involvement in the SSC is valued by parents, nurses and physicians and may improve patient safety.ConclusionsA SSC implementation strategy focused on paediatric patients and their families can achieve high acceptability and good compliance. SSCs’ role in improving measures of paediatric surgical outcome is not well established, but they may be effective when used within a comprehensive implementation strategy especially for high-risk patients in low-resource settings.

Download Full-text

Success and failure: a case study of two rural telemedicine projects

Journal of Telemedicine and Telecare ◽

10.1258/135763303767149906 ◽

2003 ◽

Vol 9 (3) ◽

pp. 125-129 ◽

Cited By ~ 26

Author(s):

Pamela Whitten ◽

Inez Adams

Keyword(s):

Data Collection ◽

Organizational Structure ◽

Health Organizations ◽

Telemedicine Application ◽

Clinical Utilization ◽

Steady Growth ◽

Multiple Data ◽

One Year ◽

Collection Strategies

We studied two rural telemedicine projects in the state of Michigan: one that enjoyed success and steady growth in activity, and one that experienced frustration and a lack of clinical utilization. Multiple data collection strategies were employed during study periods, which lasted approximately one year. Both projects enjoyed a grassroots approach and had dedicated project coordinators. However, the more successful project benefited from resources and expertise not available to the less successful project. In addition, the more successful project possessed a more formalized organizational structure for the telemedicine application. A comparison of the two projects leads to a simple conclusion. Telemedicine programmes are positioned within larger health organizations and do not operate in a vacuum. It is crucial that the organization in which it is intended to launch telemedicine is examined carefully first. Each organization operates within a larger environment, which is often constrained by fiscal, geographical and personnel factors. All these will affect the introduction of telemedicine.

Download Full-text