An Overview of Implementation Fidelity

There is substantial variability in the implementation of evidence-based interventions across the United States, which leads to inconsistent access to evidence-based prevention and treatment strategies at a population level. Increased dissemination and implementation of evidence-based interventions could result in significant public health gains. While the availability of evidence-based interventions is increasing, study of implementation, adaptation, and dissemination has only recently gained attention in public health. To date, insufficient attention has been given to the issue of fidelity. Consideration of fidelity is necessary to balance the need for internal and external validity across the research continuum. There is also a need for a more robust literature to increase knowledge about factors that influence fidelity, strategies for maximizing fidelity, methods for measuring and analyzing fidelity, and examining sources of variability in implementation fidelity.

Download Full-text

Formative process evaluation of a guideline-driven process for improving the cultural responsiveness of alcohol and drug treatment services

BMC Health Services Research ◽

10.1186/s12913-021-06367-7 ◽

2021 ◽

Vol 21 (1) ◽

Author(s):

Sara Farnbach ◽

Julaine Allan ◽

Raechel Wallace ◽

Alexandra Aiken ◽

Anthony Shakeshaft

Keyword(s):

Culturally Responsive ◽

Implementation Fidelity ◽

Healthcare Services ◽

Cultural Responsiveness ◽

Free Text ◽

Structured Interviews ◽

Culturally Responsive Practice ◽

Step Process ◽

Treatment Services ◽

Service Staff

Abstract Background To improve Australian Aboriginal and Torres Strait Islander people’s access to, and experience of, healthcare services, including Alcohol and other Drug (AoD) treatment services, principles and frameworks have been developed to optimise cultural responsiveness. Implementing those principles in practice, however, can be difficult to achieve. This study has five aims: i) to describe a five-step process developed to operationalise improvements in culturally responsive practice in AoD services; ii) to evaluate the fidelity of implementation for this five-step process; iii) to identify barriers and enablers to implementation; iv) to assess the feasibility and acceptability of this approach; and v) to describe iterative adaptation of implementation processes based on participant feedback. Methods Participating services were 15 non-Aboriginal AoD services in New South Wales, Australia. Implementation records were used to assess the implementation fidelity of the project. Structured interviews with chief executive officers or senior management were conducted, and interview data were thematically analysed to identify project acceptability, and the key enablers of, and barriers to, project implementation. Quantitative descriptive analyses were performed on the post-implementation workshop survey data, and responses to the free text questions were thematically analysed. Results A high level of implementation fidelity was achieved. Key enablers to improving culturally responsive practice were the timing of the introduction of the five-step process, the active interest of staff across a range of seniority and the availability of resources and staff time to identify and implement activities. Key barriers included addressing the unique needs of a range of treatment sub-groups, difficulty adapting activities to different service delivery models, limited time to implement change in this evaluation (three months) and the varied skill level across staff. The project was rated as being highly acceptable and relevant to service CEOs/managers and direct service staff, with planned changes perceived to be achievable and important. Based on CEO/management feedback after the project was implemented at the initial services, several improvements to processes were made. Conclusion The operationalisation of the five-step process developed to improve cultural responsiveness was feasible and acceptable and may be readily applicable to improving the cultural responsiveness of a wide variety of health and human services.

Download Full-text

A systematic approach to context-mapping to prepare for health interventions: development and validation of the SETTING-tool in four countries

BMJ Global Health ◽

10.1136/bmjgh-2020-003221 ◽

2021 ◽

Vol 6 (1) ◽

pp. e003221

Author(s):

Evelyn A Brakema ◽

Rianne MJJ van der Kleij ◽

Charlotte C Poot ◽

Niels H Chavannes ◽

Ioanna Tsiligianni ◽

...

Keyword(s):

Health Workers ◽

Rapid Assessment ◽

Implementation Strategies ◽

Implementation Fidelity ◽

Chronic Respiratory Disease ◽

Health Interventions ◽

Successful Implementation ◽

Mapping Tool ◽

Implementation Failure ◽

Development And Validation

Effectiveness of health interventions can be substantially impaired by implementation failure. Context-driven implementation strategies are critical for successful implementation. However, there is no practical, evidence-based guidance on how to map the context in order to design context-driven strategies. Therefore, this practice paper describes the development and validation of a systematic context-mapping tool. The tool was cocreated with local end-users through a multistage approach. As proof of concept, the tool was used to map beliefs and behaviour related to chronic respiratory disease within the FRESH AIR project in Uganda, Kyrgyzstan, Vietnam and Greece. Feasibility and acceptability were evaluated using the modified Conceptual Framework for Implementation Fidelity. Effectiveness was assessed by the degree to which context-driven adjustments were made to implementation strategies of FRESH AIR health interventions. The resulting Setting-Exploration-Treasure-Trail-to-Inform-implementatioN-strateGies (SETTING-tool) consisted of six steps: (1) Coset study priorities with local stakeholders, (2) Combine a qualitative rapid assessment with a quantitative survey (a mixed-method design), (3) Use context-sensitive materials, (4) Collect data involving community researchers, (5) Analyse pragmatically and/or in-depth to ensure timely communication of findings and (6) Continuously disseminate findings to relevant stakeholders. Use of the tool proved highly feasible, acceptable and effective in each setting. To conclude, the SETTING-tool is validated to systematically map local contexts for (lung) health interventions in diverse low-resource settings. It can support policy-makers, non-governmental organisations and health workers in the design of context-driven implementation strategies. This can reduce the risk of implementation failure and the waste of resource potential. Ultimately, this could improve health outcomes.

Download Full-text

Rapid Realist Review of School-Based Physical Activity Interventions in 7- to 11-Year-Old Children

Children ◽

10.3390/children8010052 ◽

2021 ◽

Vol 8 (1) ◽

pp. 52

Author(s):

Emmanuel Defever ◽

Michelle Jones

Keyword(s):

Physical Activity ◽

School Leadership ◽

School Environment ◽

Meta Analysis ◽

Implementation Fidelity ◽

Realist Review ◽

School Level ◽

School Settings ◽

Physical Activity Interventions ◽

Rapid Realist Review

Meta-analysis of physical activity interventions in school settings have revealed low efficacy and that there is a need to explore implementation fidelity. The aim of this rapid realist review was to determine, what physical activity interventions in school settings for children aged 7- to 11-years-old works, for whom, and in what circumstances. The realist synthesis was conducted following RAMESES guidelines. Relevant studies were identified following a systematic search process and data from 28 studies was extracted for evidence to form context-mechanism-outcome configurations that were clustered and refined. Using the five-level socioecological model, the program theories were classified into the levels of intrapersonal (child), interpersonal (teachers), institutional (program content, school administration, and school environment), community (home and neighborhood), and policy. The school level led to most context-mechanism-outcome configurations related to school leadership and policy, workforce structure, program characteristics, and school environment. At each level, we identified features of interventions, alongside implementation considerations that might work to promote efficacy and sustainability. The need to recognize the school environment as part of a complex system with multi-level interaction and influences was a key finding. In line with realist philosophy, the researchers encouraged primary research to confirm, refute, and refine the program theories presented.

Download Full-text

After the Race to the Top: State and District Capacity to Sustain Professional Development Innovation in Florida

Educational Policy ◽

10.1177/08959048211015619 ◽

2021 ◽

pp. 089590482110156

Author(s):

Motoko Akiba ◽

Cassandra Howard

Keyword(s):

Professional Development ◽

Lesson Study ◽

Student Outcomes ◽

Instructional System ◽

Implementation Fidelity ◽

Race To The Top ◽

Mixed Methods Study ◽

Policy Implications ◽

Federally Funded ◽

Strategic Adaptation

The Race to the Top (RTTT) program incentivized states to use innovation for systemwide improvement of student outcomes, but little is known about how RTTT-funded innovation was sustained after the RTTT program ended. This mixed-methods study examined state and district approaches to sustaining an international innovation called lesson study, a teacher-driven, collaborative, inquiry-based teacher learning process imported from Japan and promoted statewide in Florida. While the state’s role in sustaining lesson study was limited, we found that districts that integrated lesson study into the district instructional system through a clear expectation and strategic adaptation, supported school and teacher ownership of lesson study practice, and provided necessary support and funding were more likely to sustain lesson study. In contrast, the districts that focused on implementation fidelity and district-led facilitation eventually phased out lesson study. Policy implications for sustaining federally funded professional development innovations are discussed.

Download Full-text

Why Does Construct Validity Matter in Measuring Implementation Fidelity? A Methodological Case Study

Assessment for Effective Intervention ◽

10.1177/1534508421998772 ◽

2021 ◽

pp. 153450842199877

Author(s):

Wilhelmina van Dijk ◽

A. Corinne Huggins-Manley ◽

Nicholas A. Gage ◽

Holly B. Lane ◽

Michael Coyne

Keyword(s):

Construct Validity ◽

Reading Instruction ◽

Model Building ◽

Implementation Fidelity ◽

Modeling Framework ◽

Relative Rank ◽

Rank Ordering ◽

Adherence Data ◽

Funded Project

In reading intervention research, implementation fidelity is assumed to be positively related to student outcomes, but the methods used to measure fidelity are often treated as an afterthought. Fidelity has been conceptualized and measured in many different ways, suggesting a lack of construct validity. One aspect of construct validity is the fidelity index of a measure. This methodological case study examined how different decisions in fidelity indices influence relative rank ordering of individuals on the construct of interest and influence our perception of the relation between the construct and intervention outcomes. Data for this study came from a large State-funded project to implement multi-tiered systems of support for early reading instruction. Analyses were conducted to determine whether the different fidelity indices are stable in relative rank ordering participants and if fidelity indices of dosage and adherence data influence researcher decisions on model building within a multilevel modeling framework. Results indicated that the fidelity indices resulted in different relations to outcomes with the most commonly used fidelity indices for both dosage and adherence being the worst performing. The choice of index to use should receive considerable thought during the design phase of an intervention study.

Download Full-text

Effects of Teachers as Coaches for Paraprofessionals Implementing Functional Communication Training

Journal of Positive Behavior Interventions ◽

10.1177/1098300720983538 ◽

2021 ◽

pp. 109830072098353

Author(s):

Emily Gregori ◽

Mandy J. Rispoli ◽

Catharine Lory ◽

So Yeon Kim ◽

Marie David

Keyword(s):

Special Education Teachers ◽

Challenging Behavior ◽

Professional Growth ◽

Implementation Fidelity ◽

Multiple Baseline ◽

Functional Communication ◽

Baseline Design ◽

Behavioral Needs ◽

Teacher Coaching ◽

Child Engagement

Young children with intensive behavioral needs are often served by paraprofessionals, who are typically among the least trained and least supported staff in the school. While professional development opportunities to address challenging behavior are generally limited for special education teachers, such professional growth opportunities are often completely unavailable for paraprofessionals. The purpose of this study was to evaluate a program in which teachers served as coaches for paraprofessionals. Using a multiple-baseline design across paraprofessional-and-child dyads, we evaluated the effects of the teachers-as-coaches program on paraprofessional implementation fidelity and child engagement in challenging behavior and appropriate communication. The results showed that with teacher coaching, paraprofessionals increased their behavior intervention implementation fidelity to 100%. During the maintenance probes, two of the paraprofessionals implemented the intervention with 100% fidelity, and the third paraprofessional implemented the intervention with higher fidelity than baseline. Results also showed corresponding decreases in child challenging behavior and increases in appropriate communication. Implications for research and practice in supporting paraprofessionals are discussed.

Download Full-text