Assessing Causal Pathways and Targets of Implementation Variability for EBP use (Project ACTIVE): a study protocol

Background Advancing causal implementation theory is critical for designing tailored implementation strategies that target specific mechanisms associated with evidence-based practice (EBP) use. This study will test the generalizability of a conceptual model that integrates organizational constructs and behavioral theory to predict clinician use of cognitive-behavioral therapy (CBT) techniques in community mental health centers. CBT is a leading psychosocial EBP for psychiatric disorders that remains underused despite substantial efforts to increase its implementation. Methods We will leverage ongoing CBT implementation efforts in two large public health systems (Philadelphia and Texas) to recruit 300 mental health clinicians and 600 of their clients across 40 organizations. Our primary implementation outcomes of interest are clinician intentions to use CBT and direct observation of clinician use of CBT. As CBT comprises discrete components that vary in complexity and acceptability, we will measure clinician use of six discrete components of CBT. After finishing their CBT training, participating clinicians will complete measures of organizational and behavior change constructs delineated in the model. Clinicians also will be observed twice via audio recording delivering CBT with a client. Within 48 h of each observation, theorized moderators of the intention-behavior gap will be collected via survey. A subset of clinicians who report high intentions to use CBT but demonstrate low use will be purposively recruited to complete semi-structured interviews assessing reasons for the intention-behavior gap. Multilevel path analysis will test the extent to which intentions and determinants of intention predict the use of each discrete CBT component. We also will test the extent to which theorized determinants of intention that include psychological, organizational, and contextual factors explain variation in intention and moderate the association between intentions and CBT use. Discussion Project ACTIVE will advance implementation theory, currently in its infancy, by testing the generalizability of a promising causal model of implementation. These results will inform the development of implementation strategies targeting modifiable factors that explain substantial variance in intention and implementation that can be applied broadly across EBPs.

Social impact measurement in social housing: a theory-based investigation into the context, mechanisms and outcomes of implementation

Purpose English social housing providers are increasingly turning to social impact measurement to assess their social value. This paper aims to understand the contextual factors causing this rise in the practice, specifically within this sector; the mechanisms that enable it to be effectively implemented within an individual organisation and the outcomes of successful implementation for individual organisations and more widely across the sector and beyond. Design/methodology/approach A realist theory-based approach is applied to the study of a small number of social housing organisations and leaders within the sector to explore the use of social impact measurement. The paper addresses three questions: Why is social impact measurement being adopted in this sector? How is it successfully implemented? And what happens (outcomes) when it is successfully implemented? Addressing these questions necessitates deeper insight into the contextual pressures that have brought to the fore social impact measurement within the sector and the beneficial outcomes the practice provides (or is anticipated to provide) to social housing providers. The methodological approach of Realist Evaluation (Pawson and Tilley, 1997, 2004) is used to structure and analyse the empirical data and findings into a programme theory for social impact measurement. Realist Evaluation provides a programme theory perspective, seeking to answer the question “what works, for whom and in what circumstances?”. In this research, the “whom” refers to English social housing providers and the circumstances are the contextual conditions experienced by the sector over the past decade. The programme theory aims to set out the links between the contextual drivers for social impact measurement, the mechanisms that bring about its implementation and the outcomes that occur as a result. Within this, greater detail on the implementation perspective is provided by developing an implementation theory using a Theory of Change approach (Connell et al., 1995; Fulbright-Anderson et al., 1998). The implementation theory is then embedded within the wider programme theory so as to bring the two elements together, thereby creating a refinement of the overall theory for social impact measurement. In turn, this paper demonstrates its importance (the outcomes that it can achieve for organisations and the sector) and how it can effectively be implemented to bring about those outcomes. Findings Social housing providers use social impact measurement both internally, to determine their organisational priorities and externally, to demonstrate their value to local and national governments and cross-sector partners then to shape and influence resource allocation. The practice itself is shown to be an open and active programme, rather than a fixed calculative practice. Research limitations/implications The intensive nature of the research means that only a limited number of cases were explored. Further research could test theories developed here against evidence collected from a wider range of cases, e.g. other types of providers or non-adopters. Practical implications The research makes a strong contribution to practice in the form of a re-conceptualisation of how social impact measurement can be shown to be effective, based on a deeper understanding of causal mechanisms, how they interact and the outcomes that result. This is of value to the sector as such information could help other organisations both to understand the value of social impact measurement and to provide practical guidance on how to implement it effectively. Social implications As the practice of impact measurement continues to develop, practitioners will need to be aware of any changes to these contextual factors and consider questions such as: is the context still supportive of impact measurement? Does the practice need to be adjusted to meet the needs of the current context? For instance, the recent tragedy at Grenfell Tower has led to a reconsideration of the role of social housing; a new Green Paper is currently being drafted (Ministry for Housing, Communities and Local Government, 2018). This may have a number of implications for social impact measurement, such as a rebalancing of emphasis on outcomes relating to environmental improvements, towards outcomes relating to the well-being of tenants. Originality/value Existing literature is largely limited to technical guides. This paper links theory-based evaluation to practice contributing to social housing practice.

Using Integrative Implementation Theory to Understand New Jersey's 2015 Opt-Out Movement

Background/Context In the spring of 2015, about 135,000 New Jersey students—almost 20% of the test-eligible children—did not take the state's test. Opposition of this magnitude directly contradicted a central stipulation of the federal No Child Left Behind Act of 2001 (NCLB), which required states to test 95% of their eligible students to receive federal education funding. This article examines what happened in New Jersey in 2015 to produce such a large change in the implementation of state testing policy. Purpose/Objective/Research Question/Focus of Study In this article, I use Richard Matland's integrative policy implementation theory to explain the circumstances that produced such a large change in the enactment of New Jersey testing policy in 2015. More specifically, I focus on two research questions: (1) What were the major national, state, and local factors that contributed to parent and student opt-out decisions in New Jersey in 2015? (2) How does integrative implementation theory help us to understand the different circumstances contributing to the opt-out movement in New Jersey in 2015? Population/Participants/Subjects The article is based upon extensive document review and 33 interviews with New Jersey state policymakers, professional education association representatives, advocacy group leaders, school administrators, teachers, parents, and students. Intervention/Program/Practice: Since the 1990s, testing has become an increasingly important function of state education policy. High-stakes state testing policies were further expanded under the NCLB, which required states to test 95% of their eligible students to receive streams of federal education funding. The opt-out movement across the nation in 2015 was a major departure from this well-established policy. Research Design This study employs qualitative interpretive research to examine how multiple actors at different levels of the New Jersey education system understood and interpreted the 2015 opt-out movement. Using Matland's ambiguity-conflict model of policy implementation I interpret the interactions between the policy design and local implementers’ beliefs and goals. Findings/Results The findings illuminate the shifting national, state, and local factors that contributed to district opt-out rates as well as variation across school levels and districts with different socioeconomic conditions. The combination of increased federal press on states, New Jersey's fast timelines for new standards and assessment adoption even as it ratcheted up accountability, and inconsistent state policy signals all contributed to backlash from teachers, community members, and anti-testing advocates. These factors illuminate many of the changing circumstances in New Jersey that fueled the dramatic opt-out movement in 2015. Conclusions/Recommendations While Matland's integrative implementation model helps to explain the dynamics of the 2015 New Jersey opt-out movement, it does not account for additional contributing factors including changes over time, dynamics at different system levels, and consideration of a broader range of actors beyond policymakers and policy targets. Incorporating these factors can help make integrative implementation theories even more robust.

Exploring frontliners' knowledge, participation and evaluation in the implementation of a pay-for-performance program (PMAQ) in primary health care in Brazil

PurposeThis paper employs implementation theory and the political literature on performance measurement to understand how frontline health workers know, participate and evaluate the Brazilian National Program for Improving Access and Quality of Primary Care (PMAQ, 2nd round).Design/methodology/approachThis paper develops an implementation theory-driven qualitative analysis. The research is developed in the city of Goiania (Brazil): a challenging organizational context in primary care (PHC). Interviews were carried out with 25 frontliners – managers, doctors, nurses and community health workers. Data were thematically and hierarchically analysed according to theoretical concepts such as policy knowledge, policy adherence, forms of accountability, alternative logics, organizational capacity and policy feedback.FindingsResults show the need to foster organizational capacity, knowledge, participation and policy feedback at the frontline. Successful implementation would require those adaptations to counteract policy challenges/failures or the emergence of alternative logics.Research limitations/implicationsThe study was conducted in only one setting, however, our sample includes different types of professionals working in units with different levels of organization capacity, located in distinct HDs, expressing well the implementation of PMAQ/P4P. Qualitative researches need to be developed for further exploring the same/other factors.Social implicationsFindings can be used to improve discussions/planning and design of P4P programs in the city and State of Goias.Originality/valueThe majority of analysis of PMAQ are of a quantitative or results-based nature. This article focuses on politically significant and unanswered questions regarding the implementation of PMAQ.

Understanding the implementation of evidence-informed policies and practices from a policy perspective: a critical interpretive synthesis

Background The fields of implementation science and knowledge translation have evolved somewhat independently from the field of policy implementation research, despite calls for better integration. As a result, implementation theory and empirical work do not often reflect the implementation experience from a policy lens nor benefit from the scholarship in all three fields. This means policymakers, researchers, and practitioners may find it challenging to draw from theory that adequately reflects their implementation efforts. Methods We developed an integrated theoretical framework of the implementation process from a policy perspective by combining findings from these fields using the critical interpretive synthesis method. We began with the compass question: How is policy currently described in implementation theory and processes and what aspects of policy are important for implementation success? We then searched 12 databases as well as gray literature and supplemented these documents with other sources to fill conceptual gaps. Using a grounded and interpretive approach to analysis, we built the framework constructs, drawing largely from the theoretical literature and then tested and refined the framework using empirical literature. Results A total of 11,434 documents were retrieved and assessed for eligibility and 35 additional documents were identified through other sources. Eighty-six unique documents were ultimately included in the analysis. Our findings indicate that policy is described as (1) the context, (2) a focusing lens, (3) the innovation itself, (4) a lever of influence, (5) an enabler/facilitator or barrier, or (6) an outcome. Policy actors were also identified as important participants or leaders of implementation. Our analysis led to the development of a two-part conceptual framework, including process and determinant components. Conclusions This framework begins to bridge the divide between disciplines and provides a new perspective about implementation processes at the systems level. It offers researchers, policymakers, and implementers a new way of thinking about implementation that better integrates policy considerations and can be used for planning or evaluating implementation efforts.

Factors influencing allied health professionals’ implementation of upper limb sensory rehabilitation for stroke survivors: a qualitative study to inform knowledge translation

ObjectivesSomatosensory loss is common after stroke with one-in-two individuals affected. Although clinical practice guidelines recommend providing somatosensory rehabilitation, this impairment often remains unassessed and untreated. To address the gap between guideline recommendations and clinical practice, this study sought to understand the factors influencing delivery of evidence-based upper limb sensory rehabilitation after stroke.DesignQualitative study involving focus groups and interviews. Data analysis used an inductive approach (thematic analysis) and deductive analysis using implementation theory (the Theoretical Domains Framework and Normalisation Process Theory).SettingEight healthcare organisations in metropolitan and regional areas of Victoria and New South Wales, Australia.ParticipantsEighty-seven rehabilitation therapists (79% occupational therapists and 21% physiotherapists) were purposively sampled and participated in a knowledge translation study with staggered recruitment from 2014 to 2018.ResultsThree types of factors influenced therapists’ delivery of upper limb somatosensory rehabilitation: individual (‘The uncertain, unskilled therapist’), patient (‘Patient understanding and priorities’) and organisational (‘System pressures and resources’). Deductive analysis using implementation theory identified key determinants of practice change, such as opportunities to consolidate new skills, the anticipated benefits of upskilling as a therapy team and the work anticipated by therapists to incorporate a new somatosensory rehabilitation approach.ConclusionsOccupational therapists and physiotherapists hold valuable insights towards practice change in somatosensory rehabilitation from the ‘frontline’. Therapists experience barriers to change including a lack of knowledge and skills, lack of resources and organisational pressures. Facilitators for change were identified, including social support and therapists’ perceived legitimacy in using new somatosensory rehabilitation approaches. Results will inform the design of a tailored implementation strategy to increase the use of evidence-based somatosensory rehabilitation in Australia.Trial registration numberAustralian New Zealand Clinical Trials Registry (ACTRN12615000933550).

Understanding the implementation of evidence-informed policies and practices from a policy perspective: a critical interpretive synthesis

Background: The fields of implementation science and knowledge translation have evolved somewhat independently from the field of policy implementation research, despite calls for better integration. As a result, implementation theory and empirical work do not often reflect the implementation experience from a policy lens nor benefit from the scholarship in all three fields. This means policymakers, researchers and practitioners may find it challenging to draw from theory that adequately reflects their implementation efforts.Methods: We developed an integrated theoretical framework of the implementation process from a policy perspective by combining findings from these fields using the critical interpretive synthesis method. We began with the compass question: how is policy currently described in implementation theory and processes and what aspects of policy are important for implementation success? We then searched 12 databases as well as grey literature and supplemented these documents with other sources to fill conceptual gaps. Using a grounded and interpretive approach to analysis, we built the framework constructs, drawing largely from the theoretical literature and then tested and refined the framework using empirical literature.Results: A total of 11,434 documents were retrieved and assessed for eligibility and 35 additional documents were identified through other sources. Eighty-six unique documents were ultimately included in the analysis. Our findings indicate that policy is described as: 1) the context; 2) a focusing lens; 3) the innovation itself; 4) a lever of influence; 5) an enabler/facilitator or barrier; or 6) an outcome. Policy actors were also identified as important participants or leaders of implementation. Our analysis led to the development of a two-part conceptual framework, including process and determinant components. Conclusions: This framework begins to bridge the divide between disciplines and provides a new perspective about implementation processes at the systems level. It offers researchers, policymakers and implementers a new way of thinking about implementation that better integrates policy considerations and can be used for planning or evaluating implementation efforts.

Understanding the implementation of evidence-informed policies and practices from a policy perspective: a critical interpretive synthesis

Background: The fields of implementation science and knowledge translation have evolved somewhat independently from the field of policy implementation research, despite calls for better integration. As a result, implementation theory and empirical work do not often reflect the implementation experience from a policy lens nor benefit from the scholarship in all three fields. This means policymakers, researchers and practitioners may find it challenging to draw from theory that adequately reflects their implementation efforts.Methods: We developed an integrated theoretical framework of the implementation process from a policy perspective by combining findings from these fields using the critical interpretive synthesis method. We began with the compass question: how is policy currently described in implementation theory and processes and what aspects of policy are important for implementation success? We then searched 12 databases as well as grey literature and supplemented these documents with other sources to fill conceptual gaps. Using a grounded and interpretive approach to analysis, we built the framework constructs, drawing largely from the theoretical literature and then tested and refined the framework using empirical literature.Results: A total of 11,434 documents were retrieved and assessed for eligibility and 35 additional documents were identified through other sources. Eighty-six unique documents were ultimately included in the analysis. Our findings indicate that policy is described as: 1) the context; 2) a focusing lens; 3) the innovation itself; 4) a lever of influence; 5) an enabler/facilitator or barrier; or 6) an outcome. Policy actors were also identified as important participants or leaders of implementation. Our analysis led to the development of a two-part conceptual framework, including process and determinant components. Conclusions: This framework begins to bridge the divide between disciplines and offers a new way of thinking about implementation processes at the systems level. It offers researchers, policymakers and implementers with a new way of thinking about implementation and can be used for planning or evaluating implementation efforts.

Understanding the implementation of evidence-informed policies and practices from a policy perspective: a critical interpretive synthesis

Background: The fields of implementation science and knowledge translation have evolved somewhat independently from the field of policy implementation research, despite calls for better integration. As a result, implementation theory and empirical work do not often reflect the implementation experience from a policy lens nor benefit from the scholarship in all three fields. This means policy-makers, researchers and practitioners may find it challenging to draw from theory that adequately reflects their implementation efforts.Methods: We developed an integrated theoretical framework of the implementation process from a policy perspective by combining findings from these fields using the critical interpretive synthesis method. We began with the compass question: how is policy currently described in implementation theory and processes and what aspects of policy are important for implementation success? We then searched 12 databases as well as grey literature and supplemented these documents with other sources to fill conceptual gaps. Using a grounded and interpretive approach to analysis, we built the framework constructs, drawing largely from the theoretical literature and then tested and refined the framework using empirical literature.Results: A total of 7850 documents were retrieved and assessed for eligibility and 34 additional documents were identified through other sources. Eighty-two unique documents were ultimately included in the analysis. Our findings indicate that policy is described as: 1) the context; 2) a focusing lens; 3) the innovation itself; 4) a lever of influence; 5) an enabler/facilitator or barrier; or 6) an outcome. Policy actors were also identified as important participants or leaders of implementation. Our analysis led to the development of a two-part conceptual framework, including process and determinant components. Conclusions: This framework begins to bridge the divide between disciplines and offers a new way of thinking about implementation processes at the systems level. It offers researchers, policy-makers and implementers with a new way of thinking about implementation and can be used for planning or evaluating implementation efforts.