scholarly journals Measures of organizational culture, organizational climate, and implementation climate in behavioral health: A systematic review

2021 ◽  
Vol 2 ◽  
pp. 263348952110188
Author(s):  
Byron J Powell ◽  
Kayne D Mettert ◽  
Caitlin N Dorsey ◽  
Bryan J Weiner ◽  
Cameo F Stanick ◽  
...  
Keyword(s):  
Organizational Culture ◽  
Organizational Climate ◽  
Psychometric Properties ◽  
Behavioral Health ◽  
Implementation Research ◽  
Learning Climate ◽  
Evidence Based ◽  
Implementation Climate ◽  
Relative Priority ◽  
Evidence Based Practices

Background: Organizational culture, organizational climate, and implementation climate are key organizational constructs that influence the implementation of evidence-based practices. However, there has been little systematic investigation of the availability of psychometrically strong measures that can be used to assess these constructs in behavioral health. This systematic review identified and assessed the psychometric properties of measures of organizational culture, organizational climate, implementation climate, and related subconstructs as defined by the Consolidated Framework for Implementation Research (CFIR) and Ehrhart and colleagues. Methods: Data collection involved search string generation, title and abstract screening, full-text review, construct assignment, and citation searches for all known empirical uses. Data relevant to nine psychometric criteria from the Psychometric and Pragmatic Evidence Rating Scale (PAPERS) were extracted: internal consistency, convergent validity, discriminant validity, known-groups validity, predictive validity, concurrent validity, structural validity, responsiveness, and norms. Extracted data for each criterion were rated on a scale from −1 (“poor”) to 4 (“excellent”), and each measure was assigned a total score (highest possible score = 36) that formed the basis for head-to-head comparisons of measures for each focal construct. Results: We identified full measures or relevant subscales of broader measures for organizational culture ( n = 21), organizational climate ( n = 36), implementation climate ( n = 2), tension for change ( n = 2), compatibility ( n = 6), relative priority ( n = 2), organizational incentives and rewards ( n = 3), goals and feedback ( n = 3), and learning climate ( n = 2). Psychometric evidence was most frequently available for internal consistency and norms. Information about other psychometric properties was less available. Median ratings for psychometric properties across categories of measures ranged from “poor” to “good.” There was limited evidence of responsiveness or predictive validity. Conclusion: While several promising measures were identified, the overall state of measurement related to these constructs is poor. To enhance understanding of how these constructs influence implementation research and practice, measures that are sensitive to change and predictive of key implementation and clinical outcomes are required. There is a need for further testing of the most promising measures, and ample opportunity to develop additional psychometrically strong measures of these important constructs. Plain Language Summary Organizational culture, organizational climate, and implementation climate can play a critical role in facilitating or impeding the successful implementation and sustainment of evidence-based practices. Advancing our understanding of how these contextual factors independently or collectively influence implementation and clinical outcomes requires measures that are reliable and valid. Previous systematic reviews identified measures of organizational factors that influence implementation, but none focused explicitly on behavioral health; focused solely on organizational culture, organizational climate, and implementation climate; or assessed the evidence base of all known uses of a measure within a given area, such as behavioral health–focused implementation efforts. The purpose of this study was to identify and assess the psychometric properties of measures of organizational culture, organizational climate, implementation climate, and related subconstructs that have been used in behavioral health-focused implementation research. We identified 21 measures of organizational culture, 36 measures of organizational climate, 2 measures of implementation climate, 2 measures of tension for change, 6 measures of compatibility, 2 measures of relative priority, 3 measures of organizational incentives and rewards, 3 measures of goals and feedback, and 2 measures of learning climate. Some promising measures were identified; however, the overall state of measurement across these constructs is poor. This review highlights specific areas for improvement and suggests the need to rigorously evaluate existing measures and develop new measures.

scholarly journals Organizational Factors Influencing Implementation of Evidence-Based Practices for Integrated Treatment in Behavioral Health Agencies

2014 ◽  
Vol 2014 ◽  
pp. 1-9 ◽  
Author(s):  
Caroline A. Bonham ◽  
David Sommerfeld ◽  
Cathleen Willging ◽  
Gregory A. Aarons
Keyword(s):  
New Mexico ◽  
Organizational Climate ◽  
Behavioral Health ◽  
Organizational Factors ◽  
Integrated Treatment ◽  
Financial Resources ◽  
Evidence Based ◽  
Mixed Method Research ◽  
Evidence Based Practices ◽  
Community Agencies

Objective. In recent years, New Mexico has prioritized integrated treatment for cooccurring mental health and substance use disorders within its public behavioral health system. This report describes factors likely to be important when implementing evidence-based practices (EBPs) in community agencies.Methods. Our mixed-method research design consisted of observations, semistructured interviews, and surveys undertaken with employees at 14 agencies at baseline and after 18 months. We developed four-agency typologies based on iterative coding and analysis of observations and interviews. We then examined survey data from employees at the four exemplar agencies to validate qualitative findings.Results. Financial resources and strong leadership impacted agency capacity to train providers and implement EBPs. Quantitative analysis of service provider survey responses from these agencies (N= 38) supported qualitative findings and demonstrated significant mean score differences in leadership, organizational climate, and attitudes toward EBPs in anticipated directions.Conclusion. The availability of strong leadership and financial resources were key components to initial implementation success in this study of community agencies in New Mexico. Reliance only on external funding poses risks for sustainment when demoralizing work climates precipitate employee turnover. Strong agency leadership does not always compensate for deficient financial resources in vulnerable communities.

scholarly journals Measuring implementation climate: psychometric properties of the Implementation Climate Scale (ICS) in Norwegian mental health care services

2022 ◽  
Vol 22 (1) ◽  
Author(s):  
Nadina Peters ◽  
Randi Hovden Borge ◽  
Ane- Marthe Solheim Skar ◽  
Karina M. Egeland
Keyword(s):  
Mental Health ◽  
Psychometric Properties ◽  
Concurrent Validity ◽  
Evidence Based ◽  
Implementation Climate ◽  
Mental Health Clinics ◽  
Evidence Based Practices ◽  
Health Clinics ◽  
Norwegian Version ◽  
Climate Scale

AbstractBackgroundEmployees’ perceptions of organizational climate for implementation of new methods are important in assessing and planning for implementation efforts. More specifically, feedback from employees’ points to which implementation strategies to select, adopt, and tailor in building positive climate for implementation of new evidence-based practices within the organization. Implementation climate can be measured with the Implementation Climate Scale (ICS). The purpose of this study was to investigate the psychometric properties of the Norwegian version of the ICS in outpatient mental health clinics.MethodsThe ICS was administered to 383 clinicians within 47 different child and adult mental health clinics across the country. We conducted confirmatory factor analysis to assess the psychometric functioning of the ICS. Cronbach’s alpha was examined to assess internal consistency. We also examined criterion related validity of the scale by comparing it with an alternative measure of implementation climate (concurrent validity) and by examining correlations with clinicians’ intentions to use evidence-based practices.ResultsResults supported the 6-factor structure and the internal consistency reliability of the ICS. One exception was poor functioning of the Reward scale. Concurrent validity was stronger at the group than at the individual level, and assessment of associations with clinicians’ intentions to use evidence- based practices showed positive correlations.ConclusionsThe Norwegian version of the ICS is a promising tool for assessing implementation climate which can provide organizations with specific feedback concerning which aspects of the implementation climate to attend to. Due to poor functioning of the Reward scale, adaptations and further testing of this is recommended.

What Makes or Breaks Evidence-Based Supervision? Staff and Organizational Predictors of Evidence-Based Practice in Probation

2021 ◽  
pp. 0306624X2110491
Author(s):  
Sara Debus-Sherrill ◽  
Alex Breno ◽  
Faye S. Taxman
Keyword(s):  
Organizational Culture ◽  
Survey Data ◽  
Evidence Based Practice ◽  
Organizational Factors ◽  
Evidence Based ◽  
Evidence Based Practices ◽  
Weak Evidence ◽  
The Impact

Research on staff and organizational factors that affect receptivity, adoption, feasibility, and utilization of innovations in justice settings is limited. This study uses survey data from 349 employees in one probation agency to assess how staff and perceived organizational factors influence attitudes related to evidence-based practices (EBPs) and their self-reported use. Staff characteristics, including education and knowledge about EBPs, and perceptions of the organization, including cynicism about the organization’s ability to change, predicted EBP outcomes. Staff age, tenure at the agency, and caseload size affected perceptions of organizational culture, but did not predict attitudes or use of EBPs. There is weak evidence for a relationship between self-reported use of EBPs with attitudinal support for EBPs, prior EBP training, and knowledge of EBPs. This study contributes to an emerging body of literature about the impact of various individual and organizational factors on support for EBPs with important lessons for implementation.

scholarly journals Non-participants in policy efforts to promote evidence-based practices in a large behavioral health system

2017 ◽  
Vol 12 (1) ◽  
Author(s):  
Rebecca E. Stewart ◽  
Danielle R. Adams ◽  
David S. Mandell ◽  
Gayatri Nangia ◽  
Lauren Shaffer ◽  
...  
Keyword(s):  
Health System ◽  
Behavioral Health ◽  
Evidence Based ◽  
Evidence Based Practices ◽  
Behavioral Health System

scholarly journals A scoping review of strategies for financing the implementation of evidence-based practices in behavioral health systems: State of the literature and future directions

2020 ◽  
Vol 1 ◽  
pp. 263348952093998
Author(s):  
Alex R Dopp ◽  
Marie-Rachelle Narcisse ◽  
Peter Mundey ◽  
Jane F Silovsky ◽  
Allison B Smith ◽  
...  
Keyword(s):  
Behavioral Health ◽  
Health Systems ◽  
Cost Sharing ◽  
Research Literature ◽  
Evidence Based ◽  
Fee For Service ◽  
Evidence Based Practices ◽  
Level Of Use ◽  
System Leaders ◽  
Ebp Implementation

Background: Increased availability of evidence-based practices (EBPs) is essential to alleviating the negative public health and societal effects of behavioral health problems. A major challenge to implementing and sustaining EBPs broadly is the limited and fragmented nature of available funding. Method: We conducted a scoping review that assessed the current state of evidence on EBP financing strategies for behavioral health based on recent literature (i.e., post-Affordable Care Act). We defined financing strategies as techniques that secure and direct financial resources to support EBP implementation. This article introduces a conceptualization of financing strategies and then presents a compilation of identified strategies, following established reporting guidelines for the implementation strategies. We also describe the reported level of use for each financing strategy in the research literature. Results: Of 23 financing strategies, 13 were reported as being used within behavioral health services, 4 had potential for use, 5 had conceptual use only, and 1 was potentially contraindicated. Examples of strategies reported being used include increased fee-for-service reimbursement, grants, cost sharing, and pay-for-success contracts. No strategies had been evaluated in ways that allowed for strong conclusions about their impact on EBP implementation outcomes. Conclusion: The existing literature on EBP financing strategies in behavioral health raises far more questions than answers. Therefore, we propose a research agenda that will help better understand these financing strategies. We also discuss the implications of our findings for behavioral health professionals, system leaders, and policymakers who want to develop robust, sustainable financing for EBP implementation in behavioral health systems. Plain language abstract: Organizations that treat behavioral health problems (mental health and substance use) often seek to adopt and use evidence-based practices (EBPs). A challenge to adopting EBPs broadly is the limited funding available, often from various sources that are poorly coordinated with one another. To help organizations plan effectively to adopt EBPs, we conducted a review of recent evidence (i.e., since the passage of the 2010 Affordable Care Act) on strategies for financing EBP adoption in behavioral health systems. We present definitions of 23 identified strategies and describe each strategy’s reported (in the research literature) level of use to fund EBP adoption in behavioral health services. Of the 23 financing strategies, 13 strategies had evidence of use, 4 had potential for use, 5 had conceptual use only, and 1 was potentially contraindicated. Examples of strategies with evidence of use include increased fee-for-service reimbursement, grants, cost sharing, and pay-for-success contracts. This comprehensive list of EBP financing strategies may help guide decision-making by behavioral health professionals, system leaders, and policymakers. The article also presents a research agenda for building on the current research literature by (1) advancing methods to evaluate financing strategies’ effects, (2) partnering with stakeholders and decision-makers to examine promising financing strategies, (3) focusing on strategies and service systems with the greatest needs, (4) improving methods to guide the selection of financing strategies, and (5) paying greater attention to sustainable long-term financing of EBPs.

Furthering Dissemination and Implementation Research

2017 ◽  
Author(s):  
Lawrence W. Green ◽  
Mona Nasser
Keyword(s):  
Systematic Reviews ◽  
Implementation Research ◽  
External Validity ◽  
Evidence Based Practice ◽  
Internal Validity ◽  
Implementation Process ◽  
Dissemination And Implementation ◽  
Evidence Based ◽  
Evidence Based Practices ◽  
Dissemination And Implementation Research

This chapter raises questions about the reliability of much “evidence-based practice” disseminated from the original studies and systematic reviews of those studies, insofar as they were often conducted and reviewed with inadequate attention to external validity. Important issues are raised for dissemination and implementation researchers. Indeed, the pressure on investigators to provide for increasingly rigorous controls on threats to internal validity, and to exclude studies that fall below standards for internal validity, has made many such sources of evidence more suspect in their external validity and less credible to the practitioners or policymakers who would adopt them. Greater attention is needed to ways to incorporate considerations of external validity into studies and in systematic reviews of studies to produce more generalizable evidence, and greater attention to practice-based evidence that can complement the more formal evidence-based practices in the process of implementing and evaluating the dissemination and implementation process

Sign in / Sign up

Export Citation Format

Share Document