“It’s good to feel like you’re doing something”: a qualitative study examining state health department employees’ views on why ineffective programs continue to be implemented in the USA

Background Mis-implementation, the inappropriate continuation of programs or policies that are not evidence-based or the inappropriate termination of evidence-based programs and policies, can lead to the inefficient use of scarce resources in public health agencies and decrease the ability of these agencies to deliver effective programs and improve population health. Little is known about why mis-implementation occurs, which is needed to understand how to address it. This study sought to understand the state health department practitioners’ perspectives about what makes programs ineffective and the reasons why ineffective programs continue. Methods Eight state health departments (SHDs) were selected to participate in telephone-administered qualitative interviews about decision-making around ending or continuing programs. States were selected based on geographic representation and on their level of mis-implementation (low and high) categorized from our previous national survey. Forty-four SHD chronic disease staff participated in interviews, which were audio-recorded and transcribed verbatim. Transcripts were consensus coded, and themes were identified and summarized. This paper presents two sets of themes, related to (1) what makes a program ineffective and (2) why ineffective programs continue to be implemented according to SHD staff. Results Participants considered programs ineffective if they were not evidence-based or if they did not fit well within the population; could not be implemented well due to program restraints or a lack of staff time and resources; did not reach those who could most benefit from the program; or did not show the expected program outcomes through evaluation. Practitioners described several reasons why ineffective programs continued to be implemented, including concerns about damaging the relationships with partner organizations, the presence of program champions, agency capacity, and funding restrictions. Conclusions The continued implementation of ineffective programs occurs due to a number of interrelated organizational, relational, human resources, and economic factors. Efforts should focus on preventing mis-implementation since it limits public health agencies’ ability to conduct evidence-based public health, implement evidence-based programs effectively, and reduce the high burden of chronic diseases. The use of evidence-based decision-making in public health agencies and supporting adaptation of programs to improve their fit may prevent mis-implementation. Future work should identify effective strategies to reduce mis-implementation, which can optimize public health practice and improve population health.

A program evaluation of the Innovative Teen Pregnancy Prevention Programs (iTP3) Project

Background Teen pregnancy prevention in the United States has traditionally focused on the development, testing, and subsequent implementation of a set of evidence-based programs (EBPs), recommended nationally. However, these existing EBPs often do not prioritize the most at-risk or vulnerable populations. Methods The Innovative Teen Pregnancy Prevention Programs (iTP3) project was funded to facilitate the development of new, innovative programs to reach disparate populations. Through a mixed methods design, iTP3 evaluated the process and resulting innovative programs from five iterative cohorts of funded organizations, referred to as Innovators. iTP3 utilized both a traditional funding model with more traditional methods of capacity building assistance, but transitioned over time to a design-focused funding model in which organizations and individuals developed innovative programs through an intensive human centered design process. Results Evaluation results showed that the resulting portfolio of programs had differences in the types of programs resulting from the differing funding models. Notable differences among programs from the two funding models include program length, along with personnel, time, and resources needed to develop and manage. Conclusion Both traditional and design funding models led to innovative programs, with notable differences in the development process and resulting programs.

Expanding Arizona’s Dementia Capable System

Currently, 5.8 million US adults live with Alzheimer’s disease (ADRD); the number is expected to double by 2050. Arizona will experience the greatest percent increase in ADRD by 2025. This project targeted three underserved groups in order to expand Arizona’s dementia capable system: people living alone with ADRD; people with Down Syndrome or another intellectual/developmental disability (DS/IDD) aging with ADRD and their family caregivers; and people with ADRD and their caregivers in the Latino community. This presentation describes the development and delivery of the project’s educational workshops, case management services, and evidence-based programs. Over 2,220 participants have participated in workshops to date with the largest percentage being case managers, care coordinators, and discharge planners. Evaluations have been extremely positive with 86.1% being “very likely” to recommend the project to others. The presentation concludes with findings and lessons learned regarding the delivery of the project’s evidence-based programs and case management services.

Transforming Dementia Care: Implementation Challenges Moving Evidence-Based Programs to Health Care

Although evidence for dementia care programs continues to grow, families and health providers do not have ready access to programs, nor have they been widely disseminated and routinized in healthcare. Understanding implementation considerations when embedding evidence-based programs in healthcare systems can inform ways to effectively transform dementia care. This symposium will examine similarities and differences in implementation challenges encountered and strategies used when implementing four evidence-based programs being tested in different healthcare environments using distinct study designs. Dr. Gaugler et al., will discuss implementation challenges encountered with a staff-delivered intervention (ADS Plus) to support caregivers in adult day services that is being tested using a mixed methods hybrid trial design in >50 sites nationally. Dr. Hodgson et al., will discuss adaptions and their measurement to COPE, a home-based dyadic support program being embedded in 10 PACEs of a large healthcare system using a noninferiority trial testing staff training strategies. Dr. Forester et al., will examine implementation of the Care Ecosystem for dementia patients in a high-risk, integrated care management program using a pilot embedded pragmatic trial. Dr. Hepburn et al., will explore tactical challenges of implementing Tele-Savvy, an online caregiver psychoeducation program, within the context of a pilot pragmatic clinical trial. Drawing upon implementation science, themes discussed include balancing adaptations and fidelity, measurement of implementation outcomes and organizational readiness, and staff training implications. Also highlighted are research design considerations. Dr. Czaja, an expert in the design and implementation of dementia care interventions from in-person to technology-based will be the discussant.

Experiences From Healthcare and Community Organizations Delivering Evidence-Based Dementia Caregiving Programs

Best Practice Caregiving surveyed 324 healthcare and community organizations that replicated one or more of the 44 evidence-based programs about delivery organization characteristics, delivery staff, caregivers and persons with dementia served, funding sources, delivery challenges, perceived impact, and satisfaction. 211 (65.1%) organizations completed surveys about 30 different evidence-based programs. The most common types of organizations that delivered programs were healthcare organizations (23.8%) and Area Agencies on Aging (23.8%). Results showed on average organizations delivered programs for 49 months and served 68 families/year. The most common program delivery challenges were marketing (69.8%) and engaging participants (66.3%). Organizations generally agreed that programs had positive impacts on caregivers (59.5% strongly agree) but were less positive about benefits for persons with dementia (25.1% strongly agree). Discussion provides insights into successes and challenges organizations face when adopting evidence-based dementia caregiving programs in their communities.

Implementation Science Translation: Program Sustainment for Managing Distress Behavior in Dementia

Distress behaviors in dementia (DBD) are common in nursing home settings, are distressing, and result in poorer outcomes for residents and staff. We present on the implementation of STAR-VA, an interdisciplinary intervention for effective management of DBD in Veterans Health Administration (VA) nursing home settings, called Community Living Centers (CLCs). A primary focus of this symposium is the use of implementation science concepts to improve and sustain evidence-based programs through tailored implementation strategies and key partnerships. Key implementation science concepts from conceptual frameworks, including the Consolidated Framework for Implementation Research (CFIR) and the use of organizational Knowledge Reservoirs (KR) for sustaining new clinical practices, formed the basis of this work. Their application in health care practice will be discussed using STAR-VA as an exemplar. Interdisciplinary CLC staff feedback during STAR-VA implementation and sustainment is presented, including feedback regarding barriers to integrating new program interventions into usual care processes. Mapping key implementation strategies onto reported barriers informed development of implementation tools and strategies designed to guide adaptions tailored to the needs of the residents and frontline staff, increasing the chances of successful sustainment. Finally, we highlight the importance of key leadership partnerships in implementation of evidence-based programs to improve care of residents with DBD and present strategies for developing these partnerships. Discussion will include the importance of using implementation science to implement evidence-based interventions for effective management of DBD and strategies for sustainment of these effective practices into usual care.

Exploring Well-Being Outcomes Among Persons Living with Dementia: Self-Reports Versus Caregiver Proxy Reports

A vast amount of research has focused on the development of dementia caregiving evidence-based programs (EBPs) in order to mitigate the well-known negative impacts of providing care for persons living with dementia (PLWD). In order to support efficacious results, many programs utilize a variety of both objective and subjective outcomes as typically reported by the caregiver or collected via medical history and clinical assessments. However, more recent research has attempted to understand the subjective illness experience of PLWD through self-report measures. Using data obtained from an online database (i.e., Best Practice Caregiving), that currently provides in-depth information on 44 dementia caregiving EBPs, the current study aimed to examine the number of EBPs that incorporated PLWD self-report measures on a variety of well-being outcomes. Results indicate that out of the 44 EBPs included for analysis: 1) Four out of 19 programs evaluating PLWD symptom severity/difficulty/distress used a PLWD self-report measure; 2) Eight out of 10 programs evaluating PLWD symptoms of depression used a PLWD self-report measure; 3) Five out of 10 programs evaluating PLWD quality of life/care utilized a PLWD self-report measure; and 4) One out of 9 programs evaluating ADL/IADL dependencies utilized a PLWD self-report measure. Discussion will focus on potential gaps and limitations of primarily using caregiver proxy reports to collect personal well-being outcomes for PLWD and ways in which researchers can integrate more self-report measures into intervention and programmatic evaluations.

Outcome Priorities Among Evidence-Based Dementia Caregiving Programs: A Closer Look

Currently, the Best Practice Caregiving website provides information on 231 published studies from 44 dementia caregiving evidence-based programs that have demonstrated beneficial outcomes for dementia caregivers within health care and community-based settings. Across all programs, a total of 34 biopsychosocial outcomes were identified. Supported by the commonly used stress-related frameworks (e.g., Stress-Health Process, Cognitive Behavioral Theory) for which the programs were developed, the most frequently utilized program outcomes included: 1) Caregiver stress, strain, and/or burden (84.1%); 2) Caregiver depressive symptomology (79.5%); and 3) Caregiving efficacy, skills, and/or confidence (63.6%). The least common programmatic outcomes included: 1) Access to support information/Community service use (9.1%); 2) Unmet needs (6.8%); and 3) Respite/break from care (2.3%). The lesser utilized outcomes provide critical insight into current evidence-based programmatic priorities and ways in which professionals can seek to fill gaps in dementia caregiving interventions. Discussion will also focus on future directions of caregiver-related outcome assessments.

Support From the Top: Diverse Leadership Partners for Sustained STAR-VA Implementation

The STAR-VA program was an initiative out of what is now called the VA Office of Mental Health and Suicide Prevention, partnering with the national Offices of Geriatrics and Extended Care and Nursing Services. Ongoing collaboration with these national, as well as regional and medical-center-level leaders, has been critical for informing program implementation and dissemination strategies. We will discuss several key partnered strategies, including (1) linking STAR-VA to national CLC systematic quality improvement efforts; (2) engaging national inter-office program leaders in decisions about outreach to and inclusion of facilities in STAR-VA training and implementation; (3) training local STAR-VA champions on strategies for engaging local leadership support; (4) briefing leaders across the system with program updates; and (5) using national VA data to inform STAR-VA sustained implementation. Discussion will address challenges and opportunities for engaging leadership stakeholders in facilitating sustained implementation of evidence-based programs.

Implementation Strategies for Widespread Scaling of Effective Programs into Healthcare systems

Translation of effective evidence-based programs into practice is critical to promoting and preserving older adults’ function and independence. This presentation will provide an introduction to implementation strategies, defined as the “methods or techniques used to enhance the adoption, implementation, and sustainability of a clinical program or practice.” Some examples of implementation strategies include education and training, stakeholder engagement, patient and/or consumer involvement, adaptation, and technical assistance. Application of these implementation strategies will be illustrated using examples from local and national scale out of evidence-based health promotion programs for older adults within the VA Healthcare System. This presentation will close with guidance on how to select, track, and evaluate implementation strategies.