Improving Uptake of a National Web-Based Psychoeducational Workshop for Informal Caregivers of Veterans: Mixed Methods Implementation Evaluation

Background Although web-based psychoeducational programs may be an efficient, accessible, and scalable option for improving participant well-being, they seldom are sustained beyond trial publication. Implementation evaluations may help optimize program uptake, but few are performed. When the US Department of Veterans Affairs (VA) launched the web-based psychoeducational workshop Building Better Caregivers (BBC) for informal caregivers of veterans nationwide in 2013, the workshop did not enroll as many caregivers as anticipated. Objective This study aims to identify the strengths and weaknesses of initial implementation, strategies likely to improve workshop uptake, whether the VA adopted these strategies, and whether workshop enrollment changed. Methods We used mixed methods and the Promoting Action on Research Implementation in Health Services (PARIHS) implementation evaluation framework. In stage 1, we conducted semistructured interviews with caregivers, local staff, and regional and national VA leaders and surveys with caregivers and staff. We collected and analyzed survey and interview data concurrently and integrated the results to identify implementation strengths and weaknesses, and strategies likely to improve workshop uptake. In stage 2, we reinterviewed national leaders to determine whether the VA adopted recommended strategies and used national data to determine whether workshop enrollment changed over time. Results A total of 54 caregivers (n=32, 59%), staff (n=13, 24%), and regional (n=5, 9%) and national (n=4, 7%) leaders were interviewed. We received survey responses from 72% (23/32) of caregivers and 77% (10/13) of local staff. In stage 1, survey and interview results were consistent across multiple PARIHS constructs. Although participants from low-enrollment centers reported fewer implementation strengths and more weaknesses, qualitative themes were consistent across high- and low-enrollment centers, and across caregiver, staff, and leadership respondent groups. Identified strengths included belief in a positive workshop impact and the use of some successful outreach approaches. Implementation weaknesses included missed opportunities to improve outreach and to better support local staff. From these, we identified and recommended new and enhanced implementation strategies—increased investment in outreach and marketing capabilities; tailoring outreach strategies to multiple stakeholder groups; use of campaigns that are personal, repeated, and detailed, and have diverse delivery options; recurrent training and mentoring for new staff; and comprehensive data management and reporting capabilities. In stage 2, we determined that the VA had adopted several of these strategies in 2016. In the 3 years before and after adoption, cumulative BBC enrollment increased from 2139 (2013-2015) to 4030 (2016-2018) caregivers. Conclusions This study expands the limited implementation science literature on best practices to use when implementing web-based psychoeducational programs. We found that robust outreach and marketing strategies and support for local staff were critical to the implementation success of the BBC workshop. Other health systems may want to deploy these strategies when implementing their web-based programs.

Improving Uptake of a National Web-Based Psychoeducational Workshop for Informal Caregivers of Veterans: Mixed Methods Implementation Evaluation (Preprint)

BACKGROUND Although web-based psychoeducational programs may be an efficient, accessible, and scalable option for improving participant well-being, they seldom are sustained beyond trial publication. Implementation evaluations may help optimize program uptake, but few are performed. When the US Department of Veterans Affairs (VA) launched the web-based psychoeducational workshop Building Better Caregivers (BBC) for informal caregivers of veterans nationwide in 2013, the workshop did not enroll as many caregivers as anticipated. OBJECTIVE This study aims to identify the strengths and weaknesses of initial implementation, strategies likely to improve workshop uptake, whether the VA adopted these strategies, and whether workshop enrollment changed. METHODS We used mixed methods and the Promoting Action on Research Implementation in Health Services (PARIHS) implementation evaluation framework. In stage 1, we conducted semistructured interviews with caregivers, local staff, and regional and national VA leaders and surveys with caregivers and staff. We collected and analyzed survey and interview data concurrently and integrated the results to identify implementation strengths and weaknesses, and strategies likely to improve workshop uptake. In stage 2, we reinterviewed national leaders to determine whether the VA adopted recommended strategies and used national data to determine whether workshop enrollment changed over time. RESULTS A total of 54 caregivers (n=32, 59%), staff (n=13, 24%), and regional (n=5, 9%) and national (n=4, 7%) leaders were interviewed. We received survey responses from 72% (23/32) of caregivers and 77% (10/13) of local staff. In stage 1, survey and interview results were consistent across multiple PARIHS constructs. Although participants from low-enrollment centers reported fewer implementation strengths and more weaknesses, qualitative themes were consistent across high- and low-enrollment centers, and across caregiver, staff, and leadership respondent groups. Identified strengths included belief in a positive workshop impact and the use of some successful outreach approaches. Implementation weaknesses included missed opportunities to improve outreach and to better support local staff. From these, we identified and recommended new and enhanced implementation strategies—increased investment in outreach and marketing capabilities; tailoring outreach strategies to multiple stakeholder groups; use of campaigns that are personal, repeated, and detailed, and have diverse delivery options; recurrent training and mentoring for new staff; and comprehensive data management and reporting capabilities. In stage 2, we determined that the VA had adopted several of these strategies in 2016. In the 3 years before and after adoption, cumulative BBC enrollment increased from 2139 (2013-2015) to 4030 (2016-2018) caregivers. CONCLUSIONS This study expands the limited implementation science literature on best practices to use when implementing web-based psychoeducational programs. We found that robust outreach and marketing strategies and support for local staff were critical to the implementation success of the BBC workshop. Other health systems may want to deploy these strategies when implementing their web-based programs.