Tracking the Randomized Rollout of a Veterans Affairs Opioid Risk Management Tool: A Multi-method Implementation Evaluation Using the Consolidated Framework for Implementation Research
Abstract BackgroundThe Veterans Health Administration (VHA) developed the Stratification Tool for Opioid Risk Mitigation (STORM) dashboard to assist VHA clinicians in identifying Veterans at risk for adverse opioid overdose or suicide-related events. In 2018, a national policy was implemented requiring providers at all VHA facilities to complete case reviews of Veterans identified by STORM as very high risk for adverse events. Nationally, facilities were randomized by the type of oversight required when sufficient case reviews were not completed and also by the timing of an increase in the number of required case reviews. As part of a comprehensive assessment of this policy intervention, we aimed to 1) identify barriers and facilitators to implementing case reviews as required in the policy; 2) assess variation across the four arms of the study; and 3) evaluate associations between facility characteristics and implementation barriers and facilitators.MethodsUsing the Consolidated Framework for Implementation Research (CFIR), we developed a semi-structured interview guide to examine barriers to and facilitators of implementing the STORM policy. Staff from 40 purposefully selected facilities who were involved in implementation were invited to participate in telephone interviews. Interview transcripts were coded and then organized into memos, which were numerically rated using the -2 to +2 CFIR rating system for each construct. Descriptive statistics were used to evaluate the mean ratings on each CFIR construct, the associations between ratings and study arm, and three facility characteristics (size, rurality, and level of academic detailing) associated with CFIR ratings. We used the mean CFIR rating for each site to determine which constructs differed between the sites with highest and lowest overall CFIR scores, and these constructs were described in detail. ResultsInterviews with 78 staff at 39 VHA facilities identified a slightly positive (+0.2) overall mean CFIR rating. CFIR ratings were not significantly different between the four study arms, nor associated with facility characteristics. Overall, two important barriers to implementation were CFIR constructs Access to knowledge and information and Evaluating and reflecting. Having time to complete the reviews was a pervasive barrier. Sites with higher overall CFIR scores showed three important facilitators: Leadership engagement, Engaging, and Implementation climate. ConclusionAlthough there was variability in implementation barriers and facilitators across facilities, these were unrelated to study arms and facility characteristics. Leadership, resources, and overall implementation climate were the strongest facilitators of policy implementation.