Assessing the Usability of Complex Psychosocial Interventions: The Intervention Usability Scale

BackgroundUsability – the extent to which an intervention can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction – is a key determinant of implementation success. However, usability is rarely assessed in implementation research and no instruments have been developed to measure the design quality of complex health interventions, such as the evidence-based psychosocial interventions that characterize the majority of effective practices in mental and behavioral health services. This study evaluated the structural validity of the Intervention Usability Scale (IUS), an adapted version of the well-established System Usability Scale for digital technologies, when measuring the usability of complex health interventions. Prior studies of the original System Usability Scale have found both one- and two-factor solutions, both of which were examined in the current study of the IUS.MethodsA survey was administered to 205 healthcare professionals working at 11 primary care sites. Surveys collected demographic information, including each participant’s professional role (i.e., medical provider, mental/behavioral health provider, pharmacist), and IUS ratings for one of six common evidence-based psychosocial interventions (e.g., cognitive behavioral therapy, motivational interviewing) that they reported using most regularly. Factor analyses replicated the procedures used in prior research on the System Usability Scale, and a sensitivity analysis using analyses of variance compared IUS scores across different groups of respondents and interventions assessed.ResultsAnalyses indicated that a two-factor solution (with “usable” and “learnable” subscales) in which one item was removed best fit the data. This solution accounted for 52.6% of the variance observed. Inter-item reliabilities for the total score, usable subscale, and learnable subscale were α = .83, α = .82, and α = .63, respectively. Resulting scores indicated that usability ranged from below acceptable standards to good, depending on the intervention. On average, behavioral health providers found the interventions to be more usable that other types of healthcare providers.ConclusionsThe current study provides evidence for a two-factor IUS structure consistent with some prior research, as well as acceptable reliability and sensitivity to role and intervention. Future directions for implementation research evaluating the usability of complex health interventions are discussed.Contributions to the Literature• The ease with which interventions can be readily adopted by service providers is a key predictor of implementation success, but very little implementation research has attended to intervention usability.• No instruments exist to evaluate the usability of complex health interventions, such as the evidence-based practices that are commonly used to integrate mental and behavioral health services into primary care.• The current study evaluated the first instrument for assessing the usability of complex health interventions and found that its factor structure replicated some research with the original version of the instrument, a scale developed to assess the usability of digital systems.