Measurement of Sustainment of Prevention Programs and Initiatives: The Sustainment Measurement System Scale
Abstract Background A large knowledge gap exists regarding the measurement of sustainability of evidence-based prevention programs for mental and behavioral health. This paper describes an instrument that is a component of the Sustainment Measurement System (SMS), designed to assess both determinants and outcomes of efforts to sustain prevention programs and initiatives. Methods A 42-item scale was administered to 186 representatives of 145 programs funded by 7 SAMHSA prevention grant initiatives. Scale items were identified from qualitative data collected from 45 representatives of 10 programs and 8 SAMHSA program officers and organized into 9 categories of sustainment determinants and sustainment outcomes. Factor structure identified from principal axis factor analysis and elimination of items with CFA coefficients less than .300 resulted in a 35-item scale with 8 determinant factors and 1 outcome factor. Results Confirmatory Factor Analysis provided support for a 35-item model fit to the data. The SMSS demonstrated good inter-item reliability and all determinant factors were significantly associated with sustainment outcome individual and global measures. Conclusions The SMSS is easy to use and demonstrates good reliability and convergent and discriminant validity in assessing likelihood of sustainment of SAMHSA funded prevention programs and initiatives upon termination of original funding. The measure demonstrates potential in identifying predictors of program sustainment and as a tool for enhancing the likelihood of successful sustainment through ongoing evaluation and feedback.