Objective:
Quality of transitional care is associated with important health outcomes such as rehospitalization, increased medical errors, and costs. A widely used measure of the construct, the Care Transitions Measure (CTM-15), was developed and validated with relatively small samples and classical test theory approach. The short version of the measure (CTM-3) was recently included in the CAHPS® Hospital Survey. We aimed to conduct a psychometric evaluation of the CTM-15 in a large sample and evaluate whether an item response theory (IRT) based scoring would lead to better measurement precision.
Methods:
As part of TRACE-CORE 1545 participants were interviewed during hospitalization for ACS providing information on general health status (SF-36). At 1 month following discharge, patients completed CTM items, health utilization and care process questions. We evaluated the psychometric properties of the CTM using descriptive statistics, factor analyses, and item response theory analyses. We compared the measurement precision of CTM-15, CTM-3, and a CTM-IRT based score using relative validity (RV) coefficients.
Results:
Participants were 79% non-Hispanic white, 67% male, 27% with a college education or higher (27%) and average age of 62 years. The CTM scale had good internal consistency (Cronbach’s alpha=0.95), but demonstrated strong acquiescence bias (8.7% participants responded “Strongly agree”, 19% “Agree” to all 15 items) and limited score variability (5% <50score). A bi-factor model with one general factor with high item loadings and four secondary factors with lower item loadings, fit the data best supporting the assumption of essential unidimensionality of the measure. Review of item characteristic curves indicated presence of non-discriminating response categories in all items. IRT based item parameters were estimated for all items, but for five of the items there was evidence for model misfit. The CTM-15 differentiated between groups of patients defined by self-reported health status, health care utilization, and care transition process indicators. Differences between groups were small (2-3 points). There was no gain in measurement precision for the scale from IRT scoring. The CTM-3 was not significantly lower for patients reporting rehospitalization or emergency department visits.
Conclusion:
We identified psychometric challenges of the CTM, which may limit its value in research and practice. The strong acquiescence bias in the measure leads to highly skewed, clustered scores with restricted score variance and makes it difficult to differentiate between levels of care transition quality. In the absence of guidelines on meaningfully important differences, it is hard to determine whether detected statistically significant differences in care transition quality are important. These results are in line with emerging evidence of gaps in the validity of the measure.
Comparing five depression measures in depressed Chinese patients using item response theory: an examination of item properties, measurement precision and score comparability
