Examining Hospital Variation on Multiple Indicators of Stroke Quality of Care

Background: Provider profiling involves comparing the performance of hospitals on indicators of quality of care. Typically, provider profiling examines the performance of hospitals on each quality indicator in isolation. Consequently, one cannot formally examine whether hospitals that have poor performance on one indicator also have poor performance on a second indicator. Methods: We used Bayesian multivariate response random effects logistic regression model to simultaneously examine variation and covariation in multiple binary indicators across hospitals. We considered 7 binary patient-level indicators of quality of care for patients presenting to hospital with a diagnosis of acute stroke. We examined between-hospital variation in these 7 indicators across 86 hospitals in Ontario, Canada. Results: The number of patients eligible for each indicator ranged from 1321 to 14 079. There were 7 pairs of indicators for which there was a strong correlation between a hospital’s performance on each of the 2 indicators. Twenty-nine of the 86 hospitals had a probability higher than 0.90 of having worse performance than average on at least 4 of the 7 indicators. Seven of the 86 of hospitals had a probability higher than 0.90 of having worse performance than average on at least 5 indicators. Fourteen of the 86 of hospitals had a probability higher than 0.50 of having worse performance than average on at least 6 indicators. No hospitals had a probability higher than 0.50 of having worse performance than average on all 7 indicators. Conclusions: These findings suggest that there are a small number of hospitals that perform poorly on at least half of the quality indicators, and that certain indicators tend to cluster together. The described methods allow for targeting quality improvement initiatives at these hospitals.

Assessing the Impacts of Misclassified Case-Mix Factors on Health Care Provider Profiling: Performance of Dialysis Facilities

Quantitative metrics are used to develop profiles of health care institutions, including hospitals, nursing homes, and dialysis clinics. These profiles serve as measures of quality of care, which are used to compare institutions and determine reimbursement, as a part of a national effort led by the Center for Medicare and Medicaid Services in the United States. However, there is some concern about how misclassification in case-mix factors, which are typically accounted for in profiling, impacts results. We evaluated the potential effect of misclassification on profiling results, using 20 744 patients from 2740 dialysis facilities in the US Renal Data System. In this case study, we compared 30-day readmission as the profiling outcome measure, using comorbidity data from either the Center for Medicare and Medicaid Services Medical Evidence Report (error-prone) or Medicare claims (more accurate). Although the regression coefficient of the error-prone covariate demonstrated notable bias in simulation, the outcome measure—standardized readmission ratio—and profiling results were quite robust; for example, correlation coefficient of 0.99 in standardized readmission ratio estimates. Thus, we conclude that misclassification on case-mix did not meaningfully impact overall profiling results. We also identified both extreme degree of case-mix factor misclassification and magnitude of between-provider variability as 2 factors that can potentially exert enough influence on profile status to move a clinic from one performance category to another (eg, normal to worse performer).

Detecting and Visualizing Outliers in Provider Profiling Using Funnel Plots and Mixed Effects Models—An Example from Prescription Claims Data

When prescribing a drug for a patient, a physician also has to consider economic aspects. We were interested in the feasibility and validity of profiling based on funnel plots and mixed effect models for the surveillance of German ambulatory care physicians’ prescribing. We analyzed prescriptions issued to patients with a health insurance card attending neurologists’ and psychiatrists’ ambulatory practices in the German federal state of Saarland. The German National Association of Statutory Health Insurance Physicians developed a prescribing assessment scheme (PAS) which contains a systematic appraisal of the benefit of drugs for so far 12 different indications. The drugs have been classified on the basis of their clinical evidence as “standard”, “reserve” or “third level” medication. We had 152.583 prescriptions in 56 practices available for analysis. A total of 38.796 patients received these prescriptions. The funnel plot approach with additive correction for overdispersion was almost equivalent to a mixed effects model which directly took the multilevel structure of the data into account. In the first case three practices were labeled as outliers, the mixed effects model resulted in two outliers. We suggest that both techniques should be routinely applied within a surveillance system of prescription claims data.