Assessing the validity of using claims data compared to medical chart reviews for measuring care quality in Japan.

264 Background: Health insurance claims data have been used extensively as a less labor-intensive method of collecting data than medical record reviews, which are the preferred source of data collection in most medical studies. Although recent reports have raised questions about its validity of use in measuring care quality, validity of using claims data may differ by health systems and should therefore be assessed by country. We aimed to test the validity of using claims data in Japan by comparing quality performance scores obtained from claims data to those derived from medical record reviews. Methods: We reviewed medical records from Apr 2013 to Apr 2014 of gastric and colorectal cancer patients who were diagnosed in 2011 from 4 cancer care hospitals in Okinawa. We calculated the proportion of patients who received adjuvant chemotherapy for gastric and colorectal cancer using claims data, and compared the results with those obtained from medical record reviews. Chart reviews were performed by certified tumor registrars. We used kappa coefficients to measure the level of agreement between claims data and medical record reviews. Results: Analysis using claims data resulted in 14 Stage II and III gastric cancer patients who had undergone surgery, with 50% receiving adjuvant chemotherapy; whereas medical record reviews resulted in 19 patients, 94.7% of whom either received or had a clinically valid rationale for not undergoing adjuvant therapy. For colorectal patients, claims data resulted in 48.5% of 68 surgical stage III colorectal patients receiving adjuvant therapy, compared to 74.4% of 78 patients using medical record reviews who either received adjuvant therapy or had a valid reason for not undergoing therapy. Agreement between claims data and chart reviews was low (kappa=0.14) for gastric cancer, but fair (kappa =0.37) for colorectal cancer. Conclusions: Our analysis showed that use of claims data may greatly underestimate quality performance measures if they are not compensated by medical record reviews. Claims data alone cannot capture a large proportion of patients who either choose not to, or has a clinically valid reason for not undergoing standard care.