Accuracy of emergency physicians’ self-estimates of CT scan utilization and its potential effect on an audit and feedback intervention: a randomized trial

Background Audit and feedback (A&F) has been used as a strategy to modify clinician behavior with moderate success. Although A&F is theorized to work by improving the accuracy of clinicians’ estimates of their own behavior, few interventions have included assessment of clinicians’ estimates at baseline to examine whether they account for intervention success or failure. We tested an A&F intervention to reduce computed tomography (CT) ordering by emergency physicians, while also examining the physicians’ baseline estimates of their own behavior compared to peers. Methods Our study was a prospective, multi-site, 20-month, randomized trial to examine the effect of an A&F intervention on CT ordering rates, overall and by test subtype. From the electronic health record, we obtained 12 months of baseline CT ordering per 100 patients treated for every physician from four emergency departments. Those who were randomized to receive A&F were shown a de-identified graph of the group’s baseline CT utilization, asked to estimate wherein the distribution of their own CT order practices fell, and then shown their actual performance. All participants also received a brief educational intervention. CT ordering rates were collected for all physicians for 6 months after the intervention. Pre-post ordering rates were compared using independent and repeated measures t tests. Results Fifty-one of 52 eligible physicians participated. The mean CT ordering rate increased significantly in both experimental conditions after the intervention (intervention pre = 35.7, post = 40.3, t = 4.13, p < 0.001; control pre = 33.9, post = 38.9, t = 3.94, p = 0.001), with no significant between-group difference observed at follow-up (t = 0.43, p = 0.67). Within the intervention group, physicians had poor accuracy in estimating their own ordering behavior at baseline: most overestimated and all guessed that they were in the upper half of the distribution of their peers. CT ordering increased regardless of self-estimate accuracy. Conclusions Our A&F intervention failed to reduce physician CT ordering: our feedback to the physicians showed most of them that they had overestimated their CT ordering behavior, and they were therefore unlikely to reduce it as a result. After “audit,” it may be prudent to assess baseline clinician awareness of behavior before moving toward a feedback intervention.

A74 INCREASING RATES OF CT IMAGING IN THE EMERGENCY DEPARTMENT AMONG PATIENTS WITH INFLAMMATORY BOWEL DISEASE

Background Computed tomography (CT) imaging of the abdomen is often performed in the emergency department (ED) to exclude urgent pathology in patients with Inflammatory Bowel Disease (IBD). However, concerns over excessive radiation exposure from repeated use have led to expert consensus guidelines advising against CT imaging unless there is suspicion of complications (obstruction, perforation, abscess) or a non-IBD cause for symptoms. Aims Our study aimed to determine trends in abdominal CT utilization and findings among patients with IBD in the ED. Methods We performed a retrospective cohort study between 01/01/2009 and 31/12/2018 at a single academic center. We identified ED encounters for adults (age ≥17 years) with a pre-existing diagnosis of IBD from our institutional database and determined the proportion resulting in an abdominal CT scan within 72 hours of presentation. IBD subtypes were classified based on ICD-10 claims: K50.90* for Crohn’s disease (CD), K51.90* for Ulcerative colitis (UC) and IBD undifferentiated (IBDU) for patients with both claims. A time trend analysis was performed using a generalized linear model that assumed a Poisson distribution. CT scans were classified according to the dominant imaging finding. For this part we excluded studies performed within 1-month of surgery or those re-assessing a known abscess or malignancy. Results A total of 14783 encounters were identified. Among these encounters 3000 CT scans were performed: 2000 for patients with CD (21.9%), 652 for UC (16.5%) and 348 for IBDU (20.4%). The rates of CT utilization significantly increased by 2.7% (95% CI, 1.2–4.3; p=0.0004) in patients with CD, by 4.2% (95% CI, 1.7–6.7; p=0.0009) in patients with UC and by 6.3% in patients with IBDU (95% CI, 2.5–10.0; p=0.0011). Among the eligible CT scans performed for CD, the following dominant findings were reported: normal (25%), inflammation (23%), obstruction (23%), penetrating (18%) and unrelated to IBD (8.6%). In contrast, the following findings for patients with UC were reported: normal (20%), inflammation (39%), obstructive (19%), penetrating (8%) and unrelated to IBD (15%). Conclusions In this single center study, a steady increase in CT utilization in the ED was observed in patients with IBD. Interestingly, only a small proportion of the CT scans demonstrated urgent findings. Future studies are required to determine the factors that contribute to the ongoing increase in CT utilization in this patient population. Funding Agencies None

Development and Investigation of an Appropriateness Guideline-based Mobile Application to Support Physicians' Head CT Scan Ordering: A Quasi-experimental Study (Preprint)

BACKGROUND Although many medical apps are already available for the use of health care professionals (HCP), there is scarce objective evidence regarding the intention of using mobile apps by HCP; and the effectiveness of using mobile applications among physicians is still unclear. Thus, understanding the contributing factors in the successful adoption of mobile applications by healthcare professionals is critical for various stakeholders, including patients, health care decision-makers, insurance companies, and industry such as major stakeholders in mobile app stores. OBJECTIVE To address this challenge, this study seeks two objectives: 1) design & implement a head CT scan appropriateness guideline mobile app (HAC app) for residents CT scan ordering; 2) investigate the impact of the HAC app on physicians' ordering behavior. METHODS Following the study conducted on the CT utilization by "utilization management and cost-conscious care committee (UM3C)", we performed a quasi-experimental study with all neurology and neurosurgery residents in a given general teaching hospital with 510 beds in Iran. We selected the Care Core Head CT Scan guideline for the mobile application. HAC app impact on CT utilization was investigated for seven months at three points: before the intervention, during the intervention, and after the intervention. Data for CT scan utilization were collected using a checklist and medical records and then analyzed using descriptive statistics, Chi-square, and Fisher's exact test. At the end of the study, we conducted a reflection session through the focus group discussion with residents to review and digest residents' experiences during interaction with the HAC app. RESULTS Total brain CT utilization for seven months was 920. The median CT ordered per patient in the baseline phase (before intervention) was 2 (Q1=1, Q3=4), during the intervention phase was 2 (Q1= 1, Q3=2) and in two months after cessation of the intervention phase was 2 (Q1=1, Q3=2). The decrease of total CT scan utilization was statistically significant during pre-intervention and intervention phases (P= 0.027). No significant differences were found for total brain CT utilization during the intervention and after the intervention phase (P= 0.075). Two key themes were extracted during the reflection session and categorized into the future HAC app and its drawbacks. CONCLUSIONS Mobile devices affect residents' CT scan ordering behavior; whereas this impact was not long-lasting, the direct cause- and effect relationship between the mobile apps and changing physicians' behavior remains open to debate. The lack of objective studies in evaluating mobile apps' impact on clinical care outcome prevents individual judgment on its effectiveness. Further studies based on real interactive experiences with mobile systems is advisable to investigate contributing factors in physicians' mobile intention of use before it can be recommended for widespread use by HCP.