Objective:
Prognostication after cardiac arrest is challenging. We tested if providers’ confidence in their neuroprognostic assessments correlates with accuracy.
Methods:
We presented physicians with clinical synopses using data from real-time, post-arrest patients being treated at a single academic medical center. We allowed providers to ask for any additional data available at the time the assessment was performed. We asked providers: (1) will the patient survive to hospital discharge?; (2) will the patient have favourable function at discharge?; and, (3) their confidence in each prediction (0-100%). We repeated assessments daily until death or post-arrest day 5.
Results:
We completed 414 assessments of 51 patients with 59 providers. Of patients, 79% died, 8% were discharged with unfavourable function and 12% had functionally favourable survival. Providers accurately predicted survival in 257/414 (62%) assessments. In most errors (136/141, 96%), providers predicted survival in a patient who died. Providers accurately predicted function in 282/414 (68%) assessments. In most errors (125/132, 95%), providers incorrectly predicted a favourable outcome. Providers were confident in their assessments (median confidence predicting survival 80 [IQR 60 - 90]; median confidence predicting function 80 [IQR 60 - 95]). Accuracy predicting survival and function were both positively correlated with confidence (both P<0.001), but confidence explained only 7% and 15% of observed variance in accuracy, respectively. When providers reported 100% confidence predicting survival, they were correct in 31/42 (74%) cases. Accuracy did not vary over time. Attending physicians were not more accurate than trainees predicting survival (65% vs 60% accurate) and were less accurate prediciting functional outcome (62% vs 84% accurate, P< 0.001). Confidence did not differ between attendings and trainees.
Conclusions:
Providers were overly optimistic predicting outcomes at discharge. Self-reported confidence explained only a small percentage of variance in accuracy. Even when extremely confident, providers were often wrong. Our future work will explore patient and provider factors that contribute to error.
