Variation in referral rates to emergency departments and inpatient services from a GP out of hours service and the potential impact of alternative staffing models

IntroductionOut of hours (OOHs) primary care is a critical component of the acute care system overnight and at weekends. Referrals from OOH services to hospital will add to the burden on hospital assessment in the ED and on-call specialties.MethodsWe studied the variation in referral rates (to the ED and direct specialty admission) of individual clinicians working in the Oxfordshire, UK OOH service covering a population of 600 000 people. We calculated the referral probability for each clinician over a 13-month period of practice (1 December 2014 to 31 December 2015), stratifying by clinician factors and location and timing of assessment. We used Simul8 software to determine the range of hospital referrals potentially due to variation in clinician referral propensity.ResultsAmong the 119 835 contacts with the service, 5261 (4.4%) were sent directly to the ED and 3474 (3.7%) were admitted directly to specialties. More referrals were made to ED by primary care physicians if they did not work in the local practices (5.5% vs 3.5%, p=0.011). For clinicians with >1000 consultations, percentage of patients referred varied from 1% to 21% of consultations. Simulations where propensity to refer was made less extreme showed a difference in maximum referrals of 50 patients each week.ConclusionsThere is substantial variation in clinician referral rates from OOHs primary care to the acute hospital setting. The number of patients referred could be influenced by this variation in clinician behaviour. Referral propensity should be studied including casemix adjustment to determine if interventions targeting such behaviour are effective.

Can valid and practical risk-prediction or casemix adjustment models, including adjustment for comorbidity, be generated from English hospital administrative data (Hospital Episode Statistics)? A national observational study

BackgroundNHS hospitals collect a wealth of administrative data covering accident and emergency (A&E) department attendances, inpatient and day case activity, and outpatient appointments. Such data are increasingly being used to compare units and services, but adjusting for risk is difficult.ObjectivesTo derive robust risk-adjustment models for various patient groups, including those admitted for heart failure (HF), acute myocardial infarction, colorectal and orthopaedic surgery, and outcomes adjusting for available patient factors such as comorbidity, using England’s Hospital Episode Statistics (HES) data. To assess if more sophisticated statistical methods based on machine learning such as artificial neural networks (ANNs) outperform traditional logistic regression (LR) for risk prediction. To update and assess for the NHS the Charlson index for comorbidity. To assess the usefulness of outpatient data for these models.Main outcome measuresMortality, readmission, return to theatre, outpatient non-attendance. For HF patients we considered various readmission measures such as diagnosis-specific and total within a year.MethodsWe systematically reviewed studies comparing two or more comorbidity indices. Logistic regression, ANNs, support vector machines and random forests were compared for mortality and readmission. Models were assessed using discrimination and calibration statistics. Competing risks proportional hazards regression and various count models were used for future admissions and bed-days.ResultsOur systematic review and empirical analysis suggested that for general purposes comorbidity is currently best described by the set of 30 Elixhauser comorbidities plus dementia. Model discrimination was often high for mortality and poor, or at best moderate, for other outcomes, for examplec = 0.62 for readmission andc = 0.73 for death following stroke. Calibration was often good for procedure groups but poorer for diagnosis groups, with overprediction of low risk a common cause. The machine learning methods we investigated offered little beyond LR for their greater complexity and implementation difficulties. For HF, some patient-level predictors differed by primary diagnosis of readmission but not by length of follow-up. Prior non-attendance at outpatient appointments was a useful, strong predictor of readmission. Hospital-level readmission rates for HF did not correlate with readmission rates for non-HF; hospital performance on national audit process measures largely correlated only with HF readmission rates.ConclusionsMany practical risk-prediction or casemix adjustment models can be generated from HES data using LR, though an extra step is often required for accurate calibration. Including outpatient data in readmission models is useful. The three machine learning methods we assessed added little with these data. Readmission rates for HF patients should be divided by diagnosis on readmission when used for quality improvement.Future workAs HES data continue to develop and improve in scope and accuracy, they can be used more, for instance A&E records. The return to theatre metric appears promising and could be extended to other index procedures and specialties. While our data did not warrant the testing of a larger number of machine learning methods, databases augmented with physiological and pathology information, for example, might benefit from methods such as boosted trees. Finally, one could apply the HF readmissions analysis to other chronic conditions.FundingThe National Institute for Health Research Health Services and Delivery Research programme.