Can decision support combat incompleteness and bias in routine primary care data?

AbstractObjectiveRoutine primary care data may be used for the derivation of clinical prediction rules and risk scores. We sought to measure the impact of a decision support system (DSS) on data completeness and freedom from bias.Materials and MethodsWe used the clinical documentation of 34 UK general practitioners who took part in a previous study evaluating the DSS. They consulted with 12 standardized patients. In addition to suggesting diagnoses, the DSS facilitates data coding. We compared the documentation from consultations with the electronic health record (EHR) (baseline consultations) vs consultations with the EHR-integrated DSS (supported consultations). We measured the proportion of EHR data items related to the physician’s final diagnosis. We expected that in baseline consultations, physicians would document only or predominantly observations related to their diagnosis, while in supported consultations, they would also document other observations as a result of exploring more diagnoses and/or ease of coding.ResultsSupported documentation contained significantly more codes (incidence rate ratio [IRR] = 5.76 [4.31, 7.70] P < .001) and less free text (IRR = 0.32 [0.27, 0.40] P < .001) than baseline documentation. As expected, the proportion of diagnosis-related data was significantly lower (b = −0.08 [−0.11, −0.05] P < .001) in the supported consultations, and this was the case for both codes and free text.ConclusionsWe provide evidence that data entry in the EHR is incomplete and reflects physicians’ cognitive biases. This has serious implications for epidemiological research that uses routine data. A DSS that facilitates and motivates data entry during the consultation can improve routine documentation.

Download Full-text

Impact of a diagnostic decision support system on GP clinical documentation

British Journal of General Practice ◽

10.3399/bjgp19x703037 ◽

2019 ◽

Vol 69 (suppl 1) ◽

pp. bjgp19X703037

Author(s):

Chris Tracey ◽

Brendan Delaney ◽

Olga Kostopoulou

Keyword(s):

Decision Support ◽

Decision Support System ◽

Support System ◽

Final Diagnosis ◽

Differential Diagnoses ◽

Delphi Panel ◽

Free Text ◽

Clinical Documentation ◽

Diagnostic Decision ◽

The Impact

BackgroundKostopoulou and colleagues designed and evaluated a diagnostic decision support system (DSS) that presented GPs with differential diagnoses to consider at the start of the consultation. The DSS was integrated with the electronic health record (EHR) and evaluated in simulated consultations (34 GPs consulting with 12 actors), where it was found to improve diagnostic accuracy.AimTo evaluate the impact of the DSS on GP clinical documentation.MethodSecondary data analysis. The analysis dataset consisted of all data items that the 34 GPs recorded during their consultations with the 12 actors (408 consultations). Each GP had conducted six consultations with the EHR alone, and six with the integrated DSS. The 12 patient scenarios had a pre-defined set of clinical cues and differential diagnoses. For each patient scenario, a Delphi panel of five GPs identified which cues were consistent with each differential diagnosis. The data items recorded, either in free text or code, were counted. These were referenced against the Delphi panel’s matrix of cues and diagnoses. For each GP, it was estimated what proportion of the recorded data was consistent with the diagnosis that they finally gave.ResultsUsing the DSS increased the total amount of information documented (b = 3.58 [95% confidence interval {CI} = 2.81 to 4.35] P<0.001) and reduced the proportion of documentation that was consistent with the GP’s final diagnosis (b = −0.08 [95% CI = −0.11 to −0.05] P<0.001), suggesting less bias.ConclusionUsing the DSS led to more complete and less biased documentation. This has implications for the use of routine EHR data to create clinical prediction rules.

Download Full-text

Assessing and predicting adolescent and early adulthood common mental disorders using electronic primary care data: analysis of a prospective cohort study (ALSPAC) in Southwest England

BMJ Open ◽

10.1136/bmjopen-2021-053624 ◽

2021 ◽

Vol 11 (10) ◽

pp. e053624

Author(s):

Daniel Smith ◽

Kathryn Willan ◽

Stephanie L Prady ◽

Josie Dickerson ◽

Gillian Santorelli ◽

...

Keyword(s):

Primary Care ◽

Cohort Study ◽

Mental Disorders ◽

Outcome Measures ◽

Prospective Cohort Study ◽

Prospective Cohort ◽

Common Mental Disorders ◽

Free Text ◽

Primary Care Data ◽

Time Point

ObjectivesWe aimed to examine agreement between common mental disorders (CMDs) from primary care records and repeated CMD questionnaire data from ALSPAC (the Avon Longitudinal Study of Parents and Children) over adolescence and young adulthood, explore factors affecting CMD identification in primary care records, and construct models predicting ALSPAC-derived CMDs using only primary care data.Design and settingProspective cohort study (ALSPAC) in Southwest England with linkage to electronic primary care records.ParticipantsPrimary care records were extracted for 11 807 participants (80% of 14 731 eligible). Between 31% (3633; age 15/16) and 11% (1298; age 21/22) of participants had both primary care and ALSPAC CMD data.Outcome measuresALSPAC outcome measures were diagnoses of suspected depression and/or CMDs. Primary care outcome measure were Read codes for diagnosis, symptoms and treatment of depression/CMDs. For each time point, sensitivities and specificities for primary care CMD diagnoses were calculated for predicting ALSPAC-derived measures of CMDs, and the factors associated with identification of primary care-based CMDs in those with suspected ALSPAC-derived CMDs explored. Lasso (least absolute selection and shrinkage operator) models were used at each time point to predict ALSPAC-derived CMDs using only primary care data, with internal validation by randomly splitting data into 60% training and 40% validation samples.ResultsSensitivities for primary care diagnoses were low for CMDs (range: 3.5%–19.1%) and depression (range: 1.6%–34.0%), while specificities were high (nearly all >95%). The strongest predictors of identification in the primary care data for those with ALSPAC-derived CMDs were symptom severity indices. The lasso models had relatively low prediction rates, especially in the validation sample (deviance ratio range: −1.3 to 12.6%), but improved with age.ConclusionsPrimary care data underestimate CMDs compared to population-based studies. Improving general practitioner identification, and using free-text or secondary care data, is needed to improve the accuracy of models using clinical data.

Download Full-text

THE IMPACT OF ATRIAL FIBRILLATION ON OUTCOMES OF PERIPHERAL ARTERIAL DISEASE: ANALYSIS OF ROUTINELY COLLECTED PRIMARY CARE DATA

The American Journal of Medicine ◽

10.1016/j.amjmed.2021.10.021 ◽

2021 ◽

Author(s):

Antonios Vitalis ◽

Krishnarajah Nirantharakumar ◽

Rasiah Thayakaran ◽

Rajiv K. Vohra ◽

MarK. Kay ◽

...

Keyword(s):

Atrial Fibrillation ◽

Primary Care ◽

Peripheral Arterial Disease ◽

Arterial Disease ◽

Primary Care Data ◽

Peripheral Arterial ◽

The Impact ◽

Disease Analysis

Download Full-text

A Proof of Concept for Assessing Emergency Room Use with Primary Care Data and Natural Language Processing

Methods of Information in Medicine ◽

10.3414/me12-01-0012 ◽

2013 ◽

Vol 52 (01) ◽

pp. 33-42 ◽

Cited By ~ 11

Author(s):

M.-H. Kuo ◽

P. Gooch ◽

J. St-Maurice

Keyword(s):

Mental Health ◽

Primary Care ◽

Natural Language Processing ◽

Natural Language ◽

Emergency Room ◽

Language Processing ◽

Free Text ◽

Proof Of Concept ◽

Term Extraction ◽

Primary Care Data

SummaryObjective: The objective of this study was to undertake a proof of concept that demonstrated the use of primary care data and natural language processing and term extraction to assess emergency room use. The study extracted biopsychosocial concepts from primary care free text and related them to inappropriate emergency room use through the use of odds ratios.Methods: De-identified free text notes were extracted from a primary care clinic in Guelph, Ontario and analyzed with a software toolkit that incorporated General Architecture for Text Engineering (GATE) and MetaMap components for natural language processing and term extraction.Results: Over 10 million concepts were extracted from 13,836 patient records. Codes found in at least 1% percent of the sample were regressed against inappropriate emergency room use. 77 codes fell within the realm of biopsychosocial, were very statistically significant (p < 0.001) and had an OR > 2.0. Thematically, these codes involved mental health and pain related concepts.Conclusions: Analyzed thematically, mental health issues and pain are important themes; we have concluded that pain and mental health problems are primary drivers for inappropriate emergency room use. Age and sex were not significant. This proof of concept demonstrates the feasibly of combining natural language processing and primary care data to analyze a system use question. As a first work it supports further research and could be applied to investigate other, more complex problems.

Download Full-text

Validation and impact of a simplified clinical decision rule for diagnosing pulmonary embolism in primary care: design of the PECAN prospective diagnostic cohort management study

BMJ Open ◽

10.1136/bmjopen-2019-031639 ◽

2019 ◽

Vol 9 (10) ◽

pp. e031639 ◽

Cited By ~ 4

Author(s):

Rosanne van Maanen ◽

Frans H Rutten ◽

Frederikus A Klok ◽

Menno V Huisman ◽

Jeanet W Blom ◽

...

Keyword(s):

Primary Care ◽

Pulmonary Embolism ◽

False Negative ◽

Diagnostic Algorithm ◽

Clinical Decision ◽

Final Diagnosis ◽

Hospital Environment ◽

Diagnostic Study ◽

The Impact ◽

D Dimer

IntroductionCombined with patient history and physical examination, a negative D-dimer can safely rule-out pulmonary embolism (PE). However, the D-dimer test is frequently false positive, leading to many (with hindsight) ‘unneeded’ referrals to secondary care. Recently, the novel YEARS algorithm, incorporating flexible D-dimer thresholds depending on pretest risk, was developed and validated, showing its ability to safely exclude PE in the hospital environment. Importantly, this was accompanied with 14% fewer computed tomographic pulmonary angiography than the standard, fixed D-dimer threshold. Although promising, in primary care this algorithm has not been validated yet.Methods and analysisThe PECAN (DiagnosingPulmonaryEmbolism in the context ofCommonAlternative diagNoses in primary care) study is a prospective diagnostic study performed in Dutch primary care. Included patients with suspected acute PE will be managed by their general practitioner according to the YEARS diagnostic algorithm and followed up in primary care for 3 months to establish the final diagnosis. To study the impact of the use of the YEARS algorithm, the primary endpoints are the safety and efficiency of the YEARS algorithm in primary care. Safety is defined as the proportion of false-negative test results in those not referred. Efficiency denotes the proportion of patients classified in this non-referred category. Additionally, we quantify whether C reactive protein measurement has added diagnostic value to the YEARS algorithm, using multivariable logistic and polytomous regression modelling. Furthermore, we will investigate which factors contribute to the subjective YEARS item ‘PE most likely diagnosis’.Ethics and disseminationThe study protocol was approved by the Medical Ethical Committee Utrecht, the Netherlands. Patients eligible for inclusion will be asked for their consent. Results will be disseminated by publication in peer-reviewed journals and presented at (inter)national meetings and congresses.Trial registrationNTR 7431.

Download Full-text

A Closer Look at the “Right” Format for Clinical Decision Support: Methods for Evaluating a Storyboard BestPractice Advisory

Journal of Personalized Medicine ◽

10.3390/jpm10040142 ◽

2020 ◽

Vol 10 (4) ◽

pp. 142

Author(s):

Brian J. Douthit ◽

R. Clayton Musser ◽

Kay S. Lytle ◽

Rachel L. Richesson

Keyword(s):

Decision Support ◽

Clinical Decision Support ◽

Local Level ◽

Clinical Decision ◽

System Level ◽

Evaluation Methodology ◽

Task Completion ◽

Free Text ◽

Alert Fatigue ◽

The Impact

(1) Background: The five rights of clinical decision support (CDS) are a well-known framework for planning the nuances of CDS, but recent advancements have given us more options to modify the format of the alert. One-size-fits-all assessments fail to capture the nuance of different BestPractice Advisory (BPA) formats. To demonstrate a tailored evaluation methodology, we assessed a BPA after implementation of Storyboard for changes in alert fatigue, behavior influence, and task completion; (2) Methods: Data from 19 weeks before and after implementation were used to evaluate differences in each domain. Individual clinics were evaluated for task completion and compared for changes pre- and post-redesign; (3) Results: The change in format was correlated with an increase in alert fatigue, a decrease in erroneous free text answers, and worsened task completion at a system level. At a local level, however, 14% of clinics had improved task completion; (4) Conclusions: While the change in BPA format was correlated with decreased performance, the changes may have been driven primarily by the COVID-19 pandemic. The framework and metrics proposed can be used in future studies to assess the impact of new CDS formats. Although the changes in this study seemed undesirable in aggregate, some positive changes were observed at the level of individual clinics. Personalized implementations of CDS tools based on local need should be considered.

Download Full-text

Evaluating the impact of computer-based decision support for the management of asthma in primary care

http://isrctn.org/> ◽

10.1186/isrctn58726678 ◽

2013 ◽

Author(s):

Robyn Tamblyn

Keyword(s):

Primary Care ◽

Decision Support ◽

Computer Based ◽

The Impact

Download Full-text

46 Primary Care Practitioners Have A High Level of Satisfaction with The Current Breast Referral Pathway but The Majority Would Support A Change to Patient Self-Referral

British Journal of Surgery ◽

10.1093/bjs/znab259.199 ◽

2021 ◽

Vol 108 (Supplement_6) ◽

Author(s):

T Russell ◽

J Cooper ◽

M McIntyre ◽

S Ramzi

Keyword(s):

Primary Care ◽

Free Text ◽

Referral Rates ◽

Care Services ◽

Primary Care Services ◽

Level Of Satisfaction ◽

And Gender ◽

Primary Care Practitioners ◽

High Level ◽

The Impact

Abstract Aim Currently, patients must consult with a primary care practitioner (PCP) prior to being referred to secondary care breast services. A change to patient self-referral would arguably reduce primary care workload, improve access for patients, and allow breast units to allocate resources more appropriately; no data currently supports this. This study aims to explore PCP's views on breast referral, evaluate the community breast workload, and to investigate the impact of COVID-19 on referral rates. Method An electronic survey was designed on SurveyMonkey.com which aimed to collect both quantitative and qualitative data. The weblink to the survey was sent out via two electronic newsletters. Participants were asked: their role and gender, their level of confidence surrounding breast care, details surrounding their breast workload, how they felt COVID-19 had affected their referral rates, their level of satisfaction with the current pathway, and their opinions on a potential change to patient self-referral. Results 79 responses were received. PCPs estimated that 7.0% (median) of their total consultations were regarding a breast-related issue and that COVID-19 had not had a significant impact on the rate of referral to breast units (P = 0.75). 84.8% of PCPs were satisfied with the current referral pathway. Whilst 74.5% felt a change to patient self-referral would benefit patients and primary care services, their free text comments highlighted some of their reservations. Conclusions PCPs have a high level of satisfaction with the current breast referral pathway, but the majority would be open to a change to patient self-referral to specialist breast units.

Download Full-text

Implementation of wide-scale pharmacogenetic testing in primary care

Pharmacogenomics ◽

10.2217/pgs-2019-0043 ◽

2019 ◽

Vol 20 (12) ◽

pp. 903-913 ◽

Cited By ~ 3

Author(s):

Natasha Petry ◽

Jordan Baye ◽

Aissa Aifaoui ◽

Russell A Wilke ◽

Roxana A Lupu ◽

...

Keyword(s):

Primary Care ◽

Decision Support ◽

Clinical Decision Support ◽

Healthcare Delivery ◽

Clinical Decision ◽

Interdisciplinary Teams ◽

Gene Interactions ◽

Support Tools ◽

Wide Scale ◽

The Impact

The convergence of translational genomics and biomedical informatics has changed healthcare delivery. Institutional consortia have begun implementing lab testing and decision support for drug–gene interactions. Aggregate datasets are now revealing the impact of clinical decision support for drug–gene interactions. Given the pleiotropic nature of pharmacogenes, interdisciplinary teams and robust clinical decision support tools must exist within an informatics framework built to be flexible and capable of cross-talk between clinical specialties. Navigation of the challenges presented with the implementation of five steps to build a genetics program infrastructure requires the expertise of multiple healthcare professionals. Ultimately, this manuscript describes our efforts to place pharmacogenomics in the hands of the primary care provider integrating this information into a patient’s healthcare over their lifetime.

Download Full-text