scholarly journals 169. Development of a Real Time Electronic Algorithm to Identify Hospitalized Patients with Community-Acquired Pneumonia

2020 ◽  
Vol 7 (Supplement_1) ◽  
pp. S92-S92
Author(s):  
Valeria Fabre ◽  
Valeria Fabre ◽  
George Jones ◽  
Joe Amoah ◽  
Eili Klein ◽  
...  

Abstract Background Syndrome-based antibiotic stewardship can be limited by difficulty in finding cases for evaluation. We developed an electronic extraction algorithm to prospectively identify CAP patients. Methods We included non-oncology patients ≥18 years old admitted to The Johns Hopkins Hospital from 12/2018 to 3/2019 who 1) received common CAP antibiotics for ≥48 hours after admission and 2) had a bacterial urinary antigen and chest imaging ordered within 48 hours of admission that was not for assessment of endotracheal tube or central line placement. Charts of patients meeting these criteria were reviewed by 2 authors to identify true cases of CAP based on IDSA guidelines. Cases identified in 12/2018 (n=111) were used to explore potential indicators of CAP, and cases identified 1–3/2019 (n=173) were used to evaluate combinations of indicators that could identify patients treated for CAP who did have CAP (true CAP) and did not have CAP (false CAP). This cohort was divided into a training and a validation set (2/3 and 1/3, respectively). Potential indicators included vitals signs, laboratory data and free text extracted via natural language processing (NLP). Predictive performance of composite indicators for true CAP were assessed using receiver-operating characteristics (ROC) curves. The Hosmer-Lemeshow goodness fit test was used to test model fit and the Akaike Information Criteria was used to determine model selection. Results True CAP was observed in 41% (71/173) of cases and 14 potential individual indicators were identified (Table). These were combined to make 45 potential composite indicators. ROC curves for selected composite indicators are shown in the Figure. Models without use of NLP-derived variables had poor discriminative ability. The best model included fever, hypoxemia, leukocytosis, and “consolidation” on imaging with a sensitivity and positive predictive value 78.7% and specificity and negative predictive value of 85.7%. Table. Indicators evaluated to identify patients with CAP Figure. ROC curves for composite indicators Conclusion Patients with CAP can be identified using electronic data but use of NLP-derived radiographic criteria is required. These data can be linked with data on antibiotic use and duration to develop reports for clinicians regarding appropriate CAP diagnosis and treatment. Disclosures All Authors: No reported disclosures

2020 ◽  
Author(s):  
Amy Y X Yu ◽  
Zhongyu A Liu ◽  
Chloe Pou-Prom ◽  
Kaitlyn Lopes ◽  
Moira K Kapral ◽  
...  

BACKGROUND Diagnostic neurovascular imaging data are important in stroke research, but obtaining these data typically requires laborious manual chart reviews. OBJECTIVE We aimed to determine the accuracy of a natural language processing (NLP) approach to extract information on the presence and location of vascular occlusions as well as other stroke-related attributes based on free-text reports. METHODS From the full reports of 1320 consecutive computed tomography (CT), CT angiography, and CT perfusion scans of the head and neck performed at a tertiary stroke center between October 2017 and January 2019, we manually extracted data on the presence of proximal large vessel occlusion (primary outcome), as well as distal vessel occlusion, ischemia, hemorrhage, Alberta stroke program early CT score (ASPECTS), and collateral status (secondary outcomes). Reports were randomly split into training (n=921) and validation (n=399) sets, and attributes were extracted using rule-based NLP. We reported the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and the overall accuracy of the NLP approach relative to the manually extracted data. RESULTS The overall prevalence of large vessel occlusion was 12.2%. In the training sample, the NLP approach identified this attribute with an overall accuracy of 97.3% (95.5% sensitivity, 98.1% specificity, 84.1% PPV, and 99.4% NPV). In the validation set, the overall accuracy was 95.2% (90.0% sensitivity, 97.4% specificity, 76.3% PPV, and 98.5% NPV). The accuracy of identifying distal or basilar occlusion as well as hemorrhage was also high, but there were limitations in identifying cerebral ischemia, ASPECTS, and collateral status. CONCLUSIONS NLP may improve the efficiency of large-scale imaging data collection for stroke surveillance and research.


10.2196/24381 ◽  
2021 ◽  
Vol 9 (5) ◽  
pp. e24381
Author(s):  
Amy Y X Yu ◽  
Zhongyu A Liu ◽  
Chloe Pou-Prom ◽  
Kaitlyn Lopes ◽  
Moira K Kapral ◽  
...  

Background Diagnostic neurovascular imaging data are important in stroke research, but obtaining these data typically requires laborious manual chart reviews. Objective We aimed to determine the accuracy of a natural language processing (NLP) approach to extract information on the presence and location of vascular occlusions as well as other stroke-related attributes based on free-text reports. Methods From the full reports of 1320 consecutive computed tomography (CT), CT angiography, and CT perfusion scans of the head and neck performed at a tertiary stroke center between October 2017 and January 2019, we manually extracted data on the presence of proximal large vessel occlusion (primary outcome), as well as distal vessel occlusion, ischemia, hemorrhage, Alberta stroke program early CT score (ASPECTS), and collateral status (secondary outcomes). Reports were randomly split into training (n=921) and validation (n=399) sets, and attributes were extracted using rule-based NLP. We reported the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and the overall accuracy of the NLP approach relative to the manually extracted data. Results The overall prevalence of large vessel occlusion was 12.2%. In the training sample, the NLP approach identified this attribute with an overall accuracy of 97.3% (95.5% sensitivity, 98.1% specificity, 84.1% PPV, and 99.4% NPV). In the validation set, the overall accuracy was 95.2% (90.0% sensitivity, 97.4% specificity, 76.3% PPV, and 98.5% NPV). The accuracy of identifying distal or basilar occlusion as well as hemorrhage was also high, but there were limitations in identifying cerebral ischemia, ASPECTS, and collateral status. Conclusions NLP may improve the efficiency of large-scale imaging data collection for stroke surveillance and research.


2019 ◽  
Author(s):  
Daniel Leightley ◽  
David Pernet ◽  
Sumithra Velupillai ◽  
Robert J Stewart ◽  
Katharine M Mark ◽  
...  

BACKGROUND Electronic health care records (EHRs) are a rich source of health-related information, with potential for secondary research use. In the United Kingdom, there is no national marker for identifying those who have previously served in the Armed Forces, making analysis of the health and well-being of veterans using EHRs difficult. OBJECTIVE This study aimed to develop a tool to identify veterans from free-text clinical documents recorded in a psychiatric EHR database. METHODS Veterans were manually identified using the South London and Maudsley (SLaM) Biomedical Research Centre Clinical Record Interactive Search—a database holding secondary mental health care electronic records for the SLaM National Health Service Foundation Trust. An iterative approach was taken; first, a structured query language (SQL) method was developed, which was then refined using natural language processing and machine learning to create the Military Service Identification Tool (MSIT) to identify if a patient was a civilian or veteran. Performance, defined as correct classification of veterans compared with incorrect classification, was measured using positive predictive value, negative predictive value, sensitivity, F1 score, and accuracy (otherwise termed Youden Index). RESULTS A gold standard dataset of 6672 free-text clinical documents was manually annotated by human coders. Of these documents, 66.00% (4470/6672) were then used to train the SQL and MSIT approaches and 34.00% (2202/6672) were used for testing the approaches. To develop the MSIT, an iterative 2-stage approach was undertaken. In the first stage, an SQL method was developed to identify veterans using a keyword rule–based approach. This approach obtained an accuracy of 0.93 in correctly predicting civilians and veterans, a positive predictive value of 0.81, a sensitivity of 0.75, and a negative predictive value of 0.95. This method informed the second stage, which was the development of the MSIT using machine learning, which, when tested, obtained an accuracy of 0.97, a positive predictive value of 0.90, a sensitivity of 0.91, and a negative predictive value of 0.98. CONCLUSIONS The MSIT has the potential to be used in identifying veterans in the United Kingdom from free-text clinical documents, providing new and unique insights into the health and well-being of this population and their use of mental health care services.


2021 ◽  
Author(s):  
Junpeng Wang ◽  
Xin Fan ◽  
Shanshan Qin ◽  
Han Zhang ◽  
Fei Yu

Abstract Purpose: To explore the feasibility and efficacy of radiomics with left ventricular tomograms obtained from D-SPECT myocardial perfusion imaging (MPI) for auxiliary diagnosis of myocardial ischemia in coronary artery disease (CAD).Methods: The images of 103 patients with CAD myocardial ischemia between September 2020 and April 2021 were retrospectively selected. After information desensitization processing, format conversion, annotation using the Labelme tool on an open-source platform, lesion classification, and establishment of a database, the images were cropped for analysis. The ResNet18 model was used to automate two steps (classification and segmentation) with five randomization, training and validation steps. Sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, positive predictive value, negative predictive value, Youden’s index, agreement rate, and kappa value were calculated as evaluation indexes of the classification results for each training-validation step; then, receiver operating characteristics (ROC) curves were drawn, and the areas under the curve (AUCs) were calculated. The Dice coefficient, intersection over union, and Hausdorff distance were calculated as evaluation indexes of the segmentation results for each training-validation step; then, the predicted images were exported.Results: Under the existing conditions, the radiomics model can distinguish myocardial ischemia quite accurately, with AUCs all exceeding 0.95, and predict the areas of myocardial ischemia quite accurately; all evaluated indexes were close to those of the gold standard.Conclusion: Radiomics can be feasibly applied to left ventricular tomograms obtained from D-SPECT MPI for auxiliary diagnosis, with quite good results. Patients may benefit from this approach as technology evolves and associated software is developed.


2021 ◽  
Vol 19 (1) ◽  
Author(s):  
Yu Yan ◽  
Lina Qiao ◽  
Yimin Hua ◽  
Shuran Shao ◽  
Nanjun Zhang ◽  
...  

Abstract Background Intravenous immunoglobulin (IVIG) resistance prediction is one of the primary clinical issues and study hotspots in KD. This study aimed to prospectively investigate the value of albumin-bilirubin grade (ALBI) in predicting IVIG resistance in KD and to assess whether ALBI has more predictive value or accuracy than either ALB or TBil alone in predicting IVIG resistance. Methods A total of 823 patients with KD were prospectively enrolled. The clinical and laboratory data were compared between the IVIG-response group (n = 708) and the IVIG-resistance group (n = 115). Multivariate logistic regression analysis was performed to identify the independent risk factors for IVIG resistance. Receiver operating characteristic (ROC) curves analysis was applied to assess the validity of ALBI, ALB, and TBil in predicting IVIG resistance. Results ALBI was significantly higher in patients with IVIG resistance and was identified as an independent risk factor for IVIG resistance in KD. The parameter of ALBI ≥ − 2.57 (AUC: 0.705, 95 %CI: 0.672–0.736), ALB ≤ 33.0 g/L (AUC: 0.659, 95 %CI: 0.626–0.692), and TBil ≥ 16.0µmol/L (AUC: 0.626, 95 %CI: 0.592–0.659), produced a sensitivity, specificity, PPV, and NPV of 0.617, 0.657, 0.226 and 0.914; 0.374, 0.850, 0.289 and 0.893; 0.269, 0.941, 0.425 and 0.888, respectively. Conclusions A higher ALBI was an independent risk factor for IVIG resistance in KD. It yielded better predictive ability than ALB and TBil alone for initial IVIG resistance.


2021 ◽  
Author(s):  
Dane Gunter ◽  
Paulo Puac-Polanco ◽  
Olivier Miguel ◽  
Rebecca E. Thornhill ◽  
Amy Y. X. Yu ◽  
...  

BACKGROUND Data extraction from radiology free-text reports is time-consuming when performed manually. Recently, more automated extraction methods using natural language processing (NLP) are proposed. A previously developed rule-based NLP algorithm showed promise in its ability to extract stroke-related data from radiology reports. OBJECTIVE We aimed to externally validate the accuracy of CHARTextract, a rule-based NLP algorithm, to extract stroke-related data from free-text radiology reports. METHODS Free-text reports of CT angiography (CTA) and perfusion (CTP) studies of consecutive patients with acute ischemic stroke admitted to a regional Stroke center for endovascular thrombectomy were analyzed from January 2015 - 2021. Stroke-related variables were manually extracted (reference standard) from the reports, including proximal and distal anterior circulation occlusion, posterior circulation occlusion, presence of ischemia, hemorrhage, Alberta stroke program early CT score (ASPECTS), and collateral status. These variables were simultaneously extracted using a rule-based NLP algorithm. The NLP algorithm's accuracy, specificity, sensitivity, positive predictive value (PPV), and negative predictive value (NPV) were assessed. RESULTS The NLP algorithm's accuracy was >90% for identifying distal anterior occlusion, posterior circulation occlusion, hemorrhage, and ASPECTS. Accuracy was 85%, 74%, and 79% for proximal anterior circulation occlusion, presence of ischemia, and collateral status respectively. The algorithm had an accuracy of 87-100% for the detection of variables not reported in radiology reports. CONCLUSIONS Rule-based NLP has a moderate to good performance for stroke-related data extraction from free-text imaging reports. The algorithm's accuracy was affected by inconsistent report styles and lexicon among reporting radiologists.


10.2196/15852 ◽  
2020 ◽  
Vol 8 (5) ◽  
pp. e15852
Author(s):  
Daniel Leightley ◽  
David Pernet ◽  
Sumithra Velupillai ◽  
Robert J Stewart ◽  
Katharine M Mark ◽  
...  

Background Electronic health care records (EHRs) are a rich source of health-related information, with potential for secondary research use. In the United Kingdom, there is no national marker for identifying those who have previously served in the Armed Forces, making analysis of the health and well-being of veterans using EHRs difficult. Objective This study aimed to develop a tool to identify veterans from free-text clinical documents recorded in a psychiatric EHR database. Methods Veterans were manually identified using the South London and Maudsley (SLaM) Biomedical Research Centre Clinical Record Interactive Search—a database holding secondary mental health care electronic records for the SLaM National Health Service Foundation Trust. An iterative approach was taken; first, a structured query language (SQL) method was developed, which was then refined using natural language processing and machine learning to create the Military Service Identification Tool (MSIT) to identify if a patient was a civilian or veteran. Performance, defined as correct classification of veterans compared with incorrect classification, was measured using positive predictive value, negative predictive value, sensitivity, F1 score, and accuracy (otherwise termed Youden Index). Results A gold standard dataset of 6672 free-text clinical documents was manually annotated by human coders. Of these documents, 66.00% (4470/6672) were then used to train the SQL and MSIT approaches and 34.00% (2202/6672) were used for testing the approaches. To develop the MSIT, an iterative 2-stage approach was undertaken. In the first stage, an SQL method was developed to identify veterans using a keyword rule–based approach. This approach obtained an accuracy of 0.93 in correctly predicting civilians and veterans, a positive predictive value of 0.81, a sensitivity of 0.75, and a negative predictive value of 0.95. This method informed the second stage, which was the development of the MSIT using machine learning, which, when tested, obtained an accuracy of 0.97, a positive predictive value of 0.90, a sensitivity of 0.91, and a negative predictive value of 0.98. Conclusions The MSIT has the potential to be used in identifying veterans in the United Kingdom from free-text clinical documents, providing new and unique insights into the health and well-being of this population and their use of mental health care services.


2019 ◽  
Vol 1 (1) ◽  
pp. 11-15 ◽  
Author(s):  
Sarah Yaziz ◽  
Ahmad Sobri Muda ◽  
Wan Asyraf Wan Zaidi ◽  
Nik Azuan Nik Ismail

Background : The clot burden score (CBS) is a scoring system used in acute ischemic stroke (AIS) to predict patient outcome and guide treatment decision. However, CBS is not routinely practiced in many institutions. This study aimed to investigate the feasibility of CBS as a relevant predictor of good clinical outcome in AIS cases. Methods:  A retrospective data collection and review of AIS patients in a teaching hospital was done from June 2010 until June 2015. Patients were selected following the inclusion and exclusion criteria. These patients were followed up after 90 days of discharge. The Modified Rankin scale (mRS) was used to assess their outcome (functional status). Linear regression Spearman Rank correlation was performed between the CBS and mRS. The quality performance of the correlations was evaluated using Receiver operating characteristic (ROC) curves. Results: A total of 89 patients with AIS were analysed, 67.4% (n=60) male and 32.6% (n=29) female. Twenty-nine (29) patients (33.7%) had a CBS ?6, 6 patients (6.7%) had CBS <6, while 53 patients (59.6%) were deemed clot free. Ninety (90) days post insult, clinical assessment showed that 57 (67.6%) patients were functionally independent, 27 (30.3%) patients functionally dependent, and 5 (5.6%) patients were deceased. Data analysis reported a significant negative correlation (r= -0.611, p<0.001). ROC curves analysis showed an area under the curve of 0.81 at the cut-off point of 6.5. This showed that a CBS of more than 6 predicted a good mRS clinical outcome in AIS patients; with sensitivity of 98.2%, specificity of 53.1%, positive predictive value (PPV) of 76%, and negative predictive value (NPV) of 21%. Conclusion: CBS is a useful additional variable for the management of AIS cases, and should be incorporated into the routine radiological reporting for acute ischemic stroke (AIS) cases.


2011 ◽  
Vol 42 (2) ◽  
pp. 56-64 ◽  
Author(s):  
Remigiusz Szczepanowski

Conscious access to fear-relevant information is mediated by thresholdThe present report proposed a model of access consciousness to fear-relevant information according to which there is a threshold for emotional perception beyond that the subject makes hits with no false alarm. The model was examined by having the participants performed a confidence-ratings masking task with fearful faces. Measures of the thresholds for conscious access were taken by looking at the receiver operating characteristics (ROC) curves generated from a three-state low- and high-threshold (3-LHT) model by Krantz. Indeed, the analysis of the masking data revealed that the ROCs had threshold-like-nature (a two-limb shape) rather continuous (a curvilinear shape) challenging in this fashion the classical signal-detection view on perceptual processing. Moreover, the threshold ROC curve exhibited the specific y-intercepts relevant to conscious access performance. The study suggests that the threshold can be an intrinsic property of conscious access, mediating emotional contents between perceptual states and consciousness.


2021 ◽  
Vol 4 (Supplement_1) ◽  
pp. 92-93
Author(s):  
M Sey ◽  
O Siddiqi ◽  
C McDonald ◽  
S cocco ◽  
Z Hindi ◽  
...  

Abstract Background Performing a minimum number of colonoscopies annually has been proposed by some jurisdictions as a requirement for maintaining privileges. However, this practice is supported by limited evidence. Aims The objective of this study was to determine if annual colonoscopy volume was associated with colonoscopy quality metrics. Methods A population-based study was performed using the Southwest Ontario Colonoscopy cohort, which consists of all adult patients who underwent colonoscopy between April 2017 and Oct 2018 at 21 academic and community hospitals within the health region. Data were collected through a mandatory quality assurance form completed after each procedure and pathology reports were manually reviewed. Physician annualized colonoscopy volumes were compared by correlation analysis to each quality-related outcome, by means of the area under the receiver operating characteristics curve (AUROC), and logistic regression. The prognostic value of colonoscopy volume was also adjusted for case-mix and potential confounders in separate regression analyses for each outcome. The primary outcome was ADR. Secondary outcomes were polyp detection rate (PDR), sessile serrated polyp detection rate (SSPDR), and cecal intubation. Results A total of 47,195 colonoscopies were performed by 75 physicians (37.5% by gastroenterologists, 60% by general surgeons, 2.5% others). There were no clear relationships between annual colonoscopy volumes and study outcomes. Colonoscopy volume was not associated with ADR (OR 1.03, 95% CI 0.96–1.10, p=0.48) and corresponded to an AUROC not significantly different from the null (AUROC 0.52, 95% CI 0.43–0.61, p=0.65). Multi-variable regression adjusting for case-mix also demonstrated no predictive value of annual colonoscopy volume for the primary outcome (OR 1.03, 95% CI 0.94–1.12, p=0.55). Similarly, analyses of secondary outcomes failed to find an association between colonoscopy volume and PDR, SSPDR, or cecal intubation (Table 1). Conclusions Annual colonoscopy volumes do not predict ADR, PDR, SSPDR, or cecal intubation rate. Results of unconditional and conditional approaches for examining the predictive value of annual colonoscopy volume for quality related outcomes. Funding Agencies None


Sign in / Sign up

Export Citation Format

Share Document