scholarly journals Developing and validating a primary care EMR-based frailty definition using machine learning

2020 ◽  
Vol 5 (1) ◽  
Author(s):  
Tyler Williamson ◽  
Sylvia Aponte-Hao ◽  
Bria Mele ◽  
Brendan Cord Lethebe ◽  
Charles Leduc ◽  
...  
Keyword(s):  
Machine Learning ◽  
Primary Care ◽  
Sensitivity And Specificity ◽  
Health Spending ◽  
Case Definition ◽  
Complex Nature ◽  
Case Definitions ◽  
Surveillance Network ◽  
Clinical Frailty Scale ◽  
Cross Sectional

Introduction. Individuals who have been identified as frail have an increased state of vulnerability, often leading to adverse health events, increased health spending, and potentially detrimental outcomes. Objective. The objective of this work is to develop and validate a case definition for frailty that can be used in a primary care electronic medical record database. Methods. This is a cross-sectional validation study using data from the Canadian Primary Care Sentinel Surveillance Network (CPCSSN) in Southern Alberta. 52 CPCSSN sentinels assessed a random sample of their own patients using the Rockwood Clinical Frailty scale, resulting in a total of 875 patients to be used as reference standard. Patients must be over the age of 65 and have had a clinic visit within the last 24 months. The case definition for frailty was developed using machine learning methods using CPCSSN records for the 875 patients. Results. Of the 875 patients, 155 (17.7%) were frail and 720 (84.2%) were not frail. Validation metrics of the case definition were: sensitivity and specificity of 0.28, 95% CI (0.21 to 0.36) and 0.94, 95% CI (0.93 to 0.96), respectively; PPV and NPV of 0.53, 95% CI (0.42 to 0.64) and 0.86, 95% CI (0.83 to 0.88), respectively. Conclusion. The low sensitivity and specificity results could be because frailty as a construct remains under-developed and relatively poorly understood due to its complex nature. These results contribute to the literature by demonstrating that case definitions for frailty require expert consensus and potentially more sophisticated algorithms to be successful

scholarly journals Developing a Primary Care EMR-based Frailty Definition using Machine Learning

2018 ◽  
Vol 3 (4) ◽  
Author(s):  
Sylvia Aponte-Hao ◽  
Bria Mele ◽  
Dave Jackson ◽  
Alan Katz ◽  
Charles Leduc ◽  
...  
Keyword(s):  
Machine Learning ◽  
Primary Care ◽  
Positive Predictive Value ◽  
Predictive Value ◽  
Laboratory Data ◽  
Screening Tools ◽  
Geriatric Syndrome ◽  
Surveillance Network ◽  
Clinical Frailty Scale ◽  
Definition Of

IntroductionFrailty is a geriatric syndrome that is predictive of heightened vulnerability for disability, hospitalization, and mortality. Annually an estimated 250,000 frail Canadians die, and this estimate is expected to double in the next 40 years, as Canadians grow older. Currently there is no single accepted clinical definition of frailty. Objectives and ApproachThe objective of this study was to develop an operational definition of frailty using machine learning that can be applied to a primary care electronic medical record (EMR) database. The Canadian Primary Care Sentinel Surveillance Network (CPCSSN) is a pan-Canadian network of primary care practices that collect de-identified patient information (such as encounter diagnoses, health conditions, and laboratory data) from EMRs. 780 patients from CPCSSN have were randomly selected and assessed by physicians using the Rockwood Clinical Frailty Scale (as frail or not frail), and their clinical characteristics from CPCSSN used to develop the definition using machine-learning. ResultsA total of 8,044 clinical features were extracted from these tables: billing, problem list, encounter diagnosis, labs, medications and referrals. A chi-squared automatic interaction detector (CHAID) approach was selected as the best approach. The bootstrapping process used a cost matrix that prioritized high sensitivity and positive predictive value. 10-fold cross validation was used for validity measures. Key features factored into the algorithm included: diagnosis of dementia (ICD-9 code 290), medications furosemide and vitamins, and use of key word “obstruction” within the billing table. The validation measures with 95% confidence intervals are as follows: sensitivity of 28% (95% CI: 21% to 36%), specificity of 94% (95% CI: 93% to 96%), positive predictive value of 53% (95% CI: 42% to 64%), negative predictive value of 86% (95% CI: 83% to 88%). Conclusion/ImplicationsNo other primary care specific frailty screening tools have sufficient validity. These results suggest heterogeneous diseases require clearly defined features and potentially more sophisticated algorithms to account for heterogeneity. Further research utilizing continuous features and continuous frailty scores may be more suitable in the creation of a case detection algorithm.

scholarly journals Guidance for Studies Evaluating the Accuracy of Sputum-Based Tests to Diagnose Tuberculosis

2019 ◽  
Vol 220 (Supplement_3) ◽  
pp. S99-S107 ◽  
Author(s):  
Samuel G Schumacher ◽  
William A Wells ◽  
Mark P Nicol ◽  
Karen R Steingart ◽  
Grant Theron ◽  
...  
Keyword(s):  
Test Performance ◽  
Scale Up ◽  
Case Definition ◽  
World Health ◽  
Complex Nature ◽  
Sputum Smear ◽  
Smear Microscopy ◽  
Test Accuracy ◽  
Consecutive Series ◽  
Cross Sectional

Abstract Tests that can replace sputum smear microscopy have been identified as a top priority diagnostic need for tuberculosis by the World Health Organization. High-quality evidence on diagnostic accuracy for tests that may meet this need is an essential requirement to inform decisions about policy and scale-up. However, test accuracy studies are often of low and inconsistent quality and poorly reported, leading to uncertainty about true test performance. Here we provide guidance for the design of diagnostic test accuracy studies of sputum smear-replacement tests. Such studies should have a cross-sectional or cohort design, enrolling either a consecutive series or a random sample of patients who require evaluation for tuberculosis. Adults with respiratory symptoms are the target population. The reference standard should at a minimum be a single, automated, liquid culture, but additional cultures, follow-up, clinical case definition, and specific measures to understand discordant results should also be included. Inclusion of smear microscopy and Xpert MTB/RIF (or MTB/RIF Ultra) as comparators is critical to allow broader comparability and generalizability of results, because disease spectrum can vary between studies and affects relative test performance. Given the complex nature of sputum (the primary specimen type used for pulmonary TB), careful design and reporting of the specimen flow is essential. Test characteristics other than accuracy (such as feasibility, implementation considerations, and data on impact on patient, population and health systems outcomes) are also important aspects.

scholarly journals Machine learning for identification of frailty in Canadian primary care practices

2021 ◽  
Vol 6 (1) ◽  
Author(s):  
Sylvia Aponte-Hao ◽  
Sabrina T. Wong ◽  
Manpreet Thandi ◽  
Paul Ronksley ◽  
Kerry McBrien ◽  
...  
Keyword(s):  
Machine Learning ◽  
Primary Care ◽  
Predictive Value ◽  
Case Definition ◽  
Machine Learning Algorithms ◽  
Supervised Machine Learning ◽  
Sensitivity Analyses ◽  
Screening Tests ◽  
Training Dataset ◽  
Total N

IntroductionFrailty is a medical syndrome, commonly affecting people aged 65 years and over and is characterized by a greater risk of adverse outcomes following illness or injury. Electronic medical records contain a large amount of longitudinal data that can be used for primary care research. Machine learning can fully utilize this wide breadth of data for the detection of diseases and syndromes. The creation of a frailty case definition using machine learning may facilitate early intervention, inform advanced screening tests, and allow for surveillance. ObjectivesThe objective of this study was to develop a validated case definition of frailty for the primary care context, using machine learning. MethodsPhysicians participating in the Canadian Primary Care Sentinel Surveillance Network across Canada were asked to retrospectively identify the level of frailty present in a sample of their own patients (total n = 5,466), collected from 2015-2019. Frailty levels were dichotomized using a cut-off of 5. Extracted features included previously prescribed medications, billing codes, and other routinely collected primary care data. We used eight supervised machine learning algorithms, with performance assessed using a hold-out test set. A balanced training dataset was also created by oversampling. Sensitivity analyses considered two alternative dichotomization cut-offs. Model performance was evaluated using area under the receiver-operating characteristic curve, F1, accuracy, sensitivity, specificity, negative predictive value and positive predictive value. ResultsThe prevalence of frailty within our sample was 18.4%. Of the eight models developed to identify frail patients, an XGBoost model achieved the highest sensitivity (78.14%) and specificity (74.41%). The balanced training dataset did not improve classification performance. Sensitivity analyses did not show improved performance for cut-offs other than 5. ConclusionSupervised machine learning was able to create well performing classification models for frailty. Future research is needed to assess frailty inter-rater reliability, and link multiple data sources for frailty identification.

scholarly journals MEASLES IN LARUT, MATANG AND SELAMA: ANALYSIS AND EVALUATION OF CLINICAL CASE DEFINITIONS FOR 2015-2019

2020 ◽  
Vol 20 (1) ◽  
pp. 130-140
Author(s):  
Asraf Ahmad Qamruddin ◽  
Afiq Malek ◽  
Asnita Rozali ◽  
Norsihimah Wahid
Keyword(s):  
Positive Predictive Value ◽  
Incidence Rate ◽  
Negative Predictive Value ◽  
Predictive Value ◽  
Clinical Case ◽  
Case Definition ◽  
Case Definitions ◽  
Cross Sectional ◽  
Sensitivity Specificity ◽  
Clinical Case Definition

An accurate system of identifying measles cases is critical for the measles surveillance system. The objectives were: 1) To determine the incidence rate of measles in Larut, Matang and Selama district in Perak from 2015 to 2019 2) To evaluate the measles clinical case definition by comparing the performance of the measles clinical case definition in predicting laboratory-confirmed measles case. A cross-sectional analysis was carried out looking at all suspected and laboratory-confirmed measles cases in Larut, Matang and Selama District registered on the online measles surveillance reporting system between 2015 to 2019. The sensitivity, specificity, positive predictive value and negative predictive value of the clinical case definition as confirmed by the laboratory result were calculated. The incidence rate for suspected measles showed an increasing trend from 3.96 per 100,000 population in 2015 to 28.82 per 100,000 population in 2019. For laboratory-confirmed measles cases, the incidence rate showed more variation with an increase to 36.11 per million population in 2017 from 5.67 per million population in 2015. The incidence rate later decreased to 10.99 per million population in 2018 and increased again to 24.47 per million population in 2019. The sensitivity of the clinical case definition in confirming measles was 86.67% (95% CI: 69.28%, 96.24%) , specificity 47.52% (95% CI: 41.56%, 53.52%), positive predictive value 14.95% (95% CI 12.81%, 17.36%)  and negative predictive value 97.10% (93.03%, 98.83%). Measles incidence is increasing in trend. The clinical case definition is an effective tool to rule out measles in cases that failed to meet the criteria due to the high negative predictive value of the definition. However, for cases that meet the clinical case definition, laboratory confirmation or epidemiological link to a confirmed case is needed.

scholarly journals Chikungunya: important lessons from the Jamaican experience

2017 ◽  
Vol 41 ◽  
pp. 1 ◽  
Author(s):  
Jacqueline Duncan ◽  
Kelly Ann Gordon-Johnson ◽  
Marshall K Tulloch-Reid ◽  
Colette Cunningham-Myrie ◽  
Kacey Ernst ◽  
...  
Keyword(s):  
Cross Sectional Study ◽  
Clinical Findings ◽  
Case Definition ◽  
Health Centers ◽  
Case Definitions ◽  
Ministry Of Health ◽  
Cross Sectional ◽  
Major Health ◽  
Viral Illness ◽  
Health Organization

Objectives. To describe the clinical presentation of chikungunya virus (CHIKV) illness in adults during the 2014 outbreak in Jamaica and to determine the predictive value of the case definition. Methods. A cross-sectional study was conducted using clinical data from suspected cases of CHIKV that were reported to the Ministry of Health in April – December 2014. In addition, charts were reviewed of all individuals over 15 years of age with suspected CHIKV based on a diagnosis of CHIKV or “acute viral illness” that presented to four major health centers in Jamaica during the week prior to and the peak week of the epidemic. Data abstracted from these charts using a modified CHIKV Case Investigation Form included demographics, clinical findings, and laboratory tests. Results. In 2014, the Ministry of Health of Jamaica received 4 447 notifications of CHIKV infection. PCR testing was conducted on 137 suspected CHIKV cases (56 men and 81 women; median age 28 years) and was positive for 89 (65%) persons. In all, 205 health charts were identified that met the selection criteria (51 men and 154 women, median age 43 years). The most commonly reported symptoms were arthralgia (86%) and fever (76%). Of those who met the epidemiologic case definition for CHIKV as defined by the Pan American Health Organization, only 34% had this diagnosis recorded. Acute viral illness was the most frequently recorded diagnosis (n = 79; 58%). Conclusions. Broader case definitions for acute CHIKV illness may be needed to identify suspected cases during an outbreak. Standardized data collection forms and validation of case definitions may be useful for future outbreaks.

scholarly journals 1644. Performance of Symptom-Based Case Definitions to Identify Influenza Virus Infection among Pregnant Women in Middle-Income Countries: Findings from the Pregnancy and Influenza Multinational Epidemiologic (PRIME) Study

2019 ◽  
Vol 6 (Supplement_2) ◽  
pp. S600-S600
Author(s):  
Meredith G Wesley ◽  
Yeny Tinoco ◽  
Archana Patel ◽  
Piyarat Suntarattiwong ◽  
Danielle R Hunt ◽  
...  
Keyword(s):  
Pregnant Women ◽  
Sensitivity And Specificity ◽  
Univariate Analysis ◽  
Influenza Virus Infection ◽  
Case Definition ◽  
World Health ◽  
Influenza Surveillance ◽  
Multivariable Model ◽  
Case Definitions ◽  
Runny Nose

Abstract Background The World Health Organization (WHO) recommends case definitions for influenza surveillance that are also used in public health research, though their performance has not been assessed in many risk groups, including pregnant women in whom influenza may manifest differently. Â We evaluated the performance of symptom-based case definitions to detect influenza in a cohort of pregnant women in India, Peru, and Thailand. Methods In 2017, we contacted 4774 pregnant women twice a week during the influenza season to identify illnesses with new or worsened cough, runny nose, sore throat, difficulty breathing or myalgia, and collected data on other symptoms and nasal swabs for influenza rRT–PCR testing. To identify symptom predictors of influenza, we used multivariable logistic regression with forward selection of symptoms significant in univariate analysis after controlling for country, chronic conditions, influenza vaccination, and time from symptom onset to swab collection. We calculated sensitivity and specificity of each symptom, WHO respiratory illness case definitions and a case definition based on significant predictors from the multivariable model. Results Of 2431 eligible illness episodes among 1,716 participants, 142 (5.8%) were positive for influenza. Among individual symptoms, runny nose was most sensitive and measured fever ≥ 38° Celsius was most specific (Figure 1). In a multivariable model, measured fever ≥ 38° Celsius [adjusted odds ratio = 3.8, 95% confidence interval [CI] = 2.0–7.2], cough [2.7, CI 1.6–4.7], chills [2.2, CI 1.2–3.8], and myalgia [1.2, CI 2.2, 5.3] were independently associated with influenza illness. A case definition based on these four (measured fever, cough, chills or myalgia), was 91%-sensitive and 37% specific. Sensitivity and specificity of case definitions varied (Figure 2). Conclusion While a case definition based on one or more of fever, chills, cough or myalgia is highly-sensitive and moderately specific among pregnant women, case definitions requiring measured or subjective fever may miss many influenza cases making them sub-optimal for studies of burden or vaccine efficacy. The intended use of case definitions should be considered when evaluating the tradeoff between sensitivity and specificity. Disclosures All authors: No reported disclosures.

scholarly journals Prevalence of multimorbidity in general practice: a cross-sectional study within the Swiss Sentinel Surveillance System (Sentinella)

BMJ Open ◽  
2018 ◽  
Vol 8 (3) ◽  
pp. e019616 ◽  
Author(s):  
Sophie Excoffier ◽  
Lilli Herzig ◽  
Alexandra A N’Goran ◽  
Anouk Déruaz-Luyet ◽  
Dagmar M Haller
Keyword(s):  
Primary Care ◽  
Health System ◽  
Chronic Conditions ◽  
Cross Sectional Study ◽  
Sentinel Surveillance ◽  
Psychological Disorder ◽  
Secondary Outcome ◽  
Endocrine Disorders ◽  
Surveillance Network ◽  
Cross Sectional

ObjectivesTo estimate the prevalence of multimorbidity using a list of 75 chronic conditions derived from the International Classification for Primary Care, Second edition and developed specifically to assess multimorbidity in primary care. Our aim was also to provide prevalence data for multimorbidity in primary care in a country in which general practitioners (GPs) do not play a gatekeeping role in the health system.SettingA representative sample of GPs within the Swiss Sentinel Surveillance Network.Participants118 GPs completed a paper-based questionnaire about 25 consecutive patients of all ages between September and November 2015. There were no patient exclusion criteria. Recorded data included date of birth, gender and the patients’ chronic conditions.Primary and secondary outcome measuresWe estimated the prevalence of multimorbidity, defined as ≥2, and ≥3 chronic conditions stratified by gender and age group, and adjusted for clustering by GPs. We also computed the prevalence of each chronic condition individually and grouped by system.ResultsData from 2904 patients were included (mean age (SD)=56.5 (20.5) years; male=43.7%). Prevalence was 52.1% (95% CI 48.6% to 55.5%) for ≥2 and 35.0% (95% CI 31.6% to 38.5%) for ≥3 chronic conditions, with no significant gender differences. Prevalence of two or more chronic conditions was low (6.2%, 95% CI 2.8% to 13.0%) in those below 20 but affected more than 85% (85.8%, 95% CI 79.6% to 90.3%) of those above the age of 80. The most prevalent conditions were cardiovascular (42.7%, 95% CI 39.7% to 45.7%), psychological (28.5%, 95% CI 26.1% to 31.1%) and metabolic or endocrine disorders (24.1%, 95% CI 21.6% to 26.7%). Elevated blood pressure was the most prevalent cardiovascular condition and depression the most common psychological disorder.ConclusionIn a country in which GPs do not play a gatekeeping role within the health system, the prevalence of multimorbidity, as assessed using a list of chronic conditions specifically relevant to primary care, is high and increases with age.

scholarly journals Validation of four AIDS-case definitions in HIV-infected intravenous drug users in Barcelona, Spain

1998 ◽  
Vol 1 (2) ◽  
pp. 170-176 ◽  
Author(s):  
Paulo Sérgio C. Miranda ◽  
Andrés Marco ◽  
Joan Arthur Caylà ◽  
Hernando Galdós-Tangüis ◽  
Patricia García de Olalla
Keyword(s):  
Cell Count ◽  
Sensitivity And Specificity ◽  
Drug Users ◽  
Case Definition ◽  
Intravenous Drug ◽  
Intravenous Drug Users ◽  
Case Definitions ◽  
Predictive Values ◽  
Cd4 Cell Count ◽  
Cd4 Cell

The objective of the study is to assess the sensitivity and specificity of four epidemiological AIDS-Case Definitions (CDC-87, CDC-93, Europe-93 and Revised Caracas) in HIV-infected intravenous drug users (IDU). The authors carried out a cross-sectional study with 136 IDUs, HIV-infected from a Men Penitentiary Center and from a drug addiction treatment center of Barcelona, Spain, between October/93 and April/94. A protocol, including demographic, clinical and laboratory variables was used by one doctor and the laboratory tests were done in the same institution. After that, the patients were classified in the four Epidemiological AIDS-Case Definitions used by this study. As gold standard we used the CD4 Cell Count (out point 200 or 14% CD4+). The number of AIDS cases varied between 31 and 84 according to the type of AIDS definition. The CDC-93 AIDS definition implied an increase of 170.9% in the number of cases in relation to CDC-87 AIDS-Case Definition. The sensitivities of the CDC-87, CDC-93, Europe-93 and Revised Caracas Epidemiological AIDS - Case Definitions were 34.2, 88.6, 45.6 and 56.9% while the specificities were 93.0, 75.4, 75.4 and 77.2%, respectively. The positive predictive values were between 72.0% (Europe-93) and 87.1% (CDC-87) and the negative predictive values were between 50.0% (Europe-93) and 82.7% (CDC-93). The authors concluded: the sensitivity and specificity of Caracas Revised Epidemiological AIDS-Case Definition was better than Europe-93 AIDS Case Definition. So this Definition can be very useful in countries and situations where the CD4 Cell Count is not available for technical or economical reasons.

Sign in / Sign up

Export Citation Format

Share Document