Can we predict failure in licensure exams from medical students’ undergraduate academic performance?

Background: In 2015, the Medical Council of Canada increased the minimum pass level for the Medical Council of Canada Qualifying Examination Part I, and students had a higher rate of failure than in previous years. The purpose of this study was to predict students at an increased odds of examination failure to allow for early, targeted interventions. Methods: We divided our dataset into a derivation cohort and two validation cohorts and used multiple logistic regression to predict licensing examination failure. We then performed receiver operating characteristics and a sensitivity analysis using different cutoffs for explanatory variables to identify the cutoff threshold with the best predictive value at identifying students at increased odds of failure. Results: After multivariate analysis, only pre-clerkship GPA was a significant independent predictor of failure (OR 0.76, 95% CI [0.66, 0.88], p < 0.001). The probability of failure increased steeply when the pre-clerkship GPA fell below 80% and 76% was found to be the most efficient cutoff for predicting failure (OR 9.37, 95% CI [3.08, 38.41]). Conclusions: Pre-clerkship performance can predict students at increased odds of licensing examination failure. Further studies are needed to explore whether early interventions for at-risk students alter their examination performance.

Download Full-text

Does the Medical College Admission Test (MCAT) predict licensing examination performance in the Canadian context?

Canadian Medical Education Journal ◽

10.36834/cmej.42307 ◽

2019 ◽

Vol 10 (1) ◽

pp. e13-19 ◽

Cited By ~ 1

Author(s):

Maitreyi Raman ◽

Sara Lukmanji ◽

Ian Walker ◽

Douglas Myhre ◽

Sylvain Coderre ◽

...

Keyword(s):

Predictive Validity ◽

Odds Ratio ◽

Validation Cohort ◽

Medical Council ◽

Successful Performance ◽

Derivation Cohort ◽

Medical College ◽

Examination Performance ◽

Canadian Context ◽

Licensing Examination

Background: Research on the predictive validity of the Medical College Admissions Test (MCAT) on licensing examination performance is varied in its conclusions, with only a few studies examining this relationship in a Canadian context. We assessed the predictive validity of the MCAT on successful performance on the Medical Council of Canada Qualifying Examination (MCCQE) Part 1 by students attending the Cumming School of Medicine. Methods: Prospective data were collected on MCAT score and sub-section scores, MCCQE decision, multiple mini interview (MMI) performance, gender, and age. The cohort was divided into a derivation cohort (2013 and 2014) and validation cohort (2015 and 2016). Students were dichotomized into pass or fail on MCCQE. Multiple logistic regression in which our dependent variable was MCCQE Part I examination success at the first attempt was used, and potential explanatory variables were age, gender, MCAT total score, and sub-scores for the biological sciences (MCAT-BS), physical sciences, and verbal reasoning, GPA, and MMI ratings.Results: For the derivation cohort MCAT-BS was associated with success on the MCCQE Part I. The odds ratio for this association of 1.37 (95% confidence interval [1.01, 1.85], p = 0.04). When we applied the MCAT-BS to our validation cohort the odds ratio of MCCQE Part I examination success was 1.42 [1.10, 1.83], p = 0.007) and the area under the ROC curve was 0.66 [0.54, 0.79]).Conclusion: The MCAT-BS predicted successful performance on the MCCQE Part 1 Examination in the Canadian setting.

Download Full-text

Development and validation of an ECG-based cardiac syncope risk calculator

European Heart Journal ◽

10.1093/ehjci/ehaa946.0703 ◽

2020 ◽

Vol 41 (Supplement_2) ◽

Author(s):

T Zimmermann ◽

J Du Fay De Lavallaz ◽

T Nestelberger ◽

D Gualandro ◽

I Strebel ◽

...

Keyword(s):

Diagnostic Accuracy ◽

Clinical Information ◽

Funding Source ◽

Risk Calculator ◽

Operating Characteristics ◽

Derivation Cohort ◽

Cardiac Syncope ◽

Predicted Probability ◽

Ecg Criteria ◽

Rule Out

Abstract Background The early diagnosis of cardiac syncope is often challenging. We therefore developed an ECG-based risk calculator as an aid for rapid rule-out or rule-in of cardiac syncope and aimed to validate this decision tool. Methods In a prospective diagnostic international multicenter study (derivation cohort), 2007 patients, 40 years or older, presenting with syncope to the emergency department were recruited. The primary diagnostic outcome, cardiac syncope, was centrally adjudicated by two independent cardiologists using all clinical information obtained during syncope work-up including 12-month follow up. 12-lead ECG was recorded at presentation and read by residents blinded to clinical information. Significant ECG predictors of cardiac syncope were identified using penalized backward selection. Findings were validated in an independent US multicenter cohort with 2'269 syncope patients. Results In the derivation cohort (median age 71 years, 40% women), centrally adjudicated cardiac syncope was present in 267 patients (16%). Seven ECG criteria (rhythm, heart rate, corrected QT-interval, ST-segment depression, atrioventricular-block, bundle-branch-block and ventricular extrasystole/non-sustained ventricular tachycardia) were identified as significant predictors for cardiac syncope and combined into the bAseL Ecg Risk calculaTor for Cardiac Syncope (ALERT-CS). Diagnostic accuracy of ALERT-CS for cardiac syncope, as quantified by the area under the receiver-operating characteristics curve (AUC), was high (0.80, 95%-confidence interval (CI) 0.77–0.83) and significantly higher compared to the EGSYS score (0.73, 95% CI 0.70–0.76, p<0.001). In combination, ALERT-CS significantly increased the AUC of BNP (0.82, 95% CI 0.79–0.85 vs 0.77, 95% CI 0.74–0.81, p=0.003), hs-cTnT (0.84, 95% CI 0.0.81–0.87 vs 0.77, 95% CI 0.74–0.80, p<0.001) and integrated clinical judgment in the ED (0.90, 95% CI 0.89–0.92 vs 0.87, 95% CI 0.84–0.90, p<0.001). A predicted probability for cardiac syncope below 5.5% by ALERT-CS identified 138 patients (8%) eligible for triage towards rapid rule-out of cardiac syncope with a sensitivity of 99%. A predicted probability above 37.5% identified 181 patients (11%) eligible for triage towards rapid rule-in of cardiac syncope with a specificity of 95%. Prognostic verification for 30-day major adverse cardiac events (MACE) showed a high rate of MACE in the rule-in group and a very low rate of MACE in the rule-out group (Figure). External validation (median age 72 years, 48% women) showed similar diagnostic accuracy (AUC 0.76, 95% CI 0.73–0.79) and prognostic results. Conclusion Combining seven ECG criteria within the simple ALERT-CS may aid ED physicians in the early rule-out or rule-in of cardiac syncope. Figure 1 Funding Acknowledgement Type of funding source: Public grant(s) – National budget only. Main funding source(s): Swiss National Science Foundation, Swiss Heart Foundation

Download Full-text

Integration of Remotely Sensed Soil Sealing Data in Landslide Susceptibility Mapping

Remote Sensing ◽

10.3390/rs12091486 ◽

2020 ◽

Vol 12 (9) ◽

pp. 1486

Author(s):

Tania Luti ◽

Samuele Segoni ◽

Filippo Catani ◽

Michele Munafò ◽

Nicola Casagli

Keyword(s):

Landslide Susceptibility ◽

Landslide Susceptibility Mapping ◽

Categorical Variable ◽

Operating Characteristics ◽

Mapping Unit ◽

Explanatory Variables ◽

The Road ◽

Soil Sealing ◽

Susceptibility Model ◽

Remote Sensing Techniques

Soil sealing is the destruction or covering of natural soils by totally or partially impermeable artificial material. ISPRA (Italian Institute for Environmental Protection Research) uses different remote sensing techniques to monitor this process and updates yearly a national-scale soil sealing map of Italy. In this work, for the first time, we tried to combine soil sealing indicators as additional parameters within a landslide susceptibility assessment. Four new parameters were derived from the raw soil sealing map: Soil sealing aggregation (percentage of sealed soil within each mapping unit), soil sealing (categorical variable expressing if a mapping unit is mainly natural or sealed), urbanization (categorical variable subdividing each unit into natural, semi-urbanized, or urbanized), and roads (expressing the road network disturbance). These parameters were integrated with a set of well-established explanatory variables in a random forest landslide susceptibility model and different configurations were tested: Without the proposed soil-sealing-derived variables, with all of them contemporarily, and with each of them separately. Results were compared in terms of AUC ((area under receiver operating characteristics curve, expressing the overall effectiveness of each configuration) and out-of-bag-error (estimating the relative importance of each variable). We found that the parameter “soil sealing aggregation” significantly enhanced the model performances. The results highlight the potential relevance of using soil sealing maps on landslide hazard assessment procedures.

Download Full-text

A novel predictive model based on inflammatory markers to assess the prognosis of patients with HBV-related acute-on-chronic liver failure：a retrospective cohort study

10.21203/rs.2.14221/v3 ◽

2020 ◽

Author(s):

Li Qiang ◽

Jiao Qin ◽

Changfeng Sun ◽

Yunjian Sheng ◽

Wen Chen ◽

...

Keyword(s):

Liver Failure ◽

Predictive Value ◽

Inflammatory Markers ◽

Validation Cohort ◽

Meld Score ◽

Chronic Liver ◽

Assessment Model ◽

Operating Characteristics ◽

Distribution Width ◽

Derivation Cohort

Abstract Background: Systemic inflammatory response is closely related to the development and prognosis of liver failure. This study aimed to establish a new model combing the inflammatory markers including neutrophil/lymphocyte ratio (NLR) and red blood cell distribution width (RDW) with several hematological testing indicators to assess the prognosis of patients with hepatitis B virus-related acute-on-chronic liver failure (HBV-ACLF). Methods: A derivation cohort with 421 patients and a validation cohort with 156 patients were recruited from three hospitals. Retrospectively collecting their clinical data and laboratory testing indicators. Medcalc-15.10 software was employed for Data analyses.Results: Multivariate analysis indicated that RDW, NLR, INR, TBIL and Cr were risk factors for 90-day mortality in patients with HBV-ACLF. The risk assessment model isCOXRNTIC=0.053×RDW+0.027×NLR+0.003×TBIL+0.317×INR+0.003×Cr (RNTIC) with a cut-off value of 3.08 (sensitivity: 77.89%, specificity: 86.04%). The area under the receiver operating characteristics curve (AUC) of the RNTIC was 0.873 [95%CI(0.837–0.903)], better than the predictive value of MELD score [0.732, 95%CI(0.687–0.774)], MELD-Na [0.714, 95%CI(0.668-0.757)], CTP[0.703, 95%CI(0.657-0.747)]. In the validation cohort, RNTIC also performed a better prediction value than MELD score, MELD-Na and CTP with the AUC of [0.845, 95%CI(0.778-0.898)], [0.768, 95%CI (0.694-0.832)], [0.759, 95%CI(0.684-0.824)] and [0.718, 95%CI(0.641-0.787)] respectively. Conclusions: The inflammatory markers RDW and NLR could be used as independent predictors of 90-day mortality in patients with HBV-ACLF. Compared with MELD score, RNTIC had a more powerful predictive value for prognosis of patients with HBV-ACLF.

Download Full-text

P3794MOODS: a novel risk score to identify patients with atrial fibrillation and sleep apnoea

European Heart Journal ◽

10.1093/eurheartj/ehz745.0639 ◽

2019 ◽

Vol 40 (Supplement_1) ◽

Author(s):

K Kadhim ◽

A Elliott ◽

M Middeldorp ◽

J Hendriks ◽

C Gallagher ◽

...

Keyword(s):

Atrial Fibrillation ◽

Risk Score ◽

Clinical Characteristics ◽

Validation Cohort ◽

Multivariate Logistic Regression Analysis ◽

Positive Likelihood Ratio ◽

Resource Utilisation ◽

Operating Characteristics ◽

Derivation Cohort ◽

Sleep Studies

Abstract Background Sleep-disordered breathing (SDB) is an important risk factor for developing atrial fibrillation (AF), and treatment of concomitant SDB can improve AF rhythm outcomes. Diagnosis of SDB requires sleep studies which can pose a significant time and resource burden. We sought to develop a prediction score based on clinical characteristics that can help identify AF patients who require further assessment for SDB. Methods Prospectively-collected data for 442 consecutive patients treated for AF from 2009 to 2017 were analysed. All patients were considered candidates for rhythm-control and therefore referred for sleep studies. The diagnosis of SDB was confirmed using in-lab polysomnography and classified using the apnoea-hypopnoea-index (AHI), with cut-offs of ≥15/hr and ≥30/hr indicating moderate-to-severe and severe SDB respectively. Patients treated up to 2015 formed the derivation cohort (n=311) and the remainder (n=113) formed the validation cohort. Multivariate logistic regression analysis was used to identify clinical variables predictive of moderate-to-severe SDB. A risk score model was developed based on regression coefficients and tested using receiver-operating-characteristics analyses on the validation cohort. Results Overall, mean age was 60±11 years, mean body mass index (BMI) was 30±5 kg/m2 and 69% were men. The prevalence of moderate-to-severe SDB was 33.7%. There were no significant differences in baseline characteristics between the derivation and validation cohorts. Male gender (score=1), overweight (BMI: 25–29 kg/m2, score=2), obesity (BMI≥30 kg/m2, score=3), diabetes (score=1), and stroke (score=2) were significantly independently predictive of moderate-to-severe SDB and formulated the score. The score performed well to predict moderate-to-severe SDB with a C-statistic of 0.73 (95% CI: 0.67–0.79, P<0.001) in the derivation cohort, and 0.67 (95% CI: 0.57–0.77, P<0.001) in the validation cohort. As a rule-out test, a score of ≤3 had a negative predictive value of 77% for moderate-to-severe SDB (91% for severe SDB). A score of ≥4 had an intermediate positive likelihood ratio (PLR) of 2 for moderate-to-severe SDB (2.2 for severe SDB), while a score of ≥5 had a high PLR of 6.5 and 6.8 for moderate-to-severe SDB and severe SDB respectively. Sensitivity and specificity table Conclusion A novel risk score comprising clinical characteristics can identify patients with AF likely to benefit from further assessment for SDB. Application of this model may aid optimise resource utilisation and facilitate timely patient care.

Download Full-text

Emergence of digital biomarkers to predict and modify treatment efficacy: machine learning study

BMJ Open ◽

10.1136/bmjopen-2019-030710 ◽

2019 ◽

Vol 9 (7) ◽

pp. e030710 ◽

Cited By ~ 1

Author(s):

Nicole L Guthrie ◽

Jason Carpenter ◽

Katherine L Edwards ◽

Kevin J Appelbaum ◽

Sourav Dey ◽

...

Keyword(s):

Machine Learning ◽

Blood Pressure ◽

Treatment Response ◽

Response To Treatment ◽

Behavioural Intervention ◽

Operating Characteristics ◽

Explanatory Variables ◽

Er Model ◽

Digital Biomarkers ◽

Predict Treatment Response

ObjectivesDevelopment of digital biomarkers to predict treatment response to a digital behavioural intervention.DesignMachine learning using random forest classifiers on data generated through the use of a digital therapeutic which delivers behavioural therapy to treat cardiometabolic disease. Data from 13 explanatory variables (biometric and engagement in nature) generated in the first 28 days of a 12-week intervention were used to train models. Two levels of response to treatment were predicted: (1) systolic change ≥10 mm Hg (SC model), and (2) shift down to a blood pressure category of elevated or better (ER model). Models were validated using leave-one-out cross validation and evaluated using area under the curve receiver operating characteristics (AUROC) and specificity- sensitivity. Ability to predict treatment response with a subset of nine variables, including app use and baseline blood pressure, was also tested (models SC-APP and ER-APP).SettingData generated through ad libitum use of a digital therapeutic in the USA.ParticipantsDeidentified data from 135 adults with a starting blood pressure ≥130/80, who tracked blood pressure for at least 7 weeks using the digital therapeutic.ResultsThe SC model had an AUROC of 0.82 and a sensitivity of 58% at a specificity of 90%. The ER model had an AUROC of 0.69 and a sensitivity of 32% at a specificity at 91%. Dropping explanatory variables related to blood pressure resulted in an AUROC of 0.72 with a sensitivity of 42% at a specificity of 90% for the SC-APP model and an AUROC of 0.53 for the ER-APP model.ConclusionsMachine learning was used to transform data from a digital therapeutic into digital biomarkers that predicted treatment response in individual participants. Digital biomarkers have potential to improve treatment outcomes in a digital behavioural intervention.

Download Full-text

Probability assessment of intracerebral hemorrhage in prehospital emergency patients

Neurological Research and Practice ◽

10.1186/s42466-020-00100-1 ◽

2021 ◽

Vol 3 (1) ◽

Author(s):

Frederik Geisler ◽

Medschid Wesirow ◽

Martin Ebinger ◽

Alexander Kunz ◽

Michal Rozanski ◽

...

Keyword(s):

Intracerebral Hemorrhage ◽

Validation Cohort ◽

Receiver Operating Characteristics ◽

Operating Characteristics ◽

Systolic Blood Pressure Level ◽

Emergency Patients ◽

Derivation Cohort ◽

Mobile Stroke Unit ◽

Ich Score ◽

Receiver Operating

Abstract Background Routing of patients with intracerebral hemorrhage (ICH) and acute ischemic stroke (AIS) to the most appropriate hospital is challenging for emergency medical services particularly when specific treatment options are only provided by specialized hospitals and determination of the exact diagnosis is difficult. We aimed to develop a prehospital score – called prehospital-intracerebral hemorrhage score (ph-ICH score) – to assist in discriminating between both conditions. Methods The ph-ICH score was developed with data from patients treated aboard a mobile stroke unit in Berlin, Germany, between 2011 and 2013 (derivation cohort) and in 2018 (validation cohort). Diagnosis of ICH or AIS was established using clinical data and neuroradiological cerebral imaging. Diagnostic accuracy was measured with significance testing, Cohen’s d and receiver-operating-characteristics. Results We analyzed 416 patients (32 ICH, 224 AIS, 41 transient ischemic attack, 119 stroke mimic) in the derivation cohort and 285 patients (33 ICH and 252 AIS) in the validation cohort. Systolic blood pressure, level of consciousness and severity of neurological deficits (i. e. certain items of the National Institutes of Health Stroke Scale) were used to calculate the ph-ICH score that showed higher values in the ICH compared to the AIS group (derivation cohort: 1.8 ± 1.2 vs. 1.0 ± 0.9 points; validation cohort: 1.8 ± 0.9 vs. 0.8 ± 0.7 points; d = 0.9 and 1.4, both p < 0.01). Receiver-operating-characteristics showed fair and good accuracy with an area under the curve of 0.71 for the derivation and 0.81 for the validation cohort. Conclusions The ph-ICH score can assist medical personnel in the field to assess the likelihood of ICH and AIS in emergency patients.

Download Full-text

National scale soil sealing monitoring data as a new explanatory variable for landslide susceptibility models

10.5194/egusphere-egu21-861 ◽

2021 ◽

Author(s):

Tania Luti ◽

Samuele Segoni ◽

Michele Munafò ◽

Nicola Casagli

Keyword(s):

Land Cover ◽

Landslide Susceptibility ◽

State Of The Art ◽

Continuous Variable ◽

Categorical Variable ◽

Operating Characteristics ◽

Mapping Unit ◽

Explanatory Variables ◽

Soil Sealing ◽

Remote Sensing Techniques

<div> <p>It is widely known that human activities can negatively affect the equilibrium of slope systems, triggering or predisposing to landslides. In Italy, ISPRA (Italian Institute for Environmental Protection Research) uses remote sensing techniques to monitor the expansion of artificialization of the territory and releases every year an updated map of soil sealing, which is defined as the destruction or covering of natural soils by totally or partially impermeable artificial material. The soil sealing map covers the entire national territory and has a fine spatial resolution (10 m).</p> <p>In this work, for the first time, soil sealing indicators are used as explanatory variables in a landslide susceptibility assessment. Three new parameters were derived from the raw soil sealing map: &#8220;soil sealing aggregation&#8221; (continuous variable expressing the percentage of sealed soil within each mapping unit), &#8220;soil sealing&#8221; (categorical variable expressing if a mapping unit is mainly natural or sealed), &#8220;urbanization&#8221; (categorical variable subdividing each unit into natural, semi-urbanized, or urbanized).</p> <p>These parameters were added to a set of state-of-the-art explanatory variables in a random forest landslide susceptibility model. In particular, the parameters derived from soil sealing were compared with two state-of-the-art parameters widely used to account for human disturbance: land cover/land use (as derived from a CORINE land cover map) and road network. &#160;</p> <p>Results were compared in terms of AUC (area under receiver operating characteristics curve, expressing the overall effectiveness of the configurations tested) and out-of-bag-error (used to quantify the relative importance of each variable). We found that the parameter &#8220;soil sealing aggregation&#8221; significantly enhanced the model performances. The results open new perspectives for the use of data derived from soil sealing monitoring programs to improve landslide hazard studies.&#160;&#160;</p> </div>

Download Full-text

Prediction of Unplanned 30- day Readmission for ICU Patients with Heart Failure

10.1101/2021.10.06.21264643 ◽

2021 ◽

Author(s):

Maryam Pishgar ◽

Houshang Darabi ◽

Julian Theis ◽

Hadis Anahideh ◽

Amer Ardati ◽

...

Keyword(s):

Heart Failure ◽

Intensive Care ◽

Financial Burden ◽

Operating Characteristics ◽

Icu Patients ◽

Targeted Interventions ◽

Risk Of Death ◽

Severity Scores ◽

Patients With Heart Failure ◽

Time Information

ABSTRACT Background Intensive Care Unit (ICU) readmissions in patients with Heart Failure (HF) result in a significant risk of death and financial burden for patients and healthcare systems. Prediction of at-risk patients for readmission allows for targeted interventions that reduce morbidity and mortality. Methods and Results We presented a process mining approach for the prediction of unplanned 30-day readmission of ICU patients with HF. A patients health record can be understood as a sequence of observations called event logs; used to discover a process model. Time information was extracted using the DREAM (Decay Replay Mining) algorithm. Demographic information and severity scores upon admission were then combined with the time information and fed to a Neural Network (NN) model to further enhance the prediction efficiency. Results By using the Medical Information Mart for Intensive Care III (MIMIC-III) dataset of 3411 ICU patients with HF, our proposed model yielded an Area Under the Receiver Operating Characteristics (AUROC) of 0.920. Conclusions The proposed approach was capable of modeling the time-related variables and incorporating the medical history of patients from prior hospital visits for prediction. Thus, our approach significantly improved the outcome prediction compared to that of other ML-based models and health calculators.

Download Full-text

The 2CAN Score

Stroke ◽

10.1161/strokeaha.118.022130 ◽

2018 ◽

Vol 49 (12) ◽

pp. 2866-2871 ◽

Cited By ~ 2

Author(s):

Philip Chang ◽

Ilana Ruff ◽

Scott J. Mendelson ◽

Fan Caprio ◽

Deborah L. Bergman ◽

...

Keyword(s):

Risk Factors ◽

Predictive Value ◽

Validation Cohort ◽

Vital Signs ◽

Area Under The Curve ◽

Clinical Score ◽

Final Diagnosis ◽

Operating Characteristics ◽

Derivation Cohort ◽

Academic Center

Background and Purpose— A quarter of acute strokes occur in patients hospitalized for another reason. A stroke recognition instrument may be useful for non-neurologists to discern strokes from mimics such as seizures or delirium. We aimed to derive and validate a clinical score to distinguish stroke from mimics among inhospital suspected strokes. Methods— We reviewed consecutive inpatient stroke alerts in a single academic center from January 9, 2014, to December 7, 2016. Data points, including demographics, stroke risk factors, stroke alert reason, postoperative status, neurological examination, vital signs and laboratory values, and final diagnosis, were collected. Using multivariate logistic regression, we derived a weighted scoring system in the first half of patients (derivation cohort) and validated it in the remaining half of patients (validation cohort) using receiver operating characteristics testing. Results— Among 330 subjects, 116 (35.2%) had confirmed stroke, 43 (13.0%) had a neurological mimic (eg, seizure), and 171 (51.8%) had a non-neurological mimic (eg, encephalopathy). Four risk factors independently predicted stroke: clinical deficit score (clinical deficit score 1: 1 point; clinical deficit score ≥2: 3 points), recent cardiac procedure (1 point), history of atrial fibrillation (1 point), and being a new patient (<24 hours from admission: 1 point). The score showed excellent discrimination in the first 165 patients (derivation cohort, area under the curve=0.93) and remaining 165 patients (validation cohort, area under the curve=0.88). A score of ≥2 had 92.2% sensitivity, 69.6% specificity, 62.2% positive predictive value, and 94.3% negative predictive value for identifying stroke. Conclusions— The 2CAN score for recognizing inpatient stroke performs well in a single-center study. A future prospective multicenter study would help validate this score.

Download Full-text