scholarly journals Prognostic models for identifying risk of poor outcome in people with acute ankle sprains: the SPRAINED development and external validation study

2018 ◽  
Vol 22 (64) ◽  
pp. 1-112 ◽  
Author(s):  
David J Keene ◽  
Michael M Schlüssel ◽  
Jacqueline Thompson ◽  
Daryl A Hagan ◽  
Mark A Williams ◽  
...  

BackgroundAnkle sprains are very common injuries. Although recovery can occur within weeks, around one-third of patients have longer-term problems.ObjectivesTo develop and externally validate a prognostic model for identifying people at increased risk of poor outcome after an acute ankle sprain.DesignDevelopment of a prognostic model in a clinical trial cohort data set and external validation in a prospective cohort study.SettingEmergency departments (EDs) in the UK.ParticipantsAdults with an acute ankle sprain (within 7 days of injury).Sample sizeThere were 584 clinical trial participants in the development data set and 682 recruited for the external validation study.PredictorsCandidate predictor variables were chosen based on availability in the clinical data set, clinical consensus, face validity, a systematic review of the literature, data quality and plausibility of predictiveness of the outcomes.Main outcome measuresModels were developed to predict two composite outcomes representing poor outcome. Outcome 1 was the presence of at least one of the following symptoms at 9 months after injury: persistent pain, functional difficulty or lack of confidence. Outcome 2 included the same symptoms as outcome 1, with the addition of recurrence of injury. Rates of poor outcome in the external data set were lower than in the development data set, 7% versus 20% for outcome 1 and 16% versus 24% for outcome 2.AnalysisMultiple imputation was used to handle missing data. Logistic regression models, together with multivariable fractional polynomials, were used to select variables and identify transformations of continuous predictors that best predicted the outcome based on a nominal alpha of 0.157, chosen to minimise overfitting. Predictive accuracy was evaluated by assessing model discrimination (c-statistic) and calibration (flexible calibration plot).Results(1) Performance of the prognostic models in development data set – the combinedc-statistic for the outcome 1 model across the 50 imputed data sets was 0.74 [95% confidence interval (CI) 0.70 to 0.79], with good model calibration across the imputed data sets. The combinedc-statistic for the outcome 2 model across the 50 imputed data sets was 0.70 (95% CI 0.65 to 0.74), with good model calibration across the imputed data sets. Updating these models, which used baseline data collected at the ED, with an additional variable at 4 weeks post injury (pain when bearing weight on the ankle) improved the discriminatory ability (c-statistic 0.77, 95% CI 0.73 to 0.82, for outcome 1 and 0.75, 95% CI 0.71 to 0.80, for outcome 2) and calibration of both models. (2) Performance of the models in the external data set – the combinedc-statistic for the outcome 1 model across the 50 imputed data sets was 0.73 (95% CI 0.66 to 0.79), with a calibration plot intercept of –0.91 (95% CI –0.98 to 0.44) and slope of 1.13 (95% CI 0.76 to 1.50). The combinedc-statistic for the outcome 2 model across the 50 imputed data sets was 0.63 (95% CI 0.58 to 0.69), with a calibration plot intercept of –0.25 (95% CI –0.27 to 0.11) and slope of 1.03 (95% CI 0.65 to 1.42). The updated models with the additional pain variable at 4 weeks had improved discriminatory ability over the baseline models but not better calibration.ConclusionsThe SPRAINED (Synthesising a clinical Prognostic Rule for Ankle Injuries in the Emergency Department) prognostic models performed reasonably well, and showed benefit compared with not using any model; therefore, the models may assist clinical decision-making when managing and advising ankle sprain patients in the ED setting. The models use predictors that are simple to obtain.LimitationsThe data used were from a randomised controlled trial and so were not originally intended to fulfil the aim of developing prognostic models. However, the data set was the best available, including data on the symptoms and clinical events of interest.Future workFurther model refinement, including recalibration or identifying additional predictors, may be required. The effect of implementing and using either model in clinical practice, in terms of acceptability and uptake by clinicians and on patient outcomes, should be investigated.Trial registrationCurrent Controlled Trials ISRCTN12726986.FundingThis project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full inHealth Technology Assessment; Vol. 22, No. 64. See the NIHR Journals Library website for further project information. Funding was also recieved from the NIHR Collaboration for Leadership in Applied Health Research, Care Oxford at Oxford Health NHS Foundation Trust, NIHR Biomedical Research Centre, Oxford, and the NIHR Fellowship programme.

BMJ Open ◽  
2018 ◽  
Vol 8 (11) ◽  
pp. e022802 ◽  
Author(s):  
Michael M Schlussel ◽  
David J Keene ◽  
Gary S Collins ◽  
Jennifer Bostock ◽  
Christopher Byrne ◽  
...  

ObjectivesTo develop and externally validate a prognostic model for poor recovery after ankle sprain.Setting and participantsModel development used secondary data analysis of 584 participants from a UK multicentre randomised clinical trial. External validation used data from 682 participants recruited in 10 UK emergency departments for a prospective observational cohort.Outcome and analysisPoor recovery was defined as presence of pain, functional difficulty or lack of confidence in the ankle at 9 months after injury. Twenty-three baseline candidate predictors were included together in a multivariable logistic regression model to identify the best predictors of poor recovery. Relationships between continuous variables and the outcome were modelled using fractional polynomials. Regression parameters were combined over 50 imputed data sets using Rubin’s rule. To minimise overfitting, regression coefficients were multiplied by a heuristic shrinkage factor and the intercept re-estimated. Incremental value of candidate predictors assessed at 4 weeks after injury was explored using decision curve analysis and the baseline model updated. The final models included predictors selected based on the Akaike information criterion (p<0.157). Model performance was assessed by calibration and discrimination.ResultsOutcome rate was lower in the development (6.7%) than in the external validation data set (19.9%). Mean age (29.9 and 33.6 years), body mass index (BMI; 26.3 and 27.1 kg/m2), pain when resting (37.8 and 38.5 points) or bearing weight on the ankle (75.4 and 71.3 points) were similar in both data sets. Age, BMI, pain when resting, pain bearing weight, ability to bear weight, days from injury until assessment and injury recurrence were the selected predictors. The baseline model had fair discriminatory ability (C-statistic 0.72; 95% CI 0.66 to 0.79) but poor calibration. The updated model presented better discrimination (C-statistic 0.78; 95% CI 0.72 to 0.84), but equivalent calibration.ConclusionsThe models include predictors easy to assess clinically and show benefit when compared with not using any model.Trial registration numberISRCTN12726986; Results.


BMJ Open ◽  
2021 ◽  
Vol 11 (1) ◽  
pp. e040778
Author(s):  
Vineet Kumar Kamal ◽  
Ravindra Mohan Pandey ◽  
Deepak Agrawal

ObjectiveTo develop and validate a simple risk scores chart to estimate the probability of poor outcomes in patients with severe head injury (HI).DesignRetrospective.SettingLevel-1, government-funded trauma centre, India.ParticipantsPatients with severe HI admitted to the neurosurgery intensive care unit during 19 May 2010–31 December 2011 (n=946) for the model development and further, data from same centre with same inclusion criteria from 1 January 2012 to 31 July 2012 (n=284) for the external validation of the model.Outcome(s)In-hospital mortality and unfavourable outcome at 6 months.ResultsA total of 39.5% and 70.7% had in-hospital mortality and unfavourable outcome, respectively, in the development data set. The multivariable logistic regression analysis of routinely collected admission characteristics revealed that for in-hospital mortality, age (51–60, >60 years), motor score (1, 2, 4), pupillary reactivity (none), presence of hypotension, basal cistern effaced, traumatic subarachnoid haemorrhage/intraventricular haematoma and for unfavourable outcome, age (41–50, 51–60, >60 years), motor score (1–4), pupillary reactivity (none, one), unequal limb movement, presence of hypotension were the independent predictors as its 95% confidence interval (CI) of odds ratio (OR)_did not contain one. The discriminative ability (area under the receiver operating characteristic curve (95% CI)) of the score chart for in-hospital mortality and 6 months outcome was excellent in the development data set (0.890 (0.867 to 912) and 0.894 (0.869 to 0.918), respectively), internal validation data set using bootstrap resampling method (0.889 (0.867 to 909) and 0.893 (0.867 to 0.915), respectively) and external validation data set (0.871 (0.825 to 916) and 0.887 (0.842 to 0.932), respectively). Calibration showed good agreement between observed outcome rates and predicted risks in development and external validation data set (p>0.05).ConclusionFor clinical decision making, we can use of these score charts in predicting outcomes in new patients with severe HI in India and similar settings.


2021 ◽  
Vol 28 (1) ◽  
pp. e100267
Author(s):  
Keerthi Harish ◽  
Ben Zhang ◽  
Peter Stella ◽  
Kevin Hauck ◽  
Marwa M Moussa ◽  
...  

ObjectivesPredictive studies play important roles in the development of models informing care for patients with COVID-19. Our concern is that studies producing ill-performing models may lead to inappropriate clinical decision-making. Thus, our objective is to summarise and characterise performance of prognostic models for COVID-19 on external data.MethodsWe performed a validation of parsimonious prognostic models for patients with COVID-19 from a literature search for published and preprint articles. Ten models meeting inclusion criteria were either (a) externally validated with our data against the model variables and weights or (b) rebuilt using original features if no weights were provided. Nine studies had internally or externally validated models on cohorts of between 18 and 320 inpatients with COVID-19. One model used cross-validation. Our external validation cohort consisted of 4444 patients with COVID-19 hospitalised between 1 March and 27 May 2020.ResultsMost models failed validation when applied to our institution’s data. Included studies reported an average validation area under the receiver–operator curve (AUROC) of 0.828. Models applied with reported features averaged an AUROC of 0.66 when validated on our data. Models rebuilt with the same features averaged an AUROC of 0.755 when validated on our data. In both cases, models did not validate against their studies’ reported AUROC values.DiscussionPublished and preprint prognostic models for patients infected with COVID-19 performed substantially worse when applied to external data. Further inquiry is required to elucidate mechanisms underlying performance deviations.ConclusionsClinicians should employ caution when applying models for clinical prediction without careful validation on local data.


2020 ◽  
Vol 17 (6) ◽  
pp. 916-925
Author(s):  
Niyati Behera ◽  
Guruvayur Mahalakshmi

Attributes, whether qualitative or non-qualitative are the formal description of any real-world entity and are crucial in modern knowledge representation models like ontology. Though ample evidence for the amount of research done for mining non-qualitative attributes (like part-of relation) extraction from text as well as the Web is available in the wealth of literature, on the other side limited research can be found relating to qualitative attribute (i.e., size, color, taste etc.,) mining. Herein this research article an analytical framework has been proposed to retrieve qualitative attribute values from unstructured domain text. The research objective covers two aspects of information retrieval (1) acquiring quality values from unstructured text and (2) then assigning attribute to them by comparing the Google derived meaning or context of attributes as well as quality value (adjectives). The goal has been accomplished by using a framework which integrates Vector Space Modelling (VSM) with a probabilistic Multinomial Naive Bayes (MNB) classifier. Performance Evaluation has been carried out on two data sets (1) HeiPLAS Development Data set (106 adjective-noun exemplary phrases) and (2) a text data set in Medicinal Plant Domain (MPD). System is found to perform better with probabilistic approach compared to the existing pattern-based framework in the state of art


2019 ◽  
Vol 4 (6) ◽  
pp. e001801
Author(s):  
Sarah Hanieh ◽  
Sabine Braat ◽  
Julie A Simpson ◽  
Tran Thi Thu Ha ◽  
Thach D Tran ◽  
...  

IntroductionGlobally, an estimated 151 million children under 5 years of age still suffer from the adverse effects of stunting. We sought to develop and externally validate an early life predictive model that could be applied in infancy to accurately predict risk of stunting in preschool children.MethodsWe conducted two separate prospective cohort studies in Vietnam that intensively monitored children from early pregnancy until 3 years of age. They included 1168 and 475 live-born infants for model development and validation, respectively. Logistic regression on child stunting at 3 years of age was performed for model development, and the predicted probabilities for stunting were used to evaluate the performance of this model in the validation data set.ResultsStunting prevalence was 16.9% (172 of 1015) in the development data set and 16.4% (70 of 426) in the validation data set. Key predictors included in the final model were paternal and maternal height, maternal weekly weight gain during pregnancy, infant sex, gestational age at birth, and infant weight and length at 6 months of age. The area under the receiver operating characteristic curve in the validation data set was 0.85 (95% Confidence Interval, 0.80–0.90).ConclusionThis tool applied to infants at 6 months of age provided valid prediction of risk of stunting at 3 years of age using a readily available set of parental and infant measures. Further research is required to examine the impact of preventive measures introduced at 6 months of age on those identified as being at risk of growth faltering at 3 years of age.


2021 ◽  
pp. 304-314
Author(s):  
Christen R. Elledge ◽  
Anna W. LaVigne ◽  
Jacob Fiksel ◽  
Jean L. Wright ◽  
Todd McNutt ◽  
...  

PURPOSE The Bone Metastases Ensemble Trees for Survival (BMETS) model uses a machine learning algorithm to estimate survival time following consultation for palliative radiation therapy for symptomatic bone metastases (SBM). BMETS was developed at a tertiary-care, academic medical center, but its validity and stability when applied to external data sets are unknown. PATIENTS AND METHODS Patients treated with palliative radiation therapy for SBM from May 2013 to May 2016 at two hospital-based community radiation oncology clinics were included, and medical records were retrospectively reviewed to collect model covariates and survival time. The Kaplan-Meier method was used to estimate overall survival from consultation to death or last follow-up. Model discrimination was estimated using time-dependent area under the curve (tAUC), which was calculated using survival predictions from BMETS based on the initial training data set. RESULTS A total of 216 sites of SBM were treated in 182 patients. Most common histologies were breast (27%), lung (23%), and prostate (23%). Compared with the BMETS training set, the external validation population was older (mean age, 67 v 62 years; P < .001), had more primary breast (27% v 19%; P = .03) and prostate cancer (20% v 12%; P = .01), and survived longer (median, 10.7 v 6.4 months). When the BMETS model was applied to the external data set, tAUC values at 3, 6, and 12 months were 0.82, 0.77, and 0.77, respectively. When refit with data from the combined training and external validation sets, tAUC remained > 0.79. CONCLUSION BMETS maintained high discriminative ability when applied to an external validation set and when refit with new data, supporting its generalizability, stability, and the feasibility of dynamic modeling.


2020 ◽  
Author(s):  
Mark Tamisiea ◽  
Benjamin Krichman ◽  
Himanshu Save ◽  
Srinivas Bettadpur ◽  
Zhigui Kang ◽  
...  

&lt;p&gt;To assess the quality of the CSR solutions, we compare results against external data sets that have contemporaneous availability.&amp;#160; These evaluations fall into three categories: changes in terrestrial water storage against data from the North American and Global Land Data Assimilation Systems, variations in ocean bottom pressure against data from the Deep Ocean Assessment of Tsunami Network, and estimates of the low degree and order Stokes coefficients compared against those inferred from satellite laser ranging observations (i.e. the CSR monthly 5x5 gravity harmonics from the MEaSUREs project).&amp;#160; &amp;#160;As the mission provides a unique measurement of mass changes in the Earth system, evaluation of the new solutions against other data sets and models is challenging.&amp;#160; Thus, we primarily focus on relative agreement with these data set with the GRACE-FO solutions in relation to the historic agreement of the data sets with the GRACE solutions.&lt;/p&gt;


2020 ◽  
pp. bjophthalmol-2020-316984
Author(s):  
Tyler Hyungtaek Rim ◽  
Aaron Y Lee ◽  
Daniel S Ting ◽  
Kelvin Teo ◽  
Bjorn Kaijun Betzler ◽  
...  

BackgroundThe ability of deep learning (DL) algorithms to identify eyes with neovascular age-related macular degeneration (nAMD) from optical coherence tomography (OCT) scans has been previously established. We herewith evaluate the ability of a DL model, showing excellent performance on a Korean data set, to generalse onto an American data set despite ethnic differences. In addition, expert graders were surveyed to verify if the DL model was appropriately identifying lesions indicative of nAMD on the OCT scans.MethodsModel development data set—12 247 OCT scans from South Korea; external validation data set—91 509 OCT scans from Washington, USA. In both data sets, normal eyes or eyes with nAMD were included. After internal testing, the algorithm was sent to the University of Washington, USA, for external validation. Area under the receiver operating characteristic curve (AUC) and precision–recall curve (AUPRC) were calculated. For model explanation, saliency maps were generated using Guided GradCAM.ResultsOn external validation, AUC and AUPRC remained high at 0.952 (95% CI 0.942 to 0.962) and 0.891 (95% CI 0.875 to 0.908) at the individual level. Saliency maps showed that in normal OCT scans, the fovea was the main area of interest; in nAMD OCT scans, the appropriate pathological features were areas of model interest. Survey of 10 retina specialists confirmed this.ConclusionOur DL algorithm exhibited high performance for nAMD identification in a Korean population, and generalised well to an ethnically distinct, American population. The model correctly focused on the differences within the macular area to extract features associated with nAMD.


2017 ◽  
Vol 21 (18) ◽  
pp. 1-100 ◽  
Author(s):  
Shakila Thangaratinam ◽  
John Allotey ◽  
Nadine Marlin ◽  
Ben W Mol ◽  
Peter Von Dadelszen ◽  
...  

BackgroundThe prognosis of early-onset pre-eclampsia (before 34 weeks’ gestation) is variable. Accurate prediction of complications is required to plan appropriate management in high-risk women.ObjectiveTo develop and validate prediction models for outcomes in early-onset pre-eclampsia.DesignProspective cohort for model development, with validation in two external data sets.SettingModel development: 53 obstetric units in the UK. Model transportability: PIERS (Pre-eclampsia Integrated Estimate of RiSk for mothers) and PETRA (Pre-Eclampsia TRial Amsterdam) studies.ParticipantsPregnant women with early-onset pre-eclampsia.Sample sizeNine hundred and forty-six women in the model development data set and 850 women (634 in PIERS, 216 in PETRA) in the transportability (external validation) data sets.PredictorsThe predictors were identified from systematic reviews of tests to predict complications in pre-eclampsia and were prioritised by Delphi survey.Main outcome measuresThe primary outcome was the composite of adverse maternal outcomes established using Delphi surveys. The secondary outcome was the composite of fetal and neonatal complications.AnalysisWe developed two prediction models: a logistic regression model (PREP-L) to assess the overall risk of any maternal outcome until postnatal discharge and a survival analysis model (PREP-S) to obtain individual risk estimates at daily intervals from diagnosis until 34 weeks. Shrinkage was used to adjust for overoptimism of predictor effects. For internal validation (of the full models in the development data) and external validation (of the reduced models in the transportability data), we computed the ability of the models to discriminate between those with and without poor outcomes (c-statistic), and the agreement between predicted and observed risk (calibration slope).ResultsThe PREP-L model included maternal age, gestational age at diagnosis, medical history, systolic blood pressure, urine protein-to-creatinine ratio, platelet count, serum urea concentration, oxygen saturation, baseline treatment with antihypertensive drugs and administration of magnesium sulphate. The PREP-S model additionally included exaggerated tendon reflexes and serum alanine aminotransaminase and creatinine concentration. Both models showed good discrimination for maternal complications, with anoptimism-adjustedc-statistic of 0.82 [95% confidence interval (CI) 0.80 to 0.84] for PREP-L and 0.75 (95% CI 0.73 to 0.78) for the PREP-S model in the internal validation. External validation of the reduced PREP-L model showed good performance with ac-statistic of 0.81 (95% CI 0.77 to 0.85) in PIERS and 0.75 (95% CI 0.64 to 0.86) in PETRA cohorts for maternal complications, and calibrated well with slopes of 0.93 (95% CI 0.72 to 1.10) and 0.90 (95% CI 0.48 to 1.32), respectively. In the PIERS data set, the reduced PREP-S model had ac-statistic of 0.71 (95% CI 0.67 to 0.75) and a calibration slope of 0.67 (95% CI 0.56 to 0.79). Low gestational age at diagnosis, high urine protein-to-creatinine ratio, increased serum urea concentration, treatment with antihypertensive drugs, magnesium sulphate, abnormal uterine artery Doppler scan findings and estimated fetal weight below the 10th centile were associated with fetal complications.ConclusionsThe PREP-L model provided individualised risk estimates in early-onset pre-eclampsia to plan management of high- or low-risk individuals. The PREP-S model has the potential to be used as a triage tool for risk assessment. The impacts of the model use on outcomes need further evaluation.Trial registrationCurrent Controlled Trials ISRCTN40384046.FundingThe National Institute for Health Research Health Technology Assessment programme.


Ocean Science ◽  
2014 ◽  
Vol 10 (4) ◽  
pp. 731-744 ◽  
Author(s):  
M.-H. Rio ◽  
A. Pascual ◽  
P.-M. Poulain ◽  
M. Menna ◽  
B. Barceló ◽  
...  

Abstract. The accurate knowledge of the ocean's mean dynamic topography (MDT) is a crucial issue for a number of oceanographic applications and, in some areas of the Mediterranean Sea, important limitations have been found pointing to the need of an upgrade. We present a new MDT that was computed for the Mediterranean Sea. It profits from improvements made possible by the use of extended data sets and refined processing. The updated data set spans the 1993–2012 period and consists of drifter velocities, altimetry data, hydrological profiles and model data. The methodology is similar to the previous MDT by Rio et al. (2007). However, in Rio et al. (2007) no hydrological profiles had been taken into account. This required the development of dedicated processing. A number of sensitivity studies have been carried out to obtain the most accurate MDT as possible. The main results from these sensitivity studies are the following: moderate impact to the choice of correlation scales but almost negligible sensitivity to the choice of the first guess (model solution). A systematic external validation to independent data has been made to evaluate the performance of the new MDT. Compared to previous versions, SMDT-MED-2014 (Synthetic Mean Dynamic Topography of the MEDiterranean sea) features shorter-scale structures, which results in an altimeter velocity variance closer to the observed velocity variance and, at the same time, gives better Taylor skills.


Sign in / Sign up

Export Citation Format

Share Document