Evaluating Methodological Quality of Prognostic Prediction Models on Patient Reported Outcome Measurements after Total Hip Replacement and Total Knee Replacement Surgery: a systematic review protocol

Abstract Background Prediction models for poor patient-reported surgical outcomes after total hip replacement (THR) and total knee replacement (TKR) may provide a method for improving appropriate surgical care for hip and knee osteoarthritis. There are concerns about methodological issues and the risk of bias of studies producing prediction models. A critical evaluation of the methodological quality of prediction modelling studies in THR and TKR is needed to ensure their clinical usefulness. This systematic review aims to: 1) evaluate and report the quality of risk stratification and prediction modelling studies that predict patient-reported outcomes after THR and TKR; 2) identify areas of methodological deficit and provide recommendations for future research; and 3) synthesise the evidence on prediction models associated with post-operative patient-reported outcomes after THR and TKR surgeries. Methods MEDLINE, EMBASE and CINAHL electronic databases will be searched to identify relevant studies. Title and abstract and full-text screening will be performed by two independent reviewers. We will include: 1) prediction model development studies without external validation; 2) prediction model development studies with external validation of independent data; 3) external model validation studies; and 4) studies updating a previously developed prediction model. Data extraction spreadsheets will be developed based on the CHARMS checklist and TRIPOD statement and piloted on two relevant studies. Study quality and risk of bias will be assessed using the PROBAST tool. Prediction models will be summarised qualitatively. Meta-analyses on the predictive performance of included models will be conducted if appropriate. Discussion This systematic review will evaluate the methodological quality and usefulness of prediction models for poor outcomes after THR or TKR. This information is essential to provide evidence-based healthcare for end-stage hip and knee osteoarthritis. Findings of this review will contribute to the identification of key areas for improvement in conducting prognostic research in this field and facilitate the progress in evidence-based tailored treatments for hip and knee osteoarthritis. Systematic review registration: Submitted to PROSPERO on 30 August 2021.

Download Full-text

A systematic review of methodological quality of model development studies predicting prognostic outcome for resectable pancreatic cancer

BMJ Open ◽

10.1136/bmjopen-2018-027192 ◽

2019 ◽

Vol 9 (8) ◽

pp. e027192 ◽

Cited By ~ 2

Author(s):

Alison Bradley ◽

Robert Van Der Meer ◽

Colin J McKay

Keyword(s):

Systematic Review ◽

Methodological Quality ◽

External Validation ◽

Model Development ◽

Predictive Modelling ◽

Secondary Outcome ◽

Development Studies ◽

Cancer Data ◽

Modelling Studies

ObjectivesTo assess the methodological quality of prognostic model development studies pertaining to post resection prognosis of pancreatic ductal adenocarcinoma (PDAC).Design/settingA narrative systematic review of international peer reviewed journalsData sourceSearches were conducted of: MEDLINE, Embase, PubMed, Cochrane database and Google Scholar for predictive modelling studies applied to the outcome of prognosis for patients with PDAC post resection. Predictive modelling studies in this context included prediction model development studies with and without external validation and external validation studies with model updating. Data was extracted following the Checklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) checklist.Primary and secondary outcome measuresPrimary outcomes were all components of the CHARMS checklist. Secondary outcomes included frequency of variables included across predictive models.Results263 studies underwent full text review. 15 studies met the inclusion criteria. 3 studies underwent external validation. Multivariable Cox proportional hazard regression was the most commonly employed modelling method (n=13). 10 studies were based on single centre databases. Five used prospective databases, seven used retrospective databases and three used cancer data registry. The mean number of candidate predictors was 19.47 (range 7 to 50). The most commonly included variables were tumour grade (n=9), age (n=8), tumour stage (n=7) and tumour size (n=5). Mean sample size was 1367 (range 50 to 6400). 5 studies reached statistical power. None of the studies reported blinding of outcome measurement for predictor values. The most common form of presentation was nomograms (n=5) and prognostic scores (n=5) followed by prognostic calculators (n=3) and prognostic index (n=2).ConclusionsAreas for improvement in future predictive model development have been highlighted relating to: general aspects of model development and reporting, applicability of models and sources of bias.Trial registration numberCRD42018105942

Download Full-text

P–419 On prognosis after unexplained recurrent pregnancy losses (RPL); a systematic review and external validation of clinical prediction models

Human Reproduction ◽

10.1093/humrep/deab130.418 ◽

2021 ◽

Vol 36 (Supplement_1) ◽

Author(s):

A Youssef

Keyword(s):

Systematic Review ◽

Supportive Care ◽

Prediction Model ◽

Pregnancy Outcome ◽

Prediction Models ◽

External Validation ◽

Model Development ◽

Predictive Performance ◽

Pregnancy Rates ◽

Cochrane Library

Abstract Study question Which models that predict pregnancy outcome in couples with unexplained RPL exist and what is the performance of the most used model? Summary answer We identified seven prediction models; none followed the recommended prediction model development steps. Moreover, the most used model showed poor predictive performance. What is known already RPL remains unexplained in 50–75% of couples For these couples, there is no effective treatment option and clinical management rests on supportive care. Essential part of supportive care consists of counselling on the prognosis of subsequent pregnancies. Indeed, multiple prediction models exist, however the quality and validity of these models varies. In addition, the prediction model developed by Brigham et al is the most widely used model, but has never been externally validated. Study design, size, duration We performed a systematic review to identify prediction models for pregnancy outcome after unexplained RPL. In addition we performed an external validation of the Brigham model in a retrospective cohort, consisting of 668 couples with unexplained RPL that visited our RPL clinic between 2004 and 2019. Participants/materials, setting, methods A systematic search was performed in December 2020 in Pubmed, Embase, Web of Science and Cochrane library to identify relevant studies. Eligible studies were selected and assessed according to the TRIPOD) guidelines, covering topics on model performance and validation statement. The performance of predicting live birth in the Brigham model was evaluated through calibration and discrimination, in which the observed pregnancy rates were compared to the predicted pregnancy rates. Main results and the role of chance Seven models were compared and assessed according to the TRIPOD statement. This resulted in two studies of low, three of moderate and two of above average reporting quality. These studies did not follow the recommended steps for model development and did not calculate a sample size. Furthermore, the predictive performance of neither of these models was internally- or externally validated. We performed an external validation of Brigham model. Calibration showed overestimation of the model and too extreme predictions, with a negative calibration intercept of –0.52 (CI 95% –0.68 – –0.36), with a calibration slope of 0.39 (CI 95% 0.07 – 0.71). The discriminative ability of the model was very low with a concordance statistic of 0.55 (CI 95% 0.50 – 0.59). Limitations, reasons for caution None of the studies are specifically named prediction models, therefore models may have been missed in the selection process. The external validation cohort used a retrospective design, in which only the first pregnancy after intake was registered. Follow-up time was not limited, which is important in counselling unexplained RPL couples. Wider implications of the findings: Currently, there are no suitable models that predict on pregnancy outcome after RPL. Moreover, we are in need of a model with several variables such that prognosis is individualized, and factors from both the female as the male to enable a couple specific prognosis. Trial registration number Not applicable

Download Full-text

Trends in the conduct and reporting of clinical prediction model development and validation: a systematic review

10.1101/2021.10.22.21265374 ◽

2021 ◽

Author(s):

Cynthia Yang ◽

Jan A. Kors ◽

Solomon Ioannou ◽

Luis H. John ◽

Aniek F. Markus ◽

...

Keyword(s):

Systematic Review ◽

Prediction Model ◽

Prediction Models ◽

External Validation ◽

Model Development ◽

Clinical Prediction ◽

Clinical Prediction Model ◽

Final Model ◽

Development And Validation ◽

Over Time

Objectives This systematic review aims to provide further insights into the conduct and reporting of clinical prediction model development and validation over time. We focus on assessing the reporting of information necessary to enable external validation by other investigators. Materials and Methods We searched Embase, Medline, Web-of-Science, Cochrane Library and Google Scholar to identify studies that developed one or more multivariable prognostic prediction models using electronic health record (EHR) data published in the period 2009-2019. Results We identified 422 studies that developed a total of 579 clinical prediction models using EHR data. We observed a steep increase over the years in the number of developed models. The percentage of models externally validated in the same paper remained at around 10%. Throughout 2009-2019, for both the target population and the outcome definitions, code lists were provided for less than 20% of the models. For about half of the models that were developed using regression analysis, the final model was not completely presented. Discussion Overall, we observed limited improvement over time in the conduct and reporting of clinical prediction model development and validation. In particular, the prediction problem definition was often not clearly reported, and the final model was often not completely presented. Conclusion Improvement in the reporting of information necessary to enable external validation by other investigators is still urgently needed to increase clinical adoption of developed models.

Download Full-text

Risk Prediction Models for Patients With Acute Heart Failure: A Protocol for Systematic Review, Critical Appraisal, and Meta-Analysis

10.21203/rs.3.rs-948336/v1 ◽

2021 ◽

Author(s):

Xuecheng Zhang ◽

Kehua Zhou ◽

Jingjing Zhang ◽

Ying Chen ◽

Hengheng Dai ◽

...

Keyword(s):

Heart Failure ◽

Systematic Review ◽

Prediction Model ◽

Acute Heart Failure ◽

Risk Prediction ◽

Methodological Quality ◽

Critical Appraisal ◽

Prediction Models ◽

Cochrane Library ◽

Risk Prediction Models

Abstract Background Nearly a third of patients with acute heart failure (AHF) die or are readmitted within three months after discharge, accounting for the majority of costs associated with heart failure-related care. A considerable number of risk prediction models, which predict outcomes for mortality and readmission rates, have been developed and validated for patients with AHF. These models could help clinicians stratify patients by risk level and improve decision making, and provide specialist care and resources directed to high-risk patients. However, clinicians sometimes reluctant to utilize these models, possibly due to their poor reliability, the variety of models, and/or the complexity of statistical methodologies. Here, we describe a protocol to systematically review extant risk prediction models. We will describe characteristics, compare performance, and critically appraise the reporting transparency and methodological quality of risk prediction models for AHF patients. Method Embase, Pubmed, Web of Science, and the Cochrane Library will be searched from their inception onwards. A back word will be searched on derivation studies to find relevant external validation studies. Multivariable prognostic models used for AHF and mortality and/or readmission rate will be eligible for review. Two reviewers will conduct title and abstract screening, full-text review, and data extraction independently. Included models will be summarized qualitatively and quantitatively. We will also provide an overview of critical appraisal of the methodological quality and reporting transparency of included studies using the Prediction model Risk of Bias Assessment Tool(PROBAST tool) and the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis(TRIPOD statement). Discussion The result of the systematic review could help clinicians better understand and use the prediction models for AHF patients, as well as make standardized decisions about more precise, risk-adjusted management. Systematic review registration : PROSPERO registration number CRD42021256416.

Download Full-text

Persian participation measures for adolescents: A systematic review

British Journal of Occupational Therapy ◽

10.1177/0308022620910400 ◽

2020 ◽

pp. 030802262091040

Author(s):

Ghodsiyeh Joveini ◽

Armin Zareiyan ◽

Laleh Lajevardi ◽

Mitra Khalafbeigi ◽

Afsoon Hassani Mehraban

Keyword(s):

Systematic Review ◽

Methodological Quality ◽

Leisure Activities ◽

Patient Reported Outcome Measures ◽

Patient Reported Outcome ◽

Bibliographic Databases ◽

Patient Reported ◽

Age Features ◽

Adolescent Participation

Introduction Enhancing participation is the focus of occupational therapy. Comprehensive and accurate assessment ensures that clinicians can tailor an intervention to the client’s needs. This systematic review was completed to identify Persian adolescents’ participation measures and critically appraise them. It would be helpful in the selection of the most appropriate instrument to use in adolescent-related research and clinical practice. Method Ten bibliographic databases, four Iranian and six international, without year limits were searched up to June 2019. A systematic search was directed according to COSMIN guidelines for systematic reviews of patient-reported outcome measures and PRISMA guidelines (Systematic review registration: CRD42017073581). Results Seven measures were extracted from the articles. Reviewing the content and psychometric properties of the measures as well as the methodological quality of the studies indicated that the Modified Activity Questionnaire is the only measure with consistent and moderately reliable results. It measures adolescent participation in leisure activities but not all domains of participation. Conclusion There may be a growing need for adapting existing Persian measures or developing new ones based on specific age features related to puberty-stage alongside cultural, social and academic demands, which have a significant effect on adolescents’ participation in meaningful occupations. High methodological quality in designing such studies also has great importance.

Download Full-text

A systematic review of the quality of clinical prediction models in in vitro fertilisation

Human Reproduction ◽

10.1093/humrep/dez258 ◽

2020 ◽

Vol 35 (1) ◽

pp. 100-116 ◽

Cited By ~ 8

Author(s):

M B Ratna ◽

S Bhattacharya ◽

B Abdulrahim ◽

D J McLernon

Keyword(s):

Systematic Review ◽

Live Birth ◽

Prediction Models ◽

Conflicts Of Interest ◽

External Validation ◽

Post Treatment ◽

Treatment Model ◽

Clinical Prediction ◽

Clinical Prediction Models

Abstract STUDY QUESTION What are the best-quality clinical prediction models in IVF (including ICSI) treatment to inform clinicians and their patients of their chance of success? SUMMARY ANSWER The review recommends the McLernon post-treatment model for predicting the cumulative chance of live birth over and up to six complete cycles of IVF. WHAT IS KNOWN ALREADY Prediction models in IVF have not found widespread use in routine clinical practice. This could be due to their limited predictive accuracy and clinical utility. A previous systematic review of IVF prediction models, published a decade ago and which has never been updated, did not assess the methodological quality of existing models nor provided recommendations for the best-quality models for use in clinical practice. STUDY DESIGN, SIZE, DURATION The electronic databases OVID MEDLINE, OVID EMBASE and Cochrane library were searched systematically for primary articles published from 1978 to January 2019 using search terms on the development and/or validation (internal and external) of models in predicting pregnancy or live birth. No language or any other restrictions were applied. PARTICIPANTS/MATERIALS, SETTING, METHODS The PRISMA flowchart was used for the inclusion of studies after screening. All studies reporting on the development and/or validation of IVF prediction models were included. Articles reporting on women who had any treatment elements involving donor eggs or sperm and surrogacy were excluded. The CHARMS checklist was used to extract and critically appraise the methodological quality of the included articles. We evaluated models’ performance by assessing their c-statistics and plots of calibration in studies and assessed correct reporting by calculating the percentage of the TRIPOD 22 checklist items met in each study. MAIN RESULTS AND THE ROLE OF CHANCE We identified 33 publications reporting on 35 prediction models. Seventeen articles had been published since the last systematic review. The quality of models has improved over time with regard to clinical relevance, methodological rigour and utility. The percentage of TRIPOD score for all included studies ranged from 29 to 95%, and the c-statistics of all externally validated studies ranged between 0.55 and 0.77. Most of the models predicted the chance of pregnancy/live birth for a single fresh cycle. Six models aimed to predict the chance of pregnancy/live birth per individual treatment cycle, and three predicted more clinically relevant outcomes such as cumulative pregnancy/live birth. The McLernon (pre- and post-treatment) models predict the cumulative chance of live birth over multiple complete cycles of IVF per woman where a complete cycle includes all fresh and frozen embryo transfers from the same episode of ovarian stimulation. McLernon models were developed using national UK data and had the highest TRIPOD score, and the post-treatment model performed best on external validation. LIMITATIONS, REASONS FOR CAUTION To assess the reporting quality of all included studies, we used the TRIPOD checklist, but many of the earlier IVF prediction models were developed and validated before the formal TRIPOD reporting was published in 2015. It should also be noted that two of the authors of this systematic review are authors of the McLernon model article. However, we feel we have conducted our review and made our recommendations using a fair and transparent systematic approach. WIDER IMPLICATIONS OF THE FINDINGS This study provides a comprehensive picture of the evolving quality of IVF prediction models. Clinicians should use the most appropriate model to suit their patients’ needs. We recommend the McLernon post-treatment model as a counselling tool to inform couples of their predicted chance of success over and up to six complete cycles. However, it requires further external validation to assess applicability in countries with different IVF practices and policies. STUDY FUNDING/COMPETING INTEREST(S) The study was funded by the Elphinstone Scholarship Scheme and the Assisted Reproduction Unit, University of Aberdeen. Both D.J.M. and S.B. are authors of the McLernon model article and S.B. is Editor in Chief of Human Reproduction Open. They have completed and submitted the ICMJE forms for Disclosure of potential Conflicts of Interest. The other co-authors have no conflicts of interest to declare. REGISTRATION NUMBER N/A

Download Full-text

Systematic review of the psychometric properties of patient-reported outcome measures for rheumatoid arthritis in the foot and ankle

Clinical Rehabilitation ◽

10.1177/0269215519862328 ◽

2019 ◽

Vol 33 (11) ◽

pp. 1788-1799 ◽

Cited By ~ 2

Author(s):

Ana Belen Ortega-Avila ◽

Laura Ramos-Petersen ◽

Pablo Cervera-Garvi ◽

Christopher J Nester ◽

José Miguel Morales-Asencio ◽

...

Keyword(s):

Rheumatoid Arthritis ◽

Systematic Review ◽

Psychometric Properties ◽

Outcome Measures ◽

Methodological Quality ◽

Measurement Properties ◽

Foot And Ankle ◽

Inclusion Criteria ◽

Patient Reported

Objective: To identify self-reported outcome measures specific to the foot and ankle in patients with rheumatoid arthritis and to investigate the methodological quality and psychometric properties of these measures. Method: A systematic review focusing on patients with rheumatoid arthritis. Setting: The search was conducted in the PubMed, SCOPUS, CINAHL, PEDro and Google Scholar databases, based on the following inclusion criteria: population (with rheumatoid arthritis) > 18 years; psychometric or clinimetric validation studies of patient-reported outcomes specific to the foot and ankle, in different languages, with no time limit. Two of the present authors independently assessed the quality of the studies located and extracted the relevant data. Terwee’s criteria and the COSMIN checklist were employed to ensure adequate methodological quality. Results: Of the initial 431 studies considered, 14 met the inclusion criteria, representing 7,793 patients (56.8 years). These instruments were grouped into three dimensions (pain, perceived health status and quality of life and disability). The time to complete any of the PROMs varies around 15 minutes. PROMs criterias with the worst scores by COSMIN, 92.85% and 85.71% were criterion validity, measurement error, internal consistency and responsiveness. 28.57% of PROMs were compared with the measurement properties. Conclusion: the Self-Reported Foot and Ankle Score achieved the highest number of positive criteria (according to Terwee and COSMIN), and is currently the most appropriate for patients with Rheumatoid arthritis.

Download Full-text

Prediction Models for Severe Manifestations and Mortality due to COVID-19: A Rapid Systematic Review

10.1101/2021.01.28.21250718 ◽

2021 ◽

Author(s):

Jamie L. Miller ◽

Masafumi Tada ◽

Michihiko Goto ◽

Nicholas Mohr ◽

Sangil Lee

Keyword(s):

Systematic Review ◽

Prediction Model ◽

Prediction Models ◽

External Validation ◽

Clinical Signs ◽

Risk Of Bias ◽

Low Risk ◽

Prognostic Models ◽

Organ Support ◽

Public Data

ABSTRACTBackgroundThroughout 2020, the coronavirus disease 2019 (COVID-19) has become a threat to public health on national and global level. There has been an immediate need for research to understand the clinical signs and symptoms of COVID-19 that can help predict deterioration including mechanical ventilation, organ support, and death. Studies thus far have addressed the epidemiology of the disease, common presentations, and susceptibility to acquisition and transmission of the virus; however, an accurate prognostic model for severe manifestations of COVID-19 is still needed because of the limited healthcare resources available.ObjectiveThis systematic review aims to evaluate published reports of prediction models for severe illnesses caused COVID-19.MethodsSearches were developed by the primary author and a medical librarian using an iterative process of gathering and evaluating terms. Comprehensive strategies, including both index and keyword methods, were devised for PubMed and EMBASE. The data of confirmed COVID-19 patients from randomized control studies, cohort studies, and case-control studies published between January 2020 and July 2020 were retrieved. Studies were independently assessed for risk of bias and applicability using the Prediction Model Risk Of Bias Assessment Tool (PROBAST). We collected study type, setting, sample size, type of validation, and outcome including intubation, ventilation, any other type of organ support, or death. The combination of the prediction model, scoring system, performance of predictive models, and geographic locations were summarized.ResultsA primary review found 292 articles relevant based on title and abstract. After further review, 246 were excluded based on the defined inclusion and exclusion criteria. Forty-six articles were included in the qualitative analysis. Inter observer agreement on inclusion was 0.86 (95% confidence interval: 0.79 - 0.93). When the PROBAST tool was applied, 44 of the 46 articles were identified to have high or unclear risk of bias, or high or unclear concern for applicability. Two studied reported prediction models, 4C Mortality Score from hospital data and QCOVID from general public data from UK, and were rated as low risk of bias and low concerns for applicability.ConclusionSeveral prognostic models are reported in the literature, but many of them had concerning risks of biases and applicability. For most of the studies, caution is needed before use, as many of them will require external validation before dissemination. However, two articles were found to have low risk of bias and low applicability can be useful tools.

Download Full-text

Methodological quality of model development studies predicting prognostic outcome for Hepatocellular Carcinoma: Protocol for systematic review

10.31219/osf.io/pv4e9 ◽

2021 ◽

Author(s):

Ali Haider Bangash

Keyword(s):

Hepatocellular Carcinoma ◽

Systematic Review ◽

Methodological Quality ◽

Model Development ◽

Development Studies ◽

Prognostic Outcome

Hepatocellular carcinoma (HCC) is being explored. Specifically, we shall be looking into the methodological quality of those studies that attempt to develop prognostication models for HCC.

Download Full-text

Evaluating methodological quality of Prognostic models Including Patient-reported HeAlth outcomes iN oncologY (EPIPHANY): a systematic review protocol

BMJ Open ◽

10.1136/bmjopen-2018-025054 ◽

2018 ◽

Vol 8 (10) ◽

pp. e025054 ◽

Cited By ~ 2

Author(s):

Nina Deliu ◽

Francesco Cottone ◽

Gary S Collins ◽

Amélie Anota ◽

Fabio Efficace

Keyword(s):

Systematic Review ◽

Systematic Reviews ◽

Methodological Quality ◽

Data Extraction ◽

Prognostic Models ◽

Published Data ◽

Evidence Based ◽

Patient Reported ◽

Data Extraction Form

IntroductionWhile there is mounting evidence of the independent prognostic value of patient-reported outcomes (PROs) for overall survival (OS) in patients with cancer, it is known that the conduct of these studies may hold a number of methodological challenges. The aim of this systematic review is to evaluate the quality of published studies in this research area, in order to identify methodological and statistical issues deserving special attention and to also possibly provide evidence-based recommendations.Methods and analysisAn electronic search strategy will be performed in PubMed to identify studies developing or validating a prognostic model which includes PROs as predictors. Two reviewers will independently be involved in data collection using a predefined and standardised data extraction form including information related to study characteristics, PROs measures used and multivariable prognostic models. Studies selection will be reported following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, with data extraction form using fields from the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) checklist for multivariable models. Methodological quality assessment will also be performed and will be based on prespecified domains of the CHARMS checklist. As a substantial heterogeneity of included studies is expected, a narrative evidence synthesis will also be provided.Ethics and disseminationGiven that this systematic review will use only published data, ethical permissions will not be required. Findings from this review will be published in peer-reviewed scientific journals and presented at major international conferences. We anticipate that this review will contribute to identify key areas of improvement for conducting and reporting prognostic factor analyses with PROs in oncology and will lay the groundwork for developing future evidence-based recommendations in this area of research.Prospero registration numberCRD42018099160.

Download Full-text