scholarly journals Validating and Updating GRASP: An Evidence-Based Framework for Grading and Assessment of Clinical Predictive Tools

2020 ◽  
Author(s):  
Mohamed Khalifa ◽  
Farah Magrabi ◽  
Blanca Gallego
Keyword(s):  
Interrater Reliability ◽  
Online Survey ◽  
Evaluation Criteria ◽  
Predictive Performance ◽  
Potential Effect ◽  
Three Dimensions ◽  
Evidence Based ◽  
Level Of Evidence ◽  
Predictive Tools ◽  
Published Evidence

Abstract Background: When selecting predictive tools, clinicians are challenged with an overwhelming and ever-growing number, most of which have never been implemented or evaluated for comparative effectiveness. To overcome this challenge, the authors developed an evidence-based framework for grading and assessment of predictive tools (GRASP). The objective of this study is to update GRASP and evaluate its reliability. Methods: An online survey was developed to collect responses of a wide international group of experts, who published studies on developing, implementing or evaluating clinical decision support tools. The interrater reliability of the framework, to assign grades to eight predictive tools by two independent users, was evaluated. Results: Among 882 invited experts, 81 provided valid responses. On a five-points Likert scale, experts overall strongly agreed to GRASP evaluation criteria (4.35). Experts strongly agreed to six criteria: predictive performance (4.87) and predictive performance levels (4.44), usability (4.68) and potential effect (4.61), post-implementation impact (4.78) and evidence direction (4.26). Experts somewhat agreed to one criterion: post-implementation impact levels (4.16). Experts were neutral about one criterion; usability is higher than potential effect (2.97). Sixty-four respondents provided recommendations to open-ended questions regarding adding, removing or changing evaluation criteria. Forty-three respondents suggested the potential effect should be higher than the usability. Experts highlighted the importance of reporting quality of studies and strength of evidence supporting grades assigned to predictive tools. Accordingly, GRASP concept and its detailed report were updated. The updated framework’s interrater reliability, to produce accurate and consistent results by two independent users, was tested and found to be initially reliable. Conclusion: The GRASP framework grades predictive tools based on critical appraisal of published evidence across three dimensions: phase of evaluation, level of evidence, and direction of evidence. The final grade of a tool is based on the highest phase of evaluation, supported by the highest level of positive evidence, or mixed evidence that supports positive conclusion. GRASP aims to provide clinicians with a high-level, evidence-based, and comprehensive, yet simple and feasible, approach to evaluate predictive tools, considering their predictive performance before implementation, usability and potential effect during planning for implementation, and post-implementation impact on healthcare outcomes.

scholarly journals Developing a framework for evidence-based grading and assessment of predictive tools for clinical decision support

2019 ◽  
Vol 19 (1) ◽  
Author(s):  
Mohamed Khalifa ◽  
Farah Magrabi ◽  
Blanca Gallego
Keyword(s):  
Predictive Performance ◽  
Patient Characteristics ◽  
Clinical Decision ◽  
Potential Effect ◽  
Three Dimensions ◽  
Early Warning Score ◽  
Evidence Based ◽  
Level Of Evidence ◽  
Predictive Tools ◽  
Published Evidence

Abstract Background Clinical predictive tools quantify contributions of relevant patient characteristics to derive likelihood of diseases or predict clinical outcomes. When selecting predictive tools for implementation at clinical practice or for recommendation in clinical guidelines, clinicians are challenged with an overwhelming and ever-growing number of tools, most of which have never been implemented or assessed for comparative effectiveness. To overcome this challenge, we have developed a conceptual framework to Grade and Assess Predictive tools (GRASP) that can provide clinicians with a standardised, evidence-based system to support their search for and selection of efficient tools. Methods A focused review of the literature was conducted to extract criteria along which tools should be evaluated. An initial framework was designed and applied to assess and grade five tools: LACE Index, Centor Score, Well’s Criteria, Modified Early Warning Score, and Ottawa knee rule. After peer review, by six expert clinicians and healthcare researchers, the framework and the grading of the tools were updated. Results GRASP framework grades predictive tools based on published evidence across three dimensions: 1) Phase of evaluation; 2) Level of evidence; and 3) Direction of evidence. The final grade of a tool is based on the highest phase of evaluation, supported by the highest level of positive evidence, or mixed evidence that supports a positive conclusion. Ottawa knee rule had the highest grade since it has demonstrated positive post-implementation impact on healthcare. LACE Index had the lowest grade, having demonstrated only pre-implementation positive predictive performance. Conclusion GRASP framework builds on widely accepted concepts to provide standardised assessment and evidence-based grading of predictive tools. Unlike other methods, GRASP is based on the critical appraisal of published evidence reporting the tools’ predictive performance before implementation, potential effect and usability during implementation, and their post-implementation impact. Implementing the GRASP framework as an online platform can enable clinicians and guideline developers to access standardised and structured reported evidence of existing predictive tools. However, keeping GRASP reports up-to-date would require updating tools’ assessments and grades when new evidence becomes available, which can only be done efficiently by employing semi-automated methods for searching and processing the incoming information.

scholarly journals Validating and Updating GRASP: An Evidence-Based Framework for Grading and Assessment of Clinical Predictive Tools

2020 ◽  
Author(s):  
Mohamed Khalifa ◽  
Farah Magrabi ◽  
Blanca Gallego
Keyword(s):  
Interrater Reliability ◽  
External Validation ◽  
Evaluation Criteria ◽  
Predictive Performance ◽  
Potential Effect ◽  
Evidence Based ◽  
Web Based ◽  
Predictive Tools ◽  
High Level

Abstract Background: When selecting predictive tools, clinicians are challenged with an overwhelming and ever-growing number, most of which have never been implemented or evaluated for comparative effectiveness. The authors developed an evidence-based framework for grading and assessment of predictive tools (GRASP). The objective of this study is to update GRASP and evaluate its reliability.Methods: A web-based survey was developed to collect responses of a wide international group of experts, who published studies on clinical prediction tools. Experts were invited via email and their responses were quantitatively and qualitatively analysed using NVivo software. The interrater reliability of the framework, to assign grades to eight predictive tools by two independent users, was evaluated.Results: We received 81 valid responses. On five-points Likert scale, experts overall strongly agreed with GRASP evaluation criteria=4.35/5, SD=1.01, 95%CI [4.349, 4.354]. Experts strongly agreed with six criteria: predictive performance=4.88/5, SD=0.43, 95%CI [4.87, 488] and evidence levels of predictive performance=4.44/5, SD=0.87, 95%CI [4.44, 4.45], usability=4.68/5, SD=0.70, 95%CI [4.67, 4.68] and potential effect=4.62/5, SD=0.68, 95%CI [4.61, 4.62], post-implementation impact=4.78/5, SD=0.57, 95%CI [4.78, 4.79] and evidence direction=4.25/5, SD=0.78, 95%CI [4.25, 4.26]. Experts somewhat agreed with one criterion: post-implementation impact levels=4.18/5, SD=1.14, 95%CI [4.17, 4.19]. Experts were neutral about one criterion; usability is higher than potential effect=2.96/5, SD=1.23, 95%CI [2.95, 2.97]. Sixty-four respondents provided recommendations to six open-ended questions regarding updating evaluation criteria. Forty-three suggested potential effect is higher than usability. Experts highlighted the importance of quality of studies and strength of evidence. Accordingly, GRASP concept and its detailed report were updated. The framework’s interrater reliability was tested, and two independent reviewers produced accurate and consistent results in grading eight predictive tools using the framework.Conclusion: Before implementation, internal and external validation of predictive performance of tools is essential in evaluating sensitivity and specificity. During planning for implementation, potential effect is more important that usability to evaluate acceptability of tools by users. Post-implementation, it is crucial to evaluate tools’ impact on healthcare processes and clinical outcomes. The GRASP framework aims to provide clinicians with a high-level, evidence-based, and comprehensive, yet simple and feasible, approach to evaluate, compare, and select predictive tools.

scholarly journals Brazilian Society of Rheumatology 2020 guidelines for psoriatic arthritis

2021 ◽  
Vol 61 (1) ◽  
Author(s):  
Sueli Carneiro ◽  
Penelope Esther Palominos ◽  
Sônia Maria Alvarenga Anti ◽  
Rodrigo Luppino Assad ◽  
Rafaela Silva Guimarães Gonçalves ◽  
...  
Keyword(s):  
Psoriatic Arthritis ◽  
Interrater Reliability ◽  
Biologic Therapy ◽  
Evidence Based ◽  
Review Of The Literature ◽  
Immune Disease ◽  
Level Of Evidence ◽  
Brazilian Society ◽  
Conventional Drug ◽  
Clinical Questions

AbstractPsoriatic arthritis (PsA) is a chronic and systemic immune disease characterized by inflammation of peripheral and/or axial joints and entheses in patients with psoriasis (PsO). Extra-articular and extracutaneous manifestations and numerous comorbidities can also be present. These recommendations replace the previous version published in May 2013. A systematic review of the literature retrieved 191 articles that were used to formulate 12 recommendations in response to 12 clinical questions, divided into 4 sections: diagnosis, non-pharmacological treatment, conventional drug therapy and biologic therapy. These guidelines provide evidence-based information on the clinical management for PsA patients. For each recommendation, the level of evidence (highest available), degree of strength (Oxford) and degree of expert agreement (interrater reliability) are reported.

scholarly journals LO18: The state of the evidence for emergency medical services (EMS) care of prehospital hypoglycemia: an analysis of appraised research from the Prehospital Evidence-based Practice Program

CJEM ◽  
2020 ◽  
Vol 22 (S1) ◽  
pp. S13-S13
Author(s):  
J. Greene ◽  
J. Goldstein ◽  
R. Brown ◽  
J. Swain ◽  
D. Fidgen ◽  
...  
Keyword(s):  
Primary Outcome ◽  
Assessment Tool ◽  
Evidence Based Practice ◽  
Oral Glucose ◽  
Evidence Based ◽  
High Quality ◽  
Level Of Evidence ◽  
Glucose Testing ◽  
Published Evidence ◽  
Quality Evidence

Introduction: The Prehospital Evidence-based Practice (PEP) program is an online, freely accessible, continuously updated repository of appraised EMS research evidence. This report is an analysis of published evidence for EMS interventions used to assess and treat patients suffering from hypoglycemia. Methods: PubMed was systematically searched in June 2019. One author screened titles, abstracts and full-texts for relevance. Trained appraisers reviewed full text articles, scored each on a three-point Level of Evidence (LOE) scale (based on study design and quality) and three-point Direction of Evidence (DOE) scale (supportive, neutral, or opposing findings for each intervention's primary outcome), abstracted the primary outcome, setting and assigned an outcome category (patient or process). Second party appraisal was conducted for all included studies. The level and direction of each intervention was plotted in an evidence matrix, based on appraisals. Results: Twenty-nine studies were included and appraised for seven interventions: 5 drugs (Dextrose 50% (D50), Dextrose 10% (D10), glucagon, oral glucose and thiamine), one assessment tool (point-of-care (POC) glucose testing) and one call disposition (treat-and-release). The most frequently reported study primary outcomes were related to: clinical improvement (n = 15, 51.7%), feasibility/safety (n = 8, 27.6%), and diagnostics (n = 6, 20.7%). The majority of outcomes were patient focused (n = 18, 62.0%). Conclusion: EMS interventions for treating hypoglycemia are informed by high-quality supportive evidence. Both D50 and D10 are supported by high-quality evidence; suggesting D10 may be an effective alternative to the standard D50. “Treat-and-release” practices for hypoglycemia are supported by moderate-quality evidence for the patient related outcomes of relapse, patient preference and complications. This body of evidence is high-quality, patient-focused and conducted in the prehospital setting thus generalizable paramedic practice.

scholarly journals Burnout Determinants among Nurses Working in Palliative Care during the Coronavirus Disease 2019 Pandemic

2021 ◽  
Vol 18 (7) ◽  
pp. 3358
Author(s):  
José Vítor Gonçalves ◽  
Luísa Castro ◽  
Guilhermina Rêgo ◽  
Rui Nunes
Keyword(s):  
Palliative Care ◽  
Health Care Professionals ◽  
Online Survey ◽  
Three Dimensions ◽  
Linear Regression Models ◽  
Perceived Health Status ◽  
Cross Sectional ◽  
National Network ◽  
Work Related ◽  
High Level

Nurses working in palliative care are at risk of burnout. The Copenhagen Burnout Inventory was used to determine burnout levels of nurses working in the Portuguese national network of palliative care. We evaluated the contribution of personal, work, and COVID-19 variables in three burnout subclasses: personal, work, and patient-related. A cross-sectional, exploratory, and quantitative design was employed and participants were sampled using convenience and snowball technique. An online survey was conducted and 153 nurses participated in our study. Socio-demographic characterization was conducted, levels of burnout and determinants were explored through multiple linear regression models for its three dimensions. High levels of personal, working, and patient burnout were present in 71 (46%), 68 (44%), and 33 nurses (22%), respectively. Most of them agreed that COVID-19 had an impact on their activities. Significant personal and work related burnout factors found were specialization in palliative care, self-perceived health status, unit type, weekly hours of work, and allocation to COVID-19 units. Gender was found to be a significant factor in patient-related burnout. There is a high level of burnout among nurses working in the Portuguese national network of palliative care. Measures for identification and mitigation of burnout are necessary to protect health care professionals.

scholarly journals Older and Wiser? The Need to Reexamine the Impact of Health Professionals Age and Experience on Competency-Based Practices

2021 ◽  
Vol 7 ◽  
pp. 237796082110290
Author(s):  
Jing Xu ◽  
Kristen Hicks-Roof ◽  
Chloe E. Bailey ◽  
Hanadi Y. Hamadi
Keyword(s):  
Health Professionals ◽  
Online Survey ◽  
Evidence Based Practice ◽  
Core Competencies ◽  
Healthcare Services ◽  
Interdisciplinary Teams ◽  
Evidence Based ◽  
Health Providers ◽  
Cross Sectional ◽  
The Impact

Introduction Delivery of healthcare services makes up a complex system and it requires providers to be competent and to be able to integrate each of the institute of medicine’s (IOM) 5 core competencies into practice. However, healthcare providers are challenged with the task to be able to understand and apply the IOM core competencies into practice. Objective The purpose of the study was to examine the factors that influence health professional’s likelihood of accomplishing the IOM core competencies. Methods A cross-sectional study design was used to administer a validated online survey to health providers. This survey was distributed to physicians, nursing professionals, specialists, and allied healthcare professionals. The final sample included 3,940 participants who completed the survey. Results The study findings show that younger health professionals more consistently practice daily competencies than their older counterparts, especially in the use of evidence-based practice, informatics, and working in interdisciplinary teams. Less experienced health professionals more consistently applied quality improvement methods but less consistently used evidence-based practice compared to their more experienced counterparts. Conclusion There is a need to understand how health professionals’ age and experience impact their engagement with IOM’s core competencies. This study highlights the need for educational resources on the competencies to be tailored to health providers’ age and experience.

scholarly journals Nonattendance at Scheduled Appointments in Outpatient Clinics Due to COVID-19 and Related Factors in Taiwan: A Health Belief Model Approach

2021 ◽  
Vol 18 (9) ◽  
pp. 4445
Author(s):  
Yi-Ping Hsieh ◽  
Cheng-Fang Yen ◽  
Chia-Fen Wu ◽  
Peng-Wei Wang
Keyword(s):  
Health Behaviors ◽  
Health Belief Model ◽  
Health Beliefs ◽  
Online Survey ◽  
Health Belief ◽  
Three Dimensions ◽  
Outpatient Clinics ◽  
Related Factors ◽  
Study Results ◽  
Belief Model

During the COVID-19 pandemic, the number of hospital visits and attendance at scheduled appointments have dropped significantly. We used the health belief model (in three dimensions) to examine the determinants of non-attendance of scheduled appointments in outpatient clinics due to the COVID-19 pandemic. Participants in Taiwan (n = 1954) completed an online survey from 10 April 10 to 23 April 2020, which assessed how people perceived and responded to the outbreak of a fast-spreading infectious disease. We performed both univariate and multivariate logistic regression to examine the roles of cognitive, affective, and behavioral health belief constructs in nonattendance at scheduled appointments. The results indicated that individuals who perceived high confidence in coping with COVID-19 were less likely to miss or cancel their doctor’s appointments, whereas individuals who reported high anxiety and practiced more preventive health behaviors, including avoiding crowded places, washing hands more often, and wearing a mask more often, were more likely to miss or cancel their appointments due to the COVID-19 pandemic. Non-heterosexual participants had a lower rate of nonattendance at scheduled appointments compared with heterosexual ones. The study results increase our understanding of the patients’ cognitive health beliefs, psychological distress, and health behaviors when assessing adherence to medical appointments during a pandemic.

Intra- and Inter-Rater Reliability of a Ballet-based Dance Technique Screening Instrument

2020 ◽  
Vol 35 (1) ◽  
pp. 28-34
Author(s):  
Shaw Bronner ◽  
Ivetta Lassey ◽  
Jessie R Lesar ◽  
Zachary G Shaver ◽  
Catherine Turner
Keyword(s):  
Standardized Testing ◽  
Injury Risk ◽  
Online Survey ◽  
Physical Therapists ◽  
Screening Instrument ◽  
Risk Level ◽  
Level Of Evidence ◽  
Rater Reliability ◽  
Training Video ◽  
Dance Medicine

OBJECTIVES: To investigate intra- and inter-rater reliability of a ballet-based Dance Technique Screening Instrument used by physical therapists (PTs) and student PTs (SPTs) with prior dance medicine or dance experience. METHODS: Ten pre-professional dancers were video-recorded in the sagittal and frontal planes while performing four dance sequences: 1) second position grand plié; 2) développé à la seconde; 3) single-limb passé relevé balance; and 4) jumps in first position. Dance videos and electronic versions of the demographics and scoring forms were provided through a secure online survey to 28 PTs and SPTs who served as raters. Raters reviewed a training video prior to scoring the 10 dancers. Raters were asked to repeat their assessments 1–2 wks later. Intraclass correlations (ICC) were assessed for all-raters, PTs, and SPTs for total and sequence scores. RESULTS: Twenty-eight raters assessed the videos one time. Inter-rater reliability was ICC=0.98 (CI95=0.96–0.99) (all-raters), with PTs and SPTs displaying similar values (ICC=0.96 and 0.96, respectively). Eighteen raters (11 PTs, 7 SPTs) repeated the video analysis. Intra-rater reliability was ICC=0.78 (CI95=0.72–0.83) with PTs ICC=0.81 and SPTs ICC=0.70. CONCLUSIONS: Correlations were high for all-raters. SPTs were as reliable as PTs in inter-rater comparisons. PTs exhibited higher intra-rater reliability compared to SPTs. These results substantiate the reliability of a standardized testing instrument to conduct dance technique assessment. Validity of this instrument was demonstrated in a previous study which found dancers with better technique were less likely to sustain injury. The ability to identify technique deficits can guide preventative programs that may reduce injury risk. LEVEL OF EVIDENCE: Level III.

2014 Chinese guidelines for secondary prevention of ischemic stroke and transient ischemic attack

2017 ◽  
Vol 12 (3) ◽  
pp. 302-320 ◽  
Author(s):  
Yongjun Wang ◽  
Ming Liu ◽  
Chuanqiang Pu
Keyword(s):  
Ischemic Stroke ◽  
Secondary Prevention ◽  
Transient Ischemic Attack ◽  
Recurrent Stroke ◽  
Stroke Recurrence ◽  
Evidence Based ◽  
Cerebrovascular Disorder ◽  
Level Of Evidence ◽  
Vascular Stenosis ◽  
Ischemic Attack

Ischemic stroke and transient ischemic attack (TIA) are the most common cerebrovascular disorder and leading cause of death in China. The Effective secondary prevention is the vital strategy for reducing stroke recurrence. The aim of this guideline is to provide the most updated evidence-based recommendation to clinical physicians from the prior version. Control of risk factors, intervention for vascular stenosis/occlusion, antithrombotic therapy for cardioembolism, and antiplatelet therapy for noncardioembolic stroke are all recommended, and the prevention of recurrent stroke in a variety of uncommon causes and subtype provided as well. We modified the level of evidence and recommendation according to part of results from domestic RCT in order to facility the clinical practice.

Sign in / Sign up

Export Citation Format

Share Document