scholarly journals Developing a framework for evidence-based grading and assessment of predictive tools for clinical decision support

2019 ◽  
Vol 19 (1) ◽  
Author(s):  
Mohamed Khalifa ◽  
Farah Magrabi ◽  
Blanca Gallego
Keyword(s):  
Predictive Performance ◽  
Patient Characteristics ◽  
Clinical Decision ◽  
Potential Effect ◽  
Three Dimensions ◽  
Early Warning Score ◽  
Evidence Based ◽  
Level Of Evidence ◽  
Predictive Tools ◽  
Published Evidence

Abstract Background Clinical predictive tools quantify contributions of relevant patient characteristics to derive likelihood of diseases or predict clinical outcomes. When selecting predictive tools for implementation at clinical practice or for recommendation in clinical guidelines, clinicians are challenged with an overwhelming and ever-growing number of tools, most of which have never been implemented or assessed for comparative effectiveness. To overcome this challenge, we have developed a conceptual framework to Grade and Assess Predictive tools (GRASP) that can provide clinicians with a standardised, evidence-based system to support their search for and selection of efficient tools. Methods A focused review of the literature was conducted to extract criteria along which tools should be evaluated. An initial framework was designed and applied to assess and grade five tools: LACE Index, Centor Score, Well’s Criteria, Modified Early Warning Score, and Ottawa knee rule. After peer review, by six expert clinicians and healthcare researchers, the framework and the grading of the tools were updated. Results GRASP framework grades predictive tools based on published evidence across three dimensions: 1) Phase of evaluation; 2) Level of evidence; and 3) Direction of evidence. The final grade of a tool is based on the highest phase of evaluation, supported by the highest level of positive evidence, or mixed evidence that supports a positive conclusion. Ottawa knee rule had the highest grade since it has demonstrated positive post-implementation impact on healthcare. LACE Index had the lowest grade, having demonstrated only pre-implementation positive predictive performance. Conclusion GRASP framework builds on widely accepted concepts to provide standardised assessment and evidence-based grading of predictive tools. Unlike other methods, GRASP is based on the critical appraisal of published evidence reporting the tools’ predictive performance before implementation, potential effect and usability during implementation, and their post-implementation impact. Implementing the GRASP framework as an online platform can enable clinicians and guideline developers to access standardised and structured reported evidence of existing predictive tools. However, keeping GRASP reports up-to-date would require updating tools’ assessments and grades when new evidence becomes available, which can only be done efficiently by employing semi-automated methods for searching and processing the incoming information.

scholarly journals Validating and Updating GRASP: An Evidence-Based Framework for Grading and Assessment of Clinical Predictive Tools

2020 ◽  
Author(s):  
Mohamed Khalifa ◽  
Farah Magrabi ◽  
Blanca Gallego
Keyword(s):  
Interrater Reliability ◽  
Online Survey ◽  
Evaluation Criteria ◽  
Predictive Performance ◽  
Potential Effect ◽  
Three Dimensions ◽  
Evidence Based ◽  
Level Of Evidence ◽  
Predictive Tools ◽  
Published Evidence

Abstract Background: When selecting predictive tools, clinicians are challenged with an overwhelming and ever-growing number, most of which have never been implemented or evaluated for comparative effectiveness. To overcome this challenge, the authors developed an evidence-based framework for grading and assessment of predictive tools (GRASP). The objective of this study is to update GRASP and evaluate its reliability. Methods: An online survey was developed to collect responses of a wide international group of experts, who published studies on developing, implementing or evaluating clinical decision support tools. The interrater reliability of the framework, to assign grades to eight predictive tools by two independent users, was evaluated. Results: Among 882 invited experts, 81 provided valid responses. On a five-points Likert scale, experts overall strongly agreed to GRASP evaluation criteria (4.35). Experts strongly agreed to six criteria: predictive performance (4.87) and predictive performance levels (4.44), usability (4.68) and potential effect (4.61), post-implementation impact (4.78) and evidence direction (4.26). Experts somewhat agreed to one criterion: post-implementation impact levels (4.16). Experts were neutral about one criterion; usability is higher than potential effect (2.97). Sixty-four respondents provided recommendations to open-ended questions regarding adding, removing or changing evaluation criteria. Forty-three respondents suggested the potential effect should be higher than the usability. Experts highlighted the importance of reporting quality of studies and strength of evidence supporting grades assigned to predictive tools. Accordingly, GRASP concept and its detailed report were updated. The updated framework’s interrater reliability, to produce accurate and consistent results by two independent users, was tested and found to be initially reliable. Conclusion: The GRASP framework grades predictive tools based on critical appraisal of published evidence across three dimensions: phase of evaluation, level of evidence, and direction of evidence. The final grade of a tool is based on the highest phase of evaluation, supported by the highest level of positive evidence, or mixed evidence that supports positive conclusion. GRASP aims to provide clinicians with a high-level, evidence-based, and comprehensive, yet simple and feasible, approach to evaluate predictive tools, considering their predictive performance before implementation, usability and potential effect during planning for implementation, and post-implementation impact on healthcare outcomes.

scholarly journals Validating and Updating GRASP: An Evidence-Based Framework for Grading and Assessment of Clinical Predictive Tools

2020 ◽  
Author(s):  
Mohamed Khalifa ◽  
Farah Magrabi ◽  
Blanca Gallego
Keyword(s):  
Interrater Reliability ◽  
External Validation ◽  
Evaluation Criteria ◽  
Predictive Performance ◽  
Potential Effect ◽  
Evidence Based ◽  
Web Based ◽  
Predictive Tools ◽  
High Level

Abstract Background: When selecting predictive tools, clinicians are challenged with an overwhelming and ever-growing number, most of which have never been implemented or evaluated for comparative effectiveness. The authors developed an evidence-based framework for grading and assessment of predictive tools (GRASP). The objective of this study is to update GRASP and evaluate its reliability.Methods: A web-based survey was developed to collect responses of a wide international group of experts, who published studies on clinical prediction tools. Experts were invited via email and their responses were quantitatively and qualitatively analysed using NVivo software. The interrater reliability of the framework, to assign grades to eight predictive tools by two independent users, was evaluated.Results: We received 81 valid responses. On five-points Likert scale, experts overall strongly agreed with GRASP evaluation criteria=4.35/5, SD=1.01, 95%CI [4.349, 4.354]. Experts strongly agreed with six criteria: predictive performance=4.88/5, SD=0.43, 95%CI [4.87, 488] and evidence levels of predictive performance=4.44/5, SD=0.87, 95%CI [4.44, 4.45], usability=4.68/5, SD=0.70, 95%CI [4.67, 4.68] and potential effect=4.62/5, SD=0.68, 95%CI [4.61, 4.62], post-implementation impact=4.78/5, SD=0.57, 95%CI [4.78, 4.79] and evidence direction=4.25/5, SD=0.78, 95%CI [4.25, 4.26]. Experts somewhat agreed with one criterion: post-implementation impact levels=4.18/5, SD=1.14, 95%CI [4.17, 4.19]. Experts were neutral about one criterion; usability is higher than potential effect=2.96/5, SD=1.23, 95%CI [2.95, 2.97]. Sixty-four respondents provided recommendations to six open-ended questions regarding updating evaluation criteria. Forty-three suggested potential effect is higher than usability. Experts highlighted the importance of quality of studies and strength of evidence. Accordingly, GRASP concept and its detailed report were updated. The framework’s interrater reliability was tested, and two independent reviewers produced accurate and consistent results in grading eight predictive tools using the framework.Conclusion: Before implementation, internal and external validation of predictive performance of tools is essential in evaluating sensitivity and specificity. During planning for implementation, potential effect is more important that usability to evaluate acceptability of tools by users. Post-implementation, it is crucial to evaluate tools’ impact on healthcare processes and clinical outcomes. The GRASP framework aims to provide clinicians with a high-level, evidence-based, and comprehensive, yet simple and feasible, approach to evaluate, compare, and select predictive tools.

scholarly journals Dantrolene in the treatment of MDMA-related hyperpyrexia: a systematic review

CJEM ◽  
2010 ◽  
Vol 12 (05) ◽  
pp. 435-442 ◽  
Author(s):  
Brian E. Grunau ◽  
Matthew O. Wiens ◽  
Jeffrey R. Brubacher
Keyword(s):  
Systematic Review ◽  
Clinical Decision Making ◽  
Case Reports ◽  
Data Extraction ◽  
Patient Characteristics ◽  
Clinical Decision ◽  
Poison Control ◽  
Published Data ◽  
Published Evidence

ABSTRACTObjective:The use of dantrolene in the treatment of hyperpyrexia related to MDMA (3,4-methylenedioxymethamphetamine) is controversial, with little data available to guide clinical decision-making. Although the treatment is recommended by several poison control centres, published data are primarily in the form of case reports and animal and in vitro experiments. We conducted a systematic review to investigate the published evidence regarding the safety and benefits of dantrolene for MDMA-related hyperpyrexia in humans.Data sources:A systematic search of Embase and MEDLINE was conducted from the earliest possible date to November 2008.Study selection:All human trials and case reports of MDMA-related hyperpyrexia were considered.Data extraction:Data were abstracted systematically and characteristics including use of dantrolene, adverse reactions attributed to dantrolene, peak temperature, complications from MDMA-related hyperpyrexia and survival were recorded.Data synthesis:Our search yielded 668 articles of which 53, reporting 71 cases of MDMA-induced hyperpyrexia, met our inclusion criteria. No clinical trials, randomized controlled trials, observational studies or meta-analyses were identified. Dantrolene was used in 26 cases. Patient characteristics were similar in the dantrolene and no dantrolene groups. The proportion of survivors was higher in the dantrolene group (21/26) than in the no dantrolene group (25/45). This difference was especially pronounced in those with extreme (≥ 42°C) and severe (≥ 40°C) fever, with a survival rate of 8 of 13 and 10 of 10, respectively, in the dantrolene group compared with 0 of 4 and 15 of 27 in the no dantrolene group. There were no reports of adverse events attributable to dantrolene with the exception of a possible association with an episode of transient hypoglycemia.Conclusion:Our systematic review suggests that dantrolene is safe for patients with MDMA-related hyperpyrexia. Dantrolene may also be associated with improved survival and reduced complications, especially in patients with extreme (≥ 42°C) or severe (≥ 40°C) hyperpyrexia, although this conclusion must be interpreted with caution given the risk of reporting or publication bias.

scholarly journals Contrasting evidence to reimbursement reality for off-label use (OLU) of drug treatments in cancer care: rationale and design of the CEIT-OLU project

ESMO Open ◽  
2019 ◽  
Vol 4 (6) ◽  
pp. e000596
Author(s):  
Amanda Katherina Herbrand ◽  
Andreas Michael Schmitt ◽  
Matthias Briel ◽  
Stefan Diem ◽  
Hannah Ewald ◽  
...  
Keyword(s):  
Cancer Care ◽  
Clinical Evidence ◽  
Patient Characteristics ◽  
Evidence Based ◽  
Cancer Drugs ◽  
Off Label Use ◽  
Level Of Evidence ◽  
Treatment Benefit ◽  
Off Label ◽  
Label Use

BackgroundOff-label use (OLU) of a drug reflects a perceived unmet medical need, which is common in oncology. Cancer drugs are often highly expensive and their reimbursement is a challenge for many healthcare systems. OLU is frequently regulated by reimbursement restrictions. For evidence-based healthcare, treatment ought to be reimbursed if there is sufficient clinical evidence for treatment benefit independently of patient factors not related to the treatment indication. However, little is known about the reality of OLU reimbursement and its association with the underlying clinical evidence. Here, we aim to investigate the relationship of reimbursement decisions with the underlying clinical evidence.Methods/ designWe will extract patient characteristics and details on treatment and reimbursement of cancer drugs from over 3000 patients treated in three Swiss hospitals. We will systematically search for clinical trial evidence on benefits associated with OLU in the most common indications. We will describe the prevalence of OLU in Switzerland and its reimbursement in cancer care, and use multivariable logistic regression techniques to investigate the association of approval/rejection of a reimbursement requests to the evidence on treatment effects and to further factors, including type of drug, molecular predictive markers and the health insurer.DiscussionOur study will provide a systematic overview and assessment of OLU and its reimbursement reality in Switzerland. We may provide a better understanding of the access to cancer care that is regulated by health insurers and we hope to identify factors that determine the level of evidence-based cancer care in a highly diverse western healthcare system.

Integrating the Patient and the Clinician With the Research

2016 ◽  
pp. 189-218
Author(s):  
John C. Norcross ◽  
Thomas P. Hogan ◽  
Gerald P. Koocher ◽  
Lauren A. Maggio
Keyword(s):  
Decision Making ◽  
Stages Of Change ◽  
Clinical Decision Making ◽  
Evidence Based Practice ◽  
Patient Characteristics ◽  
Clinical Decision ◽  
Evidence Based ◽  
Clinical Expertise ◽  
Simple Extrapolation ◽  
Complex Cases

This chapter demonstrates how research is integrated with the two other pillars of evidence-based practice (EBP): clinical expertise and patient characteristics. Research alone never suffices for making clinical decisions, nor does the simple extrapolation of research qualify as EBP. The chapter begins by correcting pernicious myths about EBP and then discusses enlarging clinical decision-making by adding the clinician and the patient into the mix. The chapter describes several ways in which the three pillars of EBP can be integrated and considers what to do when they cannot be integrated, particularly in complex cases. Clinicians will ultimately adopt, adapt, or abandon a research-supported intervention for a specific case. Finally, the chapter reviews the research on being responsive to patients’ transdiagnostic features, such as preferences, stages of change, and culture.

scholarly journals Contrasting Evidence to Reimbursement Reality for Off-label use (OLU) of Drug Treatments in Cancer Care – Rationale and Design of the CEIT-OLU-project

2019 ◽  
Author(s):  
AK Herbrand ◽  
AM Schmitt ◽  
M Briel ◽  
S Diem ◽  
H Ewald ◽  
...  
Keyword(s):  
Health Care ◽  
Cancer Care ◽  
Clinical Evidence ◽  
Health Care Systems ◽  
Patient Characteristics ◽  
Evidence Based ◽  
Cancer Drugs ◽  
Level Of Evidence ◽  
Treatment Benefit ◽  
Off Label

AbstractBackgroundOff-label drug use (OLU) reflects a perceived unmet medical need, which is common in oncology. Cancer drugs are often highly expensive and their reimbursement is a challenge for many health care systems. OLU is frequently regulated by reimbursement restrictions. For evidence-based health care, treatment ought to be reimbursed if there is sufficient clinical evidence for treatment benefit independently of patient factors not related to the treatment indication. However, little is known about the reality of OLU reimbursement and its association with the underlying clinical evidence. Here we aim to investigate the relationship of reimbursement decisions with the underlying clinical evidence.Methods/DesignWe extract patient characteristics and details on treatment and reimbursement of cancer drugs from over 3000 patients treated in three Swiss hospitals. We systematically search for clinical trial evidence on benefits associated with OLU in the most common indications. We will describe the prevalence of OLU in Switzerland and its reimbursement in cancer care, and use multivariable logistic regression techniques to investigate the association of approval/rejection of a reimbursement requests to the evidence on treatment effects and to further factors, including type of drug, molecular predictive markers and the health insurer.DiscussionOur study will provide a systematic overview and assessment of OLU and its reimbursement reality in Switzerland. We may provide a better understanding of the access to cancer care that is regulated by health insurers and we hope to identify factors that determine the level of evidence-based cancer care in a highly diverse Western health care system.

Tag you’re it! Development of a clinical decision support tool to identify NCCN regimens without a costly pathways program.

2020 ◽  
Vol 38 (29_suppl) ◽  
pp. 308-308
Author(s):  
Aarti Sonia Bhardwaj ◽  
Edward Gu ◽  
Damaris Peralta Hernandez ◽  
Michael McLean ◽  
Haley Hines Theroux ◽  
...  
Keyword(s):  
Multiple Myeloma ◽  
Decision Support ◽  
Clinical Decision Support ◽  
Cancer Diagnosis ◽  
Best Practice ◽  
Clinical Pathways ◽  
Clinical Decision ◽  
Evidence Based ◽  
Tier 2 ◽  
Level Of Evidence

308 Background: Reducing variation in care can improve outcomes and decrease costs. Evidence based medicine drives cancer guidelines and adherence promotes quality cancer care. Value based programs are based on adherence to pathways. Most institutions adopt costly cloud based clinical pathways products but none are mature products that fully integrate with the EHR and they require additional data entry. We present our simple Clinical Decision Support (CDS) tool for identifying best practice treatment protocols driven by the cancer diagnosis in the EMR for our large, multi-site, mixed academic and community cancer system. Methods: Our chemotherapy council must approve all protocols that are published in the system’s Epic Beacon library using a rigorous scoring system based on level of evidence and FDA or NCCN approval. Then each protocol is “tagged” appropriately: “Tier 1A”: Preferred Regimens/NCCN Approved; “Tier 1B”: Preferred Regimens/Chemo Council Approved (but not NCCN Approved); Tier 2: Specific Disease Management Team approved regimens; and finally “Other” or research protocols. When the oncologist enters the treatment plan in EPIC, a list of protocols are suggested, ordered by level of evidence, based on the cancer diagnosis and with the easily visible level of evidence or “tag” to allow data driven decision making. Results: We implemented our CDS tool December 12, 2019. As of mid-June, 2020 a total of 1637 treatment plans have been implemented. Of those, 1323 (81%) are Tier 1A, 310 (2%) are Tier 1B and 4 (.2%) are Tier 2. Thus demonstrating 81% adherence to NCCN approved regimens across the system, regardless of the line of treatment. GI and breast cancers were responsible for the most plans with the highest adherence to Tier 1A plans, specifically 92% among the breast cancer group. Multiple Myeloma and Sarcoma were tied for the lowest adherence rate of 58%. This data can be further stratified by medical oncologist. Interestingly, Multiple Myeloma had the highest utilization of Tier 1B protocols perhaps reflecting the rapidly changing literature that is ahead of the guidelines. Conclusions: We demonstrated adherence to NCCN protocols 81% of the time over a 6 month period and over multiple cancer types. Protocol tagging and reporting utilizing the EMR alone could be used as a powerful model for value based care. We identified disease areas that will require further education regarding evidence based treatment and can consider interventions including real time feedback to clinicians and/or best practice advisories or quality based incentives.

scholarly journals LO18: The state of the evidence for emergency medical services (EMS) care of prehospital hypoglycemia: an analysis of appraised research from the Prehospital Evidence-based Practice Program

CJEM ◽  
2020 ◽  
Vol 22 (S1) ◽  
pp. S13-S13
Author(s):  
J. Greene ◽  
J. Goldstein ◽  
R. Brown ◽  
J. Swain ◽  
D. Fidgen ◽  
...  
Keyword(s):  
Primary Outcome ◽  
Assessment Tool ◽  
Evidence Based Practice ◽  
Oral Glucose ◽  
Evidence Based ◽  
High Quality ◽  
Level Of Evidence ◽  
Glucose Testing ◽  
Published Evidence ◽  
Quality Evidence

Introduction: The Prehospital Evidence-based Practice (PEP) program is an online, freely accessible, continuously updated repository of appraised EMS research evidence. This report is an analysis of published evidence for EMS interventions used to assess and treat patients suffering from hypoglycemia. Methods: PubMed was systematically searched in June 2019. One author screened titles, abstracts and full-texts for relevance. Trained appraisers reviewed full text articles, scored each on a three-point Level of Evidence (LOE) scale (based on study design and quality) and three-point Direction of Evidence (DOE) scale (supportive, neutral, or opposing findings for each intervention's primary outcome), abstracted the primary outcome, setting and assigned an outcome category (patient or process). Second party appraisal was conducted for all included studies. The level and direction of each intervention was plotted in an evidence matrix, based on appraisals. Results: Twenty-nine studies were included and appraised for seven interventions: 5 drugs (Dextrose 50% (D50), Dextrose 10% (D10), glucagon, oral glucose and thiamine), one assessment tool (point-of-care (POC) glucose testing) and one call disposition (treat-and-release). The most frequently reported study primary outcomes were related to: clinical improvement (n = 15, 51.7%), feasibility/safety (n = 8, 27.6%), and diagnostics (n = 6, 20.7%). The majority of outcomes were patient focused (n = 18, 62.0%). Conclusion: EMS interventions for treating hypoglycemia are informed by high-quality supportive evidence. Both D50 and D10 are supported by high-quality evidence; suggesting D10 may be an effective alternative to the standard D50. “Treat-and-release” practices for hypoglycemia are supported by moderate-quality evidence for the patient related outcomes of relapse, patient preference and complications. This body of evidence is high-quality, patient-focused and conducted in the prehospital setting thus generalizable paramedic practice.

Debates in Dysphagia Management: How Do You Use Evidence-Based Practice in Your Dysphagia Patient Care?

2011 ◽  
Vol 20 (4) ◽  
pp. 121-123
Author(s):  
Jeri A. Logemann
Keyword(s):  
Decision Making ◽  
Patient Care ◽  
Clinical Outcomes ◽  
Clinical Judgment ◽  
Clinical Decision Making ◽  
Evidence Based Practice ◽  
Clinical Decision ◽  
Evidence Based ◽  
Clinical Method ◽  
Do So

Evidence-based practice requires astute clinicians to blend our best clinical judgment with the best available external evidence and the patient's own values and expectations. Sometimes, we value one more than another during clinical decision-making, though it is never wise to do so, and sometimes other factors that we are unaware of produce unanticipated clinical outcomes. Sometimes, we feel very strongly about one clinical method or another, and hopefully that belief is founded in evidence. Some beliefs, however, are not founded in evidence. The sound use of evidence is the best way to navigate the debates within our field of practice.

Supporting the Development of Clinical Reasoning of Preprofessional Novices in Dysphagia Management

2019 ◽  
Vol 40 (03) ◽  
pp. 151-161 ◽  
Author(s):  
Sebastian Doeltgen ◽  
Stacie Attrill ◽  
Joanne Murray
Keyword(s):  
Clinical Reasoning ◽  
Clinical Decision Making ◽  
Management Practice ◽  
Clinical Decision ◽  
Assessment Process ◽  
Evidence Based ◽  
Sources Of Information ◽  
Multiple Sources ◽  
Implicit Processing ◽  
Reasoning Skills

AbstractProficient clinical reasoning is a critical skill in high-quality, evidence-based management of swallowing impairment (dysphagia). Clinical reasoning in this area of practice is a cognitively complex process, as it requires synthesis of multiple sources of information that are generated during a thorough, evidence-based assessment process and which are moderated by the patient's individual situations, including their social and demographic circumstances, comorbidities, or other health concerns. A growing body of health and medical literature demonstrates that clinical reasoning skills develop with increasing exposure to clinical cases and that the approaches to clinical reasoning differ between novices and experts. It appears that it is not the amount of knowledge held, but the way it is used, that distinguishes a novice from an experienced clinician. In this article, we review the roles of explicit and implicit processing as well as illness scripts in clinical decision making across the continuum of medical expertise and discuss how they relate to the clinical management of swallowing impairment. We also reflect on how this literature may inform educational curricula that support SLP students in developing preclinical reasoning skills that facilitate their transition to early clinical practice. Specifically, we discuss the role of case-based curricula to assist students to develop a meta-cognitive awareness of the different approaches to clinical reasoning, their own capabilities and preferences, and how and when to apply these in dysphagia management practice.

Sign in / Sign up

Export Citation Format

Share Document