A Systematic Approach to Identifying and Developing Implementation Strategies to Support the Uptake of a Risk Tool to Aid in Clinical Management of Patients with Syncope: A Qualitative Study

Background: The Canadian Syncope Risk Score (CSRS) is a validated risk stratification tool developed to optimize the accuracy of emergency department decisions and inform evidence-based clinical actions. While preliminary work has been undertaken to understand the barriers to CSRS use, no work to date has explored how to implement the CSRS to overcome these barriers in practice. This study aimed to identify which implementation strategies are most appropriate to address these barriers and how they should be implemented to mitigate the possibility of poor uptake. Methods: We conducted a series of three iterative online user-centered design workshops with emergency medicine physicians from three hospital sites in Ontario, Canada. The objective of the workshops was to engage participants in identifying acceptable strategies to promote CSRS uptake and how they should be operationalized. To support this, we systematically mapped previously identified barriers to corresponding behaviour change techniques to identify the most likely strategies to effect change. The sessions were audio-recorded and dialogue relating directly to the study objective were transcribed. We performed a qualitative content data analysis according to pre-defined objectives for each workshop. Results: Fourteen physicians participated across the three workshops. The main implementation strategies identified to overcome identified barriers were: education in the format of meetings, videos, journal clubs, and posters (uncertainty around when and how to apply the CSRS); an online calculator and integration of the CSRS into electronic medical record (uncertainty in how to apply the CSRS), local champion (lack of team buy-in); and dissemination of evidence summaries and feedback through email communications (lack of evidence about impact). Conclusions: The ability of the CSRS to effectively improve patient safety and syncope management relies on broad buy-in and uptake across physicians. To ensure the CSRS is well-positioned for impact, a comprehensive suite of implementation strategies was identified to address known barriers. This next phase of work will provide insight into whether these strategies facilitated better alignment with barriers, higher physician engagement with the implementation strategies, and broader uptake of the CSRS, with the objective of improving the likelihood that the CSRS will positively influence patient outcomes.

Identifying and addressing conflicting results across multiple discordant systematic reviews on the same question: Protocol for a replication study of the Jadad algorithm

Introduction: An increasing growth of systematic reviews (SRs) presents notable challenges for decision-makers seeking to answer clinical questions. In 1997, an algorithm was created by Jadad to assess discordance in results across SRs on the same question. Our study aims to (a) replicate assessments done in a sample of studies using the Jadad algorithm to determine if the same SR would have been chosen, (b) evaluate the Jadad algorithm in terms of utility, efficiency, and comprehensiveness, and (c) describe how authors address discordance in results across multiple SRs. Methods and Analysis: We will use a database of 1218 overviews (2000-2020) created from a bibliometric study as the basis of our search for studies assessing discordance (called Discordant Reviews). This bibliometric study searched MEDLINE (Ovid), Epistemonikos, and Cochrane Database of Systematic Reviews for overviews. We will include any study using Jadad (1997) or another method to assess discordance. The first 30 studies screened at the full-text stage by two independent reviewers will be included. We will replicate the authors’ Jadad assessments. We will compare our outcomes qualitatively and evaluate the differences between our Jadad assessment of discordance and the authors’ assessment. Ethics and Dissemination: No ethics approval was required as no human subjects were involved. In addition to publishing in an open-access journal, we will disseminate evidence summaries through formal and informal conferences, academic websites, and across social media platforms. This is the first study to comprehensively evaluate and replicate Jadad algorithm assessments of discordance across multiple SRs.

Translation and validation of PubMed and Embase search filters for identification of systematic reviews, intervention studies, and observational studies in the field of first aid

Objective: The aim of this project was to validate search filters for systematic reviews, intervention studies, and observational studies translated from Ovid MEDLINE and Embase syntax and used for searches in PubMed and Embase.com during the development of evidence summaries supporting first aid guidelines. We aimed to achieve a balance among recall, specificity, precision, and number needed to read (NNR).Methods: Reference gold standards were constructed per study type derived from existing evidence summaries. Search filter performance was assessed through retrospective searches and measurement of relative recall, specificity, precision, and NNR when using the translated search filters. Where necessary, search filters were optimized. Adapted filters were validated in separate validation gold standards.Results: Search filters for systematic reviews and observational studies reached recall of ≥85% in both PubMed and Embase. Corresponding specificities for systematic review filters were ≥96% in both databases, with a precision of 9.7% (NNR 10) in PubMed and 5.4% (NNR 19) in Embase. For observational study filters, specificity, precision, and NNR were 68%, 2%, and 51 in PubMed and 47%, 0.8%, and 123 in Embase, respectively. These filters were considered sufficiently effective. Search filters for intervention studies reached a recall of 85% and 83% in PubMed and Embase, respectively. Optimization led to recall of ≥95% with specificity, precision, and NNR of 49%, 1.3%, and 79 in PubMed and 56%, 0.74%, and 136 in Embase, respectively.Conclusions: We report validated filters to search for systematic reviews, observational studies, and intervention studies in guideline projects in PubMed and Embase.com.

Development of a model of medication review for use in clinical practice: Bristol medication review model

Background Medication review is a core aspect of medicine optimisation, yet existing models of review vary substantially in structure and content and are not necessarily easy to implement in clinical practice. This study aimed to use evidence from the existing literature to identify key medication review components and use this to inform the design of an improved review model. Methods A systematic review was conducted (PROSPERO: CRD42018109788) to identify randomised control trials of stand-alone medication review in adults (18+ years). The review updated that by Huiskes et al. (BMC Fam Pract. 18:5, 2017), using the same search strategy implemented in MEDLINE and Embase. Studies were assessed using the Cochrane risk of bias tool. Key review components were identified, alongside relevant clinical and health service outcomes. A working group (patients, doctors and pharmacists) developed the model through an iterative consensus process (appraisal of documents plus group discussions), working from the systematic review findings, brief evidence summaries for core review components and examples of previous models, to agree on the main purpose of the review model, overarching model structure, review components and supporting material. Results We identified 28 unique studies, with moderate bias overall. Consistent medication review components included reconciliation (26 studies), safety assessment (22), suboptimal treatment (19), patient knowledge/preferences (18), adherence (14), over-the-counter therapy (13) and drug monitoring (10). There was limited evidence from studies for improvement in key clinical outcomes. The review structure was underpinned by patient values and preferences, with parallel information gathering and evaluation stages, feeding into the final decision-making and implementation. Most key components identified in the literature were included. The final model was considered to benefit from a patient-centred, holistic approach, which captured both patient-orientated and medication-focused problems, and aligned with traditional consultation methods thus facilitating implementation in practice. Conclusions The Bristol Medication Review Model provides a framework for standardised delivery of structured reviews. The model has the potential for use by all healthcare professionals with relevant clinical experience and is designed to offer flexibility of implementation not limited to a particular healthcare setting.

Research Engagement with Policy Makers: a practical guide to writing policy briefs

Effective communication between academics and policy makers plays an important role in informing political decision making and creating impact for researchers. Policy briefs are short evidence summaries written by researchers to inform the development or implementation of policy. This guide has been developed to support researchers to write effective policy briefs. It is jointly produced by the NIHR Policy Research Unit in Behavioural Science (BehSciPRU) and the UCL Centre for Behaviour Change (CBC). It has been written in consultation with policy advisers and synthesises current evidence and expert opinion on what makes an effective policy brief. It is for any researcher who wishes to increase the impact of their work by activity that may influence the process of policy formation, implementation or evaluation. Whilst the guide has been written primarily for a UK audience, it is hoped that it will be useful to researchers in other countries.

Identifying and addressing conflicting results across multiple discordant systematic reviews on the same topic: A protocol for a replication study of the Jadad algorithm

Introduction: An increasing growth of systematic reviews (SRs) presents notable challenges for decision-makers seeking to answer clinical questions. Overviews of systematic reviews aim to address these challenges by summarising results of SRs and making sense of potentially discrepant SR results and conclusions. In 1997, an algorithm was created by Jadad to assess discordance in results across SRs on the same topic. Since this tool pre-dates the advent of overviews, it has been inconsistently applied in this context. Our study aims to (a) replicate assessments done in a sample of overviews using the Jadad algorithm to determine if the same SR would have been chosen, (b) evaluate the Jadad algorithm in terms of utility, efficiency, and comprehensiveness, and (c) describe how overviews address discordance in results across multiple SRs. Methods and Analysis: We will use a database of 1218 overviews (2000-2020) created from a bibliometric study as the basis of our search for overviews assessing discordance. This bibliometric study searched MEDLINE (Ovid), Epistemonikos, and Cochrane Database for overviews. We will include any overviews using Jadad (1997) or another method to assess discordance. The first 30 overviews screened at the full-text stage by two independent reviewers will be included. We will replicate Jadad assessments in overviews. We will compare our outcomes qualitatively and evaluate the differences between our Jadad assessment of discordance and the overviews’ assessment. Ethics and Dissemination: No ethics approval was required as no human subjects were involved. In addition to publishing in an open-access journal, we will disseminate evidence summaries through formal and informal conferences, academic websites, and across social media platforms. This is the first study to comprehensively evaluate and replicate Jadad algorithm assessments of discordance in SRs.

Development of prehospital care quality indicators for the Australian setting: a modified RAND/UCLA appropriateness method

BackgroundGlobally, the measurement of quality is an important process that supports the provision of high-quality and safe healthcare services. The requirement for valid quality measurement to gauge improvements and monitor performance is echoed in the Australian prehospital care setting. The aim of this study was to use an evidence-informed expert consensus process to identify valid quality indicators (QIs) for Australian prehospital care provided by ambulance services.MethodsA modified RAND/UCLA appropriateness method was conducted with a panel of Australian prehospital care experts from February to May 2019. The proposed QIs stemmed from a scoping review and were systematically prepared within a clinical and non-clinical classification system, and a structure/process/outcome and access/safety/effectiveness taxonomy. Rapid reviews were performed for each QI to produce evidence summaries for consideration by the panellists. QIs were deemed valid if the median score by the panel was 7–9 without disagreement.ResultsOf 117 QIs, the expert panel rated 84 (72%) as valid. This included 26 organisational/system QIs across 7 subdomains and 58 clinical QIs within 10 subdomains.Most QIs were process indicators (n=62; 74%) while QIs describing structural elements and desired outcomes were less common (n=13; 15% and n=9; 11%, respectively). Non-exclusively, 18 (21%) QIs addressed access to healthcare, 21 (25%) described safety aspects and 64 (76%) specified elements contributing to effective services and care. QIs on general time intervals, such as response time, were not considered valid by the panel.ConclusionThis study demonstrates that with consideration of best available evidence a substantial proportion of QIs scoped and synthesised from the international literature are valid for use in the Australian prehospital care context.

Evidence synthesis summary formats for clinical guideline development group members: a mixed-methods systematic review protocol

Introduction: Evidence syntheses, often in the form of systematic reviews, are essential for clinical guideline development and informing changes to health policies. However, clinical guideline development groups (CGDG) are multidisciplinary, and participants such as policymakers, healthcare professionals and patient representatives can face obstacles when trying to understand and use evidence synthesis findings. Summary formats to communicate the results of evidence syntheses have become increasingly common, but it is currently unclear which format is most effective for different stakeholders. This mixed-methods systematic review (MMSR) evaluates the effectiveness and acceptability of different evidence synthesis summary formats for CGDG members. Methods: This protocol follows guidance from the Joanna Briggs Institute on MMSRs and is reported according to the Preferred Reporting Items for Systematic Reviews (PRISMA)-P guideline. A comprehensive search of six databases will be performed with no language restrictions. Primary outcomes are those relating to the effectiveness and preferences for and attitudes towards the different summary formats. We will include qualitative research and randomised controlled trials. Two reviewers will perform title, abstract, and full-text screening. Independent double-extraction of study characteristics and critical appraisal items will be undertaken using a standardised form. We will use a convergent segregated approach to analyse quantitative and qualitative data separately; results will then be integrated. Discussion: The results of this systematic review will provide an overview of the effectiveness and acceptability of different summary formats for evidence synthesis findings. These findings can be helpful for those in or communicating to guideline development groups. The results can also inform the development and pilot-testing of summary formats for evidence summaries.

Decision aids linked to evidence summaries and clinical practice guidelines: results from user-testing in clinical encounters

Background Tools for shared decision-making (e.g. decision aids) are intended to support health care professionals and patients engaged in clinical encounters involving shared decision-making. However, decision aids are hard to produce, and onerous to update. Consequently, they often do not reflect best current evidence, and show limited uptake in practice. In response, we initiated the Sharing Evidence to Inform Treatment decisions (SHARE-IT) project. Our goal was to develop and refine a new generation of decision aids that are generically produced along digitally structured guidelines and evidence summaries. Methods Applying principles of human-centred design and following the International Patient Decision Aid Standards (IPDAS) and GRADE methods for trustworthy evidence summaries we developed a decision aid prototype in collaboration with the Developing and Evaluating Communication strategies to support Informed Decisions and practice based on Evidence project (DECIDE). We iteratively user-tested the prototype in clinical consultations between clinicians and patients. Semi-structured interviews of participating clinicians and patients were conducted. Qualitative content analysis of both user-testing sessions and interviews was performed and results categorized according to a revised Morville’s framework of user-experience. We made it possible to produce, publish and use these decision aids in an electronic guideline authoring and publication platform (MAGICapp). Results Direct observations and analysis of user-testing of 28 clinical consultations between physicians and patients informed four major iterations that addressed readability, understandability, usability and ways to cope with information overload. Participants reported that the tool supported natural flow of the conversation and induced a positive shift in consultation habits towards shared decision-making. We integrated the functionality of SHARE-IT decision aids in MAGICapp, which has since generated numerous decision aids. Conclusion Our study provides a proof of concept that encounter decision aids can be generically produced from GRADE evidence summaries and clinical guidelines. Online authoring and publication platforms can help scale up production including continuous updating of electronic encounter decision aids, fully integrated with evidence summaries and clinical practice guidelines.