scholarly journals Integrated Care Search: Development and validation of a PubMed search filter for retrieving the integrated care research evidence

2020 ◽  
Author(s):  
Raechel A. Damarell ◽  
Suzanne Lewis ◽  
Camilla Trenerry ◽  
Jennifer J. Tieman
Keyword(s):  
Integrated Care ◽  
Gold Standard ◽  
Research Evidence ◽  
Evidence Base ◽  
Standard Test ◽  
Advisory Group ◽  
Search Filter ◽  
Filter Performance ◽  
Search Filters ◽  
Development And Validation

Abstract Background Integrated care is an increasingly important principle for organising healthcare. Integrated care models show promise in reducing resource wastage and service fragmentation whilst improving the accessibility, patient-centredness and quality of care for patients. Those needing reliable access to the growing research evidence base for integrated care can be frustrated by search challenges reflective of the topic's complexity. The aim of this study is to report the empirical development and validation of two search filters for rapid and effective retrieval of integrated care evidence in PubMed. One filter is optimised for recall and the other for precision.Methods An Expert Advisory Group comprising international integrated care experts guided the study. A gold standard test set of citations was formed from screening Handbook Integrated Care chapter references for relevance. This set was divided into a Term Identification Set (20%) for determining candidate terms using frequency analysis; a Filter Development Set (40%) for testing performance of term combinations; and a Filter Validation Set (40%) reserved for confirming final filter performance. In developing the high recall filter, recall was steadily increased while maintaining precision at ≥ 50%. Similarly, the high precision filter sought to maximise precision while keeping recall ≥ 50%. For each term combination tested, an approximation of precision was obtained by reviewing the first 100 citations retrieved in Medline for relevance.Results The gold standard set comprised 534 citations. The search filter optimised for recall ('Broad Integrated Care Search') achieved 86.0%-88.3% recall with corresponding low precision (47%-53%). The search filter optimised for precise searching ('Narrow Integrated Care Search') demonstrated precision of 73%-95% with recall reduced to between 55.9% and 59.8%. These filters are now available as one-click URL hyperlinks in the website of International Foundation for Integrated Care.Conclusions The Broad and Narrow Integrated Care Search filters provide potential users, such as policy makers and researchers, seamless, reliable and ongoing access to integrated care evidence for decision making. These filters were developed according to a rigorous and transparent methodology designed to circumvent the challenges of information retrieval posed by this complex, multifaceted topic.

scholarly journals Integrated Care Search: Development and validation of a PubMed search filter for retrieving the integrated care research evidence

2019 ◽  
Author(s):  
Raechel A. Damarell ◽  
Suzanne Lewis ◽  
Camilla Trenerry ◽  
Jennifer J. Tieman
Keyword(s):  
Integrated Care ◽  
Gold Standard ◽  
Research Evidence ◽  
Evidence Base ◽  
Standard Test ◽  
Advisory Group ◽  
Search Filter ◽  
Filter Performance ◽  
Search Filters ◽  
Development And Validation

Abstract Background Integrated care is an increasingly important principle for organising healthcare. Integrated care models show promise in reducing resource wastage and service fragmentation whilst improving the accessibility, patient-centredness and quality of care for patients. Those needing reliable access to the growing research evidence base for integrated care can be frustrated by search challenges reflective of the topic's complexity. The aim of this study is to report the empirical development and validation of two search filters for rapid and effective retrieval of integrated care evidence in PubMed. One filter is optimised for recall and the other for precision.Methods An Expert Advisory Group comprising international integrated care experts guided the study. A gold standard test set of citations was formed from screening Handbook Integrated Care chapter references for relevance. This set was divided into a Term Identification Set (20%) for determining candidate terms using frequency analysis; a Filter Development Set (40%) for testing performance of term combinations; and a Filter Validation Set (40%) reserved for confirming final filter performance. In developing the high recall filter, recall was steadily increased while maintaining precision at ≥ 50%. Similarly, the high precision filter sought to maximise precision while keeping recall ≥ 50%. For each term combination tested, an approximation of precision was obtained by reviewing the first 100 citations retrieved in Medline for relevance.Results The gold standard set comprised 534 citations. The search filter optimised for recall ('Broad Integrated Care Search') achieved 86.0%-88.3% recall with corresponding low precision (47%-53%). The search filter optimised for precise searching ('Narrow Integrated Care Search') demonstrated precision of 73%-95% with recall reduced to between 55.9% and 59.8%. These filters are now available as one-click URL hyperlinks in the website of International Foundation for Integrated Care.Conclusions The Broad and Narrow Integrated Care Search filters provide potential users, such as policy makers and researchers, seamless, reliable and ongoing access to integrated care evidence for decision making. These filters were developed according to a rigorous and transparent methodology designed to circumvent the challenges of information retrieval posed by this complex, multifaceted topic.

scholarly journals Integrated Care Search: Development and validation of a PubMed search filter for retrieving the integrated care research evidence

2019 ◽  
Author(s):  
Raechel A. Damarell ◽  
Suzanne Lewis ◽  
Camilla Trenerry ◽  
Jennifer J. Tieman
Keyword(s):  
Integrated Care ◽  
Gold Standard ◽  
Research Evidence ◽  
Evidence Base ◽  
Standard Test ◽  
Advisory Group ◽  
Search Filter ◽  
Filter Performance ◽  
Search Filters ◽  
Development And Validation

Abstract Background Integrated care is an increasingly important principle for organising healthcare. Integrated care models show promise in reducing resource wastage and service fragmentation whilst improving the accessibility, patient-centredness and quality of care for patients. Those needing reliable access to the growing research evidence base for integrated care can be frustrated by search challenges reflective of the topic's complexity. The aim of this study is to report the empirical development and validation of two search filters for rapid and effective retrieval of integrated care evidence in PubMed. One filter is optimised for recall and the other for precision.Methods An Expert Advisory Group comprising international integrated care experts guided the study. A gold standard test set of citations was formed from screening Handbook Integrated Care chapter references for relevance. This set was divided into a Term Identification Set (20%) for determining candidate terms using frequency analysis; a Filter Development Set (40%) for testing performance of term combinations; and a Filter Validation Set (40%) reserved for confirming final filter performance. In developing the high recall filter, recall was steadily increased while maintaining precision at ≥ 50%. Similarly, the high precision filter sought to maximise precision while keeping recall ≥ 50%. For each term combination tested, an approximation of precision was obtained by reviewing the first 100 citations retrieved in Medline for relevance.Results The gold standard set comprised 534 citations. The search filter optimised for recall ('Broad Integrated Care Search') achieved 86.0%-88.3% recall with corresponding low precision (47%-53%). The search filter optimised for precise searching ('Narrow Integrated Care Search') demonstrated precision of 73%-95% with recall reduced to between 55.9% and 59.8%. These filters are now available as one-click URL hyperlinks in the website of International Foundation for Integrated Care.Conclusions The Broad and Narrow Integrated Care Search filters provide potential users, such as policy makers and researchers, seamless, reliable and ongoing access to integrated care evidence for decision making. These filters were developed according to a rigorous and transparent methodology designed to circumvent the challenges of information retrieval posed by this complex, multifaceted topic.

scholarly journals Translation and validation of PubMed and Embase search filters for identification of systematic reviews, intervention studies, and observational studies in the field of first aid

2021 ◽  
Vol 109 (4) ◽  
Author(s):  
Bert Avau ◽  
Hans Van Remoortel ◽  
Emmy De Buck
Keyword(s):  
Systematic Reviews ◽  
Observational Studies ◽  
Intervention Studies ◽  
First Aid ◽  
Search Filter ◽  
Filter Performance ◽  
Ovid Medline ◽  
Search Filters ◽  
Study Type ◽  
Evidence Summaries

Objective: The aim of this project was to validate search filters for systematic reviews, intervention studies, and observational studies translated from Ovid MEDLINE and Embase syntax and used for searches in PubMed and Embase.com during the development of evidence summaries supporting first aid guidelines. We aimed to achieve a balance among recall, specificity, precision, and number needed to read (NNR).Methods: Reference gold standards were constructed per study type derived from existing evidence summaries. Search filter performance was assessed through retrospective searches and measurement of relative recall, specificity, precision, and NNR when using the translated search filters. Where necessary, search filters were optimized. Adapted filters were validated in separate validation gold standards.Results: Search filters for systematic reviews and observational studies reached recall of ≥85% in both PubMed and Embase. Corresponding specificities for systematic review filters were ≥96% in both databases, with a precision of 9.7% (NNR 10) in PubMed and 5.4% (NNR 19) in Embase. For observational study filters, specificity, precision, and NNR were 68%, 2%, and 51 in PubMed and 47%, 0.8%, and 123 in Embase, respectively. These filters were considered sufficiently effective. Search filters for intervention studies reached a recall of 85% and 83% in PubMed and Embase, respectively. Optimization led to recall of ≥95% with specificity, precision, and NNR of 49%, 1.3%, and 79 in PubMed and 56%, 0.74%, and 136 in Embase, respectively.Conclusions: We report validated filters to search for systematic reviews, observational studies, and intervention studies in guideline projects in PubMed and Embase.com.

scholarly journals Youth participatory research evidence to inform health policy: a systematic review protocol

BMJ Open ◽  
2020 ◽  
Vol 10 (8) ◽  
pp. e036522
Author(s):  
Janet Njelesani ◽  
Jean Hunleth
Keyword(s):  
Systematic Review ◽  
Health Policy ◽  
Young People ◽  
Participatory Research ◽  
Health Research ◽  
Mixed Methods Research ◽  
Research Evidence ◽  
Evidence Base ◽  
Advisory Group ◽  
Active Dissemination

IntroductionYoung people’s participation in health research produces knowledge that is indispensable for creating appropriate and effective policies. However, how best to disseminate youth participatory research evidence to impact health policy is not known. Therefore, the objectives of this systematic review are to describe the evidence produced through youth participatory research, including the strategies used to disseminate youth participatory research evidence to health policymakers. These are necessary to improve policymakers’ use of youth participatory research evidence and, thereby, make programmes more impactful for young people.Methods and analysisThe meta-narrative methodology will guide the systematic review to highlight the contrasting and complementary evidence on the use of engaging youth in research to affect health policymaking. Relevant studies will be identified by searching electronic databases, including but not limited to EBSCO, PROQUEST, OVID Medline, Sociological Abstracts and Google Scholar from inception to December 2020. The methodological quality of included quantitative, qualitative and mixed-methods research studies will be assessed using valid appraisal tools. The meta-narrative approach to analysis will include identifying meta-narratives of how youth participation informed the health research findings.Ethics and disseminationAn advisory group of young people will advise on the study and dissemination of the findings. As part of our plan for active dissemination, we will produce a policy brief that builds the rationale for using research with and by youth as part of an evidence base necessary for achieving youth health outcomes.

scholarly journals LIt.search: fast tracking access to Aboriginal and Torres Strait Islander health literature

2014 ◽  
Vol 38 (5) ◽  
pp. 541 ◽  
Author(s):  
Jennifer J. Tieman ◽  
Mikaela A. Lawrence ◽  
Raechel A. Damarell ◽  
Ruth M. Sladek ◽  
Arwen Nikolof
Keyword(s):  
Torres Strait Islander ◽  
Relevant Literature ◽  
Recall Rate ◽  
Standard Test ◽  
Advisory Group ◽  
Search Filter ◽  
Fast Tracking ◽  
Health Literature ◽  
Torres Strait ◽  
Torres Strait Islander Health

Objective To develop and validate a PubMed search filter, LIt.search, that automatically retrieves Aboriginal and Torres Strait Islander health literature and to make it publicly accessible through the Lowitja Institute website. Methods Search filter development phases included: (1) scoping of the publication characteristics of Aboriginal and Torres Start Islander literature; (2) advisory group input and review; (3) systematic identification and testing of MeSH and text word terms; (4) relevance assessment of the search filter’s retrieved items; and (5) translation for use in PubMed through the web. Results Scoping study analyses demonstrated complexity in the nature and use of possible search terms and publication characteristics. The search filter achieved a recall rate of 84.8% in the full gold standard test set. To determine real-world performance, post-hoc assessment of items retrieved by the search filter in PubMed was undertaken with 87.2% of articles deemed as relevant. The search filter was constructed as a series of URL hyperlinks to enable one-click searching. Conclusion LIt.search is a search tool that facilitates research into practice for improving outcomes in Aboriginal and Torres Strait Islander health and is publicly available on the Lowitja Institute website. What is known about this topic? Health professionals, researchers and decision makers can find it difficult to retrieve published literature on Aboriginal and Torres Strait Islander health easily, effectively and in a timely way. What does this paper add? This paper describes a new web-based searching tool, LIt.search, which facilitates access to the relevant literature. What are the implications for practice? Ready access to published literature on Aboriginal and Torres Strait Islander health reduces a barrier to the use of this evidence in practice. LIt.search encourages the use of this evidence to inform clinical judgement and policy and service decision-making as well as reducing the burdens associated with searching for community practitioners, academics and policy makers.

scholarly journals Development and validation of search filters to find articles on palliative care in bibliographic databases

2019 ◽  
Vol 33 (4) ◽  
pp. 470-474 ◽  
Author(s):  
Judith AC Rietjens ◽  
Wichor M Bramer ◽  
Eric CT Geijteman ◽  
Agnes van der Heide ◽  
Wendy H Oldenmenger
Keyword(s):  
Palliative Care ◽  
Cochrane Library ◽  
Bibliographic Databases ◽  
Search Filter ◽  
Ovid Medline ◽  
Online Databases ◽  
Literature Searches ◽  
Search Filters ◽  
Validation Set ◽  
Development And Validation

Background: Healthcare professionals and researchers in the field of palliative care often have difficulties finding relevant articles in online databases. Standardized search filters may help improve the efficiency and quality of such searches, but prior developed filters showed only moderate performance. Aim: To develop and validate a specific search filter and a sensitive search filter for the field of palliative care. Design: We used a novel, objective method for search filter development. First, we created a gold standard set. This set was split into three groups: term identification, filter development, and filter validation set. After creating the filters in PubMed, we translated the filters into search filters for Ovid MEDLINE, Embase, CINAHL, PsychINFO, and Cochrane Library. We calculated specificity, sensitivity and precision of both filters. Results: The specific filter had a specificity of 97.4%, a sensitivity of 93.7%, and a precision of 45%. The sensitive filter had a sensitivity of 99.6%, a specificity of 92.5%, and a precision of 5%. Conclusion: Our search filters can support literature searches in the field of palliative care. Our specific filter retrieves 93.7% of relevant articles, while 45% of the retrieved articles are relevant. This filter can be used to find answers to questions when time is limited. Our sensitive filter finds 99.6% of all relevant articles and may, for instance, help conducting systematic reviews. Both filters perform better than prior developed search filters in the field of palliative care.

scholarly journals Assessing the performance of methodological search filters to improve the efficiency of evidence information retrieval: five literature reviews and a qualitative study

2017 ◽  
Vol 21 (69) ◽  
pp. 1-148 ◽  
Author(s):  
Carol Lefebvre ◽  
Julie Glanville ◽  
Sophie Beale ◽  
Charles Boachie ◽  
Steven Duffy ◽  
...  
Keyword(s):  
Decision Making ◽  
Performance Measures ◽  
Diagnostic Tests ◽  
Reference Standard ◽  
Performance Information ◽  
Search Filter ◽  
Filter Performance ◽  
Literature Reviews ◽  
Search Filters ◽  
Study Designs

BackgroundEffective study identification is essential for conducting health research, developing clinical guidance and health policy and supporting health-care decision-making. Methodological search filters (combinations of search terms to capture a specific study design) can assist in searching to achieve this.ObjectivesThis project investigated the methods used to assess the performance of methodological search filters, the information that searchers require when choosing search filters and how that information could be better provided.MethodsFive literature reviews were undertaken in 2010/11: search filter development and testing; comparison of search filters; decision-making in choosing search filters; diagnostic test accuracy (DTA) study methods; and decision-making in choosing diagnostic tests. We conducted interviews and a questionnaire with experienced searchers to learn what information assists in the choice of search filters and how filters are used. These investigations informed the development of various approaches to gathering and reporting search filter performance data. We acknowledge that there has been a regrettable delay between carrying out the project, including the searches, and the publication of this report, because of serious illness of the principal investigator.ResultsThe development of filters most frequently involved using a reference standard derived from hand-searching journals. Most filters were validated internally only. Reporting of methods was generally poor. Sensitivity, precision and specificity were the most commonly reported performance measures and were presented in tables. Aspects of DTA study methods are applicable to search filters, particularly in the development of the reference standard. There is limited evidence on how clinicians choose between diagnostic tests. No published literature was found on how searchers select filters. Interviewing and questioning searchers via a questionnaire found that filters were not appropriate for all tasks but were predominantly used to reduce large numbers of retrieved records and to introduce focus. The Inter Technology Appraisal Support Collaboration (InterTASC) Information Specialists’ Sub-Group (ISSG) Search Filters Resource was most frequently mentioned by both groups as the resource consulted to select a filter. Randomised controlled trial (RCT) and systematic review filters, in particular the Cochrane RCT and the McMaster Hedges filters, were most frequently mentioned. The majority indicated that they used different filters depending on the requirement for sensitivity or precision. Over half of the respondents used the filters available in databases. Interviewees used various approaches when using and adapting search filters. Respondents suggested that the main factors that would make choosing a filter easier were the availability of critical appraisals and more detailed performance information. Provenance and having the filter available in a central storage location were also important.LimitationsThe questionnaire could have been shorter and could have included more multiple choice questions, and the reviews of filter performance focused on only four study designs.ConclusionsSearch filter studies should use a representative reference standard and explicitly report methods and results. Performance measures should be presented systematically and clearly. Searchers find filters useful in certain circumstances but expressed a need for more user-friendly performance information to aid filter choice. We suggest approaches to use, adapt and report search filter performance. Future work could include research around search filters and performance measures for study designs not addressed here, exploration of alternative methods of displaying performance results and numerical synthesis of performance comparison results.FundingThe National Institute for Health Research (NIHR) Health Technology Assessment programme and Medical Research Council–NIHR Methodology Research Programme (grant number G0901496).

Integrated care: learning from East London

2018 ◽  
Vol 68 (suppl 1) ◽  
pp. bjgp18X697265
Author(s):  
Sonia Bussu
Keyword(s):  
Integrated Care ◽  
Evidence Base ◽  
Community Health Services ◽  
Health Economy ◽  
Local Health ◽  
Frontline Staff ◽  
Care Community ◽  
Organisation Development ◽  
Integrated Care Pathways ◽  
Local Health System

BackgroundDespite a growing body of literature on integrated, there remains a relatively small evidence base to suggest which elements of integrated care are most effective and how to implement them successfully. This might also be due to the fact that policy thinking around integrated care is struggling to translate into organisation change at the point of delivery. Better understanding of patterns of collaborations and integrated pathways is crucial to understand frontline staff’s OD needs and provide adequate support.AimThis paper focuses on the frontline level to assess progress towards integrated care in East London.MethodWe use admission avoidance (Rapid Response service) and discharge services (Discharge to Assess) as a lens to examine how frontline staff from secondary care, community health services and social service work together to deliver more integrated care. The study uses the Researcher in Residence (RiR), where the researcher is embedded in the in the organisations she is evaluating, as a key member of the delivery team.ResultsInitial findings suggest that while work on integrated care has enabled some level of collaborative working at strategic levels in partner organisations, on the frontline professionals are grappling with issues such as professional identity, professional boundaries, mutual trust and accountability, as new services and roles struggle to be fully embedded within the local health system.ConclusionThe paper sheds light on to the complexity on integrated care at the point of delivery. Better understanding of integrated care pathways is crucial to evidence patterns of collaboration across organisations; assess how these new roles and teams are embedding themselves within the local health economy; identify organisation development needs; and provide adequate support to frontline staff.

scholarly journals Development and validation of the Sorting non-trauMatIc adoLescent knEe pain (SMILE) tool – a development and initial validation study

2021 ◽  
Vol 19 (1) ◽  
Author(s):  
Clara Guldhammer ◽  
Sinead Holden ◽  
Marina Elmelund Sørensen ◽  
Jens Lykkegaard Olesen ◽  
Martin Bach Jensen ◽  
...  
Keyword(s):  
Knee Pain ◽  
Gold Standard ◽  
Sports Medicine ◽  
Kappa Statistic ◽  
Medical Doctors ◽  
Clinical Tests ◽  
Support Tool ◽  
Expert Clinician ◽  
Diagnostic Flowchart ◽  
Development And Validation

Abstract Background Despite the commonality of adolescent knee pain, there are no tools to support medical doctors to correctly diagnose knee pain. This study aimed to develop and evaluate a support tool for diagnosing the most common types of non-traumatic adolescent knee pain. Method A systematic search on Medline identified the literature on clinical tests and diagnoses of adolescent knee pain. The search was supplemented by textbooks and transformed into a diagnostic flowchart based on onset, symptoms, and pain localisation. This tool was revised based on feedback from general practitioners and experts in sports medicine. The tool was evaluated on two separate days with blinded assessors. Overall, 27 participants (aged 10–17 years) with non-traumatic knee pain were included. All participants were diagnosed by medical doctors or medical students, without and with the use of the tool. Diagnoses were compared to a gold standard (expert clinician). An interview to inform optimisations of the tool was performed with the assessors. Percentage agreement with the gold standard, and Kappa statistic for interrater reliability were calculated. Results The final tool improved diagnostic agreement with the gold standard from 22.7% (95% CI 10.3–35.1) to 77.3% (95% CI 64.9–89.7). Inter-rater reliability increased from poor agreement k = − 0.04 (95% CI, − 0.12-0.04) to moderate agreement k = 0.56 (95% CI, 0.40–0.72). Conclusion This simple diagnostic tool is quick to use and may assist doctors in diagnosing non-traumatic knee pain in adolescents.

Sign in / Sign up

Export Citation Format

Share Document