scholarly journals Translation and validation of PubMed and Embase search filters for identification of systematic reviews, intervention studies, and observational studies in the field of first aid

2021 ◽  
Vol 109 (4) ◽  
Author(s):  
Bert Avau ◽  
Hans Van Remoortel ◽  
Emmy De Buck
Keyword(s):  
Systematic Reviews ◽  
Observational Studies ◽  
Intervention Studies ◽  
First Aid ◽  
Search Filter ◽  
Filter Performance ◽  
Ovid Medline ◽  
Search Filters ◽  
Study Type ◽  
Evidence Summaries

Objective: The aim of this project was to validate search filters for systematic reviews, intervention studies, and observational studies translated from Ovid MEDLINE and Embase syntax and used for searches in PubMed and Embase.com during the development of evidence summaries supporting first aid guidelines. We aimed to achieve a balance among recall, specificity, precision, and number needed to read (NNR).Methods: Reference gold standards were constructed per study type derived from existing evidence summaries. Search filter performance was assessed through retrospective searches and measurement of relative recall, specificity, precision, and NNR when using the translated search filters. Where necessary, search filters were optimized. Adapted filters were validated in separate validation gold standards.Results: Search filters for systematic reviews and observational studies reached recall of ≥85% in both PubMed and Embase. Corresponding specificities for systematic review filters were ≥96% in both databases, with a precision of 9.7% (NNR 10) in PubMed and 5.4% (NNR 19) in Embase. For observational study filters, specificity, precision, and NNR were 68%, 2%, and 51 in PubMed and 47%, 0.8%, and 123 in Embase, respectively. These filters were considered sufficiently effective. Search filters for intervention studies reached a recall of 85% and 83% in PubMed and Embase, respectively. Optimization led to recall of ≥95% with specificity, precision, and NNR of 49%, 1.3%, and 79 in PubMed and 56%, 0.74%, and 136 in Embase, respectively.Conclusions: We report validated filters to search for systematic reviews, observational studies, and intervention studies in guideline projects in PubMed and Embase.com.

scholarly journals A Mixed-Methods Approach to Synthesizing Evidence on Mediators of Intervention Effects

2011 ◽  
Vol 33 (7) ◽  
pp. 870-900 ◽  
Author(s):  
Jennifer Leeman ◽  
YunKyung Chang ◽  
Corrine I. Voils ◽  
Jamie L. Crandell ◽  
Margarete Sandelowski
Keyword(s):  
Mixed Methods ◽  
Systematic Reviews ◽  
Behavioral Change ◽  
Observational Studies ◽  
Intervention Studies ◽  
Meta Analysis ◽  
Intervention Effects ◽  
Mediation Effects ◽  
Mixed Methods Approach ◽  
Using Data

Greater understanding of the mechanisms (mediators) by which behavioral-change interventions work is critical to developing theory and refining interventions. Although systematic reviews have been advocated as a method for exploring mediators, this is rarely done. One challenge is that intervention researchers typically test only two paths of the mediational model: the effect of the intervention on mediators and on outcomes. The authors addressed this challenge by drawing information not only from intervention studies but also from observational studies that provide data on associations between potential mediators and outcomes. They also reviewed qualitative studies of participants’ perceptions of why and how interventions worked. Using data from intervention ( n = 37) and quantitative observational studies ( n = 55), the authors conducted a meta-analysis of the mediation effects of eight variables. Qualitative findings ( n = 6) contributed to more in-depth explanations for findings. The methods used have potential to contribute to understanding of core mechanisms of behavioral-change interventions.

scholarly journals Integrated Care Search: Development and validation of a PubMed search filter for retrieving the integrated care research evidence

2020 ◽  
Author(s):  
Raechel A. Damarell ◽  
Suzanne Lewis ◽  
Camilla Trenerry ◽  
Jennifer J. Tieman
Keyword(s):  
Integrated Care ◽  
Gold Standard ◽  
Research Evidence ◽  
Evidence Base ◽  
Standard Test ◽  
Advisory Group ◽  
Search Filter ◽  
Filter Performance ◽  
Search Filters ◽  
Development And Validation

Abstract Background Integrated care is an increasingly important principle for organising healthcare. Integrated care models show promise in reducing resource wastage and service fragmentation whilst improving the accessibility, patient-centredness and quality of care for patients. Those needing reliable access to the growing research evidence base for integrated care can be frustrated by search challenges reflective of the topic's complexity. The aim of this study is to report the empirical development and validation of two search filters for rapid and effective retrieval of integrated care evidence in PubMed. One filter is optimised for recall and the other for precision.Methods An Expert Advisory Group comprising international integrated care experts guided the study. A gold standard test set of citations was formed from screening Handbook Integrated Care chapter references for relevance. This set was divided into a Term Identification Set (20%) for determining candidate terms using frequency analysis; a Filter Development Set (40%) for testing performance of term combinations; and a Filter Validation Set (40%) reserved for confirming final filter performance. In developing the high recall filter, recall was steadily increased while maintaining precision at ≥ 50%. Similarly, the high precision filter sought to maximise precision while keeping recall ≥ 50%. For each term combination tested, an approximation of precision was obtained by reviewing the first 100 citations retrieved in Medline for relevance.Results The gold standard set comprised 534 citations. The search filter optimised for recall ('Broad Integrated Care Search') achieved 86.0%-88.3% recall with corresponding low precision (47%-53%). The search filter optimised for precise searching ('Narrow Integrated Care Search') demonstrated precision of 73%-95% with recall reduced to between 55.9% and 59.8%. These filters are now available as one-click URL hyperlinks in the website of International Foundation for Integrated Care.Conclusions The Broad and Narrow Integrated Care Search filters provide potential users, such as policy makers and researchers, seamless, reliable and ongoing access to integrated care evidence for decision making. These filters were developed according to a rigorous and transparent methodology designed to circumvent the challenges of information retrieval posed by this complex, multifaceted topic.

scholarly journals Integrated Care Search: Development and validation of a PubMed search filter for retrieving the integrated care research evidence

2019 ◽  
Author(s):  
Raechel A. Damarell ◽  
Suzanne Lewis ◽  
Camilla Trenerry ◽  
Jennifer J. Tieman
Keyword(s):  
Integrated Care ◽  
Gold Standard ◽  
Research Evidence ◽  
Evidence Base ◽  
Standard Test ◽  
Advisory Group ◽  
Search Filter ◽  
Filter Performance ◽  
Search Filters ◽  
Development And Validation

Abstract Background Integrated care is an increasingly important principle for organising healthcare. Integrated care models show promise in reducing resource wastage and service fragmentation whilst improving the accessibility, patient-centredness and quality of care for patients. Those needing reliable access to the growing research evidence base for integrated care can be frustrated by search challenges reflective of the topic's complexity. The aim of this study is to report the empirical development and validation of two search filters for rapid and effective retrieval of integrated care evidence in PubMed. One filter is optimised for recall and the other for precision.Methods An Expert Advisory Group comprising international integrated care experts guided the study. A gold standard test set of citations was formed from screening Handbook Integrated Care chapter references for relevance. This set was divided into a Term Identification Set (20%) for determining candidate terms using frequency analysis; a Filter Development Set (40%) for testing performance of term combinations; and a Filter Validation Set (40%) reserved for confirming final filter performance. In developing the high recall filter, recall was steadily increased while maintaining precision at ≥ 50%. Similarly, the high precision filter sought to maximise precision while keeping recall ≥ 50%. For each term combination tested, an approximation of precision was obtained by reviewing the first 100 citations retrieved in Medline for relevance.Results The gold standard set comprised 534 citations. The search filter optimised for recall ('Broad Integrated Care Search') achieved 86.0%-88.3% recall with corresponding low precision (47%-53%). The search filter optimised for precise searching ('Narrow Integrated Care Search') demonstrated precision of 73%-95% with recall reduced to between 55.9% and 59.8%. These filters are now available as one-click URL hyperlinks in the website of International Foundation for Integrated Care.Conclusions The Broad and Narrow Integrated Care Search filters provide potential users, such as policy makers and researchers, seamless, reliable and ongoing access to integrated care evidence for decision making. These filters were developed according to a rigorous and transparent methodology designed to circumvent the challenges of information retrieval posed by this complex, multifaceted topic.

scholarly journals A search filter for increasing the retrieval of animal studies in Embase

2011 ◽  
Vol 45 (4) ◽  
pp. 268-270 ◽  
Author(s):  
Rob B M de Vries ◽  
Carlijn R Hooijmans ◽  
Alice Tillema ◽  
Marlies Leenaars ◽  
Merel Ritskes-Hoitinga
Keyword(s):  
Systematic Reviews ◽  
Animal Studies ◽  
Research Question ◽  
Animal Experiments ◽  
Search Process ◽  
General Requirement ◽  
Search Filter ◽  
Search Filters ◽  
Similar Search

Collecting and analysing all available literature before starting a new animal experiment is important and it is indispensable when writing systematic reviews of animal research. In practice, finding all animal studies relevant to a specific research question turns out to be anything but simple. In order to facilitate this search process, we previously developed a search filter for retrieving animal studies in the most often used biomedical database, PubMed. It is a general requirement for systematic reviews, however, that at least two databases are searched. In this report, we therefore present a similar search filter for a second important database, namely Embase. We show that our filter retrieves more animal studies than (a combination of) the options currently available in Embase. Our search filters for PubMed and Embase therefore represent valuable tools for improving the quality of (systematic) reviews and thereby of new animal experiments.

scholarly journals Laboratory animals search filter for different literature databases: PubMed, Embase, Web of Science and PsycINFO

2021 ◽  
pp. 002367722110454
Author(s):  
Stevie van der Mierden ◽  
Carlijn R Hooijmans ◽  
Alice HJ Tillema ◽  
Simone Rehn ◽  
André Bleich ◽  
...  
Keyword(s):  
Random Sample ◽  
Systematic Reviews ◽  
Animal Studies ◽  
Web Of Science ◽  
Real Life ◽  
Animal Research ◽  
Laboratory Animals ◽  
Primary Study ◽  
Search Filter ◽  
Search Filters

Systematic reviews are important tools in animal research, but the ever-increasing number of studies makes retrieval of all relevant publications challenging. Search filters aid in retrieving as many animal studies as possible. In this paper we provide updated and expanded versions of the SYRCLE animal filters for PubMed and Embase. We provide the Embase filter for both Embase.com and via Ovid. Furthermore, we provide new animal search filters for Web of Science (WoS) and APA PsycINFO via psycnet.apa.org and via Ovid. Compared with previous versions, the new filters retrieved 0.5–47.1% (19 references for PubMed, 837 for WoS) more references in a real-life example. All filters retrieved additional references, comprising multiple relevant reviews. A random sample from WoS found at least one potentially relevant primary study. These animal search filters facilitate identifying as many animal studies as possible while minimising the number of non-animal studies.

scholarly journals Development and validation of search filters to find articles on palliative care in bibliographic databases

2019 ◽  
Vol 33 (4) ◽  
pp. 470-474 ◽  
Author(s):  
Judith AC Rietjens ◽  
Wichor M Bramer ◽  
Eric CT Geijteman ◽  
Agnes van der Heide ◽  
Wendy H Oldenmenger
Keyword(s):  
Palliative Care ◽  
Cochrane Library ◽  
Bibliographic Databases ◽  
Search Filter ◽  
Ovid Medline ◽  
Online Databases ◽  
Literature Searches ◽  
Search Filters ◽  
Validation Set ◽  
Development And Validation

Background: Healthcare professionals and researchers in the field of palliative care often have difficulties finding relevant articles in online databases. Standardized search filters may help improve the efficiency and quality of such searches, but prior developed filters showed only moderate performance. Aim: To develop and validate a specific search filter and a sensitive search filter for the field of palliative care. Design: We used a novel, objective method for search filter development. First, we created a gold standard set. This set was split into three groups: term identification, filter development, and filter validation set. After creating the filters in PubMed, we translated the filters into search filters for Ovid MEDLINE, Embase, CINAHL, PsychINFO, and Cochrane Library. We calculated specificity, sensitivity and precision of both filters. Results: The specific filter had a specificity of 97.4%, a sensitivity of 93.7%, and a precision of 45%. The sensitive filter had a sensitivity of 99.6%, a specificity of 92.5%, and a precision of 5%. Conclusion: Our search filters can support literature searches in the field of palliative care. Our specific filter retrieves 93.7% of relevant articles, while 45% of the retrieved articles are relevant. This filter can be used to find answers to questions when time is limited. Our sensitive filter finds 99.6% of all relevant articles and may, for instance, help conducting systematic reviews. Both filters perform better than prior developed search filters in the field of palliative care.

scholarly journals VP195 Using The ISSG Search Filter Resource In Health Technology Assessment

2017 ◽  
Vol 33 (S1) ◽  
pp. 240-240
Author(s):  
Kath Wright ◽  
Julie Glanville ◽  
Carol Lefebvre
Keyword(s):  
Randomized Controlled Trials ◽  
Systematic Reviews ◽  
Health Technology ◽  
Controlled Trials ◽  
Health State ◽  
Search Filter ◽  
Utility Values ◽  
Randomized Controlled ◽  
Search Filters ◽  
Health State Utility Values

INTRODUCTION:Information specialists and others searching for Health Technology Assessments (HTAs) can use the ISSG Search Filter resource (SFR) to identify filters to incorporate into search strategies. This can save time and effort when designing searches and create more efficient searches that retrieve fewer and possibly more relevant database records (link available here: https://sites.google.com/a/york.ac.uk/issg-search-filters-resource/home).What are search filters? Search filters are collections of search terms designed to retrieve selections of records from bibliographic databases. Some filters are designed to retrieve records of specific study designs such as randomized controlled trials (RCTs) or systematic reviews; others aim to retrieve records relating to other features or topics such as the age or gender of study participants.Search filters may be designed to be sensitive, precise or balanced between sensitivity and precision.METHODS:When would you use a search filter in HTA? Search filters can be added to search strategies to limit to specific study types, for example, RCTs, mixed methods studies, systematic reviews. They can also be used when searching for other aspects of HTA such as patient views or specific age groups.The ISSG SFR includes sections listing search filters to help identify adverse effects, aetiology, economic evaluations, health state utility values, public views, and quality of life.RESULTS:How are filters used? A search filter is often used in combination with a topic search to restrict the search results to a specific type of record, for example, records reporting health state utility values or records of randomized controlled trials.CONCLUSIONS:Further guidance on the use of search filters can be found in the SuRe Info Search Filters chapter.

PERFORMANCE OF OVID MEDLINE SEARCH FILTERS TO IDENTIFY HEALTH STATE UTILITY STUDIES

2017 ◽  
Vol 33 (4) ◽  
pp. 472-480 ◽  
Author(s):  
Mick Arber ◽  
Sonia Garcia ◽  
Thomas Veale ◽  
Mary Edwards ◽  
Alison Shaw ◽  
...  
Keyword(s):  
High Sensitivity ◽  
Health State Utility ◽  
Health State ◽  
Search Filter ◽  
Medline Search ◽  
Ovid Medline ◽  
Search Filters ◽  
Validation Set ◽  
Health State Utility Values ◽  
Standard Sets

Objectives: This study was designed to assess the sensitivity of three Ovid MEDLINE search filters developed to identify studies reporting health state utility values (HSUVs), to improve the performance of the best performing filter, and to validate resulting search filters.Methods: Three quasi-gold standard sets (QGS1, QGS2, QGS3) of relevant studies were harvested from reviews of studies reporting HSUVs. The performance of three initial filters was assessed by measuring their relative recall of studies in QGS1. The best performing filter was then developed further using QGS2. This resulted in three final search filters (FSF1, FSF2, and FSF3), which were validated using QGS3.Results: FSF1 (sensitivity maximizing) retrieved 132/139 records (sensitivity: 95 percent) in the QGS3 validation set. FSF1 had a number needed to read (NNR) of 842. FSF2 (balancing sensitivity and precision) retrieved 128/139 records (sensitivity: 92 percent) with a NNR of 502. FSF3 (precision maximizing) retrieved 123/139 records (sensitivity: 88 percent) with a NNR of 383.Conclusions: We have developed and validated a search filter (FSF1) to identify studies reporting HSUVs with high sensitivity (95 percent) and two other search filters (FSF2 and FSF3) with reasonably high sensitivity (92 percent and 88 percent) but greater precision, resulting in a lower NNR. These seem to be the first validated filters available for HSUVs. The availability of filters with a range of sensitivity and precision options enables researchers to choose the filter which is most appropriate to the resources available for their specific research.

scholarly journals Integrated Care Search: Development and validation of a PubMed search filter for retrieving the integrated care research evidence

2019 ◽  
Author(s):  
Raechel A. Damarell ◽  
Suzanne Lewis ◽  
Camilla Trenerry ◽  
Jennifer J. Tieman
Keyword(s):  
Integrated Care ◽  
Gold Standard ◽  
Research Evidence ◽  
Evidence Base ◽  
Standard Test ◽  
Advisory Group ◽  
Search Filter ◽  
Filter Performance ◽  
Search Filters ◽  
Development And Validation

Abstract Background Integrated care is an increasingly important principle for organising healthcare. Integrated care models show promise in reducing resource wastage and service fragmentation whilst improving the accessibility, patient-centredness and quality of care for patients. Those needing reliable access to the growing research evidence base for integrated care can be frustrated by search challenges reflective of the topic's complexity. The aim of this study is to report the empirical development and validation of two search filters for rapid and effective retrieval of integrated care evidence in PubMed. One filter is optimised for recall and the other for precision.Methods An Expert Advisory Group comprising international integrated care experts guided the study. A gold standard test set of citations was formed from screening Handbook Integrated Care chapter references for relevance. This set was divided into a Term Identification Set (20%) for determining candidate terms using frequency analysis; a Filter Development Set (40%) for testing performance of term combinations; and a Filter Validation Set (40%) reserved for confirming final filter performance. In developing the high recall filter, recall was steadily increased while maintaining precision at ≥ 50%. Similarly, the high precision filter sought to maximise precision while keeping recall ≥ 50%. For each term combination tested, an approximation of precision was obtained by reviewing the first 100 citations retrieved in Medline for relevance.Results The gold standard set comprised 534 citations. The search filter optimised for recall ('Broad Integrated Care Search') achieved 86.0%-88.3% recall with corresponding low precision (47%-53%). The search filter optimised for precise searching ('Narrow Integrated Care Search') demonstrated precision of 73%-95% with recall reduced to between 55.9% and 59.8%. These filters are now available as one-click URL hyperlinks in the website of International Foundation for Integrated Care.Conclusions The Broad and Narrow Integrated Care Search filters provide potential users, such as policy makers and researchers, seamless, reliable and ongoing access to integrated care evidence for decision making. These filters were developed according to a rigorous and transparent methodology designed to circumvent the challenges of information retrieval posed by this complex, multifaceted topic.

scholarly journals Assessing the performance of methodological search filters to improve the efficiency of evidence information retrieval: five literature reviews and a qualitative study

2017 ◽  
Vol 21 (69) ◽  
pp. 1-148 ◽  
Author(s):  
Carol Lefebvre ◽  
Julie Glanville ◽  
Sophie Beale ◽  
Charles Boachie ◽  
Steven Duffy ◽  
...  
Keyword(s):  
Decision Making ◽  
Performance Measures ◽  
Diagnostic Tests ◽  
Reference Standard ◽  
Performance Information ◽  
Search Filter ◽  
Filter Performance ◽  
Literature Reviews ◽  
Search Filters ◽  
Study Designs

BackgroundEffective study identification is essential for conducting health research, developing clinical guidance and health policy and supporting health-care decision-making. Methodological search filters (combinations of search terms to capture a specific study design) can assist in searching to achieve this.ObjectivesThis project investigated the methods used to assess the performance of methodological search filters, the information that searchers require when choosing search filters and how that information could be better provided.MethodsFive literature reviews were undertaken in 2010/11: search filter development and testing; comparison of search filters; decision-making in choosing search filters; diagnostic test accuracy (DTA) study methods; and decision-making in choosing diagnostic tests. We conducted interviews and a questionnaire with experienced searchers to learn what information assists in the choice of search filters and how filters are used. These investigations informed the development of various approaches to gathering and reporting search filter performance data. We acknowledge that there has been a regrettable delay between carrying out the project, including the searches, and the publication of this report, because of serious illness of the principal investigator.ResultsThe development of filters most frequently involved using a reference standard derived from hand-searching journals. Most filters were validated internally only. Reporting of methods was generally poor. Sensitivity, precision and specificity were the most commonly reported performance measures and were presented in tables. Aspects of DTA study methods are applicable to search filters, particularly in the development of the reference standard. There is limited evidence on how clinicians choose between diagnostic tests. No published literature was found on how searchers select filters. Interviewing and questioning searchers via a questionnaire found that filters were not appropriate for all tasks but were predominantly used to reduce large numbers of retrieved records and to introduce focus. The Inter Technology Appraisal Support Collaboration (InterTASC) Information Specialists’ Sub-Group (ISSG) Search Filters Resource was most frequently mentioned by both groups as the resource consulted to select a filter. Randomised controlled trial (RCT) and systematic review filters, in particular the Cochrane RCT and the McMaster Hedges filters, were most frequently mentioned. The majority indicated that they used different filters depending on the requirement for sensitivity or precision. Over half of the respondents used the filters available in databases. Interviewees used various approaches when using and adapting search filters. Respondents suggested that the main factors that would make choosing a filter easier were the availability of critical appraisals and more detailed performance information. Provenance and having the filter available in a central storage location were also important.LimitationsThe questionnaire could have been shorter and could have included more multiple choice questions, and the reviews of filter performance focused on only four study designs.ConclusionsSearch filter studies should use a representative reference standard and explicitly report methods and results. Performance measures should be presented systematically and clearly. Searchers find filters useful in certain circumstances but expressed a need for more user-friendly performance information to aid filter choice. We suggest approaches to use, adapt and report search filter performance. Future work could include research around search filters and performance measures for study designs not addressed here, exploration of alternative methods of displaying performance results and numerical synthesis of performance comparison results.FundingThe National Institute for Health Research (NIHR) Health Technology Assessment programme and Medical Research Council–NIHR Methodology Research Programme (grant number G0901496).

Sign in / Sign up

Export Citation Format

Share Document