The NICE MEDLINE and Embase (Ovid) health apps search filters: development of validated filters to retrieve evidence about health apps

2020 ◽  
pp. 1-7
Author(s):  
Lynda Ayiku ◽  
Thomas Hudson ◽  
Sarah Glover ◽  
Nicola Walsh ◽  
Rachel Adams ◽  
...  
Keyword(s):  
Research Literature ◽  
Hand Search ◽  
Health Apps ◽  
Search Terms ◽  
Literature Searches ◽  
Search Filters ◽  
Information Professionals ◽  
Validation Set ◽  
Development And Validation ◽  
Selection Of

Abstract Objectives Health apps are software programs that are designed to prevent, diagnose, monitor, or manage conditions. Inconsistent terminology for apps is used in research literature and bibliographic database subject headings. It can therefore be challenging to retrieve evidence about them in literature searches. Information specialists at the United Kingdom's National Institute for Health and Care Excellence (NICE) have developed novel validated search filters to retrieve evidence about apps from MEDLINE and Embase (Ovid). Methods A selection of medical informatics journals was hand searched to identify a “gold standard” (GS) set of references about apps. The GS set was divided into a development and validation set. The filters’ search terms were derived from and tested against the development set. An external development set containing app references from published NICE products was also used to inform the development of the filters. The filters were then validated using the validation set. Target recall was >90 percent. The filters’ overall recall, specificity, and precision were calculated using all the references identified from the hand search. Results Both filters achieved 98.6 percent recall against their validation sets. Overall, the MEDLINE filter had 98.8 percent recall, 71.3 percent specificity, and 22.6 percent precision. The Embase filter had 98.6 percent recall, 74.9 percent specificity, and 24.5 percent precision. Conclusions The NICE health apps search filters retrieve evidence about apps from MEDLINE and Embase with high recall. They can be applied to literature searches to retrieve evidence about the interventions by information professionals, researchers, and clinicians.

OP128 Improving Literature Searching For Evidence On Health Apps: The National Institute For Health And Care Excellence (NICE) MEDLINE And Embase (Ovid) Health Apps Search Filters

2021 ◽  
Vol 37 (S1) ◽  
pp. 2-2
Author(s):  
Lynda Ayiku ◽  
Sarah Glover
Keyword(s):  
Gold Standard ◽  
Search Strategies ◽  
Bibliographic Databases ◽  
Literature Searching ◽  
Health Apps ◽  
Search Terms ◽  
Search Filters ◽  
Standard Set ◽  
Validation Set

IntroductionLiterature searching for evidence on apps in bibliographic databases is challenging because they are often described with inconsistent terminology. Information Specialists from the United Kingdom's National Institute for Health and Care Excellence (NICE) have developed validated search filters for retrieving evidence about apps from MEDLINE and Embase (Ovid) reliably.MethodsMedical informatics journals were hand-searched to create a ‘gold standard’ set of app references. The gold standard set was divided into two sets. The development set provided the search terms for the filters. The filters were validated by calculating their recall against the validation set. Target recall was >90%.A case study was then conducted to compare the number-needed-to-read (NNR) of the filters with previous non-validated MEDLINE and Embase app search strategies used for the ‘MIB214 myCOPD app’ NICE topic. NNR is the number of references screened to find each relevant reference.ResultsThe MEDLINE and Embase filters achieved 98.6 percent and 98.5 percent recall against the validation set, respectively. In the case study they achieved 100 percent recall, reducing NNR from 348 to 147 in MEDLINE and from 456 to 271 in Embase.ConclusionsThe novel NICE health apps search filters retrieve evidence on apps from MEDLINE and Embase effectively and more efficiently than previous non-validated search strategies used at NICE.

scholarly journals Development and validation of search filters to find articles on palliative care in bibliographic databases

2019 ◽  
Vol 33 (4) ◽  
pp. 470-474 ◽  
Author(s):  
Judith AC Rietjens ◽  
Wichor M Bramer ◽  
Eric CT Geijteman ◽  
Agnes van der Heide ◽  
Wendy H Oldenmenger
Keyword(s):  
Palliative Care ◽  
Cochrane Library ◽  
Bibliographic Databases ◽  
Search Filter ◽  
Ovid Medline ◽  
Online Databases ◽  
Literature Searches ◽  
Search Filters ◽  
Validation Set ◽  
Development And Validation

Background: Healthcare professionals and researchers in the field of palliative care often have difficulties finding relevant articles in online databases. Standardized search filters may help improve the efficiency and quality of such searches, but prior developed filters showed only moderate performance. Aim: To develop and validate a specific search filter and a sensitive search filter for the field of palliative care. Design: We used a novel, objective method for search filter development. First, we created a gold standard set. This set was split into three groups: term identification, filter development, and filter validation set. After creating the filters in PubMed, we translated the filters into search filters for Ovid MEDLINE, Embase, CINAHL, PsychINFO, and Cochrane Library. We calculated specificity, sensitivity and precision of both filters. Results: The specific filter had a specificity of 97.4%, a sensitivity of 93.7%, and a precision of 45%. The sensitive filter had a sensitivity of 99.6%, a specificity of 92.5%, and a precision of 5%. Conclusion: Our search filters can support literature searches in the field of palliative care. Our specific filter retrieves 93.7% of relevant articles, while 45% of the retrieved articles are relevant. This filter can be used to find answers to questions when time is limited. Our sensitive filter finds 99.6% of all relevant articles and may, for instance, help conducting systematic reviews. Both filters perform better than prior developed search filters in the field of palliative care.

scholarly journals The use of the Rotterdam Transition Profile: 10 years in review

2018 ◽  
Vol 1 (1) ◽  
Author(s):  
Sofía Zhang-Jiang ◽  
Jan Willem Gorter
Keyword(s):  
Young Adults ◽  
Developmental Stages ◽  
Grey Literature ◽  
Transition Process ◽  
Clinical Settings ◽  
Hand Search ◽  
Search Terms ◽  
Development And Validation ◽  
Google Search ◽  
Transition Profile

AbstractThe aim of this literature review was to describe the use and utility of the Rotterdam Transition Profile (RTP), as we near the tenth year since its publication in 2008. It is a tool to summarize a youth’s transition process to adulthood by classifying various domains of participation and health care into developmental stages. This review provided a comprehensive synthesis of available knowledge on the RTP by summarizing published and grey literature. Using search terms related to transition, social participation, and questionnaire, a systematic search was conducted for literature up to November 2017 in MEDLINE and Embase databases, and was supplemented with a hand-search using Google Scholar and a general internet search using Google Search. Inclusion criteria were specified to determine the papers selected for review, yielding fifty-five materials for detailed review. Combined analysis of published and grey literature identified nine papers that used some form of the RTP as a measure, 18 materials that cited the RTP or its domains but did not employ it as a measure, and twenty-eight materials that cited the original RTP development and validation paper for information not directly related to the RTP. The literature demonstrated that the RTP seems to be a useful tool to describe and monitor the transition process of adolescents and young adults, in both research and clinical settings. While it has been used with youth of 14–31 years of age across genders and health conditions, more evidence is needed to demonstrate its psychometric properties beyond construct validity in young adults with cerebral palsy.

Medical Literature Searches -How Many Bibliographic Databases are Needed for Sufficient Retrieval in Medical Topics?

1985 ◽  
Vol 24 (03) ◽  
pp. 163-165 ◽  
Author(s):  
K. John
Keyword(s):  
Medical Literature ◽  
Cost Effective ◽  
Medical Faculty ◽  
Bibliographic Databases ◽  
Literature Searches ◽  
Selection Of

SummaryAs many bibliographic services in medicine are offered, literature searches in eight databases at DIMDI were performed to find out which database is most important in medicine. The distribution of publications from members of the medical faculty of Frankfurt University was examined. No save prediction is possible as to which database will yield most articles. Overlapping from different databases is often rather low. The selection of an appropriate database mix for sufficient recall and in a cost-effective manner.is a task for an experienced searcher.

scholarly journals Successful incorporation of single reviewer assessments during systematic review screening: development and validation of sensitivity and work-saved of an algorithm that considers exclusion criteria and count

2021 ◽  
Vol 10 (1) ◽  
Author(s):  
Nassr Nama ◽  
Mirna Hennawy ◽  
Nick Barrowman ◽  
Katie O’Hearn ◽  
Margaret Sampson ◽  
...  
Keyword(s):  
Systematic Review ◽  
A Priori ◽  
Research Centre ◽  
Exclusion Criteria ◽  
Conference Abstract ◽  
Pediatric Health Care ◽  
Average Loss ◽  
Validation Set ◽  
Abstract Screening ◽  
Development And Validation

Abstract Background Accepted systematic review (SR) methodology requires citation screening by two reviewers to maximise retrieval of eligible studies. We hypothesized that records could be excluded by a single reviewer without loss of sensitivity in two conditions; the record was ineligible for multiple reasons, or the record was ineligible for one or more specific reasons that could be reliably assessed. Methods Twenty-four SRs performed at CHEO, a pediatric health care and research centre in Ottawa, Canada, were divided into derivation and validation sets. Exclusion criteria during abstract screening were sorted into 11 specific categories, with loss in sensitivity determined by individual category and by number of exclusion criteria endorsed. Five single reviewer algorithms that combined individual categories and multiple exclusion criteria were then tested on the derivation and validation sets, with success defined a priori as less than 5% loss of sensitivity. Results The 24 SRs included 930 eligible and 27390 ineligible citations. The reviews were mostly focused on pediatrics (70.8%, N=17/24), but covered various specialties. Using a single reviewer to exclude any citation led to an average loss of sensitivity of 8.6% (95%CI, 6.0–12.1%). Excluding citations with ≥2 exclusion criteria led to 1.2% average loss of sensitivity (95%CI, 0.5–3.1%). Five specific exclusion criteria performed with perfect sensitivity: conference abstract, ineligible age group, case report/series, not human research, and review article. In the derivation set, the five algorithms achieved a loss of sensitivity ranging from 0.0 to 1.9% and work-saved ranging from 14.8 to 39.1%. In the validation set, the loss of sensitivity for all 5 algorithms remained below 2.6%, with work-saved between 10.5% and 48.2%. Conclusions Findings suggest that targeted application of single-reviewer screening, considering both type and number of exclusion criteria, could retain sensitivity and significantly decrease workload. Further research is required to investigate the potential for combining this approach with crowdsourcing or machine learning methodologies.

scholarly journals Scoping review: Development and assessment of evaluation frameworks of mobile health apps for recommendations to consumers

2021 ◽  
Author(s):  
Martin Hensher ◽  
Paul Cooper ◽  
Sithara Wanni Arachchige Dona ◽  
Mary Rose Angeles ◽  
Dieu Nguyen ◽  
...  
Keyword(s):  
Full Range ◽  
Evaluation Framework ◽  
Bibliographic Databases ◽  
Health Organizations ◽  
Health Apps ◽  
Search Terms ◽  
Evaluative Criteria ◽  
Mobile Health Apps ◽  
Threshold Filter ◽  
Individual Clinician

Abstract Objective The study sought to review the different assessment items that have been used within existing health app evaluation frameworks aimed at individual, clinician, or organizational users, and to analyze the scoring and evaluation methods used in these frameworks. Materials and Methods We searched multiple bibliographic databases and conducted backward searches of reference lists, using search terms that were synonyms of “health apps,” “evaluation,” and “frameworks.” The review covered publications from 2011 to April 2020. Studies on health app evaluation frameworks and studies that elaborated on the scaling and scoring mechanisms applied in such frameworks were included. Results Ten common domains were identified across general health app evaluation frameworks. A list of 430 assessment criteria was compiled across 97 identified studies. The most frequently used scaling mechanism was a 5-point Likert scale. Most studies have adopted summary statistics to generate the total scoring of each app, and the most popular approach taken was the calculation of mean or average scores. Other frameworks did not use any scaling or scoring mechanism and adopted criteria-based, pictorial, or descriptive approaches, or “threshold” filter. Discussion There is wide variance in the approaches to evaluating health apps within published frameworks, and this variance leads to ongoing uncertainty in how to evaluate health apps. Conclusions A new evaluation framework is needed that can integrate the full range of evaluative criteria within one structure, and provide summative guidance on health app rating, to support individual app users, clinicians, and health organizations in choosing or recommending the best health app.

Development of a Performance-Related Framework for Asphalt Mixture Design for the Illinois Tollway

2021 ◽  
pp. 036119812110148
Author(s):  
Behnam Jahangiri ◽  
Punyaslok Rath ◽  
Hamed Majidifard ◽  
William G. Buttlar
Keyword(s):  
Asphalt Mixture ◽  
Field Performance ◽  
Compact Tension ◽  
T Test ◽  
Novel Approach ◽  
Final Adjustment ◽  
Development And Validation ◽  
High Degree ◽  
Selection Of

Various agencies have begun to research and introduce performance-related specifications (PRS) for the design of modern asphalt paving mixtures. The focus of most recent studies has been directed toward simplified cracking test development and evaluation. In some cases, development and validation of PRS has been performed, building on these new tests, often by comparison of test values to accelerated pavement test studies and/or to limited field data. This study describes the findings of a comprehensive research project conducted at Illinois Tollway, leading to a PRS for the design of mainline and shoulder asphalt mixtures. A novel approach was developed, involving the systematic establishment of specification requirements based on: 1) selection of baseline values based on minimally acceptable field performance thresholds; 2) elevation of thresholds to account for differences between short-term lab aging and expected long-term field aging; 3) further elevation of thresholds to account for variability in lab testing, plus variability in the testing of field cores; and 4) final adjustment and rounding of thresholds based on a consensus process. After a thorough evaluation of different candidate cracking tests in the course of the project, the Disk-shaped Compact Tension—DC(T)—test was chosen to be retained in the Illinois Tollway PRS and to be presented in this study for the design of crack-resistant mixtures. The DC(T) test was selected because of its high degree of correlation with field results and its excellent repeatability. Tailored Hamburg rut depth and stripping inflection point thresholds were also established for mainline and shoulder mixes.

scholarly journals Development and validation of a sequential two-step algorithm for the screening of individuals with potential polycythaemia vera

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Miguel Piris-Villaespesa ◽  
Alberto Álvarez-Larrán ◽  
Adolfo Saez-Marín ◽  
Claudia Nuñez-Torrón ◽  
Gloria Muñoz-Martin ◽  
...  
Keyword(s):  
Mutation Screening ◽  
Roc Curves ◽  
Polycythaemia Vera ◽  
Normal Ranges ◽  
Screening Algorithm ◽  
Number Of Patients ◽  
Step Algorithm ◽  
Development And Validation ◽  
Diagnostic Criterion ◽  
Selection Of

AbstractIn 2016, the WHO included haemoglobin values within normal ranges as a diagnostic criterion for Polycythaemia Vera (PV). Since then, concerns have arisen that a large number of patients are undergoing unnecessary screening for PV. To address this issue, we estimated the prevalence of JAK2 p.V617F in individuals with elevated haemoglobin or haematocrit and developed and validated a screening algorithm for PV. A total of 15,366 blood counts performed in seven non-consecutive days were reviewed, of which 1001 were selected for subsequent JAK2 p.V617F mutation screening. Eight (0.8%) new JAK2 p.V617F-mutated cases were detected. From ROC curves, a two-step algorithm was developed based on the optimal cut-off for the detection of the JAK2 p.V617F mutation. The algorithm was prospectively validated in an independent cohort of 15,298 blood counts. A total of 1595 (10.4%) cases met the criterion for haemoglobin or haematocrit, of whom 581 passed to step 2 (3.8% of the total). The JAK2 p.V617F mutation was detected in 7 of the 501 patients tested, which accounts for 0.04% of the total cohort and 0.4% of patients with erythrocytosis. In conclusion, this data show that our two-step algorithm improves the selection of candidates for JAK2 p.V617F testing.

scholarly journals Development and validation for research assessment of Oncotype DX® Breast Recurrence Score, EndoPredict® and Prosigna®

2021 ◽  
Vol 7 (1) ◽  
Author(s):  
Richard Buus ◽  
Zsolt Szijgyarto ◽  
Eugene F. Schuster ◽  
Hui Xiao ◽  
Ben P. Haynes ◽  
...  
Keyword(s):  
Breast Cancer ◽  
Gene Expression ◽  
Early Stage ◽  
Recurrence Score ◽  
Distant Recurrence ◽  
Research Assessment ◽  
Expression Data ◽  
Her2 Breast Cancer ◽  
Validation Set ◽  
Development And Validation

AbstractMulti-gene prognostic signatures including the Oncotype® DX Recurrence Score (RS), EndoPredict® (EP) and Prosigna® (Risk Of Recurrence, ROR) are widely used to predict the likelihood of distant recurrence in patients with oestrogen-receptor-positive (ER+), HER2-negative breast cancer. Here, we describe the development and validation of methods to recapitulate RS, EP and ROR scores from NanoString expression data. RNA was available from 107 tumours from postmenopausal women with early-stage, ER+, HER2− breast cancer from the translational Arimidex, Tamoxifen, Alone or in Combination study (TransATAC) where previously these signatures had been assessed with commercial methodology. Gene expression was measured using NanoString nCounter. For RS and EP, conversion factors to adjust for cross-platform variation were estimated using linear regression. For ROR, the steps to perform subgroup-specific normalisation of the gene expression data and calibration factors to calculate the 46-gene ROR score were assessed and verified. Training with bootstrapping (n = 59) was followed by validation (n = 48) using adjusted, research use only (RUO) NanoString-based algorithms. In the validation set, there was excellent concordance between the RUO scores and their commercial counterparts (rc(RS) = 0.96, 95% CI 0.93–0.97 with level of agreement (LoA) of −7.69 to 8.12; rc(EP) = 0.97, 95% CI 0.96–0.98 with LoA of −0.64 to 1.26 and rc(ROR) = 0.97 (95% CI 0.94–0.98) with LoA of −8.65 to 10.54). There was also a strong agreement in risk stratification: (RS: κ = 0.86, p < 0.0001; EP: κ = 0.87, p < 0.0001; ROR: κ = 0.92, p < 0.001). In conclusion, the calibrated algorithms recapitulate the commercial RS and EP scores on individual biopsies and ROR scores on samples based on subgroup-centreing method using NanoString expression data.

Text as tautology: an exploration in inference, transitivity, and logical compression

2021 ◽  
Vol 0 (0) ◽  
Author(s):  
Andrew Potter
Keyword(s):  
Knowledge Representation ◽  
Propositional Logic ◽  
Research Literature ◽  
Transitive Inference ◽  
Structure Theory ◽  
Rhetorical Structure ◽  
Logical Models ◽  
Logical Operations ◽  
Selection Of

Abstract Rhetorical structure theory (RST) and relational propositions have been shown useful in analyzing texts as expressions in propositional logic. Because these expressions are systematically derived, they may be expected to model discursive reasoning as articulated in the text. If this is the case, it would follow that logical operations performed on the expressions would be reflected in the texts. In this paper the logic of relational propositions is used to demonstrate the applicability of transitive inference to discourse. Starting with a selection of RST analyses from the research literature, analyses of the logic of relational propositions are performed to identify their corresponding logical expressions and within each expression to identify the inference path implicit within the text. By eliminating intermediary relational propositions, transitivity is then used to progressively compress the expression. The resulting compressions are applied to the corresponding texts and their compressed RST analyses. The application of transitive inference to logical expressions results in abridged texts that are intuitively coherent and logically compatible with their originals. This indicates an underlying isomorphism between the inferential structure of logical expressions and discursive coherence, and it confirms that these expressions function as logical models of the text. Potential areas for application include knowledge representation, logic and argumentation, and RST validation.

Sign in / Sign up

Export Citation Format

Share Document