scholarly journals The development and evaluation of an online application to assist in the extraction of data from graphs for use in systematic reviews

2019 ◽  
Vol 3 ◽  
pp. 157
Author(s):  
Fala Cramond ◽  
Alison O'Mara-Eves ◽  
Lee Doran-Constant ◽  
Andrew SC Rice ◽  
Malcolm Macleod ◽  
...  
Keyword(s):  
Systematic Review ◽  
Systematic Reviews ◽  
Statistical Data ◽  
Data Extraction ◽  
Evidence Synthesis ◽  
Online Application ◽  
Web Based ◽  
Graphical Data ◽  
Data Extraction Tool ◽  
Increase In Accuracy

Background: The extraction of data from the reports of primary studies, on which the results of systematic reviews depend, needs to be carried out accurately. To aid reliability, it is recommended that two researchers carry out data extraction independently. The extraction of statistical data from graphs in PDF files is particularly challenging, as the process is usually completely manual, and reviewers need sometimes to revert to holding a ruler against the page to read off values: an inherently time-consuming and error-prone process. Methods: To mitigate some of the above problems we integrated and customised two existing JavaScript libraries to create a new web-based graphical data extraction tool to assist reviewers in extracting data from graphs. This tool aims to facilitate more accurate and timely data extraction through a user interface which can be used to extract data through mouse clicks. We carried out a non-inferiority evaluation to examine its performance in comparison to standard practice. Results: We found that the customised graphical data extraction tool is not inferior to users’ prior preferred current approaches. Our study was not designed to show superiority, but suggests that there may be a saving in time of around 6 minutes per graph, accompanied by a substantial increase in accuracy. Conclusions: Our study suggests that the incorporation of this type of tool in online systematic review software would be beneficial in facilitating the production of accurate and timely evidence synthesis to improve decision-making.

scholarly journals The development and evaluation of an online application to assist in the extraction of data from graphs for use in systematic reviews

2018 ◽  
Vol 3 ◽  
pp. 157
Author(s):  
Fala Cramond ◽  
Alison O'Mara-Eves ◽  
Lee Doran-Constant ◽  
Andrew SC Rice ◽  
Malcolm Macleod ◽  
...  
Keyword(s):  
Systematic Review ◽  
Systematic Reviews ◽  
Statistical Data ◽  
Data Extraction ◽  
Evidence Synthesis ◽  
Online Application ◽  
Web Based ◽  
Graphical Data ◽  
Data Extraction Tool ◽  
Increase In Accuracy

Background: The extraction of data from the reports of primary studies, on which the results of systematic reviews depend, needs to be carried out accurately. To aid reliability, it is recommended that two researchers carry out data extraction independently. The extraction of statistical data from graphs in PDF files is particularly challenging, as the process is usually completely manual, and reviewers need sometimes to revert to holding a ruler against the page to read off values: an inherently time-consuming and error-prone process. Methods: To mitigate some of the above problems we developed a new web-based graphical data extraction tool to assist reviewers in extracting data from graphs. This tool aims to facilitate more accurate and timely data extraction through a user interface which can be used to extract data through mouse clicks. We carried out a non-inferiority evaluation to examine its performance in comparison to standard practice. Results: We found that our new graphical data extraction tool is not inferior to users’ prior preferred current approaches. Our study was not designed to show superiority, but suggests that there may be a saving in time of around 6 minutes per graph, accompanied by a substantial increase in accuracy. Conclusions: Our study suggests that the incorporation of this type of tool in online systematic review software would be beneficial in facilitating the production of accurate and timely evidence synthesis to improve decision-making.

scholarly journals The development and evaluation of an online application to assist in the extraction of data from graphs for use in systematic reviews

2019 ◽  
Vol 3 ◽  
pp. 157 ◽  
Author(s):  
Fala Cramond ◽  
Alison O'Mara-Eves ◽  
Lee Doran-Constant ◽  
Andrew SC Rice ◽  
Malcolm Macleod ◽  
...  
Keyword(s):  
Systematic Reviews ◽  
Statistical Data ◽  
Data Extraction ◽  
Evidence Synthesis ◽  
Online Application ◽  
Web Based ◽  
Standard Practice ◽  
Graphical Data ◽  
Data Extraction Tool ◽  
Increase In Accuracy

Background: The extraction of data from the reports of primary studies, on which the results of systematic reviews depend, needs to be carried out accurately. To aid reliability, it is recommended that two researchers carry out data extraction independently. The extraction of statistical data from graphs in PDF files is particularly challenging, as the process is usually completely manual, and reviewers need sometimes to revert to holding a ruler against the page to read off values: an inherently time-consuming and error-prone process. Methods: To mitigate some of the above problems we integrated and customised two existing JavaScript libraries to create a new web-based graphical data extraction tool to assist reviewers in extracting data from graphs. This tool aims to facilitate more accurate and timely data extraction through a user interface which can be used to extract data through mouse clicks. We carried out a non-inferiority evaluation to examine its performance in comparison with participants’ standard practice for extracting data from graphs in PDF documents. Results: We found that the customised graphical data extraction tool is not inferior to users’ (N=10) prior standard practice. Our study was not designed to show superiority, but suggests that, on average, participants saved around 6 minutes per graph using the new tool, accompanied by a substantial increase in accuracy. Conclusions: Our study suggests that the incorporation of this type of tool in online systematic review software would be beneficial in facilitating the production of accurate and timely evidence synthesis to improve decision-making.

scholarly journals PRISMA2020: an R package and Shiny app for producing PRISMA 2020-compliant flow diagrams, with interactivity for optimised digital transparency and Open Synthesis

2021 ◽  
Author(s):  
Neal R Haddaway ◽  
Matthew J Page ◽  
Christopher C Pritchard ◽  
Luke A McGuinness
Keyword(s):  
Systematic Review ◽  
Systematic Reviews ◽  
Science Communication ◽  
Review Process ◽  
Evidence Synthesis ◽  
R Package ◽  
Web Based ◽  
Shiny App ◽  
Data Files ◽  
Flow Diagrams

Background Reporting standards, such as PRISMA aim to ensure that the methods and results of systematic reviews are described in sufficient detail to allow full transparency. Flow diagrams in evidence syntheses allow the reader to rapidly understand the core procedures used in a review and examine the attrition of irrelevant records throughout the review process. Recent research suggests that use of flow diagrams in systematic reviews is poor and of low quality and called for standardised templates to facilitate better reporting in flow diagrams. The increasing options for interactivity provided by the Internet gives us an opportunity to support easy-to-use evidence synthesis tools, and here we report on the development of tools for the production of PRISMA 2020-compliant systematic review flow diagrams. Methods and Findings We developed a free-to-use, Open Source R package and web-based Shiny app to allow users to design PRISMA flow diagrams for their own systematic reviews. Our tools allow users to produce standardised visualisations that transparently document the methods and results of a systematic review process in a variety of formats. In addition, we provide the opportunity to produce interactive, web-based flow diagrams (exported as HTML files), that allow readers to click on boxes of the diagram and navigate to further details on methods, results or data files. We provide an interactive example here; https://driscoll.ntu.ac.uk/prisma/. Conclusions We have developed a user-friendly suite of tools for producing PRISMA 2020-compliant flow diagrams for users with coding experience and, importantly, for users without prior experience in coding by making use of Shiny. These free-to-use tools will make it easier to produce clear and PRISMA 2020-compliant systematic review flow diagrams. Significantly, users can also produce interactive flow diagrams for the first time, allowing readers of their reviews to smoothly and swiftly explore and navigate to further details of the methods and results of a review. We believe these tools will increase use of PRISMA flow diagrams, improve the compliance and quality of flow diagrams, and facilitate strong science communication of the methods and results of systematic reviews by making use of interactivity. We encourage the systematic review community to make use of these tools, and provide feedback to streamline and improve their usability and efficiency.

scholarly journals Automation of literature screening using machine learning in medical evidence synthesis: a diagnostic test accuracy systematic review protocol

2022 ◽  
Vol 11 (1) ◽  
Author(s):  
Yuelun Zhang ◽  
Siyu Liang ◽  
Yunying Feng ◽  
Qing Wang ◽  
Feng Sun ◽  
...  
Keyword(s):  
Systematic Review ◽  
Systematic Reviews ◽  
Digital Library ◽  
Data Extraction ◽  
Evidence Synthesis ◽  
Test Accuracy ◽  
Evidence Based ◽  
Medical Evidence ◽  
Evidence Collection ◽  
Automatic Methods

Abstract Background Systematic review is an indispensable tool for optimal evidence collection and evaluation in evidence-based medicine. However, the explosive increase of the original literatures makes it difficult to accomplish critical appraisal and regular update. Artificial intelligence (AI) algorithms have been applied to automate the literature screening procedure in medical systematic reviews. In these studies, different algorithms were used and results with great variance were reported. It is therefore imperative to systematically review and analyse the developed automatic methods for literature screening and their effectiveness reported in current studies. Methods An electronic search will be conducted using PubMed, Embase, ACM Digital Library, and IEEE Xplore Digital Library databases, as well as literatures found through supplementary search in Google scholar, on automatic methods for literature screening in systematic reviews. Two reviewers will independently conduct the primary screening of the articles and data extraction, in which nonconformities will be solved by discussion with a methodologist. Data will be extracted from eligible studies, including the basic characteristics of study, the information of training set and validation set, and the function and performance of AI algorithms, and summarised in a table. The risk of bias and applicability of the eligible studies will be assessed by the two reviewers independently based on Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2). Quantitative analyses, if appropriate, will also be performed. Discussion Automating systematic review process is of great help in reducing workload in evidence-based practice. Results from this systematic review will provide essential summary of the current development of AI algorithms for automatic literature screening in medical evidence synthesis and help to inspire further studies in this field. Systematic review registration PROSPERO CRD42020170815 (28 April 2020).

scholarly journals A Research Protocol for a Systematic Review of Automatic Literature Screening in Medical Evidence Synthesis

2020 ◽  
Author(s):  
Yuelun Zhang ◽  
Siyu Liang ◽  
Yunying Feng ◽  
Qing Wang ◽  
Feng Sun ◽  
...  
Keyword(s):  
Systematic Review ◽  
Systematic Reviews ◽  
Data Extraction ◽  
Evidence Synthesis ◽  
Evidence Based ◽  
Medical Evidence ◽  
Registration Number ◽  
Evidence Collection ◽  
Automatic Methods ◽  
And Performance

Abstract Background: Systematic review is an indispensable tool for optimal evidence collection and evaluation in evidence-based medicine. However, the explosive increase of the original literatures makes it difficult to accomplish critical appraisal and regular update. Artificial intelligence (AI) algorithms have been applied to automate the literature screening procedure in medical systematic reviews. In these studies, different algorithms were used and results with great variance were reported. It is therefore imperative to systematically review and analyse the developed automatic methods for literature screening and their effectiveness reported in current studies.Methods: An electronic search will be conducted using PubMed, Embase and IEEE Xplore Digital Library databases, as well as literatures found through supplementary search in Google scholar, on automatic methods for literature screening in systematic reviews. Two reviewers will independently conduct the primary screening of the articles and data extraction, in which nonconformities will be solved by discussion with a methodologist. Data will be extracted from eligible studies, including the basic characteristics of study, the information of training set and validation set, the function and performance of AI algorithms, and summarised in a table. The risk of bias and applicability of the eligible studies will be assessed by the two reviewers independently based on Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2). Quantitative analyses, if appropriate, will also be performed.Discussion: Automating systematic review process is of great help in reducing workload in evidence-based practice. Results from this systematic review will provide essential summary of the current development of AI algorithms for automatic literature screening in medical evidence synthesis, and help to inspire further studies in this field. Registration: PROSPERO registration number CRD42020170815 (28 April 2020).

scholarly journals New meniscal tears after ACL injury: what is the risk? A systematic review protocol

2017 ◽  
Vol 52 (6) ◽  
pp. 386-386 ◽  
Author(s):  
Guri Ranum Ekås ◽  
Clare Ardern ◽  
Hege Grindem ◽  
Lars Engebretsen
Keyword(s):  
Systematic Review ◽  
Systematic Reviews ◽  
Data Extraction ◽  
Evidence Synthesis ◽  
Meta Analysis ◽  
Demographic Data ◽  
Acl Injury ◽  
Risk Of Bias ◽  
Meniscal Tears ◽  
Best Evidence Synthesis

BackgroundSecondary meniscal tears after ACL injuries increase the risk of knee osteoarthritis. The current literature on secondary meniscal injuries after ACL injury is not consistent and may have methodological shortcomings. This protocol describes the methods of a systematic review investigating the rate of secondary meniscal injuries in children and adults after treatment (operative or non-operative) for ACL injury.MethodsWe will search electronic databases (Embase, Ovid Medline, Cochrane, CINAHL (Cumulative Index to Nursing and Allied Health Literature), SPORTDiscus, PEDro and Google Scholar) from database inception. Extracted data will include demographic data, methodology, intervention details and patient outcomes. Risk of bias will be assessed using the Newcastle Ottawa checklist for cohort studies. Article screening, eligibility assessment, risk of bias assessment and data extraction will be performed in duplicate by independent reviewers. A proportion meta-analysis will be performed if studies are homogeneous (I2<75%). If meta-analysis is precluded, data will be synthesised descriptively using best-evidence synthesis. The strength of recommendations and quality of evidence will be assessed using the Grading of Recommendations Assessment Development and Evaluation working group methodology.Ethics and disseminationThis protocol is written according to the Preferred Reporting Items for Systematic reviews and Meta-Analyses, and was registered in the International Prospective Register of Systematic Reviews on 22 March 2016.Trial registration numberCRD42016036788.

scholarly journals Development of the ACTIVE framework to describe stakeholder involvement in systematic reviews

2019 ◽  
Vol 24 (4) ◽  
pp. 245-255 ◽  
Author(s):  
Alex Pollock ◽  
Pauline Campbell ◽  
Caroline Struthers ◽  
Anneliese Synnot ◽  
Jack Nunn ◽  
...  
Keyword(s):  
Systematic Review ◽  
Systematic Reviews ◽  
Health Professionals ◽  
Public Involvement ◽  
Review Process ◽  
Data Extraction ◽  
Stakeholder Involvement ◽  
Decision Makers ◽  
Literature Searching ◽  
Health Decision

Objectives Involvement of patients, health professionals, and the wider public (‘stakeholders’) is seen to be beneficial to the quality, relevance and impact of research and may enhance the usefulness and uptake of systematic reviews. However, there is a lack of evidence and resources to guide researchers in how to actively involve stakeholders in systematic reviews. In this paper, we report the development of the ACTIVE framework to describe how stakeholders are involved in systematic reviews. Methods We developed a framework using methods previously described in the development of conceptual frameworks relating to other areas of public involvement, including: literature searching, data extraction, analysis, and categorization. A draft ACTIVE framework was developed and then refined after presentation at a conference workshop, before being applied to a subset of 32 systematic reviews. Data extracted from these systematic reviews, identified in a systematic scoping review, were categorized against pre-defined constructs, including: who was involved, how stakeholders were recruited, the mode of involvement, at what stage there was involvement and the level of control or influence. Results The final ACTIVE framework described whether patients, carers and/or families, and/or other stakeholders (including health professionals, health decision makers and funders) were involved. We defined: recruitment as either open or closed; the approach to involvement as either one-time, continuous or combined; and the method of involvement as either direct or indirect. The stage of involvement in reviews was defined using the Cochrane Ecosystem stages of a review. The level of control or influence was defined according to the roles and activities of stakeholders in the review process, and described as the ACTIVE continuum of involvement. Conclusions The ACTIVE framework provides a structure with which to describe key components of stakeholder involvement within a systematic review, and we have used this to summarize how stakeholders have been involved in a subset of varied systematic reviews. The ACTIVE continuum of involvement provides a new model that uses tasks and roles to detail the level of stakeholder involvement. This work has contributed to the development of learning resources aimed at supporting systematic review authors and editors to involve stakeholders in their systematic reviews. The ACTIVE framework may support the decision-making of systematic review authors in planning how to involve stakeholders in future reviews.

scholarly journals Systematic Reviews and Meta-analysis Published in Indexed Portuguese Medical Journals: Time Trends and Critical Appraisal.

2021 ◽  
Author(s):  
Luísa Prada ◽  
Ana Prada ◽  
Miguel Antunes ◽  
Ricardo Fernandes ◽  
João Costa ◽  
...  
Keyword(s):  
Systematic Reviews ◽  
Critical Appraisal ◽  
Data Extraction ◽  
Evidence Synthesis ◽  
Meta Analysis ◽  
Measurement Tool ◽  
Medical Journals ◽  
Electronic Search ◽  
Potential Impact

Abstract Introduction:Over the last years, the number of systematic reviews published is steadily increasing due to the global interest in this type of evidence synthesis. However, little is known about the characteristics of this research published in Portuguese medical journals. This study aims to evaluate the publication trends and overall quality of these systematic reviews.Material and Methods:Systematic reviews were identified through an electronic search up to August 2020, targeting Portuguese Medical journals indexed in MEDLINE. Systematic reviews selection and data extraction were done independently by three authors. The overall quality critical appraisal using the A MeaSurement Tool to Assess systematic Reviews (AMSTAR II) was independently assessed by three authors. Disagreements were solved by consensus.Results:Seventy systematic reviews published in 5 Portuguese medical journals were included. Most (n=57; 81,4%) were systematic reviews without meta-analysis. Until 2010, the number of systematic reviews per year increased. Since then, the number of reviews published has not remained stable and no less than 3 SRs were published per year. According to the systematic reviews’ typology, most have been predominantly conducted to assess the effectiveness of health interventions (n=28; 40,0%). General and Internal Medicine (n=26; 37,1%) was the most addressed field. Most systematic reviews (n=45; 64,3%) were rated as being of “critically low-quality”.Conclusions:There were consistent flaws in the methodological quality report of the systematic reviews included, particularly in establishing a prior protocol and not assessing the potential impact of the risk of bias on the results.Through the years, the number of systematic reviews published increased, yet their quality is suboptimal. There is a need to improve the reporting of systematic reviews in Portuguese medical journals, which can be achieved by better adherence to quality checklists/tools.Systematic review registration: INPLASY202090105

scholarly journals Methods of conduct and reporting of living systematic reviews: a protocol for a living methodological survey

F1000Research ◽  
2019 ◽  
Vol 8 ◽  
pp. 221 ◽  
Author(s):  
Assem M. Khamis ◽  
Lara A. Kahale ◽  
Hector Pardo-Hernandez ◽  
Holger J. Schünemann ◽  
Elie A. Akl
Keyword(s):  
Systematic Review ◽  
Systematic Reviews ◽  
Evidence Synthesis ◽  
Cochrane Library ◽  
Cross Sectional Survey ◽  
Systematic Review Methodology ◽  
Cross Sectional ◽  
Prospective Longitudinal ◽  
Over Time

Background: The living systematic review (LSR) is an emerging approach for improved evidence synthesis that uses continual updating to include relevant new evidence as soon as it is published. The objectives of this study are to: 1) assess the methods of conduct and reporting of living systematic reviews using a living study approach; and 2) describe the life cycle of living systematic reviews, i.e., describe the changes over time to their methods and findings. Methods: For objective 1, we will begin by conducting a cross-sectional survey and then update its findings every 6 months by including newly published LSRs. For objective 2, we will conduct a prospective longitudinal follow-up of the cohort of included LSRs. To identify LSRs, we will continually search the following electronic databases: Medline, EMBASE and the Cochrane library. We will also contact groups conducting LSRs to identify eligible studies that we might have missed. We will follow the standard systematic review methodology for study selection and data abstraction. For each LSR update, we will abstract information on the following: 1) general characteristics, 2) systematic review methodology, 3) living approach methodology, 4) results, and 5) editorial and publication processes. We will update the findings of both the surveys and the longitudinal follow-up of included LSRs every 6 months. In addition, we will identify articles addressing LSR methods to be included in an ‘LSR methods repository’. Conclusion: The proposed living methodological survey will allow us to monitor how the methods of conduct, and reporting as well as the findings of LSRs change over time. Ultimately this should help with ensuring the quality and transparency of LSRs.

Sign in / Sign up

Export Citation Format

Share Document