scholarly journals Reporting and methodological quality of systematic reviews and meta-analysis with protocols in Diabetes Mellitus Type II: A systematic review

PLoS ONE ◽  
2020 ◽  
Vol 15 (12) ◽  
pp. e0243091
Author(s):  
Daniel Christopher Rainkie ◽  
Zeinab Salman Abedini ◽  
Nada Nabil Abdelkader

Background Systematic reviews with or without meta-analyses (SR/MAs) are strongly encouraged to work from a protocol to facilitate high quality, transparent methodology. The completeness of reporting of a protocol (PRISMA-P) and manuscript (PRISMA) is essential to the quality appraisal (AMSTAR-2) and appropriate use of SR/MAs in making treatment decisions. Objectives The objectives of this study were to describe the completeness of reporting and quality of SR/MAs, assess the correlations between PRISMA-P, PRISMA, and AMSTAR-2, and to identify reporting characteristics between similar items of PRISMA-P and PRISMA. Methods We performed a systematic review of Type 2 Diabetes Mellitus SR/MAs of hypoglycemic agents with publicly available protocols. Cochrane reviews, guidelines, and specific types of MA were excluded. Two reviewers independently, (i) searched PubMed and Embase between 1/1/2015 to 20/3/2019; (ii) identified protocols of included studies by searching the manuscript bibliography, supplementary material, PROSPERO, and Google; (iii) completed PRISMA-P, PRISMA, and AMSTAR-2 tools. Data analysis included descriptive statistics, Pearson correlation, and multivariable linear regression. Results Of 357 relevant SR/MAs, 51 had available protocols and were included. The average score for PRISMA-P was 15.8±3.3 (66%; maximum 24) and 25.2±1.1 (93%; maximum 27) for PRISMA. The quality of SR/MAs assessed using the AMSTAR-2 tool identified an overall poor quality (63% critically low, 18% low, 8% moderate, 12% high). The correlation between the PRISMA-P and PRISMA was not significant (r = 0.264; p = 0.06). Correlation was significant between PRISMA-P and AMSTAR-2 (r = 0.333; p = 0.02) and PRISMA and AMSTAR-2 (r = 0.555; p<0.01). Discrepancies in reporting were common between similar PRISMA-P and PRISMA items. Conclusion Adherence to protocol reporting guidance was poor while manuscript reporting was comprehensive. Protocol completeness is not associated with a completely reported manuscript. Independently, PRISMA-P and PRISMA scores were weakly associated with higher quality assessments but insufficient as a surrogate for quality. Critical areas for quality improvement include protocol description, investigating causes of heterogeneity, and the impact of risk of bias on the evidence synthesis.

2019 ◽  
Author(s):  
Kaiyan Hu ◽  
Ting Zhang ◽  
Weiyi Zhang ◽  
Qi Zhou ◽  
Joey S.W. Kwong ◽  
...  

Abstract Background: Protocols of systematic reviews allow for planning and documentation of review methods, and thus improve the transparency of reviews process. However, pre-registered a protocol is not enough, the author also need to follow it. PROSPERO is an open-access online facility for the registration of non-Cochrane systematic reviews. The purpose of our research is to determined what changed were made between non-Cochrane reviews and their protocols in PROSPERO and how likely these changes impacted the quality of systematic review. Method: In this retrospective comparative study we electronically searched for protocols and their corresponding systematic reviews in the PROSPERO platform that were “completed and published” from January to December, 2018. Two reviewers independently identified and classified changes between the protocols and systematic reviews then evaluated the impact (improve/reduce/unclear)of these change on the reporting/methodology quality of reviews. Frequency (n), percentage (%) were used to analyze the number of changes categorically in each review and the distribution of different impact caused by these changes. Results: We identified 39 pre-registered protocols and their reviews, all of which exhibited alterations. All changes to only one review are considered to improve the reporting/methodology quality, and remaining 97% of reviews (n=38) contain changes that are categorically considered to reduce the methodology/reporting quality or that have an unclear impact on reviews. Conclusions: Differences between the non-Cochrane reviews and their protocols recorded in PROSPERO are widespread, and there have been many changes having an unclear impact on the quality of reviews. Guiding the author to report and explain the differences between protocol and reviews or even requiring authors to so at the level of journal are two fundamental solutions to further improve the transparency of the non-Cochrane reviews.


2013 ◽  
Vol 41 (2) ◽  
pp. 194-205 ◽  
Author(s):  
Elizabeth A.T. Ghogomu ◽  
Lara J. Maxwell ◽  
Rachelle Buchbinder ◽  
Tamara Rader ◽  
Jordi Pardo Pardo ◽  
...  

The Cochrane Musculoskeletal Group (CMSG), one of 53 groups of the not-for-profit, international Cochrane Collaboration, prepares, maintains, and disseminates systematic reviews of treatments for musculoskeletal diseases. It is important that authors conducting CMSG reviews and the readers of our reviews be aware of and use updated, state-of-the-art systematic review methodology. One hundred sixty reviews have been published. Previous method guidelines for systematic reviews of interventions in the musculoskeletal field published in 2006 have been substantially updated to incorporate methodological advances that are mandatory or highly desirable in Cochrane reviews and knowledge translation advances. The methodological advances include new guidance on searching, new risk-of-bias assessment, grading the quality of the evidence, the new Summary of Findings table, and comparative effectiveness using network metaanalysis. Method guidelines specific to musculoskeletal disorders are provided by CMSG editors for various aspects of undertaking a systematic review. These method guidelines will help improve the quality of reporting and ensure high standards of conduct as well as consistency across CMSG reviews.


2019 ◽  
Vol 2 (2) ◽  
pp. 50-57
Author(s):  
Amanda Yang Shen ◽  
Robert S Ware ◽  
Tom J O'Donohoe ◽  
Jason Wasiak

Background: An increasing number of systematic reviews are published on an annual basis. Although perusal of the full text of articles is preferable, abstracts are sometimes relied upon to guide clinical decisions. Despite this, the abstracts of systematic reviews have historically been poorly reported. We evaluated the reporting quality of systematic review abstracts within hand and wrist pathology literature. Methods: We searched MEDLINE®, EMBASE and Cochrane Library from inception to December 2017 for systematic reviews in hand and wrist pathology using the 12-item PRISMA-A checklist to assess abstract reporting quality. Results: A total of 114 abstracts were included. Most related to fracture (38%) or arthritis (17%) management. Forty-seven systematic reviews (41%) included meta-analysis. Mean PRISMA-A score was 3.6/12 with Cochrane reviews having the highest mean score and hand-specific journals having the lowest. Abstracts longer than 300 words (mean difference [MD]: 1.43, 95% CI [0.74, 2.13]; p <0.001) and systematic reviews with meta-analysis (MD: 0.64, 95% CI [0.05, 1.22]; p = 0.034) were associated with higher scores. Unstructured abstracts were associated with lower scores (MD: –0.65, 95% CI [–1.28, –0.02]; p = 0.044). A limitation of this study is the possible exclusion of relevant studies that were not published in the English language. Conclusion: Abstracts of systematic reviews pertaining to hand and wrist pathology have been suboptimally reported as assessed by the PRISMA-A checklist. Improvements in reporting quality could be achieved by endorsement of PRISMA-A guidelines by authors and journals, and reducing constraints on abstract length.


2020 ◽  
Author(s):  
Kaiyan Hu ◽  
Ting Zhang ◽  
Weiyi Zhang ◽  
Qi Zhou ◽  
Mengyao Jiang New ◽  
...  

Abstract Background: Protocols of systematic reviews allow for planning and documentation of review methods and thus improve the transparency of the reviews process. However, pre-registration of a protocol is not enough, the author also need to follow it. PROSPERO is an open-access online database for the registration of non-Cochrane systematic reviews. The purpose of this study is to compare published non-Cochrane reviews with their pre-registered protocols on PROSPERO to determine what changes, if any, have been made, and how likely these changes are to impact the quality of systematic review. Methods: This is a retrospective comparative study. We searched for protocols on PROSPERO platform that were registered in 2018 and then selected the protocols that full text have been published as of January 1st 2019. Published full texts were identified through the protocol's final publication citation. Two authors independently compared and identified changes between protocols and systematic reviews and then evaluated the impact (improve, reduce, or unclear) of these changes on the reporting or methodology quality of reviews. Descriptive statistics of percentage (%) and frequency (n) were conducted. Results: We identified 39 pairs, all of which exhibited changes. “Search strategy”(92%, n=36), “data extraction”(90%, n=35), “data synthesis”(77%, n=30), “outcome”(64%, n=24), and “subgroup analysis”(64%, n=24) all showed significant changes. All changes to only one review were considered to improve the reporting or methodology quality, and the remaining 97% of reviews (n=38) contained changes that were considered to reduce the methodology or reporting quality or that had an unclear impact on systematic reviews. Conclusions: Changes between the non-Cochrane systematic reviews and their protocols recorded on PROSPERO were widespread. Some of the changes reduced the methodology or reporting quality of systematic reviews or had an unclear impact. Measures should be taken to further improve the transparency of the non-Cochrane systematic reviews. Adding a new item in updated “Preferred Reporting Items for Systematic reviews and Meta-Analyses” (PRISMA) and “Meta-analysis of Observational Studies in Epidemiology” (MOOSE) to guide reporting and explaining the changes, as well as advising peer reviewers (and editors) to check the reviews against the protocols are two suggested fundamental solutions.


2017 ◽  
Vol 1 (S1) ◽  
pp. 42-43
Author(s):  
Melissa L. Rethlefsen ◽  
Mellanye Lackey ◽  
Michelle Fiander ◽  
Mary McFarland

OBJECTIVES/SPECIFIC AIMS: To improve the quality of evidence synthesis projects, including systematic reviews and other comparative effectiveness reviews, at the University of Utah. METHODS/STUDY POPULATION: Systematic reviews and other types of evidence syntheses are best when collaborative teams with expertise in multiple disciplines participate, including content experts, librarians and information specialists, systematic review methodologists, and statisticians. The Center for Clinical & Translational Science (CCTS), due to its interdisciplinary nature, connectivity to clinical experts, and existing Cores of methodologists, presented an opportune location for a Systematic Review Core. We designed the Systematic Review Core to focus on 2 primary aspects of evidence synthesis support: overall systematic review methodology guidance and in-depth information retrieval planning and execution. After establishing a conceptual partnership, a new position, Evidence Retrieval and Synthesis Librarian, was created to build capacity within the Core. RESULTS/ANTICIPATED RESULTS: Close connections with the CCTS’s Population Health Research Foundation have led to better interdisciplinary coverage of systematic reviews and other evidence syntheses produced by the University of Utah. We are able to partner with statisticians and clinical experts from formulating the question to completing the final manuscript. Hourly rates charged through a cost recovery model have enabled us to grow our staff able to work on the Core, as well as offset costs for major databases and resources these bibliographic data-heavy research methods require. After 1 year of existence, the Core is already at maximum capacity, with no sign of slowing. Projects have ranged from brief consultations to highly intense interactions for the duration of the research spectrum. We have also been added as key personnel to grants with systematic review components. DISCUSSION/SIGNIFICANCE OF IMPACT: Systematic reviews and other evidence syntheses are a labor-intense, interdisciplinary team effort that fit well within the scope of CTSA’s. They are a key component of the translation of science to practice, and can be used at all stages of the translational science spectrum. Quality of systematic reviews remains poor, particularly surrounding protocol development, sensitive search strategy design and reporting, and overall reporting. Librarians and information specialist involvement has been shown to positively correlate to the search strategy design and reporting aspects of systematic reviews, and librarians and information specialists increasingly act as systematic review methodologists. By including librarians and information specialists as part of the CTSA’s official Core structure, these systematic review methodologists are able to connect with statisticians, other methodologists, and clinical experts in a nexus of interdisciplinarity. At the University of Utah, the visibility and structure provided by the CCTS helps the Systematic Review Core with promotion, creating connections and opportunities for collaboration across the campus. This partnership has already led to increased uptake in services, and over time, we believe it will increase the quality of the science produced. CTSA’s have a natural partner with their health science library colleagues in translational science, as shown by this model.


2019 ◽  
Vol 37 (27_suppl) ◽  
pp. 241-241
Author(s):  
Irbaz Bin Riaz ◽  
Rabbia Siddiqi ◽  
Noureen Asghar ◽  
Elizabeth Jane Cathcart-Rake ◽  
Vitaly Herasevich ◽  
...  

241 Background: In a rapidly moving field, such as cancer immunotherapy, where immune checkpoint inhibitors (ICIs) are used across 14 different tumor types, patients may receive suboptimal treatment or even be harmed if information on toxicity is not readily translated for use in clinical practice. Every single systematic review and meta-analysis which attempted to summarize toxicity of immune checkpoint inhibitors (ICIs) quickly became outdated. A living systematic review, which is defined as a systematic review that is continually updated to incorporate relevant new evidence as it becomes available, is necessary in this situation. Methods: The process of creating a living systematic review started with the creation of a comprehensive search designed by a librarian experienced in systematic reviews in collaboration with the study’s principle investigator. Search was constantly updated every 3 months and evidence is synthesized in a series of steps (microtasks) using a combination of human and augmented intelligence. A complete infrastructure is being developed and it includes automated cumulative meta-analysis and an online reporting platform which will constantly update information for clinicians and patients in a live manner. Results: We screened 6746 studies during Sep 2018-March 2019 and identified 6746 studies and we were able to successfully maintain up-to-date toxicity estimates for immune mediated adverse events over this period while maintaining the rigor of a conventional systematic review. Eventually, we will integrate the steps of LSR into one, user-friendly, semi-automated format which can independently provide accurate estimates and feed into and support a living guidelines platform through shared Application Programing Interface (APIs). Conclusions: LSRs are feasible, efficient, and when fully developed can reduce redundancy and waste in medical research, improve the quality of evidence, reduce human effort and support living and dynamic guidelines to facilitate truly informed shared decision making.


2017 ◽  
Vol 2017 ◽  
pp. 1-12 ◽  
Author(s):  
Daniela Marchetti ◽  
Danilo Carrozzino ◽  
Federica Fraticelli ◽  
Mario Fulcheri ◽  
Ester Vitacolonna

Background and Objective. Diagnosis of Gestational Diabetes Mellitus (GDM) could significantly increase the likelihood of health problems concerning both potential risks for the mother, fetus, and child’s development and negative effects on maternal mental health above all in terms of a diminished Quality of Life (QoL). The current systematic review study is aimed at further contributing to an advancement of knowledge about the clinical link between GDM and QoL. Methods. According to PRISMA guidelines, PubMed, Web of Science, Scopus, and Cochrane databases were searched for studies aimed at evaluating and/or improving levels of QoL in women diagnosed with GDM. Results. Fifteen research studies were identified and qualitatively analyzed by summarizing results according to the following two topics: GDM and QoL and interventions on QoL in patients with GDM. Studies showed that, in women with GDM, QoL is significantly worse in both the short term and long term. However, improvements on QoL can be achieved through different intervention programs by enhancing positive diabetes-related self-management behaviors. Conclusion. Future studies are strongly recommended to further examine the impact of integrative programs, including telemedicine and educational interventions, on QoL of GDM patients by promoting their illness acceptance and healthy lifestyle behaviors.


2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Milagros Adobes Martin ◽  
Sala Santamans Faustino ◽  
Inmaculada Llario Almiñana ◽  
Riccardo Aiuto ◽  
Roberto Rotundo ◽  
...  

Abstract Background To evaluate the completeness of reporting abstracts of systematic reviews (SRs) before and after the publication of the PRISMA-A checklist in 2013 and to assess if an association exists between abstract characteristics and the completeness of reporting. Methods A systematic search of the literature was conducted in the PubMed and Scopus databases in March 2020. The search focused on the SRs of evaluations of interventions published since 2002 in the field of periodontology. The abstracts of the selected SRs were divided into two groups before and after publication of the PRISMA-A checklist in 2013, and compliance with the 12 items reported in the checklist was evaluated by three calibrated evaluators. Results A set of 265 abstracts was included in the study. The total score before (mean score, 53.78%; 95% CI, 51.56–55.90%) and after (mean score, 56.88%; 95% CI, 55.39–58.44%) the publication of the PRISMA-A statement exhibited a statistically significant improvement (P = 0.012*). Nevertheless, only the checklist items included studies and synthesis of the results displayed a statistically significant change after guideline publication. The total PRISMA-A score was higher in the meta-analysis group and in articles authored by more than four authors. Conclusions The impact of the PRISMA-A was statistically significant, but the majority of the items did not improve after its introduction. The editors and referees of periodontal journals should promote adherence to the checklist to improve the quality of the reports and provide readers with better insight into the characteristics of published studies.


2021 ◽  
pp. bmjebm-2021-111710
Author(s):  
Rebecca Abbott ◽  
Alison Bethel ◽  
Morwenna Rogers ◽  
Rebecca Whear ◽  
Noreen Orr ◽  
...  

ObjectiveThe academic and scientific community has reacted at pace to gather evidence to help and inform about COVID-19. Concerns have been raised about the quality of this evidence. The aim of this review was to map the nature, scope and quality of evidence syntheses on COVID-19 and to explore the relationship between review quality and the extent of researcher, policy and media interest.Design and settingA meta-research: systematic review of reviews.Information sourcesPubMed, Epistemonikos COVID-19 evidence, the Cochrane Library of Systematic Reviews, the Cochrane COVID-19 Study Register, EMBASE, CINAHL, Web of Science Core Collection and the WHO COVID-19 database, searched between 10 June 2020 and 15 June 2020.Eligibility criteriaAny peer-reviewed article reported as a systematic review, rapid review, overview, meta-analysis or qualitative evidence synthesis in the title or abstract addressing a research question relating to COVID-19. Articles described as meta-analyses but not undertaken as part of a systematic or rapid review were excluded.Study selection and data extractionAbstract and full text screening were undertaken by two independent reviewers. Descriptive information on review type, purpose, population, size, citation and attention metrics were extracted along with whether the review met the definition of a systematic review according to six key methodological criteria. For those meeting all criteria, additional data on methods and publication metrics were extracted.Risk of biasFor articles meeting all six criteria required to meet the definition of a systematic review, AMSTAR-2 ((A MeaSurement Tool to Assess systematic Reviews, version 2.0) was used to assess the quality of the reported methods.Results2334 articles were screened, resulting in 280 reviews being included: 232 systematic reviews, 46 rapid reviews and 2 overviews. Less than half reported undertaking critical appraisal and a third had no reproducible search strategy. There was considerable overlap in topics, with discordant findings. Eighty-eight of the 280 reviews met all six systematic review criteria. Of these, just 3 were rated as of moderate or high quality on AMSTAR-2, with the majority having critical flaws: only a third reported registering a protocol, and less than one in five searched named COVID-19 databases. Review conduct and publication were rapid, with 52 of the 88 systematic reviews reported as being conducted within 3 weeks, and a half published within 3 weeks of submission. Researcher and media interest, as measured by altmetrics and citations, was high, and was not correlated with quality.DiscussionThis meta-research of early published COVID-19 evidence syntheses found low-quality reviews being published at pace, often with short publication turnarounds. Despite being of low quality and many lacking robust methods, the reviews received substantial attention across both academic and public platforms, and the attention was not related to the quality of review methods.InterpretationFlaws in systematic review methods limit the validity of a review and the generalisability of its findings. Yet, by being reported as ‘systematic reviews’, many readers may well regard them as high-quality evidence, irrespective of the actual methods undertaken. The challenge especially in times such as this pandemic is to provide indications of trustworthiness in evidence that is available in ‘real time’.PROSPERO registration numberCRD42020188822.


2017 ◽  
Vol 32 (2) ◽  
pp. 201-208 ◽  
Author(s):  
Michelangelo Bortolin ◽  
Ilaria Morelli ◽  
Amalia Voskanyan; ◽  
Nina R. Joyce ◽  
Gregory R. Ciottone

AbstractIntroductionEarthquake-related trauma results in crush injuries and bony- and soft-tissue trauma. There are no systematic reviews analyzing the typical injury patterns and treatments in “Mega-Mass-Casualty” earthquakes. The characterization of an injury pattern specific to disaster type, be it natural or manmade, is imperative to build an effective disaster preparedness and response system.MethodsThe systematic review was conducted in accordance with Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). A comprehensive search strategy was developed to identify all publications relating to earthquakes and the orthopedic treatment in adult patients. The following databases were searched: PubMed (Medline; US National Library of Medicine, National Institutes of Health; Bethesda, Maryland USA), Ovid (Ovid Technologies; New York, New York USA), Web of Science (Thomson Reuters; New York, New York USA), and The Cochrane Library (The Cochrane Collaboration; Oxford, United Kingdom).ResultsThe searches identified 4,704 articles: 4,445 after duplicates were removed. The papers were screened for title and abstract and 65 out of those were selected for full-text analysis. The quality of data does not permit a standard-of-care (SOC) to be defined. Scarcity and poor quality of the data collected also may suggest a low level of accountability of the activity of the international hospital teams. Qualitatively, it is possible to define that there are more open fractures during daytime hours than at night. Excluding data about open and closed fractures, for all types of injuries, the results underline that the higher the impact of the earthquake, as measured by Richter Magnitude Scale (RMS), the higher is the number of injuries.DiscussionRegarding orthopedic injuries during earthquakes, special attention must be paid to the management of the lower limbs most frequently injured. Spinal cord involvement following spine fractures is an important issue: this underlines how a neurosurgeon on a disaster team could be an important asset during the response. Conservative treatment for fractures, when possible, should be encouraged in a disaster setting. Regarding amputation, it is important to underline how the response and the quality of health care delivered is different from one team to another. This study shows how important it is to improve, and to require, the accountability of international disaster teams in terms of type and quality of health care delivered, and to standardize the data collection.BortolinM, MorelliI, VoskanyanA, JoyceNR, CiottoneGR. Earthquake-related orthopedic injuries in adult population: a systematic review. Prehosp Disaster Med. 2017;32(2):201–208.


Sign in / Sign up

Export Citation Format

Share Document