scholarly journals Inconsistencies are common between non-Cochrane systematic reviews and their protocols registered in the International prospective register of systematic reviews (PROSPERO) but were seldom explained

2019 ◽  
Author(s):  
Kaiyan Hu ◽  
Ting Zhang ◽  
Weiyi Zhang ◽  
Qi Zhou ◽  
Joey S.W. Kwong ◽  
...  
Keyword(s):  
Systematic Review ◽  
Open Access ◽  
Comparative Study ◽  
Systematic Reviews ◽  
Reporting Quality ◽  
Fundamental Solutions ◽  
Cochrane Reviews ◽  
Retrospective Comparative Study ◽  
The Impact

Abstract Background: Protocols of systematic reviews allow for planning and documentation of review methods, and thus improve the transparency of reviews process. However, pre-registered a protocol is not enough, the author also need to follow it. PROSPERO is an open-access online facility for the registration of non-Cochrane systematic reviews. The purpose of our research is to determined what changed were made between non-Cochrane reviews and their protocols in PROSPERO and how likely these changes impacted the quality of systematic review. Method: In this retrospective comparative study we electronically searched for protocols and their corresponding systematic reviews in the PROSPERO platform that were “completed and published” from January to December, 2018. Two reviewers independently identified and classified changes between the protocols and systematic reviews then evaluated the impact (improve/reduce/unclear)of these change on the reporting/methodology quality of reviews. Frequency (n), percentage (%) were used to analyze the number of changes categorically in each review and the distribution of different impact caused by these changes. Results: We identified 39 pre-registered protocols and their reviews, all of which exhibited alterations. All changes to only one review are considered to improve the reporting/methodology quality, and remaining 97% of reviews (n=38) contain changes that are categorically considered to reduce the methodology/reporting quality or that have an unclear impact on reviews. Conclusions: Differences between the non-Cochrane reviews and their protocols recorded in PROSPERO are widespread, and there have been many changes having an unclear impact on the quality of reviews. Guiding the author to report and explain the differences between protocol and reviews or even requiring authors to so at the level of journal are two fundamental solutions to further improve the transparency of the non-Cochrane reviews.

scholarly journals Inconsistencies are common between non-Cochrane systematic reviews and their protocols registered in the International prospective register of systematic reviews (PROSPERO) but were seldom explained

2020 ◽  
Author(s):  
Kaiyan Hu ◽  
Ting Zhang ◽  
Weiyi Zhang ◽  
Qi Zhou ◽  
Mengyao Jiang New ◽  
...  
Keyword(s):  
Systematic Reviews ◽  
Data Extraction ◽  
Meta Analysis ◽  
Reporting Quality ◽  
Fundamental Solutions ◽  
Cochrane Reviews ◽  
Or Methodology ◽  
Retrospective Comparative Study ◽  
The Impact

Abstract Background: Protocols of systematic reviews allow for planning and documentation of review methods and thus improve the transparency of the reviews process. However, pre-registration of a protocol is not enough, the author also need to follow it. PROSPERO is an open-access online database for the registration of non-Cochrane systematic reviews. The purpose of this study is to compare published non-Cochrane reviews with their pre-registered protocols on PROSPERO to determine what changes, if any, have been made, and how likely these changes are to impact the quality of systematic review. Methods: This is a retrospective comparative study. We searched for protocols on PROSPERO platform that were registered in 2018 and then selected the protocols that full text have been published as of January 1st 2019. Published full texts were identified through the protocol's final publication citation. Two authors independently compared and identified changes between protocols and systematic reviews and then evaluated the impact (improve, reduce, or unclear) of these changes on the reporting or methodology quality of reviews. Descriptive statistics of percentage (%) and frequency (n) were conducted. Results: We identified 39 pairs, all of which exhibited changes. “Search strategy”(92%, n=36), “data extraction”(90%, n=35), “data synthesis”(77%, n=30), “outcome”(64%, n=24), and “subgroup analysis”(64%, n=24) all showed significant changes. All changes to only one review were considered to improve the reporting or methodology quality, and the remaining 97% of reviews (n=38) contained changes that were considered to reduce the methodology or reporting quality or that had an unclear impact on systematic reviews. Conclusions: Changes between the non-Cochrane systematic reviews and their protocols recorded on PROSPERO were widespread. Some of the changes reduced the methodology or reporting quality of systematic reviews or had an unclear impact. Measures should be taken to further improve the transparency of the non-Cochrane systematic reviews. Adding a new item in updated “Preferred Reporting Items for Systematic reviews and Meta-Analyses” (PRISMA) and “Meta-analysis of Observational Studies in Epidemiology” (MOOSE) to guide reporting and explaining the changes, as well as advising peer reviewers (and editors) to check the reviews against the protocols are two suggested fundamental solutions.

scholarly journals Reporting quality of systematic review abstracts in articles hand and wrist pathology: a review

2019 ◽  
Vol 2 (2) ◽  
pp. 50-57
Author(s):  
Amanda Yang Shen ◽  
Robert S Ware ◽  
Tom J O'Donohoe ◽  
Jason Wasiak
Keyword(s):  
Systematic Review ◽  
Systematic Reviews ◽  
English Language ◽  
Meta Analysis ◽  
Reporting Quality ◽  
Cochrane Library ◽  
Annual Basis ◽  
Cochrane Reviews ◽  
Abstract Reporting

Background: An increasing number of systematic reviews are published on an annual basis. Although perusal of the full text of articles is preferable, abstracts are sometimes relied upon to guide clinical decisions. Despite this, the abstracts of systematic reviews have historically been poorly reported. We evaluated the reporting quality of systematic review abstracts within hand and wrist pathology literature. Methods: We searched MEDLINE®, EMBASE and Cochrane Library from inception to December 2017 for systematic reviews in hand and wrist pathology using the 12-item PRISMA-A checklist to assess abstract reporting quality. Results: A total of 114 abstracts were included. Most related to fracture (38%) or arthritis (17%) management. Forty-seven systematic reviews (41%) included meta-analysis. Mean PRISMA-A score was 3.6/12 with Cochrane reviews having the highest mean score and hand-specific journals having the lowest. Abstracts longer than 300 words (mean difference [MD]: 1.43, 95% CI [0.74, 2.13]; p <0.001) and systematic reviews with meta-analysis (MD: 0.64, 95% CI [0.05, 1.22]; p = 0.034) were associated with higher scores. Unstructured abstracts were associated with lower scores (MD: –0.65, 95% CI [–1.28, –0.02]; p = 0.044). A limitation of this study is the possible exclusion of relevant studies that were not published in the English language. Conclusion: Abstracts of systematic reviews pertaining to hand and wrist pathology have been suboptimally reported as assessed by the PRISMA-A checklist. Improvements in reporting quality could be achieved by endorsement of PRISMA-A guidelines by authors and journals, and reducing constraints on abstract length.

scholarly journals Reporting and methodological quality of systematic reviews and meta-analysis with protocols in Diabetes Mellitus Type II: A systematic review

PLoS ONE ◽  
2020 ◽  
Vol 15 (12) ◽  
pp. e0243091
Author(s):  
Daniel Christopher Rainkie ◽  
Zeinab Salman Abedini ◽  
Nada Nabil Abdelkader
Keyword(s):  
Diabetes Mellitus ◽  
Systematic Review ◽  
Systematic Reviews ◽  
Evidence Synthesis ◽  
Pearson Correlation ◽  
Hypoglycemic Agents ◽  
Cochrane Reviews ◽  
The Impact ◽  
Completeness Of Reporting

Background Systematic reviews with or without meta-analyses (SR/MAs) are strongly encouraged to work from a protocol to facilitate high quality, transparent methodology. The completeness of reporting of a protocol (PRISMA-P) and manuscript (PRISMA) is essential to the quality appraisal (AMSTAR-2) and appropriate use of SR/MAs in making treatment decisions. Objectives The objectives of this study were to describe the completeness of reporting and quality of SR/MAs, assess the correlations between PRISMA-P, PRISMA, and AMSTAR-2, and to identify reporting characteristics between similar items of PRISMA-P and PRISMA. Methods We performed a systematic review of Type 2 Diabetes Mellitus SR/MAs of hypoglycemic agents with publicly available protocols. Cochrane reviews, guidelines, and specific types of MA were excluded. Two reviewers independently, (i) searched PubMed and Embase between 1/1/2015 to 20/3/2019; (ii) identified protocols of included studies by searching the manuscript bibliography, supplementary material, PROSPERO, and Google; (iii) completed PRISMA-P, PRISMA, and AMSTAR-2 tools. Data analysis included descriptive statistics, Pearson correlation, and multivariable linear regression. Results Of 357 relevant SR/MAs, 51 had available protocols and were included. The average score for PRISMA-P was 15.8±3.3 (66%; maximum 24) and 25.2±1.1 (93%; maximum 27) for PRISMA. The quality of SR/MAs assessed using the AMSTAR-2 tool identified an overall poor quality (63% critically low, 18% low, 8% moderate, 12% high). The correlation between the PRISMA-P and PRISMA was not significant (r = 0.264; p = 0.06). Correlation was significant between PRISMA-P and AMSTAR-2 (r = 0.333; p = 0.02) and PRISMA and AMSTAR-2 (r = 0.555; p<0.01). Discrepancies in reporting were common between similar PRISMA-P and PRISMA items. Conclusion Adherence to protocol reporting guidance was poor while manuscript reporting was comprehensive. Protocol completeness is not associated with a completely reported manuscript. Independently, PRISMA-P and PRISMA scores were weakly associated with higher quality assessments but insufficient as a surrogate for quality. Critical areas for quality improvement include protocol description, investigating causes of heterogeneity, and the impact of risk of bias on the evidence synthesis.

Updated Method Guidelines for Cochrane Musculoskeletal Group Systematic Reviews and Metaanalyses

2013 ◽  
Vol 41 (2) ◽  
pp. 194-205 ◽  
Author(s):  
Elizabeth A.T. Ghogomu ◽  
Lara J. Maxwell ◽  
Rachelle Buchbinder ◽  
Tamara Rader ◽  
Jordi Pardo Pardo ◽  
...  
Keyword(s):  
Systematic Review ◽  
Systematic Reviews ◽  
Musculoskeletal Diseases ◽  
Cochrane Collaboration ◽  
Previous Method ◽  
Systematic Review Methodology ◽  
Quality Of Reporting ◽  
For Profit ◽  
Cochrane Reviews

The Cochrane Musculoskeletal Group (CMSG), one of 53 groups of the not-for-profit, international Cochrane Collaboration, prepares, maintains, and disseminates systematic reviews of treatments for musculoskeletal diseases. It is important that authors conducting CMSG reviews and the readers of our reviews be aware of and use updated, state-of-the-art systematic review methodology. One hundred sixty reviews have been published. Previous method guidelines for systematic reviews of interventions in the musculoskeletal field published in 2006 have been substantially updated to incorporate methodological advances that are mandatory or highly desirable in Cochrane reviews and knowledge translation advances. The methodological advances include new guidance on searching, new risk-of-bias assessment, grading the quality of the evidence, the new Summary of Findings table, and comparative effectiveness using network metaanalysis. Method guidelines specific to musculoskeletal disorders are provided by CMSG editors for various aspects of undertaking a systematic review. These method guidelines will help improve the quality of reporting and ensure high standards of conduct as well as consistency across CMSG reviews.

Treating schizophrenia: the quality of evidence behind treatment recommendations and how it can improve

2019 ◽  
Vol 25 (4) ◽  
pp. 138-142
Author(s):  
Greg Aran ◽  
Chandler Hicks ◽  
Alexander Demand ◽  
Austin L Johnson ◽  
Jason Beaman ◽  
...  
Keyword(s):  
Systematic Reviews ◽  
Guideline Development ◽  
Reporting Quality ◽  
Treatment Recommendations ◽  
Post Hoc Analysis ◽  
Cochrane Reviews ◽  
Registration Number ◽  
The Mean ◽  
Post Hoc

ObjectiveTo assess the methodological and reporting quality of systematic reviews that comprise the American Psychiatric Association (APA) Practice Guideline for the Treatment of Patients with Schizophrenia and to determine the extent to which results from Cochrane systematic reviews published after guideline development would alter or confirm current recommendations.ParticipantsSystematic reviews that underpinned recommendations in the APA guidelines and Cochrane systematic reviews.Main outcomeThree independent reviewers scored all systematic reviews referenced in the guideline for quality and reporting using AMSTAR and PRISMA checklist, respectively. Items in both tools were individually graded and compared to identify consistently low-performing areas within the systematic reviews. Post hoc analysis of the Cochrane systematic reviews since the latest revision of APA’s guidelines were performed to determine whether their findings were congruent with recent recommendations.ResultsThe mean score of the 57 reviews on the PRISMA checklist was 70%. The mean AMSTAR score was 6.8, correlating with a moderate quality score. Post hoc analysis revealed that 171 Cochrane reviews had been published since the APA guideline release. Only half of the reviews of pharmacological interventions confirmed current recommendations.Conclusions and relevanceThe methodological quality of the systematic reviews included in the APA guideline was deficient in key areas. Our study brings to light the importance of using high-quality evidence in the development of clinical practice guidelines. An updated APA guideline (last updated in 2009) is necessary to provide the highest quality treatment recommendations for clinicians in the management of schizophrenia.Trial registration numberUMIN-CTR, UMIN000023099.

scholarly journals Improving peer review of systematic reviews by involving librarians and information specialists: protocol for a randomized controlled trial

Trials ◽  
2021 ◽  
Vol 22 (1) ◽  
Author(s):  
Melissa L. Rethlefsen ◽  
Sara Schroter ◽  
Lex M. Bouter ◽  
David Moher ◽  
Ana Patricia Ayala ◽  
...  
Keyword(s):  
Systematic Review ◽  
Randomized Controlled Trial ◽  
Peer Review ◽  
Systematic Reviews ◽  
Controlled Trial ◽  
Reporting Quality ◽  
Risk Of Bias ◽  
Randomized Controlled ◽  
Peer Reviewers

Abstract Background Problems continue to exist with the reporting quality and risk of bias in search methods and strategies in systematic reviews and related review types. Peer reviewers who are not familiar with what is required to transparently and fully report a search may not be prepared to review the search components of systematic reviews, nor may they know what is likely to introduce bias into a search. Librarians and information specialists, who have expertise in searching, may offer specialized knowledge that would help improve systematic review search reporting and lessen risk of bias, but they are underutilized as methodological peer reviewers. Methods This study will evaluate the effect of adding librarians and information specialists as methodological peer reviewers on the quality of search reporting and risk of bias in systematic review searches. The study will be a pragmatic randomized controlled trial using 150 systematic review manuscripts submitted to BMJ and BMJ Open as the unit of randomization. Manuscripts that report on completed systematic reviews and related review types and have been sent for peer review are eligible. For each manuscript randomized to the intervention, a librarian/information specialist will be invited as an additional peer reviewer using standard practices for each journal. First revision manuscripts will be assessed in duplicate for reporting quality and risk of bias, using adherence to 4 items from PRISMA-S and assessors’ judgements on 4 signaling questions from ROBIS Domain 2, respectively. Identifying information from the manuscripts will be removed prior to assessment. Discussion The primary outcomes for this study are quality of reporting as indicated by differences in the proportion of adequately reported searches in first revision manuscripts between intervention and control groups and risk of bias as indicated by differences in the proportions of first revision manuscripts with high, low, and unclear bias. If the intervention demonstrates an effect on search reporting or bias, this may indicate a need for journal editors to work with librarians and information specialists as methodological peer reviewers. Trial registration Open Science Framework. Registered on June 17, 2021, at 10.17605/OSF.IO/W4CK2.

Reporting of the methodological quality of search strategies in orthodontic quantitative systematic reviews

2020 ◽  
Author(s):  
Danah AlMubarak ◽  
Nikolaos Pandis ◽  
Martyn T Cobourne ◽  
Jadbinder Seehra
Keyword(s):  
Systematic Reviews ◽  
Methodological Quality ◽  
Search Strategy ◽  
Grey Literature ◽  
Reporting Quality ◽  
Search Strategies ◽  
Cochrane Library ◽  
Eligibility Criteria ◽  
Cochrane Reviews

Summary Background This study aimed to assess the reporting of the methodological quality of search strategies undertaken in orthodontic quantitative systematic reviews (SRs) and hence their reproducibility. Materials and methods A search of a single electronic database (Medline via PubMed) was undertaken to identify interventional orthodontic SRs with meta-analysis published within a 10-year period. The Cochrane Library of Systematic Reviews was also sourced. Full articles were reviewed by two assessors against the eligibility criteria. The reporting quality of each search strategy was assessed using a previously validated checklist with a score of 1 or 2 given for each of the eight items. Cumulative totals were calculated. Guided by previous research, the authors agreed the following cut-offs to categorize the overall level of quality: 8–10 (poor), 10–12 (fair), and greater than 13 (good). Results A total of 127 SRs were analysed. The overall median quality score for the reporting of the search strategy was 14 [interquartile range (IQR): 13–15]. Cochrane SRs and those originating in Europe received higher aggregate scores, whereas no difference was evident based on Prospero registration. The continent of the corresponding author predicated the overall score. Non-Cochrane reviews achieved lower overall scores compared to Cochrane reviews (−1.0, 95% confidence interval: −1.65, −0.34, P = 0.003). The most frequently searched database was EMBASE (N = 93) and the median number of authors was 5 (IQR 4–6). Authors of 26.8% of SRs searched the grey literature. Language restrictions were applied to the search strategies of 88 (69.3%) SRs. Conclusions The reporting quality of search strategies undertaken in orthodontic SRs is at a good level but differences between Cochrane and non-Cochrane reviews currently exist. The reporting of searching of the grey literature and application of no language restrictions can be improved.

Earthquake-Related Orthopedic Injuries in Adult Population: A Systematic Review

2017 ◽  
Vol 32 (2) ◽  
pp. 201-208 ◽  
Author(s):  
Michelangelo Bortolin ◽  
Ilaria Morelli ◽  
Amalia Voskanyan; ◽  
Nina R. Joyce ◽  
Gregory R. Ciottone
Keyword(s):  
Systematic Review ◽  
Health Care ◽  
New York ◽  
Systematic Reviews ◽  
Quality Of Health Care ◽  
Adult Population ◽  
Cochrane Library ◽  
Orthopedic Injuries ◽  
The Impact

AbstractIntroductionEarthquake-related trauma results in crush injuries and bony- and soft-tissue trauma. There are no systematic reviews analyzing the typical injury patterns and treatments in “Mega-Mass-Casualty” earthquakes. The characterization of an injury pattern specific to disaster type, be it natural or manmade, is imperative to build an effective disaster preparedness and response system.MethodsThe systematic review was conducted in accordance with Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). A comprehensive search strategy was developed to identify all publications relating to earthquakes and the orthopedic treatment in adult patients. The following databases were searched: PubMed (Medline; US National Library of Medicine, National Institutes of Health; Bethesda, Maryland USA), Ovid (Ovid Technologies; New York, New York USA), Web of Science (Thomson Reuters; New York, New York USA), and The Cochrane Library (The Cochrane Collaboration; Oxford, United Kingdom).ResultsThe searches identified 4,704 articles: 4,445 after duplicates were removed. The papers were screened for title and abstract and 65 out of those were selected for full-text analysis. The quality of data does not permit a standard-of-care (SOC) to be defined. Scarcity and poor quality of the data collected also may suggest a low level of accountability of the activity of the international hospital teams. Qualitatively, it is possible to define that there are more open fractures during daytime hours than at night. Excluding data about open and closed fractures, for all types of injuries, the results underline that the higher the impact of the earthquake, as measured by Richter Magnitude Scale (RMS), the higher is the number of injuries.DiscussionRegarding orthopedic injuries during earthquakes, special attention must be paid to the management of the lower limbs most frequently injured. Spinal cord involvement following spine fractures is an important issue: this underlines how a neurosurgeon on a disaster team could be an important asset during the response. Conservative treatment for fractures, when possible, should be encouraged in a disaster setting. Regarding amputation, it is important to underline how the response and the quality of health care delivered is different from one team to another. This study shows how important it is to improve, and to require, the accountability of international disaster teams in terms of type and quality of health care delivered, and to standardize the data collection.BortolinM, MorelliI, VoskanyanA, JoyceNR, CiottoneGR. Earthquake-related orthopedic injuries in adult population: a systematic review. Prehosp Disaster Med. 2017;32(2):201–208.

Sign in / Sign up

Export Citation Format

Share Document