scholarly journals A comparison of two assessment tools used in overviews of systematic reviews: ROBIS versus AMSTAR-2

2021 ◽  
Vol 10 (1) ◽  
Author(s):  
R. Perry ◽  
A. Whitmarsh ◽  
V. Leach ◽  
P. Davies

Abstract Background AMSTAR-2 is a 16-item assessment tool to check the quality of a systematic review and establish whether the most important elements are reported. ROBIS is another assessment tool which was designed to evaluate the level of bias present within a systematic review. Our objective was to compare, contrast and establish both inter-rater reliability and usability of both tools as part of two overviews of systematic reviews. Strictly speaking, one tool assesses methodological quality (AMSTAR-2) and the other assesses risk of bias (ROBIS), but there is considerable overlap between the tools in terms of the signalling questions. Methods Three reviewers independently assessed 31 systematic reviews using both tools. The inter-rater reliability of all sub-sections using each instrument (AMSTAR-2 and ROBIS) was calculated using Gwet’s agreement coefficient (AC1 for unweighted analysis and AC2 for weighted analysis). Results Thirty-one systematic reviews were included. For AMSTAR-2, the median agreement for all questions was 0.61. Eight of the 16 AMSTAR-2 questions had substantial agreement or higher (> 0.61). For ROBIS, the median agreement for all questions was also 0.61. Eleven of the 24 ROBIS questions had substantial agreement or higher. Conclusion ROBIS is an effective tool for assessing risk of bias in systematic reviews and AMSTAR-2 is an effective tool at assessing quality. The median agreement between raters for both tools was identical (0.61). Reviews that included a meta-analysis were easier to rate with ROBIS; however, further developmental work could improve its use in reviews without a formal synthesis. AMSTAR-2 was more straightforward to use; however, more response options would be beneficial.

2021 ◽  
Author(s):  
Rachel Perry ◽  
Alex Whitmarsh ◽  
Verity Leach ◽  
Philippa Davies

Abstract Background:AMSTAR-2 is a 16-item assessment tool to check the quality of a systematic review and establish whether the most important elements are reported. ROBIS is another assessment tool which was designed to evaluate the level of bias present within a systematic review. Our objective was to compare, contrast and establish both inter-rater reliability and usability of both tools as part of two overviews of systematic reviews. Strictly speaking, one tool assesses methodological quality (AMSTAR-2) and the other assesses risk of bias (ROBIS), but there is considerable overlap between the tools in terms of the signalling questions. Methods:Three reviewers independently assessed 31 systematic reviews using both tools. The inter-rater reliability of all sub-sections using each instrument (AMSTAR-2 and ROBIS) were calculated using Gwet’s agreement coefficient (AC1 for unweighted analysis and AC2 for weighted analysis).Results:Thirty-one systematic reviews were included. For AMSTAR-2 the median agreement for all questions was 0.61. Eight of the 16 AMSTAR-2 questions had substantial agreement or higher (>0.61). For ROBIS, the median agreement for all questions was also 0.61. Eleven of the 24 ROBIS questions had substantial agreement or higher. Conclusion:ROBIS is an effective tool for assessing risk of bias in systematic reviews and AMSTAR-2 is an effective tool at assessing quality. The median agreement between raters for both tools was identical (0.61). Reviews that included a meta-analysis were easier to rate with ROBIS; however, further developmental work could improve its use in reviews without a formal synthesis. AMSTAR-2 was more straightforward to use, however, more response options would be beneficial.


2021 ◽  
pp. 105381512199192
Author(s):  
Andréane Lavallée ◽  
Gwenaëlle De Clifford-Faugère ◽  
Ariane Ballard ◽  
Marilyn Aita

This systematic review and meta-analysis examined the effectiveness of parent–infant interventions for parents of preterm infants on parental sensitivity compared to standard care or active comparators. This review follows the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines and was prospectively registered in the International Prospective Register of Systematic Reviews (PROSPERO; registration ID: CRD42016047083). Database searches were performed from inception to 2020 to identify eligible randomized controlled trials. Two review authors independently selected studies, extracted data, and assessed the risk of bias using the Cochrane risk of bias assessment tool and quality of evidence using the Grades of Recommendation, Assessment, Development and Evaluation (GRADE) guidelines. A total of 19 studies ( n = 2,111 participants) were included and 14 were suitable to be pooled in our primary outcome meta-analysis. Results show no significant effect of parent–infant interventions over standard care or basic educational programs, on parental sensitivity. Results may not necessarily be due to the ineffectiveness of the interventions but rather due to implementation failure or high risk of bias of included studies.


2015 ◽  
Vol 10 (2) ◽  
pp. 150
Author(s):  
Joanne L. Jordan

A Review of: Westphal, A., Kriston, L., Hölzel, L.P., Härter, M., & von Wolff, A. (2014). Efficiency and contribution of strategies for finding randomized controlled trials: a case study from a systematic review on therapeutic interventions of chronic depression. Journal of Public Health Research, 3(2), 177. doi: 10.4081/jphr.2014.177 Abstract Objective – To evaluate the efficiency and contribution of additional searching strategies for finding randomized controlled trials (RCTs) in a systematic review. Design – A methodological case study. Setting – Biomedical literature. Methods – A sensitive search (defined as “the ratio of the number of relevant reports identified to the total number of relevant reports in existence”) was conducted of electronic databases, Cochrane CENTRAL database, MEDLINE, EMBASE, PsycInfo, CINAHL, BIOSIS, and Web of Science databases (Science and Social Science Citation Indexes). The following additional searching strategies were conducted: hand-searching contents of relevant journals (Archives of General Psychiatry, Journal of Consulting and Clinical Psychology, and Journal of Affective Disorders), citation tracking (forwards tracking using Social Science and Science Citation Index and backwards tracking by looking through reference lists of included studies), screening reference lists of relevant systematic reviews, searching clinical trials registers (ClinicalTrials.gov and ICTRP registers), and contacting first authors of included studies to find any similar unpublished studies. The number of articles identified by each of these methods was recorded and screened for inclusion in the systematic review. The authors calculated what they labelled as the ‘efficiency’ of each searching strategy (the number of included studies identified by the search method as a proportion of the full text articles screened) and the ‘contribution’ of the search strategies (the ratio of included studies identified by that method to the final number of included studies in the systematic review). The methodological quality of each included study was assessed using the Cochrane Risk of Bias Tool, which is a critical appraisal checklist used to judge the study’s value in the systematic review. The meta-analysis in the systematic review was conducted with and without the studies identified by the additional searching strategies to assess their impact on the review’s findings. Main Results – In total 50 studies were identified, 42 from electronic database searches and 8 from additional search strategies. As illustrated by the results in Table 1, the most useful additional search strategy was screening reference lists of relevant systematic reviews. Journal hand-searching and contacting authors also contributed to the review. Of the eight studies identified by the additional search strategies none were judged to have a low risk of bias (four had high risk of bias and four were unclear). Of the 42 included studies from electronic searches only 11 were judged to have a low risk of bias, whereas 9 studies had a high risk of bias and 22 were unclear. Excluding the eight studies retrieved from additional search strategies in the systematic review meta-analysis did not influence the results on the effectiveness of the different interventions for chronic depression. These studies were found to be indexed correctly on the electronic databases, but were not identified in the initial search. Conclusion – Additional search strategies, especially screening reference lists of systematic reviews and hand-searching relevant journals, retrieved a substantial number of relevant studies for a systematic review of interventions for treating chronic depression. However, results of the review’s meta-analysis did not differ when these additional studies (rated as either high or unclear risk of bias) were not included and search methods were time consuming. It might be reasonable to rely on electronic searching strategies when resources for conducting a systematic review are limited or when doing a “rapid review.” The benefits and limitations of additional search strategies should be considered particularly when resources or time for conducting a systematic review are limited. If the electronic database search is sensitive and includes the Cochrane CENTRAL database additional search strategies may not be necessary, but these findings should be tested in other research areas.


BMJ Open ◽  
2019 ◽  
Vol 9 (6) ◽  
pp. e028109 ◽  
Author(s):  
Funbi Akinola ◽  
Rudzani Muloiwa ◽  
Gregory, D Hussey ◽  
Violette Dirix ◽  
Benjamin Kagina ◽  
...  

IntroductionGlobally, some studies show a resurgence of pertussis. The risks and benefits of using whole-cell pertussis (wP) or acellular pertussis (aP) vaccines in the control of the disease have been widely debated. Better control of pertussis will require improved understanding of the immune response to pertussis vaccines. Improved understanding and assessment of the immunity induced by pertussis vaccines is thus imperative. Several studies have documented different immunological outcomes to pertussis vaccination from an array of assays. We propose to conduct a systematic review of the different immunological assays and outcomes used in the assessment of the humoraland cell-mediated immune response following pertussis vaccination.Methods and analysisThe primary outcomes for consideration are quality and quantity of immune responses (humoral and cell-mediated) post-pertussis vaccination. Of interest as secondary outcomes are types of immunoassays used in assessing immune responses post-pertussis vaccination, types of biological samples used in assessing immune responses post-pertussis vaccination, as well as the types of antigens used to stimulate these samples during post-pertussis vaccination immune response assessments. Different electronic databases (including PubMed, Cochrane, EBSCO Host, Scopus and Web of Science) will be accessed for peer-reviewed published and grey literature evaluating immune responses to pertussis vaccines between 1990 and 2019. The quality of included articles will be assessed using standardised risk and quality assessment tools specific to the study design used in each article. Data extraction will be done using a data extraction form. The extracted data will be analysed using STATA V.14.0 and RevMan V.5.3 software. A subgroup analysis will be conducted based on the study population, type of vaccine (wP or aP) and type of immune response (cell-mediated or humoral). Guidelines for reporting systematic reviews in the revised 2009 Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) statement will be used in this study.Ethics and disseminationEthics approval is not required for this study as it is a systematic review. We will only make use of data already available in the public space. Findings will be reported via publication in a peer-reviewed journal and presented at scientific meetings and workshops.Trial registration numberCRD42018102455.


2020 ◽  
Vol 9 (1) ◽  
Author(s):  
Haitham Shoman ◽  
Simone Sandler ◽  
Alexander Peters ◽  
Ameer Farooq ◽  
Magdalen Gruendl ◽  
...  

Abstract Background Gasless laparoscopy, developed in the early 1990s, was a means to minimize the clinical and financial challenges of pneumoperitoneum and general anaesthesia. It has been used in a variety of procedures such as in general surgery and gynecology procedures including diagnostic laparoscopy. There has been increasing evidence of the utility of gasless laparoscopy in resource limited settings where diagnostic imaging is not available. In addition, it may help save costs for hospitals. The aim of this study is to conduct a systematic review of the available evidence surrounding the safety and efficiency of gasless laparoscopy compared to conventional laparoscopy and open techniques and to analyze the benefits that gasless laparoscopy has for low resource setting hospitals. Methods This protocol is developed by following the Preferred Reporting Items for Systematic review and Meta-Analysis–Protocols (PRISMA-P). The PRISMA statement guidelines and flowchart will be used to conduct the study itself. MEDLINE (Ovid), Embase, Web of Science, Cochrane Central, and Global Index Medicus (WHO) will be searched and the National Institutes of Health Clinical Trials database. The articles that will be found will be pooled into Covidence article manager software where all the records will be screened for eligibility and duplicates removed. A data extraction spreadsheet will be developed based on variables of interest set a priori. Reviewers will then screen all included studies based on the eligibility criteria. The GRADE tool will be used to assess the quality of the studies and the risk of bias in all the studies will be assessed using the Cochrane Risk assessment tool. The RoB II tool will assed the risk of bias in randomized control studies and the ROBINS I will be used for the non-randomized studies. Discussion This study will be a comprehensive review on all published articles found using this search strategy on the safety and efficiency of the use of gasless laparoscopy. The systematic review outcomes will include safety and efficiency of gasless laparoscopy compared to the use of conventional laparoscopy or laparotomy. Trial registration The study has been registered in PROSPERO under registration number: CRD42017078338


BMJ Open ◽  
2019 ◽  
Vol 9 (9) ◽  
pp. e029311 ◽  
Author(s):  
Isobel Marion Harris ◽  
Sophie Beese ◽  
David Moore

ObjectiveThis systematic review aimed to evaluate the ability of risk tools to predict the future episodes of suicide/self-harm in adolescents.DesignSystematic review.Data sourcesMEDLINE, EMBASE, CINAHL and PsycINFO were searched from inception to 3 March 2018.Eligibility criteria for selecting studiesCohort studies, case–control studies and randomised controlled trials of adolescents aged 10–25 who had undergone risk assessment in a clinical setting following an episode of self-harm were included.Data extraction and synthesisTwo independent reviewers extracted data and assessed risk of bias. Data were grouped by tool and narrative synthesis undertaken, with studies appraised using a checklist combining the QUIPS (Quality In Prognosis Studies) and QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies) tools.ResultsOf the 17 137 articles initially identified, 11 studies evaluating 10 separate tools were included. The studies varied in setting, population and outcome measure. The majority of the studies were rated as having an unclear risk of bias, and meta-analysis was not possible due to high variability between studies.The ability of the tools to correctly identify those adolescents going on to make a self-harm/suicide attempt ranged from 27% (95% CI 10.7% to 50.2%) to 95.8% (95% CI 78.9% to 99.9%). A variety of metrics were provided for 1–10 points increases in various tools, for example, odds and HRs.ConclusionsThis systematic review is the first to explore the use of assessment tools in adolescents. The predictive ability of these tools varies greatly. No single tool is suitable for predicting a higher risk of suicide or self-harm in adolescent populations.PROSPERO registration numberCRD42017058686


2020 ◽  
Author(s):  
Ting Wan Tan ◽  
Han Ling Tan ◽  
Chih Ming Chang

BACKGROUND Urinary tract infection (UTI) is one of common illness in newborn and infants. An accurate urine sample to diagnosis of UTI is crucial to ensure appropriate treatment to avoid delayed and false-treatment. However, data to assess the effectiveness and safety of non-invasive stimulation technique of collecting urine in newborn and infants are yet to be critically evaluated. OBJECTIVE This protocol aimed to conduct a systematic review and meta-analysis to evaluate the evidences related to the effectiveness and safety of non-invasive stimulation technique of collecting urine in newborn and infants. METHODS This protocol follows guideline according to Preferred Reporting Items for Systematic reviews and Meta-Analysis Protocol and registered on the International Prospective Register of Systematic Reviews. Seven databases will be searched from their inception to March 2020: PubMed, EBSCO, Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE and cumulative index to nursing and allied health literature database. RESULTS This Systemic review will include randomized controlled clinical trials (RCTs) including quasi-randomized controlled trials (quasi-RCTs), observational studies (prospective cohort or retrospective cohort studies). We will perform data extraction, study selection, assessment with risk of bias and data analysis. Primary outcomes measure success in obtaining a urine sample. Secondary outcome measures included gender, age group, weight analysis, contamination rate and adverse effects. Study methodological appraisal of the studies will be assessed by the Cochrane Risk of Bias Tools. For sufficient data are available, a meta-analysis will be conducted. I2 statistics will be used to assess heterogeneity and identify potential sources. CONCLUSIONS This systematic review wills no requirement for ethical approval, without human beings involvement. This systematic review will be published electronically in a peer-reviewed journal, and will give healthcare practitioners good practical guide and information for the evidences related to the effectiveness of non-invasive stimulation techniques to collect urine in newborn and infants, aims to improve clinical outcomes.


2021 ◽  
Vol 23 (Supplement_2) ◽  
pp. ii51-ii52
Author(s):  
A M George ◽  
S Gupta ◽  
S M Keshwara ◽  
M A Mustafa ◽  
C S Gillespie ◽  
...  

Abstract BACKGROUND Systematic reviews and meta-analyses constitute the highest level of research evidence and for a disease with limited clinical trial activity, are often relied upon to help inform clinical practice. This review of reviews evaluates both the reporting & methodological quality of meningioma evidence syntheses. MATERIAL AND METHODS Potentially eligible meningioma reviews published between 1st January 1990 and 31st December 2020 were identified from eight electronic databases. Inclusion required the study to meet the Cochrane guideline definition of a systematic review or meta-analysis. Reviews concerning neurofibromatosis type 2, spinal and pediatric meningiomas were excluded. The reporting and methodological quality of articles were assessed against the following modified guidelines: Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA), A MeaSurement Tool to Assess Systematic Reviews (AMSTAR2) and the Risk of Bias in Systematic Reviews (ROBIS) guidelines. RESULTS 117 systematic reviews were identified, 57 of which included meta-analysis (48.7%). The number of meningioma systematic reviews published each year has increased with 63 studies (53.9%) published between 01/2018 and 12/2020. A median of 17 studies (IQR 9–29) were included per review. Impact factor of journals publishing a systematic review with or without a meta-analysis was similar (median 2.3 vs 1.8, P=0.397). The mean PRISMA scores for systematic reviews with a meta-analysis was 21.11 (SD 4.1, 78% adherence) and without was 13.89 (SD 3.4, 63% adherence). Twenty-nine systematic reviews with meta-analysis (51%) and 11 without meta-analysis (18%) achieved greater than 80% adherence to PRISMA recommendations. Methodological quality assessment using AMSTAR2 revealed one study (0.9%) as high quality whilst 111 (94.8%) studies were graded as critically low. One hundred and two articles (87.2%) did not utilize a comprehensive search strategy as defined by the AMSTAR2 tool. Ninety-nine studies (84.6%) obtained a high level of concern for potential bias as per the ROBIS assessment. One hundred and eight articles (92.3%) failed to present information that a protocol had been established prior to study commencement and 76 articles (65.0%) did not conduct a risk of bias assessment. Across the three tools, domains relating to the establishment of a protocol prior to review commencement and conducting appropriate risk of bias assessments were frequently low scoring. CONCLUSION Overall reporting and methodological quality of meningioma systematic reviews was sub-optimal. Established critical appraisal tools and reporting guidelines should be utilized a priori to assist in producing high-quality systematic reviews.


BMJ Open ◽  
2018 ◽  
Vol 8 (2) ◽  
pp. e019049 ◽  
Author(s):  
Nigus Gebrmedhin Asefa ◽  
Anna Neustaeter ◽  
Nomdo M Jansonius ◽  
Harold Snieder

IntroductionGlaucoma is the second leading cause of age-related vision loss worldwide; it is an umbrella term that is used to describe a set of complex ocular disorders with a multifactorial aetiology. Both genetic and lifestyle risk factors for glaucoma are well established. Thus far, however, systematic reviews on the heritability of glaucoma have focused on the heritability of primary open-angle glaucoma only. No systematic review has comprehensively reviewed or meta-analysed the heritability of other types of glaucoma, including glaucoma-related endophenotypes. The aim of this study will be to identify relevant scientific literature regarding the heritability of both glaucoma and related endophenotypes and summarise the evidence by performing a systematic review and meta-analysis.Methods and analysisThis systematic review will follow the Preferred Reporting Items for Systematic review and Meta-Analysis Protocols 2015 checklist, which provides a standardised approach for carrying out systematic reviews. To capture as much literature as possible, a comprehensive step-by-step systematic search will be undertaken in MEDLINE (PubMed), EMBASE, Web of Science and ScienceDirect, and studies published until 31 December 2017 will be included. Two reviewers will independently search the articles for eligibility according to predefined selection criteria. A database will be used for screening of eligible articles. The quality of the included studies will be rated independently by two reviewers, using the National Health Institute Quality Assessment tool for Observational Cohort and Cross-Sectional Studies. A random-effects model will be used for the meta-analysis. This systematic review is registered with the International Prospective Register of Systematic Reviews with a registration number: CRD42017064504.Ethics and disseminationWe will use secondary data from peer-reviewed published articles, and hence there is no requirement for ethics approval. The results of this systematic review will be disseminated through publication in a peer-reviewed scientific journal.


BMJ Open ◽  
2020 ◽  
Vol 10 (6) ◽  
pp. e034554
Author(s):  
Zhihan Chen ◽  
Rui Wang ◽  
Min Zhang ◽  
Yitong Wang ◽  
Yulan Ren

IntroductionOpioid use disorder (OUD) is a worldwide health problem. Clinical trials indicated that acupuncture combined with medication is effective in OUD, however, there are different conclusions presented by previous trials. This study is designed to evaluate the efficacy and safety of acupuncture combined with medication in OUD.Methods and analysisPubMed, CENTRAL, Embase, Web of Science, CINAHL, PsycINFO, ProQuest Dissertation and Theses, AMED, OpenGrey, Clinicaltrials.gov and who.int/trialsearch will be searched in September 2019 without a language restriction. Randomised controlled trials (RCTs) and quasi-RCTs which included participants with OUD receiving acupuncture therapy combined with medication versus control group will be included in this study. Two reviewers will independently screen studies, extract data, assess risk of bias by the Cochrane risk of bias assessment tool and assess quality of evidence by Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach. Any disagreements will be arbitrated by the third reviewer. Data synthesis and analysis will be conducted by using RevMan V.5.3. Subgroup analyses, sensitivity analysis, meta-regression and reporting bias assessment will be conducted if necessary and appropriate.Ethics and disseminationOn account of the nature of this systematic review and meta-analysis, ethical approval is not required. The results will be published in a peer-reviewed journal.PROSPERO registration numberCRD42019123436.


Sign in / Sign up

Export Citation Format

Share Document