scholarly journals Similarities, reliability and gaps in assessing the quality of conduct of systematic reviews using AMSTAR-2 and ROBIS: systematic survey of nutrition reviews

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Mateusz J. Swierz ◽  
Dawid Storman ◽  
Joanna Zajac ◽  
Magdalena Koperny ◽  
Paulina Weglarz ◽  
...  

Abstract Background AMSTAR-2 (‘A Measurement Tool to Assess Systematic Reviews, version 2’) and ROBIS (‘Risk of Bias in Systematic Reviews’) are independent instruments used to assess the quality of conduct of systematic reviews/meta-analyses (SR/MAs). The degree of overlap in methodological constructs together with the reliability and any methodological gaps have not been systematically assessed and summarized in the field of nutrition. Methods We performed a systematic survey of MEDLINE, EMBASE, and the Cochrane Library for SR/MAs published between January 2010 and November 2018 that examined the effects of any nutritional intervention/exposure for cancer prevention. We followed a systematic review approach including two independent reviewers at each step of the process. For AMSTAR-2 (16 items) and ROBIS (21 items), we assessed the similarities, the inter-rater reliability (IRR) and any methodological limitations of the instruments. Our protocol for the survey was registered in PROSPERO (CRD42019121116). Results We found 4 similar domain constructs based on 11 comparisons from a total of 12 AMSTAR-2 and 14 ROBIS items. Ten comparisons were considered fully overlapping. Based on Gwet’s agreement coefficients, six comparisons provided almost perfect (> 0.8), three substantial (> 0.6), and one a moderate level of agreement (> 0.4). While there is considerable overlap in constructs, AMSTAR-2 uniquely addresses explaining the selection of study designs for inclusion, reporting on excluded studies with justification, sources of funding of primary studies, and reviewers’ conflict of interest. By contrast, ROBIS uniquely addresses appropriateness and restrictions within eligibility criteria, reducing risk of error in risk of bias (RoB) assessments, completeness of data extracted for analyses, the inclusion of all necessary studies for analyses, and adherence to predefined analysis plan. Conclusions Among the questions on AMSTAR-2 and ROBIS, 70.3% (26/37 items) address the same or similar methodological constructs. While the IRR of these constructs was moderate to perfect, there are unique methodological constructs that each instrument independently addresses. Notably, both instruments do not address the reporting of absolute estimates of effect or the overall certainty of the evidence, items that are crucial for users’ wishing to interpret the importance of SR/MA results.

10.2196/16791 ◽  
2020 ◽  
Vol 22 (3) ◽  
pp. e16791 ◽  
Author(s):  
Patrick Timpel ◽  
Sarah Oswald ◽  
Peter E H Schwarz ◽  
Lorenz Harst

Background Telemedicine is defined by three characteristics: (1) using information and communication technologies, (2) covering a geographical distance, and (3) involving professionals who deliver care directly to a patient or a group of patients. It is said to improve chronic care management and self-management in patients with chronic diseases. However, currently available guidelines for the care of patients with diabetes, hypertension, or dyslipidemia do not include evidence-based guidance on which components of telemedicine are most effective for which patient populations. Objective The primary aim of this study was to identify, synthesize, and critically appraise evidence on the effectiveness of telemedicine solutions and their components on clinical outcomes in patients with diabetes, hypertension, or dyslipidemia. Methods We conducted an umbrella review of high-level evidence, including systematic reviews and meta-analyses of randomized controlled trials. On the basis of predefined eligibility criteria, extensive automated and manual searches of the databases PubMed, EMBASE, and Cochrane Library were conducted. Two authors independently screened the studies, extracted data, and carried out the quality assessments. Extracted data were presented according to intervention components and patient characteristics using defined thresholds of clinical relevance. Overall certainty of outcomes was assessed using the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) tool. Results Overall, 3564 references were identified, of which 46 records were included after applying eligibility criteria. The majority of included studies were published after 2015. Significant and clinically relevant reduction rates for glycated hemoglobin (HbA1c; ≤−0.5%) were found in patients with diabetes. Higher reduction rates were found for recently diagnosed patients and those with higher baseline HbA1c (>8%). Telemedicine was not found to have a significant and clinically meaningful impact on blood pressure. Only reviews or meta-analyses reporting lipid outcomes in patients with diabetes were found. GRADE assessment revealed that the overall quality of the evidence was low to very low. Conclusions The results of this umbrella review indicate that telemedicine has the potential to improve clinical outcomes in patients with diabetes. Although subgroup-specific effectiveness rates favoring certain intervention and population characteristics were found, the low GRADE ratings indicate that evidence can be considered as limited. Future updates of clinical care and practice guidelines should carefully assess the methodological quality of studies and the overall certainty of subgroup-specific outcomes before recommending telemedicine interventions for certain patient populations.


2021 ◽  
Vol 23 (Supplement_2) ◽  
pp. ii51-ii52
Author(s):  
A M George ◽  
S Gupta ◽  
S M Keshwara ◽  
M A Mustafa ◽  
C S Gillespie ◽  
...  

Abstract BACKGROUND Systematic reviews and meta-analyses constitute the highest level of research evidence and for a disease with limited clinical trial activity, are often relied upon to help inform clinical practice. This review of reviews evaluates both the reporting & methodological quality of meningioma evidence syntheses. MATERIAL AND METHODS Potentially eligible meningioma reviews published between 1st January 1990 and 31st December 2020 were identified from eight electronic databases. Inclusion required the study to meet the Cochrane guideline definition of a systematic review or meta-analysis. Reviews concerning neurofibromatosis type 2, spinal and pediatric meningiomas were excluded. The reporting and methodological quality of articles were assessed against the following modified guidelines: Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA), A MeaSurement Tool to Assess Systematic Reviews (AMSTAR2) and the Risk of Bias in Systematic Reviews (ROBIS) guidelines. RESULTS 117 systematic reviews were identified, 57 of which included meta-analysis (48.7%). The number of meningioma systematic reviews published each year has increased with 63 studies (53.9%) published between 01/2018 and 12/2020. A median of 17 studies (IQR 9–29) were included per review. Impact factor of journals publishing a systematic review with or without a meta-analysis was similar (median 2.3 vs 1.8, P=0.397). The mean PRISMA scores for systematic reviews with a meta-analysis was 21.11 (SD 4.1, 78% adherence) and without was 13.89 (SD 3.4, 63% adherence). Twenty-nine systematic reviews with meta-analysis (51%) and 11 without meta-analysis (18%) achieved greater than 80% adherence to PRISMA recommendations. Methodological quality assessment using AMSTAR2 revealed one study (0.9%) as high quality whilst 111 (94.8%) studies were graded as critically low. One hundred and two articles (87.2%) did not utilize a comprehensive search strategy as defined by the AMSTAR2 tool. Ninety-nine studies (84.6%) obtained a high level of concern for potential bias as per the ROBIS assessment. One hundred and eight articles (92.3%) failed to present information that a protocol had been established prior to study commencement and 76 articles (65.0%) did not conduct a risk of bias assessment. Across the three tools, domains relating to the establishment of a protocol prior to review commencement and conducting appropriate risk of bias assessments were frequently low scoring. CONCLUSION Overall reporting and methodological quality of meningioma systematic reviews was sub-optimal. Established critical appraisal tools and reporting guidelines should be utilized a priori to assist in producing high-quality systematic reviews.


Nutrients ◽  
2021 ◽  
Vol 13 (2) ◽  
pp. 470
Author(s):  
Amélie Keller ◽  
Marie Louise Rimestad ◽  
Jeanett Friis Rohde ◽  
Birgitte Holm Petersen ◽  
Christoffer Bruun Korfitsen ◽  
...  

There has been a growing interest in the gastrointestinal system and its significance for autism spectrum disorder (ASD), including the significance of adopting a gluten-free and casein-free (GFCF) diet. The objective was to investigate beneficial and safety of a GFCF diet among children with a diagnosis of ASD. We performed a systematic literature search in Medline, Embase, Cinahl, and the Cochrane Library up to January 2020 for existing systematic reviews and individual randomized controlled trials (RCTs). Studies were included if they investigated a GFCF diet compared to a regular diet in children aged 3 to 17 years diagnosed with ASD, with or without comorbidities. The quality of the identified existing reviews was assessed using A Measurement Tool to Assess Systematic Reviews (AMSTAR). The risk of bias in RCTs was assessed using the Cochrane Risk of Bias Tool, and overall quality of evidence was evaluated using Grades of Recommendation, Assessment, Development, and Evaluation (GRADE). We identified six relevant RCTs, which included 143 participants. The results from a random effect model showed no effect of a GFCF diet on clinician-reported autism core symptoms (standardized mean difference (SMD) −0.31 (95% Cl. −0.89, 0.27)), parent-reported functional level (mean difference (MD) 0.61 (95% Cl −5.92, 7.14)) or behavioral difficulties (MD 0.80 (95% Cl −6.56, 10.16)). On the contrary, a GFCF diet might trigger gastrointestinal adverse effects (relative risk (RR) 2.33 (95% Cl 0.69, 7.90)). The quality of evidence ranged from low to very low due to serious risk of bias, serious risk of inconsistency, and serious risk of imprecision. Clinical implications of the present findings may be careful consideration of introducing a GFCF diet to children with ASD. However, the limitations of the current literature hinder the possibility of drawing any solid conclusion, and more high-quality RCTs are needed. The protocol is registered at the Danish Health Authority website.


2021 ◽  
Author(s):  
Tan Jun ◽  
Jin-Hui Tian ◽  
Fei Yu ◽  
Xing Wang ◽  
lan Zheng

Abstract BackgroundType 2 Diabetes Mellitus (T2DM) is a serious disease. The intervention of High Intensity Interval Training (HIIT) on T2DM is widely concerned but controversial, which also exists in the current Systematic Reviews/Meta-Analysis (SRs/MAs). The objective of this study is to evaluate the methodology, evidence and reporting quality of SRs/MAs of the effects of HIIT on patients with T2DM. MethodsPubMed, Embase, Cochrane Library, Web of Science, CNKI, WanFang Data, SinoMed and VIP were searched. A MeaSurement Tool to Assess Systemic Reviews 2 (AMSTAR2), Risk of Bias in Systematic Review(ROBIS),Grading of Recommendations Assessment, Development and Evaluation(GRADE)were used to evaluate the included studies.ResultsSix SRs/MAs studies were included, with 79 outcomes from 21 original studies. Methodological quality of AMSTAR2 evaluation was extremely low. ROBIS evaluation showed high risk of bias. There are 39 outcomes of 3 included SRs/MAs used GRADE, of which 29 items are low, 8 items are very low, and 2 items are intermediate quality. Low quality and very low quality evidence bodies account for 95% of the total.ConclusionsSRs/MAs on the effect of HIIT intervention on patients with T2DM included a small number of original studies and the overlapping feature is obvious, outcomes are diverse and overlapping, and there is a lack of key and ending outcomes; From original study, outcomes to system reviews, quality of evaluation is low. Evidence from a large samples, long periods, better protocols, and reflecting ending outcomes and critical outcomes are needed to promote the application of HIIT in T2DM.


2017 ◽  
Vol 42 (8) ◽  
pp. 852-856 ◽  
Author(s):  
J. Wasiak ◽  
A. Y. Shen ◽  
R. Ware ◽  
T. J. O’Donohoe ◽  
C. M. Faggion

The objective of this study was to assess methodological and reporting quality of systematic reviews in hand and wrist pathology. MEDLINE, EMBASE and Cochrane Library were searched from inception to November 2016 for relevant studies. Reporting quality was evaluated using Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) and methodological quality using a measurement tool to assess systematic reviews, the Assessment of Multiple Systematic Reviews (AMSTAR). Descriptive statistics and linear regression were used to identify features associated with improved methodological quality. A total of 91 studies were included in the analysis. Most reviews inadequately reported PRISMA items regarding study protocol, search strategy and bias and AMSTAR items regarding protocol, publication bias and funding. Systematic reviews published in a plastics journal, or which included more authors, were associated with higher AMSTAR scores. A large proportion of systematic reviews within hand and wrist pathology literature score poorly with validated methodological assessment tools, which may affect the reliability of their conclusions. Level of evidence: I


2020 ◽  
Vol 2020 ◽  
pp. 1-14 ◽  
Author(s):  
Yu-Xi Li ◽  
Xi-li Xiao ◽  
Dong-Ling Zhong ◽  
Liao-Jun Luo ◽  
Han Yang ◽  
...  

Background. Migraine is a common neurological disease, which burdens individuals and society all over the world. Acupuncture, an important method in Traditional Chinese Medicine, is widely used in clinical practice as a treatment for migraine. Several systematic reviews (SRs) have investigated the effectiveness and safety of acupuncture for migraine. Objective. To summarize and critically assess the quality of relevant SRs and present an objective and comprehensive evidence on the effectiveness and safety of acupuncture for migraine. Data Sources. MEDLINE, Embase, Cochrane Library, PROSPERO database, Chinese National Knowledge Infrastructure (CNKI), Chinese Biological Medicine (CBM), China Science and Technology Journal (SCTJ), and WanFang database (WF) were searched from inception to December 2019 and grey literatures were manually searched. Selection Criteria. SRs which meet the criteria were independently selected by 2 reviewers according to a predetermined protocol. Data Extraction. Characteristics of included SRs were independently extracted by 2 reviewers following a predefined data extraction form. Review Appraisal. The methodological quality, risk of bias, and reporting quality of included SRs were assessed, respectively, by a Measurement Tool to Assess Systematic Reviews (AMSTAR) 2, the Risk of Bias in Systematic reviews (ROBIS) tool, and the Preferred Reporting Item for Systematic Review and Meta-analysis-Acupuncture (PRISMA-A) statement. The quality of outcomes was evaluated by the Grading of Recommendations Assessment, Development, and Evaluation (GRADE). Results. A total of 15 SRs were included. All the SRs were published between 2011–2019. Based on AMSTAR 2, 14 out of 15 SRs were rated critically low quality and 1 was rated low quality. According to ROBIS tool, 9 SRs (60%) were low risk of bias. With the PRISMA-A checklist, we found 11 out of 15 SRs were found adequately reported over 70%. With the GRADE tool, we found high quality of evidence indicated that the effective rate of acupuncture was superior to western medicine in treatment of migraine. Besides, acupuncture reduced more headache days and the times of using painkiller and was more effective in reducing the frequency and degree of headache than western medicine and sham acupuncture. Limitations. There might be some missing information. The accuracy of the conclusions may be decreased reduced since we were unable to synthesis all the evidence. Conclusions. Based on high quality of evidence, we concluded that acupuncture may be an effective and safe therapy for migraine. However, the quality of SRs in acupuncture for migraine still needs more improvement.


Author(s):  
Morgan Yuan ◽  
Jeremy Wu ◽  
Ryan E Austin ◽  
Frank Lista ◽  
Jamil Ahmad

Abstract Background Breast augmentation is one of the most commonly performed cosmetic surgeries worldwide. Therefore, it is imperative to have evidence with high methodological quality to guide clinical decision making. Objectives To evaluate the methodological quality of the systematic reviews (SRs) focused on breast augmentation. Methods: A comprehensive search of MEDLINE, Embase and the Cochrane Library of Systematic Reviews was performed. SRs that have a particular focus on breast augmentation and were published in the top 15 plastic and reconstructive surgery journals were included. Quality assessment was performed using A Measurement Tool to Assess Systematic Reviews (AMSTAR). Study characteristics were extracted including journal and impact factor, year of publication, country affiliation of corresponding author, reporting adherence to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, number of citations, and number of studies included. Results Among the 22 studies included for analysis, the mean AMSTAR score was moderate (5.55), with no SR achieving good quality (AMSTAR score ≥9). There were no significant associations between AMSTAR score and journal impact factor, number of citations, year of publication, or number of included studies. Studies that reported adherence to PRISMA guidelines on average scored higher on the AMSTAR tool (p=0.03). Conclusions The methodological quality of reviews about breast augmentation was found to be moderate, with no significant increase in studies or quality over time. Adherence to PRISMA guidelines and increased appraisal of SRs about breast augmentation using methodological assessment tools would further strengthen methodological quality and confidence in study findings.


2020 ◽  
Vol 163 (5) ◽  
pp. 892-905
Author(s):  
Rodrigo Martinez-Monedero ◽  
Arman Danielian ◽  
Varun Angajala ◽  
Jennifer E. Dinalo ◽  
Eric J. Kezirian

Objective To assess the methodological quality of intervention-focused systematic reviews (SRs) and meta-analyses (MAs) published in high-impact otolaryngology journals. Data Sources Ovid Medline, Embase, and Cochrane Library. Review Methods A comprehensive search was performed for SR and MA citations from 2012 to 2017 in the 10 highest impact factor otolaryngology journals. Abstracts were screened to identify published manuscripts in which the authors indicated clearly that they were performing an SR or MA. Applying a modified typology of reviews, 4 reviewers characterized the review type as SR, MA, or another review type. A simplified version of the AMSTAR 2 (A MeaSurement Tool to Assess systematic Reviews 2) tool was used to assess the reporting and methodological quality of the SRs and MAs that were focused on interventions. Results Search and abstract screening generated 499 manuscripts that identified themselves as performing an SR or MA. A substantial number (85/499, 17%) were review types other than SRs or MAs, including 34 (7%) that were literature reviews. In total, 236 SRs and MAs focused on interventions. Over 50% of these SRs and MAs had weaknesses in at least 3 of the 16 items in the AMSTAR 2, and over 40% had weaknesses in at least 2 of the 7 critical domains. Ninety-nine percent of SRs and MAs provided critically low confidence in the results of the reviews. Conclusion Intervention-focused SRs and MAs published in high-impact otolaryngology journals have important methodological limitations that diminish confidence in the results of these reviews.


PeerJ ◽  
2021 ◽  
Vol 9 ◽  
pp. e11713
Author(s):  
Ana Carolina Hovadick ◽  
Viviane Rodrigues Jardim ◽  
Constança Paúl ◽  
Adriana Pagano ◽  
Ilka Reis ◽  
...  

Background The family caregivers of patients on hemodialysis (HD) and peritoneal dialysis (PD) typically experience higher burden than the general population because of the nature of tasks these caregivers need to carry out as a part of homecare. This fact influences both the caregivers’ quality of life and the quality of their care toward the patient. Thus, this study aimed to review the effectiveness and limitations of interventions in improving the well-being of family caregivers of patients on HD and PD. Methodology A systematic review was performed according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses and the Cochrane Handbook for Systematic Reviews of Interventions (version 5.1.0). The Cochrane Library, Cumulative Index to Nursing and Allied Health Literature, Embase, MEDLINE, VHL Regional Portal, Scopus, and Web of Science databases were searched queried for randomized controlled trials that developed interventions aimed at improving the well-being of family caregivers of patients undergoing HD and/or PD from 2009 to 2020. The study protocol was registered at the International Prospective Register of Systematic Reviews (registration no. CRD42020151161). Results Six studies met the inclusion criteria, all of which addressed caregivers of patients undergoing HD. All interventions reported in the included studies were carried out in group sessions, which addressed topics such as patient assistance and care, treatment complications, coping strategies, caregiver self-care practices, problem solving, and self-efficacy. The studies found significant improvement in the caregiver’s well-being. Conclusions Group session interventions are effective in improving the well-being of family caregivers of patients undergoing HD. In regard to PD, there is insufficient evidence to make recommendations for caregivers of patients with this treatment.


2020 ◽  
Author(s):  
Elena Dragioti ◽  
Huan-Ji Dong ◽  
Britt Larsson ◽  
Björn Gerdle

BACKGROUND Interdisciplinary pain treatment (IPT) is a complex intervention; its outcomes are very diverse, as are the methodologies for handling those outcomes. This diversity may hamper evidence-based decision making. Presently, there is no gold standard recommendation of how to select reported outcomes in published systematic reviews and meta-analyses to explicitly demonstrate the effectiveness of IPT. OBJECTIVE In this systematic overview, we aim to evaluate the reported outcome domains and measurements across published systematic reviews and meta-analyses and to identify any methods, considerations, and discussion regarding the handling of the chosen outcome domains and measurements. METHODS This article describes the protocol for a systematic overview of the outcomes reported in published systematic reviews and meta-analyses of randomized control trials for the effectiveness of IPT versus any control. To this end, we searched the PubMed, Cochrane Library, and Epistemonikos databases from inception to December 2019. Two independent investigators screened the titles, the abstracts of the identified records, and the full texts of the potentially eligible systematic reviews and meta-analyses, performed data extraction according to predefined forms, and rated the quality of the included systematic reviews and meta-analyses. The quality of the included systematic reviews and meta-analyses will be rated with AMSTAR (A MeaSurement Tool to Assess systematic Reviews) 2. Data will be analyzed descriptively and stratified by AMSTAR 2. RESULTS We introduced the rationale and design of a systematic overview to summarize and map the chosen IPT outcome domains and the methods of handling these outcomes reported in published systematic reviews and meta-analyses. As of December 2019, we collected 5229 systematic reviews, of which 147 (2.81%) were examined in-depth for eligibility. Topline results are anticipated by September 2020. CONCLUSIONS The results of this study will be published as soon as they are available. Our results will fill a gap in the related literature and will be used to inform the development of a set of recommendations that can be applied in systematic reviews and hopefully serve as a gold standard. INTERNATIONAL REGISTERED REPORT PRR1-10.2196/17795


Sign in / Sign up

Export Citation Format

Share Document