Predictors of Higher Quality of Systematic Reviews Addressing Nutrition and Cancer Prevention

Systematic reviews/meta-analyses (SR/MAs) are considered a reliable source of information in healthcare. We aimed to explore the association of several characteristics of SR/MAs addressing nutrition in cancer prevention and their quality/risk of bias (using assessments from AMSTAR-2 and ROBIS tools). The analysis included 101 SR/MAs identified in a systematic survey. Associations of each specified characteristic (e.g., information about the protocol, publication year, reported use of GRADE, or other methods for assessing overall certainty of evidence) with the number of AMSTAR-2 not met (‘No’ responses) and the number of ROBIS items met (‘Probably Yes’ or “Yes’ responses) were examined. Poisson regression was used to identify predictors of the number of ‘No’ answers (indicating lower quality) for all AMSTAR-2 items and the number of ‘Yes’ or ‘Probably Yes’ answers (indicating higher quality/lower concern for bias) for all ROBIS items. Logistic regression was used to identify variables associated with at least one domain assessed as ‘low concern for bias’ in the ROBIS tool. In multivariable analysis, SR/MAs not reporting use of any quality/risk of bias assessment instrument for primary studies were associated with a higher number of ‘No’ answers for all AMSTAR-2 items (incidence rate ratio (IRR) 1.26, 95% confidence interval (CI) 1.09–1.45), and a lower number of ‘Yes’ or ‘Probably Yes’ answers for all ROBIS items (IRR 0.76, 95% CI 0.66–0.87). Providing information about the protocol and search for unpublished studies was associated with a lower number of ‘No’ answers (IRR 0.73, 95% CI 0.56–0.97 and IRR 0.75, 95% CI 0.59–0.95, respectively) and a higher number of ‘Yes’ or ‘Probably Yes’ answers (IRR 1.43, 95% CI 1.17–1.74 and IRR 1.28, 95% CI 1.07–1.52, respectively). Not using at least one quality/risk of bias assessment tool for primary studies within an SR/MA was associated with lower odds that a study would be assessed as ‘low concern for bias’ in at least one ROBIS domain (odds ratio 0.061, 95% CI 0.007–0.527). Adherence to methodological standards in the development of SR/MAs was associated with a higher overall quality of SR/MAs addressing nutrition for cancer prevention.

Download Full-text

Toolkit of methodological resources to conduct systematic reviews

F1000Research ◽

10.12688/f1000research.22032.1 ◽

2020 ◽

Vol 9 ◽

pp. 82

Author(s):

Marta Roqué ◽

Laura Martínez-García ◽

Ivan Solà ◽

Pablo Alonso-Coello ◽

Xavier Bonfill ◽

...

Keyword(s):

Diagnostic Accuracy ◽

Systematic Reviews ◽

Assessment Tool ◽

Research Question ◽

Risk Of Bias ◽

Bias Assessment ◽

Assessment Development ◽

Stage Of Development ◽

Model Study ◽

Question Development

Background: Systematic reviews (SR) can be classified by type depending on the research question they are based on. This work identifies and describes the most relevant methodological resources to conduct high-quality reviews that answer clinical questions regarding prevalence, prognosis, diagnostic accuracy and efficacy of interventions. Methods: Methodological resources have been identified from literature searches and consulting guidelines from institutions that develop SRs. The selected resources are organized by type of SR, and stage of development of the review (formulation of the research question, development of the protocol, literature search, risk of bias assessment, synthesis of findings, assessment of the quality of evidence, and report of SR results and conclusions). Results: Although the different types of SRs are developed following the same steps, each SR type requires specific methods, differing in characteristics and complexity. The extent of methodological development varies by type of SR, with more solid guidelines available for diagnostic accuracy and efficacy of interventions SRs. This methodological toolkit describes the most up-to-date risk of bias instruments: Quality in Prognostic Studies (QUIPS) tool and Prediction model study Risk Of Bias Assessment Tool (PROBAST) for prognostic SRs, Quality assessment of diagnostic accuracy studies tool (QUADAS-2) for diagnostic accuracy SRs, Cochrane risk of bias tool (ROB-2) and Risk of bias in non-randomised studies of interventions studies tool (ROBINS-I) for efficacy of interventions SRs, as well as the latest developments on the Grading of Recommendations Assessment, Development and Evaluation (GRADE) system. Conclusions: This structured compilation of the best methodological resources for each type of SR may prove to be a very useful tool for those researchers that wish to develop SRs or conduct methodological research works on SRs.

Download Full-text

Toolkit of methodological resources to conduct systematic reviews

F1000Research ◽

10.12688/f1000research.22032.3 ◽

2020 ◽

Vol 9 ◽

pp. 82

Author(s):

Marta Roqué ◽

Laura Martínez-García ◽

Ivan Solà ◽

Pablo Alonso-Coello ◽

Xavier Bonfill ◽

...

Keyword(s):

Diagnostic Accuracy ◽

Systematic Reviews ◽

Assessment Tool ◽

Research Question ◽

Risk Of Bias ◽

Bias Assessment ◽

Assessment Development ◽

Stage Of Development ◽

Model Study ◽

Question Development

Background: Systematic reviews (SR) can be classified by type depending on the research question they are based on. This work identifies and describes the most relevant methodological resources to conduct high-quality reviews that answer health care questions regarding prevalence, prognosis, diagnostic accuracy and effects of interventions. Methods: Methodological resources have been identified from literature searches and consulting guidelines from institutions that develop SRs. The selected resources are organized by type of SR, and stage of development of the review (formulation of the research question, development of the protocol, literature search, risk of bias assessment, synthesis of findings, assessment of the quality of evidence, and report of SR results and conclusions). Results: Although the different types of SRs are developed following the same steps, each SR type requires specific methods, differing in characteristics and complexity. The extent of methodological development varies by type of SR, with more solid guidelines available for diagnostic accuracy and effects of interventions SRs. This methodological toolkit describes the most up-to-date risk of bias instruments: Quality in Prognostic Studies (QUIPS) tool and Prediction model study Risk Of Bias Assessment Tool (PROBAST) for prognostic SRs, Quality assessment of diagnostic accuracy studies tool (QUADAS-2) for diagnostic accuracy SRs, Cochrane risk of bias tool (ROB-2) and Risk of bias in non-randomised studies of interventions studies tool (ROBINS-I) for effects of interventions SRs, as well as the latest developments on the Grading of Recommendations Assessment, Development and Evaluation (GRADE) system. Conclusions: This structured compilation of the best methodological resources for each type of SR may prove to be a very useful tool for those researchers that wish to develop SRs or conduct methodological research works on SRs

Download Full-text

Toolkit of methodological resources to conduct systematic reviews

F1000Research ◽

10.12688/f1000research.22032.2 ◽

2020 ◽

Vol 9 ◽

pp. 82

Author(s):

Marta Roqué ◽

Laura Martínez-García ◽

Ivan Solà ◽

Pablo Alonso-Coello ◽

Xavier Bonfill ◽

...

Keyword(s):

Diagnostic Accuracy ◽

Systematic Reviews ◽

Assessment Tool ◽

Research Question ◽

Risk Of Bias ◽

Bias Assessment ◽

Assessment Development ◽

Stage Of Development ◽

Model Study ◽

Question Development

Background: Systematic reviews (SR) can be classified by type depending on the research question they are based on. This work identifies and describes the most relevant methodological resources to conduct high-quality reviews that answer health care questions regarding prevalence, prognosis, diagnostic accuracy and effects of interventions. Methods: Methodological resources have been identified from literature searches and consulting guidelines from institutions that develop SRs. The selected resources are organized by type of SR, and stage of development of the review (formulation of the research question, development of the protocol, literature search, risk of bias assessment, synthesis of findings, assessment of the quality of evidence, and report of SR results and conclusions). Results: Although the different types of SRs are developed following the same steps, each SR type requires specific methods, differing in characteristics and complexity. The extent of methodological development varies by type of SR, with more solid guidelines available for diagnostic accuracy and effects of interventions SRs. This methodological toolkit describes the most up-to-date risk of bias instruments: Quality in Prognostic Studies (QUIPS) tool and Prediction model study Risk Of Bias Assessment Tool (PROBAST) for prognostic SRs, Quality assessment of diagnostic accuracy studies tool (QUADAS-2) for diagnostic accuracy SRs, Cochrane risk of bias tool (ROB-2) and Risk of bias in non-randomised studies of interventions studies tool (ROBINS-I) for effects of interventions SRs, as well as the latest developments on the Grading of Recommendations Assessment, Development and Evaluation (GRADE) system. Conclusions: This structured compilation of the best methodological resources for each type of SR may prove to be a very useful tool for those researchers that wish to develop SRs or conduct methodological research works on SRs

Download Full-text

A critical appraisal of the risk of bias in systematic reviews and metaanalyses pertaining to COVID-19

Coronaviruses ◽

10.2174/2666796701666201230105144 ◽

2020 ◽

Vol 01 ◽

Author(s):

Amit Dang ◽

Sheshank Madiraju ◽

Jagan Mohan Venkateswara Rao P ◽

Navya Sri Gurram ◽

Sandeep Digijarala ◽

...

Keyword(s):

High Risk ◽

Systematic Reviews ◽

Risk Of Bias ◽

Therapy Outcomes ◽

Data Synthesis ◽

Bias Assessment ◽

Meta Analyses ◽

Unclear Risk ◽

Overall Risk

Background: We critically evaluated the risk of bias in published systematic reviews (SRs) and meta-analyses (MAs) pertaining to COVID-19 using ROBIS tool. Materials And Methods: Medline and Cochrane Central Library were searched for SRs/MAs on 14th May 2020, including studies of all designs describing various facets of COVID-19 in humans; no restrictions were applied for interventions, comparators, and outcomes. Two reviewers independently assessed all the SRs/MAs with ROBIS. Results: Out of 204 identified records, 48 SRs/MAs were included. The most frequently reviewed topics were therapy outcomes, diagnosis, and comorbidities (15, 8, and 6 papers respectively). Only 29/48(60.41%) papers had made a mention of using PRISMA or other guidelines for drafting the SR/MA. Only 5/48(10.42%) of all included SRs/MAs had low overall risk of bias as per ROBIS tool; 41/48(85.42%) had high risk of bias, 2/48(4.17%) had unclear risk of bias. The highest proportion of bias was found in data synthesis and findings (30/48, 62.50% of studies had high risk of bias), followed by study identification and selection (29/48, 60.42%). The IRR for methodological quality assessment was substantial, with the Cohen’s kappa values being 0.64, 0.68, 0.62, and 0.75 for domains 1-4 of ROBIS tool, and 0.66 for overall risk of bias assessment. Conclusion: There are serious concerns about the methodology employed to generate the results of the SRs/ MAs pertaining to COVID-19, with ‘quantity’ seemingly being given more importance than ‘quality’ of the paper.

Download Full-text

Meta-Analysis of Chinese Traditional Medicine Bushen Huoxue Prescription for Endometriosis Treatment

Evidence-based Complementary and Alternative Medicine ◽

10.1155/2017/5416423 ◽

2017 ◽

Vol 2017 ◽

pp. 1-10 ◽

Cited By ~ 4

Author(s):

Jing Shan ◽

Wen Cheng ◽

Dong-xia Zhai ◽

Dan-ying Zhang ◽

Rui-pin Yao ◽

...

Keyword(s):

Western Medicine ◽

Methodological Quality ◽

Assessment Tool ◽

Meta Analysis ◽

Risk Of Bias ◽

Efficacy And Safety ◽

Bias Assessment ◽

Randomized Controlled ◽

Alternative Drug

Objectives. To evaluate the efficacy and safety of Bushen Huoxue prescription (BSHXP) for endometriosis. Methods. A meta-analysis was performed, and studies were searched from the seven databases from the date of database establishment to April 30, 2017. Randomized controlled trials (RCTs) that explored the efficacy and safety of BSHXP for patients with endometriosis were included. Two assessors independently reviewed each trial. The Cochrane Risk of Bias assessment tool was used for quality assessment. Results. In the 13 included studies, the total effectiveness rates of BSHXP were higher than those of Western medicine (RR, 1.55; 95% CI, 1.03–2.32; P=0.04), but the dysmenorrhea alleviation rates of the two treatments did not significantly differ (RR, 1.28; 95% CI, 0.70–2.34; P=0.42). The pregnancy rates of BSHXP were also higher than those of hormone therapy (RR, 1.99; 95% CI, 1.17–3.39; P=0.01). However, whether BSHXP is more effective than Western medicine in diminishing endometriotic cyst remains unknown. Conclusions. Our study provides evidence that BSHXP is effective and safe for endometriosis, but this evidence is inconclusive because of the low methodological quality of the included RCTs. Our findings suggest that BSHXP is an alternative drug for endometriosis, but it should be further examined in future clinical research.

Download Full-text

Reliability of studies published as SR/MA on nutrition in cancer prevention - a systematic survey

European Journal of Public Health ◽

10.1093/eurpub/ckaa165.1415 ◽

2020 ◽

Vol 30 (Supplement_5) ◽

Author(s):

D Storman ◽

M M Bala ◽

M Koperny ◽

J Zajac ◽

P Tobola ◽

...

Keyword(s):

Cancer Prevention ◽

Systematic Reviews ◽

Data Extraction ◽

Poor Quality ◽

Risk Of Bias ◽

Measurement Tool ◽

Nutritional Interventions ◽

Public Media ◽

Meta Analyses

Abstract Background In several fields of medicine, the quality of studies published as systematic reviews/meta-analyses (SR/MAs) is low. Similar problems may exist for SR/MA on nutrition in cancer prevention. We aimed to assess overall quality and risk of bias (RoB) of studies published as SR/MA on nutritional interventions in cancer prevention with two instruments: AMSTAR 2 ('a measurement tool to assess systematic review 2') and ROBIS ('Risk of Bias in Systematic Reviews') respectively. Methods Following a systematic search in 3 databases we included studies identified as SR/MA published between 2010 and 2018 assessing any nutritional interventions in cancer prevention in the general population or among people with cancer risk (Protocol in PROSPERO CRD42019121116). All the steps of study selection and data extraction were done by two independent reviewers with conflicts solved by discussion or by the third reviewer. Results We focused on a subsample of 101 SR/MA randomly selected from 737 included SR/MA. Included SR/MA on average searched 2 databases with Medline in 98% and included cohort studies (93%). They focused on specific food (36%), specific nutrients (27%) or beverages (24%, mostly tea and coffee). The assessment using AMSTAR 2 tool indicated that 93% of SR/MA had no pre-specified methodology, in 77% - research questions and inclusion criteria did not include the components of PICO, RoB assessment of primary studies was not used or did not contain all elements (87%) and RoB was not accounted for in the interpretation of the results (75%). Overall, the quality of 97% of studies was assessed as critically low. In the ROBIS tool for 97% of included studies, the overall high risk of bias was detected. The most important methodological flaws in ROBIS were similar to identified in AMSTAR tool. Conclusions Poor quality of SR/MA due to flawed methodology may lead to many concerns and mislead public media and consumers. Key messages Poor quality of SR/MA due to flawed methodology may lead to many concerns and mislead public media and consumers. The studies published as SR/MAs addressing nutrition for cancer prevention have major flaws, which limit the reliability of their conclusions.

Download Full-text

Methodological quality of systematic reviews on treatments for depression: a cross-sectional study

Epidemiology and Psychiatric Sciences ◽

10.1017/s2045796017000208 ◽

2017 ◽

Vol 27 (6) ◽

pp. 619-627 ◽

Cited By ~ 6

Author(s):

V. C. H. Chung ◽

X. Y. Wu ◽

Y. Feng ◽

R. S. T. Ho ◽

S. Y. S. Wong ◽

...

Keyword(s):

Systematic Reviews ◽

Methodological Quality ◽

A Priori ◽

Cross Sectional Study ◽

Risk Of Bias ◽

Sectional Study ◽

Pharmacological Treatments ◽

Cross Sectional ◽

Depression Treatments

Aims.Depression is one of the most common mental disorders and identifying effective treatment strategies is crucial for the control of depression. Well-conducted systematic reviews (SRs) and meta-analyses can provide the best evidence for supporting treatment decision-making. Nevertheless, the trustworthiness of conclusions can be limited by lack of methodological rigour. This study aims to assess the methodological quality of a representative sample of SRs on depression treatments.Methods.A cross-sectional study on the bibliographical and methodological characteristics of SRs published on depression treatments trials was conducted. Two electronic databases (the Cochrane Database of Systematic Reviews and the Database of Abstracts of Reviews of Effects) were searched for potential SRs. SRs with at least one meta-analysis on the effects of depression treatments were considered eligible. The methodological quality of included SRs was assessed using the validated AMSTAR (Assessing the Methodological Quality of Systematic Reviews) tool. The associations between bibliographical characteristics and scoring on AMSTAR items were analysed using logistic regression analysis.Results.A total of 358 SRs were included and appraised. Over half of included SRs (n = 195) focused on non-pharmacological treatments and harms were reported in 45.5% (n = 163) of all studies. Studies varied in methods and reporting practices: only 112 (31.3%) took the risk of bias among primary studies into account when formulating conclusions; 245 (68.4%) did not fully declare conflict of interests; 93 (26.0%) reported an ‘a priori’ design and 104 (29.1%) provided lists of both included and excluded studies. Results from regression analyses showed: more recent publications were more likely to report ‘a priori’ designs [adjusted odds ratio (AOR) 1.31, 95% confidence interval (CI) 1.09–1.57], to describe study characteristics fully (AOR 1.16, 95% CI 1.06–1.28), and to assess presence of publication bias (AOR 1.13, 95% CI 1.06–1.19), but were less likely to list both included and excluded studies (AOR 0.86, 95% CI 0.81–0.92). SRs published in journals with higher impact factor (AOR 1.14, 95% CI 1.04–1.25), completed by more review authors (AOR 1.12, 95% CI 1.01–1.24) and SRs on non-pharmacological treatments (AOR 1.62, 95% CI 1.01–2.59) were associated with better performance in publication bias assessment.Conclusion.The methodological quality of included SRs is disappointing. Future SRs should strive to improve rigour by considering of risk of bias when formulating conclusions, reporting conflict of interests and authors should explicitly describe harms. SR authors should also use appropriate methods to combine the results, prevent language and publication biases, and ensure timely updates.

Download Full-text

The use of posture-correcting shirts for managing musculoskeletal pain is not supported by current evidence – a scoping review of the literature

Scandinavian Journal of Pain ◽

10.1515/sjpain-2019-0005 ◽

2019 ◽

Vol 19 (4) ◽

pp. 659-670

Author(s):

Thorvaldur Skuli Palsson ◽

Mervyn J. Travers ◽

Trine Rafn ◽

Stian Ingemann-Molden ◽

J.P. Caneiro ◽

...

Keyword(s):

Musculoskeletal Pain ◽

Scoping Review ◽

Management Strategy ◽

Assessment Tool ◽

Risk Of Bias ◽

Current Evidence ◽

Cochrane Library ◽

Management Approach ◽

Clinical Populations

Abstract Background and aims The concept of bad posture being a dominant driver of pain is commonly held belief in the society. This may explain the significant attention supportive clothing such as posture-correcting shirts has recently gained in Scandinavia and the USA. The aim of this scoping review was to present an overview and synthesis of the available evidence for the use of posture-correcting shirts aimed at reducing pain or postural discomfort and optimising function/posture. Methods A systematic search was conducted for literature investigating the effect of posture-correcting shirts on musculoskeletal pain or function. PubMed, Embase, CINAHL, PEDro and the Cochrane Library were searched for relevant literature. Results of the searches were evaluated by two independent reviewers in three separate steps based on title, abstract and full text. For data synthesis, the population, intervention, comparator and outcome were extracted. The quality of the literature was evaluated using the Quality Assessment Tool for Observational Cohort and Cross-Sectional Studies and the risk of bias was assessed using the Risk Of Bias In Non-randomized Studies – of Interventions (ROBINS-I) assessment tool or the RoB 2.0 tool for individually randomized, parallel group trials. The overall confidence in the literature was determined using the Grading of Recommendations Assessment, Development and Evaluation (GRADE). Results A total of 136 articles were identified and six of these were included in the review. These studies were heterogeneous with regards to aims, outcomes and methods, presenting contrasting results. The overall findings were that posture-correcting shirts change posture and subjectively have a positive effect on discomfort, energy levels and productivity. The quality of the included literature was poor to fair with only one study being of good quality. The risk of bias was serious or critical for the included studies. Overall, this resulted in very low confidence in available evidence. An important limitation of all studies was that they were conducted in pain-free individuals. Conclusions The contrasting findings and the low quality of current literature, questions the intended effect of posture-correcting shirts and whether the changes it creates are in fact useful for clinical practice. Moreover, the findings are contrasted by the available evidence regarding posture and pain with a particular focus on whether this management strategy may have a detrimental effect on people living with musculoskeletal pain. A major limitation to the existing literature on the effect of posture-correcting shirts is that no studies have investigated their effect in clinical populations. Implications Based on the available literature and the major limitation of no studies investigating clinical populations, there is no good quality evidence to support recommendation of posture-correcting shirts as a management strategy for musculoskeletal pain. Promotion of this product may reinforce the inaccurate and unhelpful message that poor posture leads to pain. The efficacy of such garments should be tested in clinical populations and not only in pain-free individuals, to assess whether there is any meaningful benefit of this management approach. Until then, the use of posture-correcting shirts for musculoskeletal pain is not supported by current evidence.

Download Full-text

Prognostic models for predicting in-hospital paediatric mortality in resource-limited countries: a systematic review

BMJ Open ◽

10.1136/bmjopen-2019-035045 ◽

2020 ◽

Vol 10 (10) ◽

pp. e035045

Author(s):

Morris Ogero ◽

Rachel Jelagat Sarguta ◽

Lucas Malla ◽

Jalemba Aluvaala ◽

Ambrose Agweyu ◽

...

Keyword(s):

Systematic Review ◽

Low Income ◽

Assessment Tool ◽

Data Extraction ◽

Model Development ◽

Risk Of Bias ◽

Prognostic Models ◽

Risk Scores ◽

Eligibility Criteria ◽

Bias Assessment

ObjectivesTo identify and appraise the methodological rigour of multivariable prognostic models predicting in-hospital paediatric mortality in low-income and middle-income countries (LMICs).DesignSystematic review of peer-reviewed journals.Data sourcesMEDLINE, CINAHL, Google Scholar and Web of Science electronic databases since inception to August 2019.Eligibility criteriaWe included model development studies predicting in-hospital paediatric mortality in LMIC.Data extraction and synthesisThis systematic review followed the Checklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies framework. The risk of bias assessment was conducted using Prediction model Risk of Bias Assessment Tool (PROBAST). No quantitative summary was conducted due to substantial heterogeneity that was observed after assessing the studies included.ResultsOur search strategy identified a total of 4054 unique articles. Among these, 3545 articles were excluded after review of titles and abstracts as they covered non-relevant topics. Full texts of 509 articles were screened for eligibility, of which 15 studies reporting 21 models met the eligibility criteria. Based on the PROBAST tool, risk of bias was assessed in four domains; participant, predictors, outcome and analyses. The domain of statistical analyses was the main area of concern where none of the included models was judged to be of low risk of bias.ConclusionThis review identified 21 models predicting in-hospital paediatric mortality in LMIC. However, most reports characterising these models are of poor quality when judged against recent reporting standards due to a high risk of bias. Future studies should adhere to standardised methodological criteria and progress from identifying new risk scores to validating or adapting existing scores.PROSPERO registration numberCRD42018088599.

Download Full-text