scholarly journals Improving the trustworthiness of findings from nutrition evidence syntheses: assessing risk of bias and rating the certainty of evidence

2020 ◽  
Author(s):  
Lukas Schwingshackl ◽  
Holger J. Schünemann ◽  
Joerg J. Meerpohl
Keyword(s):  
Systematic Reviews ◽  
Risk Of Bias ◽  
Dietary Guidelines ◽  
United States Department ◽  
Dietary Recommendations ◽  
Assessment Development ◽  
Nutrition Research ◽  
Key Issues ◽  
Nutritional Studies ◽  
Grade Approach

AbstractSuboptimal diet is recognized as a leading modifiable risk factor for non-communicable diseases. Non-randomized studies (NRSs) with patient relevant outcomes provide many insights into diet–disease relationships. Dietary guidelines are based predominantly on findings from systematic reviews of NRSs—mostly prospective observational studies, despite that these have been repeatedly criticized for yielding potentially less trustworthy results than randomized controlled trials (RCTs). It is assumed that these are a result of bias due to prevalent-user designs, inappropriate comparators, residual confounding, and measurement error. In this article, we aim to highlight the importance of applying risk of bias (RoB) assessments in nutritional studies to improve the credibility of evidence of systematic reviews. First, we discuss the importance and challenges of dietary RCTs and NRSs, and provide reasons for potentially less trustworthy results of dietary studies. We describe currently used tools for RoB assessment (Cochrane RoB, and ROBINS-I), describe the importance of rigorous RoB assessment in dietary studies and provide examples that further the understanding of the key issues to overcome in nutrition research. We then illustrate, by comparing the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach with current approaches used by United States Department of Agriculture Dietary Guidelines for Americans, and the World Cancer Research Fund, how to establish trust in dietary recommendations. Our overview shows that the GRADE approach provides more transparency about the single domains for grading the certainty of the evidence and the strength of recommendations. Despite not increasing the certainty of evidence itself, we expect that the rigorous application of the Cochrane RoB and the ROBINS-I tools within systematic reviews of both RCTs and NRSs and their integration within the GRADE approach will strengthen the credibility of dietary recommendations.

scholarly journals Investigation of Risk Of Bias due to Unreported and SelecTively included results in meta-analyses of nutrition research: the ROBUST study protocol

F1000Research ◽  
2019 ◽  
Vol 8 ◽  
pp. 1760 ◽  
Author(s):  
Matthew J. Page ◽  
Lisa Bero ◽  
Cynthia M. Kroeger ◽  
Zhaoli Dai ◽  
Sally McDonald ◽  
...  
Keyword(s):  
Meta Analysis ◽  
Scientific Evidence ◽  
Open Science ◽  
Included Study ◽  
Risk Of Bias ◽  
Dietary Guidelines ◽  
Healthy Population ◽  
Nutrition Research ◽  
Health Related ◽  
Meta Analyses

Background: Dietary guidelines should be informed by systematic reviews (SRs) of the available scientific evidence. However, if the SRs that underpin dietary guidelines are flawed in their design, conduct or reporting, the recommendations contained therein may be misleading or harmful. To date there has been little empirical investigation of bias due to selective inclusion of results, and bias due to missing results, in SRs of food/diet-outcome relationships. Objectives: To explore in SRs with meta-analyses of the association between food/diet and health-related outcomes: (i) whether systematic reviewers selectively included study effect estimates in meta-analyses when multiple effect estimates were available; (ii) what impact selective inclusion of study effect estimates may have on meta-analytic effects, and; (iii) the risk of bias due to missing results (publication bias and selective non-reporting bias) in meta-analyses. Methods: We will systematically search for SRs with meta-analysis of the association between food/diet and health-related outcomes in a generally healthy population, published between January 2018 and June 2019. We will randomly sort titles and abstracts and screen them until we identify 50 eligible SRs. The first reported meta-analysis of a binary or continuous outcome in each SR (the ‘index meta-analysis’) will be evaluated. We will extract from study reports all study effect estimates that were eligible for inclusion in the index meta-analyses (e.g. from multiple instruments and time points) and will quantify and test for evidence of selective inclusion of results. We will also assess the risk of bias due to missing results in the index meta-analyses using a new tool (ROB-ME). Ethics and dissemination: Ethics approval is not required because information will only be extracted from published studies. Dissemination of the results will be through peer-reviewed publications and presentations at conferences. We will make all data collected from this study publicly available via the Open Science Framework.

scholarly journals Comparison of children's dietary intake patterns with US dietary guidelines

2000 ◽  
Vol 84 (3) ◽  
pp. 361-367 ◽  
Author(s):  
Louise M. Brady ◽  
Christine H. Lindquist ◽  
Sara L. Herd ◽  
Michael I. Goran
Keyword(s):  
Dietary Intake ◽  
Nutrition Education ◽  
The United States ◽  
Dietary Guidelines ◽  
United States Department ◽  
Adult Health ◽  
Dietary Intakes ◽  
Food Groups ◽  
Dietary Recommendations ◽  
American Children

Monitoring dietary intake patterns among children is important in order to explore and prevent the onset of adult health problems. The aim of the present study was to compare children's dietary intakes with national recommendations and to determine whether sex or ethnic differences were evident. This was done using a methodology that allows assessment of intake from the major components of the Food Guide Pyramid developed by the United States Department of Agriculture (USDA: ). The sample studied included 110 African-American and Caucasian males and females (mean age 9·9 years, BMI 20·1 kg/m2) from Birmingham, AL, USA, who were participating in a study investigating the development of obesity. Dietary data were based on three 24 h recalls and food group intake was determined using the USDA Pyramid Servicing Database. The results indicated that a high percentage of subjects failed to meet the recommended number of servings from each of the food groups. For example, only 5 % and 9 % met fruit and dietary group recommendations respectively. Consumption of foods from the Pyramid ‘tip’ (including discretionary fat and added sugar) contributed almost 50 % of the diet. African-Americans were more likely to meet requirements for the meat group, with a higher proportion of Caucasians meeting dietary recommendations. Males were more likely to meet the vegetable group guidelines although females consumed more energy per day from discretionary fat. In conclusion, these results suggest that implementation of nutrition education programmes may be important for promoting healthy nutrition among American children.

scholarly journals Toolkit of methodological resources to conduct systematic reviews

F1000Research ◽  
2020 ◽  
Vol 9 ◽  
pp. 82
Author(s):  
Marta Roqué ◽  
Laura Martínez-García ◽  
Ivan Solà ◽  
Pablo Alonso-Coello ◽  
Xavier Bonfill ◽  
...  
Keyword(s):  
Diagnostic Accuracy ◽  
Systematic Reviews ◽  
Assessment Tool ◽  
Research Question ◽  
Risk Of Bias ◽  
Bias Assessment ◽  
Assessment Development ◽  
Stage Of Development ◽  
Model Study ◽  
Question Development

Background: Systematic reviews (SR) can be classified by type depending on the research question they are based on. This work identifies and describes the most relevant methodological resources to conduct high-quality reviews that answer clinical questions regarding prevalence, prognosis, diagnostic accuracy and efficacy of interventions. Methods: Methodological resources have been identified from literature searches and consulting guidelines from institutions that develop SRs. The selected resources are organized by type of SR, and stage of development of the review (formulation of the research question, development of the protocol, literature search, risk of bias assessment, synthesis of findings, assessment of the quality of evidence, and report of SR results and conclusions). Results: Although the different types of SRs are developed following the same steps, each SR type requires specific methods, differing in characteristics and complexity. The extent of methodological development varies by type of SR, with more solid guidelines available for diagnostic accuracy and efficacy of interventions SRs. This methodological toolkit describes the most up-to-date risk of bias instruments: Quality in Prognostic Studies (QUIPS) tool and Prediction model study Risk Of Bias Assessment Tool (PROBAST) for prognostic SRs, Quality assessment of diagnostic accuracy studies tool (QUADAS-2) for diagnostic accuracy SRs, Cochrane risk of bias tool (ROB-2) and Risk of bias in non-randomised studies of interventions studies tool (ROBINS-I) for efficacy of interventions SRs, as well as the latest developments on the Grading of Recommendations Assessment, Development and Evaluation (GRADE) system. Conclusions: This structured compilation of the best methodological resources for each type of SR may prove to be a very useful tool for those researchers that wish to develop SRs or conduct methodological research works on SRs.

scholarly journals Investigation of Risk Of Bias due to Unreported and SelecTively included results in meta-analyses of nutrition research: the ROBUST study protocol

F1000Research ◽  
2020 ◽  
Vol 8 ◽  
pp. 1760
Author(s):  
Matthew J. Page ◽  
Lisa Bero ◽  
Cynthia M. Kroeger ◽  
Zhaoli Dai ◽  
Sally McDonald ◽  
...  
Keyword(s):  
Meta Analysis ◽  
Scientific Evidence ◽  
Open Science ◽  
Included Study ◽  
Risk Of Bias ◽  
Dietary Guidelines ◽  
Healthy Population ◽  
Nutrition Research ◽  
Health Related ◽  
Meta Analyses

Background: Dietary guidelines should be informed by systematic reviews (SRs) of the available scientific evidence. However, if the SRs that underpin dietary guidelines are flawed in their design, conduct or reporting, the recommendations contained therein may be misleading or harmful. To date there has been little empirical investigation of bias due to selective inclusion of results, and bias due to missing results, in SRs of food/diet-outcome relationships. Objectives: To explore in SRs with meta-analyses of the association between food/diet and health-related outcomes: (i) whether systematic reviewers selectively included study effect estimates in meta-analyses when multiple effect estimates were available; (ii) what impact selective inclusion of study effect estimates may have on meta-analytic effects, and; (iii) the risk of bias due to missing results (publication bias and selective non-reporting bias) in meta-analyses. Methods: We will systematically search for SRs with meta-analysis of the association between food/diet and health-related outcomes in a generally healthy population, published between January 2018 and June 2019. We will randomly sort titles and abstracts and screen them until we identify 50 eligible SRs. The first reported meta-analysis of a binary or continuous outcome in each SR (the ‘index meta-analysis’) will be evaluated. We will extract from study reports all study effect estimates that were eligible for inclusion in the index meta-analyses (e.g. from multiple instruments and time points) and will quantify and test for evidence of selective inclusion of results. We will also assess the risk of bias due to missing results in the index meta-analyses using a new tool (ROB-ME). Ethics and dissemination: Ethics approval is not required because information will only be extracted from published studies. Dissemination of the results will be through peer-reviewed publications and presentations at conferences. We will make all data collected from this study publicly available via the Open Science Framework.

Parent–Infant Interventions to Promote Parental Sensitivity During NICU Hospitalization: Systematic Review and Meta-Analysis

2021 ◽  
pp. 105381512199192
Author(s):  
Andréane Lavallée ◽  
Gwenaëlle De Clifford-Faugère ◽  
Ariane Ballard ◽  
Marilyn Aita
Keyword(s):  
Systematic Review ◽  
Systematic Reviews ◽  
Standard Care ◽  
Assessment Tool ◽  
Meta Analysis ◽  
Risk Of Bias ◽  
Parental Sensitivity ◽  
Assessment Development ◽  
Implementation Failure ◽  
Meta Analyses

This systematic review and meta-analysis examined the effectiveness of parent–infant interventions for parents of preterm infants on parental sensitivity compared to standard care or active comparators. This review follows the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines and was prospectively registered in the International Prospective Register of Systematic Reviews (PROSPERO; registration ID: CRD42016047083). Database searches were performed from inception to 2020 to identify eligible randomized controlled trials. Two review authors independently selected studies, extracted data, and assessed the risk of bias using the Cochrane risk of bias assessment tool and quality of evidence using the Grades of Recommendation, Assessment, Development and Evaluation (GRADE) guidelines. A total of 19 studies ( n = 2,111 participants) were included and 14 were suitable to be pooled in our primary outcome meta-analysis. Results show no significant effect of parent–infant interventions over standard care or basic educational programs, on parental sensitivity. Results may not necessarily be due to the ineffectiveness of the interventions but rather due to implementation failure or high risk of bias of included studies.

scholarly journals Dead People Don’t Eat: Food Governmentenomics and Conflicts-of-Interest in the USDA and FDA

2013 ◽  
Vol 7 (1) ◽  
Author(s):  
Gabriela Steier
Keyword(s):  
Consumer Protection ◽  
Conflict Of Interest ◽  
Conflicts Of Interest ◽  
The United States ◽  
Dietary Guidelines ◽  
Revolving Door ◽  
United States Department ◽  
Dietary Recommendations ◽  
Advisory Committees ◽  
Federal Advisory Committees

Conflicts of interest permeate the governance of the federal advisory committees that issue recommendations to consumer protection agencies, such as the Food and Drug Administration (FDA) and the United States Department of Agriculture (USDA), and therefore, American consumers need a federal solution to protect their health from biased recommendations. In order to promote a business-friendly food pyramid, agribusinesses and food industrialists lobby for dietary guidelines that boost their sales. The resulting guidelines cause great damage to public health, spur environmental pollution, and result in a loss of democratic freedoms. As a result, the FDA and USDA's bifurcated task of protecting both food producers and consumers, creates a conflict of interest within the agencies that often favor the food industry over consumer protection.This paper describes the problems embedded within the FDA and USDA's conflict of interest and the resulting revolving door of the heavily invested lobbyists, and finally, suggests statutory amendments to solve this problem. The proposed amendments will dispense with ineffective disclosure requirements and eliminate the possibility of waiving conflicts of interest for advisory committee members. By rebalancing the composition of the advisory committees and the scientific basis for the dietary recommendations, the proposed amendments will close the loopholes that large food industrialists currently abuse. As a result, consumer protection agencies, such as the FDA and USDA, are empowered to police the federal advisory committees issuing the dietary recommendations and prevent government officials from breaching their fiduciary duties to American consumers.

The GRADE approach, Part 1: how to assess the certainty of the evidence

Medwave ◽  
2021 ◽  
Vol 21 (02) ◽  
pp. e8109-e8109
Author(s):  
Matías Kirmayr ◽  
Carlos Quilodrán ◽  
Bárbara Valente ◽  
Cristóbal Loezar ◽  
Luis Garegnani ◽  
...  
Keyword(s):  
Dose Response ◽  
Risk Of Bias ◽  
Classification Systems ◽  
Second Step ◽  
Large Magnitude ◽  
Assessment Development ◽  
Evaluation Approach ◽  
Response Gradient ◽  
Residual Confounding ◽  
Grade Approach

The certainty of the evidence for interventions is the certainty or confidence that the true effect is within a particular range or relative to a threshold. In the new pyramid of evidence, systematic reviews represent the magnifying glass through which this certainty is evaluated. The GRADE (Grading of Recommendations Assessment, Development and Evaluation) approach arises in response to the existence of multiple evidence classification systems, and it offers a transparent and structured process to develop and present summaries of evidence considering its certainty and, in a second step, the strength of the recommendations that they inform. The GRADE process begins with an explicit question that includes all important and critical outcomes explicitly. The main domains used to assess the certainty of the evidence are risk of bias, inconsistency, indirectness of evidence, imprecision, and publication bias. The factors that can increase the certainty of the evidence are dose-response gradient, large magnitude of an effect, and effect of plausible residual confounding. Finally, the Summary of Findings tables summarize the process in a simplified way and with controlled language. This narrative review’s purpose is to address the GRADE approach’s theoretical and practical underlying concepts in a simplified way and with practical examples.

scholarly journals Applicability of evidence from previous systematic reviews on immunotherapy in current practice of childhood asthma treatment: a GRADE (Grading of Recommendations Assessment, Development and Evaluation) systematic review

BMJ Open ◽  
2017 ◽  
Vol 7 (12) ◽  
pp. e016326 ◽  
Author(s):  
Erik-Jonas van de Griendt ◽  
Mariska K Tuut ◽  
Hans de Groot ◽  
Paul L P Brand
Keyword(s):  
Asthma Control ◽  
Systematic Reviews ◽  
Childhood Asthma ◽  
Risk Of Bias ◽  
The Body ◽  
Effect Estimate ◽  
Asthma Treatment ◽  
Assessment Development ◽  
Asthma Symptoms

ObjectiveBecause most children with asthma now use inhaled corticosteroids (ICS), the added benefit of immunotherapy in asthmatic children needs to be examined. We re-assessed the effectiveness of subcutaneous (SCIT) and sublingual immunotherapy (SLIT) in childhood asthma treatment focusing on studies with patient-relevant outcome measures and children using ICS.MethodsWe used the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach to systematically search and appraise the evidence using predefined critical patient-relevant outcomes (asthma symptoms, asthma control and exacerbations). We searched to retrieve systematic reviews and randomised controlled trials on immunotherapy for asthma in children (1960–2017). We assessed the quality of the body of evidence with GRADE criteria.ResultsThe quality of the evidence for SCIT was very low due to a large risk of bias and indirectness (dated studies in children not using ICS). No effect of SCIT was found for asthma symptoms; no studies reported on asthma control. For asthma exacerbations, studies favoured SCIT. We have little confidence in this effect estimate, due to the very low quality of evidence. For SLIT, quality of the evidence was very low due to a large risk of bias, indirectness and imprecision. The outcome ‘asthma symptoms’ could not be calculated due to lack of standardisation and large clinical heterogeneity. Other predefined outcomes were not reported.ConclusionThe beneficial effects of immunotherapy in childhood asthma found in earlier reviews are no longer considered applicable, because of indirectness (studies performed in children not being treated according to current asthma guidelines with ICS). There was absence of evidence to properly determine the effectiveness or lack thereof of immunotherapy in asthma treatment in children with ICS.

scholarly journals Systematic reviews of observational studies of Risk of Thrombosis and Bleeding in General and Gynecologic Surgery (ROTBIGGS): introduction and methodology

2021 ◽  
Vol 10 (1) ◽  
Author(s):  
Lauri I. Lavikainen ◽  
Gordon H. Guyatt ◽  
Yung Lee ◽  
Rachel J. Couban ◽  
Anna L. Luomaranta ◽  
...  
Keyword(s):  
Systematic Reviews ◽  
Observational Studies ◽  
Risk Of Bias ◽  
Gynecologic Surgery ◽  
Patient Risk ◽  
Trade Off ◽  
Risk Estimates ◽  
Modeling Studies ◽  
Meta Analyses ◽  
Grade Approach

Abstract Background Venous thromboembolism (VTE) and bleeding are serious and potentially fatal complications of surgical procedures. Pharmacological thromboprophylaxis decreases the risk of VTE but increases the risk of major post-operative bleeding. The decision to use pharmacologic prophylaxis therefore represents a trade-off that critically depends on the incidence of VTE and bleeding in the absence of prophylaxis. These baseline risks vary widely between procedures, but their magnitude is uncertain. Systematic reviews addressing baseline risks are scarce, needed, and require innovations in methodology. Indeed, systematic summaries of these baseline risk estimates exist neither in general nor gynecologic surgery. We will fill this knowledge gap by performing a series of systematic reviews and meta-analyses of the procedure-specific and patient risk factor stratified risk estimates in general and gynecologic surgeries. Methods We will perform comprehensive literature searches for observational studies in general and gynecologic surgery reporting symptomatic VTE or bleeding estimates. Pairs of methodologically trained reviewers will independently assess the studies for eligibility, evaluate the risk of bias by using an instrument developed for this review, and extract data. We will perform meta-analyses and modeling studies to adjust the reported risk estimates for the use of thromboprophylaxis and length of follow up. We will derive the estimates of risk from the median estimates of studies rated at the lowest risk of bias. The primary outcomes are the risk estimates of symptomatic VTE and major bleeding at 4 weeks post-operatively for each procedure stratified by patient risk factors. We will apply the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach to rate evidence certainty. Discussion This series of systematic reviews, modeling studies, and meta-analyses will inform clinicians and patients regarding the trade-off between VTE prevention and bleeding in general and gynecologic surgeries. Our work advances the standards in systematic reviews of surgical complications, including assessment of risk of bias, criteria for arriving at the best estimates of risk (including modeling of the timing of events and dealing with suboptimal data reporting), dealing with subgroups at higher and lower risk of bias, and use of the GRADE approach. Systematic review registration PROSPERO CRD42021234119

scholarly journals Toolkit of methodological resources to conduct systematic reviews

F1000Research ◽  
2020 ◽  
Vol 9 ◽  
pp. 82
Author(s):  
Marta Roqué ◽  
Laura Martínez-García ◽  
Ivan Solà ◽  
Pablo Alonso-Coello ◽  
Xavier Bonfill ◽  
...  
Keyword(s):  
Diagnostic Accuracy ◽  
Systematic Reviews ◽  
Assessment Tool ◽  
Research Question ◽  
Risk Of Bias ◽  
Bias Assessment ◽  
Assessment Development ◽  
Stage Of Development ◽  
Model Study ◽  
Question Development

Background: Systematic reviews (SR) can be classified by type depending on the research question they are based on. This work identifies and describes the most relevant methodological resources to conduct high-quality reviews that answer health care questions regarding prevalence, prognosis, diagnostic accuracy and effects of interventions. Methods: Methodological resources have been identified from literature searches and consulting guidelines from institutions that develop SRs. The selected resources are organized by type of SR, and stage of development of the review (formulation of the research question, development of the protocol, literature search, risk of bias assessment, synthesis of findings, assessment of the quality of evidence, and report of SR results and conclusions). Results: Although the different types of SRs are developed following the same steps, each SR type requires specific methods, differing in characteristics and complexity. The extent of methodological development varies by type of SR, with more solid guidelines available for diagnostic accuracy and effects of interventions SRs. This methodological toolkit describes the most up-to-date risk of bias instruments: Quality in Prognostic Studies (QUIPS) tool and Prediction model study Risk Of Bias Assessment Tool (PROBAST) for prognostic SRs, Quality assessment of diagnostic accuracy studies tool (QUADAS-2) for diagnostic accuracy SRs, Cochrane risk of bias tool (ROB-2) and Risk of bias in non-randomised studies of interventions studies tool (ROBINS-I) for effects of interventions SRs, as well as the latest developments on the Grading of Recommendations Assessment, Development and Evaluation (GRADE) system. Conclusions: This structured compilation of the best methodological resources for each type of SR may prove to be a very useful tool for those researchers that wish to develop SRs or conduct methodological research works on SRs

Sign in / Sign up

Export Citation Format

Share Document