Impact of choice of quality appraisal tool for systematic reviews in overviews

Abstract Background In the context of the volume of mixed- and multi-methods studies in health services research, the present study sought to develop an appraisal tool to determine the methodological and reporting quality of such studies when included in systematic reviews. Evaluative evidence regarding the design and use of our existing Quality Assessment Tool for Studies with Diverse Designs (QATSDD) was synthesised to enhance and refine it for application across health services research. Methods Secondary data were collected through a literature review of all articles identified using Google Scholar that had cited the QATSDD tool from its inception in 2012 to December 2019. First authors of all papers that had cited the QATSDD (n=197) were also invited to provide further evaluative data via a qualitative online survey. Evaluative findings from the survey and literature review were synthesised narratively and these data used to identify areas requiring refinement. The refined tool was subject to inter-rater reliability, face and content validity analyses. Results Key limitations of the QATSDD tool identified related to a lack of clarity regarding scope of use of the tool and in the ease of application of criteria beyond experimental psychological research. The Quality Appraisal for Diverse Studies (QuADS) tool emerged as a revised tool to address the limitations of the QATSDD. The QuADS tool demonstrated substantial inter-rater reliability (k=0.66), face and content validity for application in systematic reviews with mixed, or multi-methods health services research. Conclusion Our findings highlight the perceived value of appraisal tools to determine the methodological and reporting quality of studies in reviews that include heterogeneous studies. The QuADS tool demonstrates strong reliability and ease of use for application to multi or mixed-methods health services research.

Download Full-text

Development and Validation of a Nutrition-Specific Set of Tools to Assess the Quality of Nutrition Studies

Current Developments in Nutrition ◽

10.1093/cdn/nzaa061_005 ◽

2020 ◽

Vol 4 (Supplement_2) ◽

pp. 1377-1377

Author(s):

Karima Benkhedda ◽

Stephen Brooks ◽

Linda Greene-Finestone ◽

Shannon Kelly ◽

Amanda MacFarlane ◽

...

Keyword(s):

Quality Assessment ◽

Study Design ◽

Systematic Reviews ◽

A Priori ◽

Reliability And Validity ◽

Quality Appraisal ◽

Case Control Studies ◽

Perfect Agreement ◽

Development And Validation

Abstract Objectives To develop and validate a set of 3 quality assessment instruments (QAls) for evaluating the quality of nutrition studies, for each of the commonly used study designs: (1) randomized controlled trials (RCTs), (2) prospective cohort, and (3) case-control studies. Methods The QAI development and validation process included 8 steps: 1) identify and evaluate existing general QAls for adaptation with nutrition-specific quality appraisal items; 2) scan the literature to identify nutrition-specific quality appraisal issues; 3) generate nutrition-specific items to be added to each of the general QAIs, adapt existing guidance for general items for nutrition applications and develop guidance for added nutrition items; 4) review, by two experts in clinical and population nutrition, of the modified general QAIs with added nutrition-specific items and guidance; 5) assess reliability and validity of the QAI for each study design; 6) improve the usability and feasibility, of the QAIs by considering feedback from the validation exercise to refine the wording of the guidance; 7) develop a worksheet to help evaluate, a priori, topic-specific methodology to address risk of bias; and 8) validate the final QAIs using five peer-reviewed studies identified from published systematic reviews with reported quality assessment. Agreement and reliability were determined for each QAI. Results Results of the validation show good to perfect agreement among evaluators for the overall study rating and across domains. When compared to the study quality assessment reported in the systematic review, nutrition- specific items had the greatest impact on study ratings, generally resulting in a downgrade of the overall rating. Conclusions A set of nutrition-specific QAls were developed to assess the quality and robustness of nutrition studies. These tools incorporate general quality issues of study design and conduct, as well as address recognised nutrition study-specific issues. They will improve consistency in how nutrition studies are assessed particularly in nutrition-related systematic reviews. This will contribute to the overall quality of assessment of diet and Funding Sources This work was supported by Health Canada.

Download Full-text

Guidance on conducting systematic reviews/meta-analyses of pharmacoepidemiological studies of safety outcomes: the gap is now filled

Epidemiology and Psychiatric Sciences ◽

10.1017/s2045796016000299 ◽

2016 ◽

Vol 25 (5) ◽

pp. 425-427 ◽

Cited By ~ 2

Author(s):

S. Cortese

Keyword(s):

Systematic Reviews ◽

Data Extraction ◽

European Medicines Agency ◽

Study Quality ◽

European Network ◽

Quality Appraisal ◽

Comprehensive Guidance ◽

Extraction Study ◽

Safety Outcomes ◽

Meta Analyses

Until recently, no comprehensive guidance specifically on the conduction of systematic reviews and meta-analyses of pharmacoepidemiological studies of safety outcomes was available. In December 2015, the European Network of Centres for Pharmacoepidemiology and Pharamacovigilance (ENCePP), a network coordinated by the European Medicines Agency, published their ‘Guidance on conducting systematic reviews and meta-analyses of completed comparative pharmacoepidemiological studies of safety outcomes’, filling an important gap in the field. This paper highlights the ENCePP recommendations in terms of study identification, data extraction, study quality appraisal and analytical plan. Although the ENCePP document should not be considered as definitive, since it will likely be refined following researchers’ feedback, it is expected that it will be highly influential and useful for the field, with the ultimate goal to improve and standardise the conduction and reporting of systematic reviews/meta-analyses of pharmacoepidemiological studies of safety outcomes.

Download Full-text

The Relationship Between Strategic Planning and Budgeting in Regional Development Planning Agency of Sukabumi City

E-Jurnal Ekonomi dan Bisnis Universitas Udayana ◽

10.24843/eeb.2019.v08.i04.p02 ◽

2019 ◽

pp. 355

Author(s):

Fahmi Adicipta ◽

Bayu Kharisma

Keyword(s):

Strategic Planning ◽

Regional Development ◽

Good Governance ◽

Development Planning ◽

Quality Appraisal ◽

Performance Quality ◽

Planning Agency ◽

The Relationship ◽

Appraisal Tool

This paper aims to analyze whether Coordination and synchronization of programs or activities and budgeting in regional development planning agency is possible. Lack of synchronization and coordination between SKPD externally so as to affect the quality of development planning. The Relationship Between Strategic Planning and Budgeting Is expected to play a role other than as a control tool, but it can also be used as a performance quality appraisal tool, as well as a tool for the realization of a clean and authoritative government (good governance). There are relationship between strategic planning in Regional Development Planning Agency of sukabumi city (Renstra) and its budgeting. The relationship particularly can be seen in the flow of Renstra (from vision to activities) and the achievement from targets.

Download Full-text

Ensuring Rigor in systematic reviews: Part 5, quality appraisal, data extraction, synthesis

Heart & Lung ◽

10.1016/j.hrtlng.2021.04.002 ◽

2021 ◽

Vol 50 (5) ◽

pp. 615-617

Author(s):

Janene Batten ◽

Alexandria Brackett

Keyword(s):

Systematic Reviews ◽

Data Extraction ◽

Quality Appraisal

Download Full-text

Effects of Nutritional Interventions on Cardiovascular Disease Health Outcomes in Aboriginal and Torres Strait Islander Australians: A Scoping Review

Nutrients ◽

10.3390/nu13114084 ◽

2021 ◽

Vol 13 (11) ◽

pp. 4084

Author(s):

Bobby Porykali ◽

Alyse Davies ◽

Cassandra Brooks ◽

Hannah Melville ◽

Margaret Allman-Farinelli ◽

...

Keyword(s):

Cardiovascular Disease ◽

Health Outcomes ◽

Torres Strait Islander ◽

Grey Literature ◽

Indigenous Research ◽

Quality Appraisal ◽

Nutritional Interventions ◽

Nutrition Programs ◽

Torres Strait ◽

Appraisal Tool

Nutrition interventions can support Aboriginal and Torres Strait Islander peoples to reduce their risk of cardiovascular disease (CVD). This review examines nutritional interventions aiming to improve CVD outcomes and appraises peer-reviewed interventions using an Aboriginal and Torres Strait Islander Quality Appraisal Tool. Five electronic databases and grey literature were searched, applying no time limit. Two reviewers completed the screening, data extraction and quality assessment independently. The study quality was assessed using the South Australian Health and Medical Research Institute and the Centre of Research Excellence in Aboriginal Chronic Disease Knowledge Translation and Exchange Aboriginal and Torres Strait Islander Quality Appraisal Tool (QAT). Twenty-one nutrition programs were included in this review. Twelve reported on anthropometric measurements, ten on biochemical and/or hematological measurements and sixteen on other outcome domains. Most programs reported improvements in measurable CVD risk factors, including reduced body mass index (BMI), waist circumference (WC), weight, blood pressure and improved lipid profiles. Most programs performed well at community engagement and capacity strengthening, but many lacked the inclusion of Indigenous research paradigms, governance and strengths-based approaches. This review highlights the need for contemporary nutrition programs aimed at improving cardiovascular health outcomes to include additional key cultural components.

Download Full-text

A case for a study quality appraisal tool in survey studies in psychology

10.31234/osf.io/xpmjy ◽

2018 ◽

Author(s):

Cleo Protogerou ◽

Martin S Hagger

Keyword(s):

Survey Research ◽

Psychological Research ◽

Study Quality ◽

Quality Appraisal ◽

Fit For Purpose ◽

Meta Analyses ◽

Quality Assessments ◽

Appraisal Tool ◽

Opinion Article

In this Opinion article, we argue that assessment of study quality is an important practice to promote greater precision, transparency, and evaluation of research in psychology. However, no quality appraisal tools have been developed by psychologists, for psychological research. Given the prominence of survey research in psychology, the development of appropriate means to assess the quality of survey research would yield considerable benefits to researchers conducting, and data analysts evaluating, survey research. To illustrate the longstanding problems resulting from the absence of a fit-for-purpose tool and the application of a variety of quality appraisal strategies, we provide examples from a brief summary of quality assessments from meta-analyses of psychological survey research. We conclude that a fit-for-purpose quality appraisal tool for survey studies in psychology is needed.

Download Full-text