scholarly journals Evaluation of effectiveness and harms reporting in overviews of systematic reviews of interventions by developing new assessment tools

2021 ◽  
Author(s):  
Κωνσταντίνος Μπουγιούκας
Keyword(s):  
Systematic Reviews ◽  
Assessment Tools ◽  
Overviews Of Systematic Reviews

Στη βιοϊατρική βιβλιογραφία, οι ανασκοπήσεις συστηματικών ανασκοπήσεων (ΑΣΑ) έχουν γίνει πρόσφατα μια δημοφιλής προσέγγιση της τεκμηριωμένης σύνθεσης, όπου η μονάδα σύνθεσης είναι η συστηματική ανασκόπηση (ΣΑ). Τα αποτελέσματα από τις ΣΑ που περιλαμβάνονται σε μια ΑΣΑ μπορούν να παρουσιαστούν ακριβώς όπως αναφέρονται στις ΣΑ (ποιοτική σύνθεση) ή μπορούν να αναλυθούν εκ νέου χρησιμοποιώντας μετα-ανάλυση (ποσοτική σύνθεση). Οι ΑΣΑ μπορούν να παρέχουν πολύτιμη πληροφορία για να υποστηρίξουν τη λήψη αποφάσεων από τους επαγγελματίες υγείας και να οδηγήσουν στην ενσωμάτωση των ερευνητικών αποτελεσμάτων στην πολιτική σχεδίασης σε σύντομο χρονικό διάστημα. Χρησιμοποιώντας ένα δείγμα 1558 ΑΣΑ που δημοσιεύθηκαν μεταξύ 2000 και 2020, η βιβλιομετρική ανάλυση αποκάλυψε ερευνητικές τάσεις και τομείς ενδιαφέροντος αυτών των μελετών. Συγκεκριμένα, διαπιστώσαμε ότι οι δημοσιεύσεις αυξάνονταν ετησίως και η ονοματολογία τους δεν ήταν ενιαία (η πιο συχνή ορολογία στον τίτλο ήταν «ανασκόπηση συστηματικών ανασκοπήσεων»). Ο μεγαλύτερος αριθμός άρθρων και αυτά με τις περισσότερες αναφορές δημοσιεύτηκαν από υπεύθυνους συγγραφείς που προέρχονται από το Ηνωμένο Βασίλειο. Τα άρθρα διανεμήθηκαν σε 737 επιστημονικά περιοδικά και πολλά από αυτά δημοσιεύθηκαν στον τομέα της συμπληρωματικής/εναλλακτικής ιατρικής, της ψυχιατρικής/ψυχολογίας, της διατροφής/διαιτητικής και της παιδιατρικής. Η ανάλυση της συν-συγγραφής αποκάλυψε συνεργασίες μεταξύ συγγραφέων διαφορετικών χωρών. Οι πιο συνηθισμένες κλινικές καταστάσεις ήταν η κατάθλιψη, ο διαβήτης, ο καρκίνος, η άνοια, ο πόνος, οι καρδιαγγειακές παθήσεις, το εγκεφαλικό επεισόδιο, η παχυσαρκία και η σχιζοφρένεια. Επιπλέον, αυτή η διατριβή παρουσιάζει εργαλεία που αξιολογούν την πληρότητα καταγραφής της αποτελεσματικότητας και της ασφάλειας των ιατρικών παρεμβάσεων στις ΑΣΑ. Αναπτύξαμε το εργαλείο PRIO-harms (56 υπο-στοιχεία) για να προωθήσουμε μια πιο διαφανή και ισορροπημένη αναφορά των ωφελειών και κινδύνων των ιατρικών παρεμβάσεων που διερευνώνται στις ΑΣΑ και το εργαλείο PRIO για περιλήψεις (20 υπο-στοιχεία) για να βοηθήσουμε τους συγγραφείς να συντάξουν ολοκληρωμένες, διαφανείς και κατατοπιστικές περιλήψεις για τις ΑΣΑ. Επίσης αυτά τα εργαλεία μπορούν να υιοθετηθούν από συντάκτες και εκδότες περιοδικών που δημοσιεύουν ΑΣΑ. Επιπροσθέτως, σε αυτήν τη διδακτορική διατριβή εισηγάγαμε νέες στατικές τεχνικές μέσω πινάκων και γραφημάτων για την οπτική παρουσίαση των αλληλεπικαλυπτόμενων ΣΑ που περιλαμβάνονται στις ΑΣΑ. Οι προτεινόμενες εκλεπτυσμένες γραφικές μέθοδοι (π.χ. “upset plots”, “heatmaps”, “node-link graphs”) ενδέχεται να βοηθήσουν τους μεθοδολόγους και συγγραφείς στην εξερεύνηση και απεικόνιση του βαθμού επικάλυψης των πρωτογενών μελετών των ΣΑ, γεγονός που με τη σειρά του μπορεί να βελτιώσει την εγκυρότητα και τη διαφάνεια στις ΑΣΑ. Ωστόσο, απαιτείται περισσότερη έρευνα για να κατανοηθεί ποιες γραφικές τεχνικές θα ήταν πιο χρήσιμες και ευκολότερες στην κατανόηση. Αυτά τα νέα εργαλεία και γραφικές τεχνικές θα μπορούσαν να βελτιωθούν από την κριτική αξιολόγηση και περαιτέρω επικύρωση από ειδικούς και ερευνητικές ομάδες που παράγουν κατευθυντήριες οδηγίες για τις ΑΣΑ.

scholarly journals A Comparison of Two Assessment Tools Used in Overviews of Systematic Reviews: ROBIS versus AMSTAR-2

2021 ◽  
Author(s):  
Rachel Perry ◽  
Alex Whitmarsh ◽  
Verity Leach ◽  
Philippa Davies
Keyword(s):  
Systematic Review ◽  
Systematic Reviews ◽  
Assessment Tool ◽  
Meta Analysis ◽  
Risk Of Bias ◽  
Assessment Tools ◽  
Substantial Agreement ◽  
Rater Reliability ◽  
Response Options ◽  
Overviews Of Systematic Reviews

Abstract Background:AMSTAR-2 is a 16-item assessment tool to check the quality of a systematic review and establish whether the most important elements are reported. ROBIS is another assessment tool which was designed to evaluate the level of bias present within a systematic review. Our objective was to compare, contrast and establish both inter-rater reliability and usability of both tools as part of two overviews of systematic reviews. Strictly speaking, one tool assesses methodological quality (AMSTAR-2) and the other assesses risk of bias (ROBIS), but there is considerable overlap between the tools in terms of the signalling questions. Methods:Three reviewers independently assessed 31 systematic reviews using both tools. The inter-rater reliability of all sub-sections using each instrument (AMSTAR-2 and ROBIS) were calculated using Gwet’s agreement coefficient (AC1 for unweighted analysis and AC2 for weighted analysis).Results:Thirty-one systematic reviews were included. For AMSTAR-2 the median agreement for all questions was 0.61. Eight of the 16 AMSTAR-2 questions had substantial agreement or higher (>0.61). For ROBIS, the median agreement for all questions was also 0.61. Eleven of the 24 ROBIS questions had substantial agreement or higher. Conclusion:ROBIS is an effective tool for assessing risk of bias in systematic reviews and AMSTAR-2 is an effective tool at assessing quality. The median agreement between raters for both tools was identical (0.61). Reviews that included a meta-analysis were easier to rate with ROBIS; however, further developmental work could improve its use in reviews without a formal synthesis. AMSTAR-2 was more straightforward to use, however, more response options would be beneficial.

scholarly journals A comparison of two assessment tools used in overviews of systematic reviews: ROBIS versus AMSTAR-2

2021 ◽  
Vol 10 (1) ◽  
Author(s):  
R. Perry ◽  
A. Whitmarsh ◽  
V. Leach ◽  
P. Davies
Keyword(s):  
Systematic Review ◽  
Systematic Reviews ◽  
Assessment Tool ◽  
Meta Analysis ◽  
Risk Of Bias ◽  
Assessment Tools ◽  
Substantial Agreement ◽  
Rater Reliability ◽  
Response Options ◽  
Overviews Of Systematic Reviews

Abstract Background AMSTAR-2 is a 16-item assessment tool to check the quality of a systematic review and establish whether the most important elements are reported. ROBIS is another assessment tool which was designed to evaluate the level of bias present within a systematic review. Our objective was to compare, contrast and establish both inter-rater reliability and usability of both tools as part of two overviews of systematic reviews. Strictly speaking, one tool assesses methodological quality (AMSTAR-2) and the other assesses risk of bias (ROBIS), but there is considerable overlap between the tools in terms of the signalling questions. Methods Three reviewers independently assessed 31 systematic reviews using both tools. The inter-rater reliability of all sub-sections using each instrument (AMSTAR-2 and ROBIS) was calculated using Gwet’s agreement coefficient (AC1 for unweighted analysis and AC2 for weighted analysis). Results Thirty-one systematic reviews were included. For AMSTAR-2, the median agreement for all questions was 0.61. Eight of the 16 AMSTAR-2 questions had substantial agreement or higher (> 0.61). For ROBIS, the median agreement for all questions was also 0.61. Eleven of the 24 ROBIS questions had substantial agreement or higher. Conclusion ROBIS is an effective tool for assessing risk of bias in systematic reviews and AMSTAR-2 is an effective tool at assessing quality. The median agreement between raters for both tools was identical (0.61). Reviews that included a meta-analysis were easier to rate with ROBIS; however, further developmental work could improve its use in reviews without a formal synthesis. AMSTAR-2 was more straightforward to use; however, more response options would be beneficial.

scholarly journals Systematic review protocol exploring the impact of the COVID-19 pandemic on the wellbeing of general practitioners

2021 ◽  
Author(s):  
Laura Jefferson ◽  
Su Golder ◽  
Veronica Dale ◽  
Holly Essex ◽  
Elizabeth McHugh ◽  
...  
Keyword(s):  
Mental Health ◽  
Systematic Review ◽  
Quality Assessment ◽  
General Practitioners ◽  
Systematic Reviews ◽  
Assessment Tools ◽  
Systematic Review Protocol ◽  
Health And Wellbeing ◽  
Review Protocol ◽  
The Impact

Background Over recent years chronic stress and burnout have been reported by doctors working in general practice in the UK NHS and internationally. The COVID-19 pandemic has changed general practitioners working lives; adding potential pressures from avoiding infection and addressing pent-up demand for care, but also changing processes such as rapidly taking up remote consultations. To date, there has been a focus on exploring the impact of the pandemic on the wellbeing of hospital clinicians. No registered systematic reviews currently focus on exploring the impact of the pandemic on the mental health and wellbeing of general practitioners. Aims and objectives To synthesise the current international evidence base exploring the impact of COVID-19 on the mental health and wellbeing of general practitioners, and which factors are associated with their reported mental health and wellbeing during the pandemic. Methods In this paper we report a systematic review protocol, following PRISMA guidance. In our search strategy we will identify primary research studies or systematic reviews exploring the mental health and wellbeing of general practitioners during the COVID-19 pandemic in four databases (MEDLINE, Embase, PsychInfo and Medrxiv) and Google Scholar. We will hand-search reference lists and grey literature. Two reviewers will undertake all stages including study selection, data extraction and quality assessment, with arbitration by a third reviewer where necessary. We will use standardised quality assessment tools to ensure transparency and reduce bias in quality assessment. Depending on the quality of included studies, we may undertake a sensitivity analysis by excluding studies from narrative synthesis that are rated as low quality using the checklists. We will describe the findings across studies using narrative thematic data synthesis, and if sufficiently homogenous data are identified, we will pool quantitative findings through meta-analysis.

Lack of Oncological Benefit from Bursectomy in Radical Gastrectomy: A Systematic Review

2021 ◽  
pp. 1-10
Author(s):  
Pankaj Kumar Garg ◽  
Ashish Jakhetiya ◽  
Kiran Kalyan Turaga ◽  
Rahul Kumar ◽  
Andreas Brandl ◽  
...  
Keyword(s):  
Overall Survival ◽  
Gastric Adenocarcinoma ◽  
Systematic Reviews ◽  
Disease Recurrence ◽  
Assessment Tools ◽  
Published Data ◽  
Radical Gastrectomy ◽  
Significant Difference ◽  
Omental Bursa ◽  
Meta Analyses

<b><i>Background:</i></b> Resection of the omental bursa has been suggested to reduce peritoneal recurrence and facilitate a complete oncological resection during a gastrectomy. The addition of this procedure increases technical complexity and prolongs the procedure. Published data regarding the oncological benefit of this procedure are conflicting. We hypothesized that a bursectomy during a radical gastrectomy does not improve overall survival. <b><i>Methods:</i></b> In accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) reporting guideline, a comprehensive literature search of 3 electronic databases (PubMed, Scopus, and Embase) was conducted to identify the clinical studies that compared bursectomy with no-bursectomy in radical gastrectomy for gastric adenocarcinoma. Qualitative and quantitative data synthesis was performed using RevMan software. A random-/fixed-effect modeling was used depending upon the heterogeneity. Bias and quality assessment tools were applied. The study was registered with the International Prospective Register of Systematic Reviews (PROSPERO) (CRD42019116556). <b><i>Results:</i></b> Of 8 studies assessing the role of bursectomy in gastric adenocarcinoma, 6 (75%) were included – of which 2 (33%) are randomized controlled trials. Of 2,904 patients, 1,273 (%) underwent a bursectomy. There was no statistically significant difference in either overall survival (hazard ratio [HR] = 0.89, 95% CI 0.75–1.06, <i>I</i><sup>2</sup> = 14%) or disease recurrence (HR = 1.01, 95% CI 0.84–1.20, <i>I</i><sup>2</sup> = 22%) in the bursectomy group compared to the no-bursectomy group. <b><i>Conclusion:</i></b> There is no additional oncological benefit of adding bursectomy to radical gastrectomy in all patients with gastric adenocarcinoma.

scholarly journals Assessment of humoral and cell-mediated immune responses to pertussis vaccination: a systematic review protocol

BMJ Open ◽  
2019 ◽  
Vol 9 (6) ◽  
pp. e028109 ◽  
Author(s):  
Funbi Akinola ◽  
Rudzani Muloiwa ◽  
Gregory, D Hussey ◽  
Violette Dirix ◽  
Benjamin Kagina ◽  
...  
Keyword(s):  
Systematic Review ◽  
Immune Response ◽  
Immune Responses ◽  
Public Space ◽  
Systematic Reviews ◽  
Data Extraction ◽  
Meta Analysis ◽  
Grey Literature ◽  
Assessment Tools ◽  
Population Type

IntroductionGlobally, some studies show a resurgence of pertussis. The risks and benefits of using whole-cell pertussis (wP) or acellular pertussis (aP) vaccines in the control of the disease have been widely debated. Better control of pertussis will require improved understanding of the immune response to pertussis vaccines. Improved understanding and assessment of the immunity induced by pertussis vaccines is thus imperative. Several studies have documented different immunological outcomes to pertussis vaccination from an array of assays. We propose to conduct a systematic review of the different immunological assays and outcomes used in the assessment of the humoraland cell-mediated immune response following pertussis vaccination.Methods and analysisThe primary outcomes for consideration are quality and quantity of immune responses (humoral and cell-mediated) post-pertussis vaccination. Of interest as secondary outcomes are types of immunoassays used in assessing immune responses post-pertussis vaccination, types of biological samples used in assessing immune responses post-pertussis vaccination, as well as the types of antigens used to stimulate these samples during post-pertussis vaccination immune response assessments. Different electronic databases (including PubMed, Cochrane, EBSCO Host, Scopus and Web of Science) will be accessed for peer-reviewed published and grey literature evaluating immune responses to pertussis vaccines between 1990 and 2019. The quality of included articles will be assessed using standardised risk and quality assessment tools specific to the study design used in each article. Data extraction will be done using a data extraction form. The extracted data will be analysed using STATA V.14.0 and RevMan V.5.3 software. A subgroup analysis will be conducted based on the study population, type of vaccine (wP or aP) and type of immune response (cell-mediated or humoral). Guidelines for reporting systematic reviews in the revised 2009 Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) statement will be used in this study.Ethics and disseminationEthics approval is not required for this study as it is a systematic review. We will only make use of data already available in the public space. Findings will be reported via publication in a peer-reviewed journal and presented at scientific meetings and workshops.Trial registration numberCRD42018102455.

scholarly journals An Overview of Systematic Reviews: Complementary Therapies for Cancer Patients

2019 ◽  
Vol 18 ◽  
pp. 153473541989002 ◽  
Author(s):  
Seong Min Lee ◽  
Ho Cheol Choi ◽  
Min Kyung Hyun
Keyword(s):  
Side Effects ◽  
Cancer Patients ◽  
Systematic Reviews ◽  
Complementary Therapies ◽  
The United States ◽  
Assessment Tools ◽  
Cochrane Library ◽  
Future Research ◽  
Cancer Type

Introduction: This article critically examines the systematic reviews (SR) and meta-analysis (MA) of complementary therapies for cancer patients to appraise the evidence level, and offers suggestions for future research and practice. Methods: The Cochrane Library and MEDLINE were searched from their inception to January 2018, to identify SR and MA of complementary therapies available for cancer patients. Final selected SR and MA were methodologically evaluated for their quality by applying the Assessing the Methodological Quality of Systematic Reviews 2 (AMSTAR2) instrument. Data extraction and risk of quality assessments were performed by 2 independent reviewers. Results: A total of 104 studies were included in the analysis. The majority of the individual clinical trials included in the SR and MA were performed in China (48%) and the United States (26.9%). Breast cancer was the most studied cancer type (25%), and acupuncture was the most studied intervention (21%). Side effects of cancer such as pain, depression, and fatigue were effectively managed with complementary therapies. The methodologically problematic items included not listing the excluded studies and lack of protocol or protocol registration. Conclusions: With increasing interest in research, complementary therapies appear to be beneficial in reducing side effects and raising the quality of life of cancer patients. Complementary therapies have generally been studied for all cancers, with acupuncture being the most researched, regardless of the cancer type. Since AMSTAR2 is a stricter assessment tool than before, future studies need to consider the risk of methodological bias with caution and discuss appropriate overall quality assessment tools.

scholarly journals Measuring test-retest reliability (TRR) of AMSTAR provides moderate to perfect agreement – a contribution to the discussion of the importance of TRR in relation to the psychometric properties of assessment tools

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Stefanie Bühn ◽  
Peggy Ober ◽  
Tim Mathes ◽  
Uta Wegewitz ◽  
Anja Jacobs ◽  
...  
Keyword(s):  
Psychometric Properties ◽  
Systematic Reviews ◽  
Methodological Quality ◽  
Assessment Tools ◽  
Measurement Properties ◽  
Perfect Agreement ◽  
Retest Reliability ◽  
Test Retest Reliability ◽  
The Impact

Abstract Background Systematic Reviews (SRs) can build the groundwork for evidence-based health care decision-making. A sound methodological quality of SRs is crucial. AMSTAR (A Measurement Tool to Assess Systematic Reviews) is a widely used tool developed to assess the methodological quality of SRs of randomized controlled trials (RCTs). Research shows that AMSTAR seems to be valid and reliable in terms of interrater reliability (IRR), but the test retest reliability (TRR) of AMSTAR has never been investigated. In our study we investigated the TRR of AMSTAR to evaluate the importance of its measurement and contribute to the discussion of the measurement properties of AMSTAR and other quality assessment tools. Methods Seven raters at three institutions independently assessed the methodological quality of SRs in the field of occupational health with AMSTAR. Between the first and second ratings was a timespan of approximately two years. Answers were dichotomized, and we calculated the TRR of all raters and AMSTAR items using Gwet’s AC1 coefficient. To investigate the impact of variation in the ratings over time, we obtained summary scores for each review. Results AMSTAR item 4 (Was the status of publication used as an inclusion criterion?) provided the lowest median TRR of 0.53 (moderate agreement). Perfect agreement of all reviewers was detected for AMSTAR-item 1 with a Gwet’s AC1 of 1, which represented perfect agreement. The median TRR of the single raters varied between 0.69 (substantial agreement) and 0.89 (almost perfect agreement). Variation of two or more points in yes-scored AMSTAR items was observed in 65% (73/112) of all assessments. Conclusions The high variation between the first and second AMSTAR ratings suggests that consideration of the TRR is important when evaluating the psychometric properties of AMSTAR.. However, more evidence is needed to investigate this neglected issue of measurement properties. Our results may initiate discussion of the importance of considering the TRR of assessment tools. A further examination of the TRR of AMSTAR, as well as other recently established rating tools such as AMSTAR 2 and ROBIS (Risk Of Bias In Systematic reviews), would be useful.

Assessment tools for palliative care.

2016 ◽  
Vol 34 (26_suppl) ◽  
pp. 66-66
Author(s):  
Sarina Isenberg ◽  
Rebecca Aslakson ◽  
Sydney Morss Dy ◽  
Renee Wilson ◽  
Julie Waldfogel ◽  
...  
Keyword(s):  
Quality Of Life ◽  
Palliative Care ◽  
Clinical Practice ◽  
Quality Indicators ◽  
Systematic Reviews ◽  
Grey Literature ◽  
Care Quality ◽  
Assessment Tools ◽  
Patient Reported

66 Background: Recent reviews have not comprehensively addressed palliative care (PC) assessment tools. This project summarizes the extent of evidence about PC assessment tools for patients and families, and how tools have been used for clinical care, quality indicators, and evaluation of interventions. Methods: We searched MEDLINE, CINAHL, and Cochrane Database of Systematic Reviews for systematic reviews of assessment tools for PC, from January 2007 to March 2016. We searched the grey literature for domains without systematic reviews, and for domains with systematic reviews > three years old. Paired investigators independently screened search results and grey literature to determine eligibility, and assessed risk of bias of systematic reviews. The team selected the most recent and highest-quality systematic reviews for each domain. One investigator abstracted information, and a second investigator checked the information. Results: Using the National Consensus Project Palliative Care Guidelines domains, we included nine systematic reviews with 167 tools, and six tools from grey literature. Most tools were in physical, psychological, psychiatric, and social aspects of care, care at the end of life, and tools that cross domains (quality of life and caregiver-reported experience). Only two tools directly addressed spiritual aspects and none addressed cultural or patient-reported experience. Internal consistency reliability was evaluated for almost all tools; most reported construct validity; and few reported responsiveness (sensitivity to change). Few studies evaluated the use of assessment tools in quality indicators or clinical practice. A systematic review of 38 PC interventions and the assessment tools used found that at least 25 interventions included physical, psychosocial and psychiatric, and quality of life tools, but the tools varied extensively, and only nine included patient experience tools. Conclusions: Although assessment tools exist in many PC domains, tools are needed to assess spiritual and cultural aspects of care, and patient-reported experience. Research is needed concerning: tools in clinical practice and quality of care; comparison of existing tools; and evaluation and dissemination tools with evidence of responsiveness.

Sign in / Sign up

Export Citation Format

Share Document