A new taxonomy was developed for overlap across ‘Overviews of systematic reviews’: a meta-research study of research waste

Background Multiple ‘overviews of reviews’ conducted on the same topic (“overlapping overviews”) represent a waste of research resources and can confuse clinicians making decisions amongst competing treatments. We aimed to assess the frequency and characteristics of overlapping overviews. Methods MEDLINE, Epistemonikos and Cochrane Database of Systematic Reviews were searched for overviews that: synthesised reviews of health interventions and conducted systematic searches. Overlap was defined as: duplication of PICO eligibility criteria, and not reported as an update nor a replication. We categorized overview topics according to 22 WHO ICD-10 medical classifications, overviews as broad or narrow in scope, and overlap as identical, nearly identical, partial, or subsumed. Subsummation was defined as when broad overviews subsumed the populations, interventions and at least one outcome of another overview. Results Of 541 overviews included, 169 (31%) overlapped across similar PICO, fell within 13 WHO ICD-10 medical classifications, and 62 topics. 148/169 (88%) overlapping overviews were broad in scope. Fifteen overviews were classified as having nearly identical overlap (9%); 123 partial overlap (73%), and 31 subsumed (18%) others. Conclusions One third of overviews overlapped in content and a majority covered broad topic areas. A multiplicity of overviews on the same topic adds to the ongoing waste of research resources, time and effort across medical disciplines. Authors of overviews can use this study and the sample of overviews to identify gaps in the evidence for future analysis, and topics that are already studied which do not need to be duplicated.

Topic overlap and research waste at the ‘Overviews of systematic reviews’ level: a meta-research study

Background Multiple ‘overviews of reviews’ conducted on the same topic (“overlapping overviews”) represent a waste of research resources and can confuse clinicians making decisions amongst competing treatments. We aimed to assess the frequency and characteristics of overlapping overviews. Methods MEDLINE, Epistemonikos and Cochrane Database of Systematic Reviews were searched for overviews that: synthesised reviews of health interventions and conducted systematic searches. Overlap was defined as: duplication of PICO eligibility criteria, and not reported as an update nor a replication. We categorized overview topics according to 22 WHO ICD-10 medical classifications, overviews as broad or narrow in scope, and overlap as identical, nearly identical, partial, or subsumed. Subsummation was defined as when broad overviews subsumed the populations, interventions and at least one outcome of another overview. Results Of 541 overviews included, 169 (31%) overlapped across similar PICO, fell within 13 WHO ICD-10 medical classifications, and 62 topics. 148/169 (88%) overlapping overviews were broad in scope. Fifteen overviews were classified as having nearly identical overlap (9%); 123 partial overlap (73%), and 31 subsumed (18%) others. Conclusions One third of overviews overlapped in content and a majority covered broad topic areas. A multiplicity of overviews on the same topic adds to the ongoing waste of research resources, time and effort across medical disciplines. Authors of overviews can use this study and the sample of overviews to identify gaps in the evidence for future analysis, and topics that are already studied which do not need to be duplicated.

Evaluation of effectiveness and harms reporting in overviews of systematic reviews of interventions by developing new assessment tools

Στη βιοϊατρική βιβλιογραφία, οι ανασκοπήσεις συστηματικών ανασκοπήσεων (ΑΣΑ) έχουν γίνει πρόσφατα μια δημοφιλής προσέγγιση της τεκμηριωμένης σύνθεσης, όπου η μονάδα σύνθεσης είναι η συστηματική ανασκόπηση (ΣΑ). Τα αποτελέσματα από τις ΣΑ που περιλαμβάνονται σε μια ΑΣΑ μπορούν να παρουσιαστούν ακριβώς όπως αναφέρονται στις ΣΑ (ποιοτική σύνθεση) ή μπορούν να αναλυθούν εκ νέου χρησιμοποιώντας μετα-ανάλυση (ποσοτική σύνθεση). Οι ΑΣΑ μπορούν να παρέχουν πολύτιμη πληροφορία για να υποστηρίξουν τη λήψη αποφάσεων από τους επαγγελματίες υγείας και να οδηγήσουν στην ενσωμάτωση των ερευνητικών αποτελεσμάτων στην πολιτική σχεδίασης σε σύντομο χρονικό διάστημα. Χρησιμοποιώντας ένα δείγμα 1558 ΑΣΑ που δημοσιεύθηκαν μεταξύ 2000 και 2020, η βιβλιομετρική ανάλυση αποκάλυψε ερευνητικές τάσεις και τομείς ενδιαφέροντος αυτών των μελετών. Συγκεκριμένα, διαπιστώσαμε ότι οι δημοσιεύσεις αυξάνονταν ετησίως και η ονοματολογία τους δεν ήταν ενιαία (η πιο συχνή ορολογία στον τίτλο ήταν «ανασκόπηση συστηματικών ανασκοπήσεων»). Ο μεγαλύτερος αριθμός άρθρων και αυτά με τις περισσότερες αναφορές δημοσιεύτηκαν από υπεύθυνους συγγραφείς που προέρχονται από το Ηνωμένο Βασίλειο. Τα άρθρα διανεμήθηκαν σε 737 επιστημονικά περιοδικά και πολλά από αυτά δημοσιεύθηκαν στον τομέα της συμπληρωματικής/εναλλακτικής ιατρικής, της ψυχιατρικής/ψυχολογίας, της διατροφής/διαιτητικής και της παιδιατρικής. Η ανάλυση της συν-συγγραφής αποκάλυψε συνεργασίες μεταξύ συγγραφέων διαφορετικών χωρών. Οι πιο συνηθισμένες κλινικές καταστάσεις ήταν η κατάθλιψη, ο διαβήτης, ο καρκίνος, η άνοια, ο πόνος, οι καρδιαγγειακές παθήσεις, το εγκεφαλικό επεισόδιο, η παχυσαρκία και η σχιζοφρένεια. Επιπλέον, αυτή η διατριβή παρουσιάζει εργαλεία που αξιολογούν την πληρότητα καταγραφής της αποτελεσματικότητας και της ασφάλειας των ιατρικών παρεμβάσεων στις ΑΣΑ. Αναπτύξαμε το εργαλείο PRIO-harms (56 υπο-στοιχεία) για να προωθήσουμε μια πιο διαφανή και ισορροπημένη αναφορά των ωφελειών και κινδύνων των ιατρικών παρεμβάσεων που διερευνώνται στις ΑΣΑ και το εργαλείο PRIO για περιλήψεις (20 υπο-στοιχεία) για να βοηθήσουμε τους συγγραφείς να συντάξουν ολοκληρωμένες, διαφανείς και κατατοπιστικές περιλήψεις για τις ΑΣΑ. Επίσης αυτά τα εργαλεία μπορούν να υιοθετηθούν από συντάκτες και εκδότες περιοδικών που δημοσιεύουν ΑΣΑ. Επιπροσθέτως, σε αυτήν τη διδακτορική διατριβή εισηγάγαμε νέες στατικές τεχνικές μέσω πινάκων και γραφημάτων για την οπτική παρουσίαση των αλληλεπικαλυπτόμενων ΣΑ που περιλαμβάνονται στις ΑΣΑ. Οι προτεινόμενες εκλεπτυσμένες γραφικές μέθοδοι (π.χ. “upset plots”, “heatmaps”, “node-link graphs”) ενδέχεται να βοηθήσουν τους μεθοδολόγους και συγγραφείς στην εξερεύνηση και απεικόνιση του βαθμού επικάλυψης των πρωτογενών μελετών των ΣΑ, γεγονός που με τη σειρά του μπορεί να βελτιώσει την εγκυρότητα και τη διαφάνεια στις ΑΣΑ. Ωστόσο, απαιτείται περισσότερη έρευνα για να κατανοηθεί ποιες γραφικές τεχνικές θα ήταν πιο χρήσιμες και ευκολότερες στην κατανόηση. Αυτά τα νέα εργαλεία και γραφικές τεχνικές θα μπορούσαν να βελτιωθούν από την κριτική αξιολόγηση και περαιτέρω επικύρωση από ειδικούς και ερευνητικές ομάδες που παράγουν κατευθυντήριες οδηγίες για τις ΑΣΑ.

A comparison of two assessment tools used in overviews of systematic reviews: ROBIS versus AMSTAR-2

Background AMSTAR-2 is a 16-item assessment tool to check the quality of a systematic review and establish whether the most important elements are reported. ROBIS is another assessment tool which was designed to evaluate the level of bias present within a systematic review. Our objective was to compare, contrast and establish both inter-rater reliability and usability of both tools as part of two overviews of systematic reviews. Strictly speaking, one tool assesses methodological quality (AMSTAR-2) and the other assesses risk of bias (ROBIS), but there is considerable overlap between the tools in terms of the signalling questions. Methods Three reviewers independently assessed 31 systematic reviews using both tools. The inter-rater reliability of all sub-sections using each instrument (AMSTAR-2 and ROBIS) was calculated using Gwet’s agreement coefficient (AC1 for unweighted analysis and AC2 for weighted analysis). Results Thirty-one systematic reviews were included. For AMSTAR-2, the median agreement for all questions was 0.61. Eight of the 16 AMSTAR-2 questions had substantial agreement or higher (> 0.61). For ROBIS, the median agreement for all questions was also 0.61. Eleven of the 24 ROBIS questions had substantial agreement or higher. Conclusion ROBIS is an effective tool for assessing risk of bias in systematic reviews and AMSTAR-2 is an effective tool at assessing quality. The median agreement between raters for both tools was identical (0.61). Reviews that included a meta-analysis were easier to rate with ROBIS; however, further developmental work could improve its use in reviews without a formal synthesis. AMSTAR-2 was more straightforward to use; however, more response options would be beneficial.

Publication practices of sub-Saharan African Cochrane authors: a bibliometric study

IntroductionCochrane Africa (https://africa.cochrane.org/) aims to increase Cochrane reviews addressing high priority questions in sub-Saharan Africa (SSA). Researchers residing in SSA, despite often drawing on Cochrane methods, training or resources, conduct and publish systematic reviews outside of Cochrane. Our objective was to investigate the extent to which Cochrane authors from SSA publish Cochrane and non-Cochrane reviews.MethodsWe conducted a bibliometric study of systematic reviews and overviews of systematic reviews from SSA, first by identifying SSA Cochrane authors, then retrieving their first and last author systematic reviews and overviews from PubMed (2008 to April 2019) and using descriptive analyses to investigate the country of origin, types of reviews and trends in publishing Cochrane and non-Cochrane systematic reviews over time. To be eligible, a review had to have predetermined objectives, eligibility criteria, at least two databases searched, data extraction, quality assessment and a first or last author with a SSA affiliation.ResultsWe identified 657 Cochrane authors and 757 eligible systematic reviews. Most authors were from South Africa (n=332; 51%), followed by Nigeria (n=126; 19%). Three-quarters of the reviews (71%) were systematic reviews of interventions. The intervention reviews were more likely to be Cochrane reviews (60.3% vs 39.7%). Conversely, the overviews (23.8% vs 76.2%), qualitative reviews (14.8% vs 85.2%), diagnostic test accuracy reviews (16.1% vs 83.9%) and the ‘other’ reviews (11.1% vs 88.9%) were more likely to be non-Cochrane reviews. During the study period, the number of non-Cochrane reviews increased more than the number of Cochrane reviews. About a quarter of the reviews covered infectious disease topics.ConclusionCochrane authors from SSA are increasingly publishing a diverse variety of systematic reviews and overviews of systematic reviews, often opting for non-Cochrane journals.

Topic overlap and research waste at the ‘Overviews of systematic reviews’ level: a meta-research study

Background Multiple ‘overviews of reviews’ conducted on the same topic (“overlapping overviews”) represent a waste of research resources, and can confuse clinicians who are required to choose among competing treatments. We aimed to assess the frequency and characteristics of overlapping overviews. Methods MEDLINE, Epistemonikos and Cochrane databases were searched for overviews that: synthesised reviews of health interventions and conducted systematic searches. Overlap in topic was defined as: duplication of PICO elements, not representing an update of a previous overview, and not a replication. We also categorized the overviews as broad or narrow in scope. Results Of 541 overviews identified (2000–2018), 172 (32%) overlapped across similar PICO. The overlapping overviews fell within 13 WHO ICD-10 medical classifications and 63 topics. The overviews may have overlapped partially or completely, such that a similar portion, major component(s), or complete representation of an overview was duplicated. 149/172 (87%) overlapping overviews were characterized as broad in scope. Most frequently, broad overviews had targeted populations for which multiple interventions were addressed (44%), or least frequently, they had a targeted intervention for multiple populations (17%). Conclusions One third of overviews overlapped in content with a majority covering broad topic areas, and fewer considering subsets of the evidence. A multiplicity of overviews on the same topic adds to the ongoing waste of research resources, time and effort across medical disciplines. This study and the database of 172 overlapping overviews can provide a guide to authors about which topics are covered, and gaps in the evidence for future analysis.

Topic duplication and research waste at the ‘Overviews of systematic reviews’ level: Survey of overlapping overviews

Background Multiple overviews of systematic reviews conducted on the same topic (“overlapping overviews”) represent a waste of research resources and can confuse or mislead clinicians and policymakers. We aimed to assess the frequency and characteristics of published overviews addressing the same clinical question or topic. Methods We used MEDLINE, Epistemonikos and Cochrane databases to locate overviews that: focused on synthesising reviews; conducted systematic searches; had a methods section; and examined a health intervention or clinical treatment. We then determined which overviews addressed the same or overlapping populations/settings, interventions, and outcomes [PIO]). Overlap in topic was defined as: duplication of PIO elements, not representing an update of a previous overview, and not a replication for quality purposes. Results Of 541 overviews located (2000–2018), 178 (33%) overlapped with another overview addressing a similar PIO. The topics of overlapping overviews fell within 13 WHO ICD-10 medical classifications, and there were 65 overlapping topics in total. The most prevalent topic with overlap across 7 overviews was smoking cessation (pharmacologic and non-pharmacologic interventions). Five overlapping overviews related to acupuncture for pain, 5 addressed cannabinoids for pain and symptoms, and 5 addressed exercise for bone and muscle health. For 15/65 (22%) of these topics, one author was involved in at least two of the overlapping overviews. Conclusions We found significant duplication and unnecessary overlap across overviews. To avoid waste and redundancy, protocols of overviews should be registered in a targeted database, and overviews should cite other studies on similar topic with a rationale.