scholarly journals Characteristics, quality and volume of the first 5 months of the COVID-19 evidence synthesis infodemic: a meta-research study

2021 ◽  
pp. bmjebm-2021-111710
Author(s):  
Rebecca Abbott ◽  
Alison Bethel ◽  
Morwenna Rogers ◽  
Rebecca Whear ◽  
Noreen Orr ◽  
...  

ObjectiveThe academic and scientific community has reacted at pace to gather evidence to help and inform about COVID-19. Concerns have been raised about the quality of this evidence. The aim of this review was to map the nature, scope and quality of evidence syntheses on COVID-19 and to explore the relationship between review quality and the extent of researcher, policy and media interest.Design and settingA meta-research: systematic review of reviews.Information sourcesPubMed, Epistemonikos COVID-19 evidence, the Cochrane Library of Systematic Reviews, the Cochrane COVID-19 Study Register, EMBASE, CINAHL, Web of Science Core Collection and the WHO COVID-19 database, searched between 10 June 2020 and 15 June 2020.Eligibility criteriaAny peer-reviewed article reported as a systematic review, rapid review, overview, meta-analysis or qualitative evidence synthesis in the title or abstract addressing a research question relating to COVID-19. Articles described as meta-analyses but not undertaken as part of a systematic or rapid review were excluded.Study selection and data extractionAbstract and full text screening were undertaken by two independent reviewers. Descriptive information on review type, purpose, population, size, citation and attention metrics were extracted along with whether the review met the definition of a systematic review according to six key methodological criteria. For those meeting all criteria, additional data on methods and publication metrics were extracted.Risk of biasFor articles meeting all six criteria required to meet the definition of a systematic review, AMSTAR-2 ((A MeaSurement Tool to Assess systematic Reviews, version 2.0) was used to assess the quality of the reported methods.Results2334 articles were screened, resulting in 280 reviews being included: 232 systematic reviews, 46 rapid reviews and 2 overviews. Less than half reported undertaking critical appraisal and a third had no reproducible search strategy. There was considerable overlap in topics, with discordant findings. Eighty-eight of the 280 reviews met all six systematic review criteria. Of these, just 3 were rated as of moderate or high quality on AMSTAR-2, with the majority having critical flaws: only a third reported registering a protocol, and less than one in five searched named COVID-19 databases. Review conduct and publication were rapid, with 52 of the 88 systematic reviews reported as being conducted within 3 weeks, and a half published within 3 weeks of submission. Researcher and media interest, as measured by altmetrics and citations, was high, and was not correlated with quality.DiscussionThis meta-research of early published COVID-19 evidence syntheses found low-quality reviews being published at pace, often with short publication turnarounds. Despite being of low quality and many lacking robust methods, the reviews received substantial attention across both academic and public platforms, and the attention was not related to the quality of review methods.InterpretationFlaws in systematic review methods limit the validity of a review and the generalisability of its findings. Yet, by being reported as ‘systematic reviews’, many readers may well regard them as high-quality evidence, irrespective of the actual methods undertaken. The challenge especially in times such as this pandemic is to provide indications of trustworthiness in evidence that is available in ‘real time’.PROSPERO registration numberCRD42020188822.

2021 ◽  
Vol 23 (Supplement_2) ◽  
pp. ii51-ii52
Author(s):  
A M George ◽  
S Gupta ◽  
S M Keshwara ◽  
M A Mustafa ◽  
C S Gillespie ◽  
...  

Abstract BACKGROUND Systematic reviews and meta-analyses constitute the highest level of research evidence and for a disease with limited clinical trial activity, are often relied upon to help inform clinical practice. This review of reviews evaluates both the reporting & methodological quality of meningioma evidence syntheses. MATERIAL AND METHODS Potentially eligible meningioma reviews published between 1st January 1990 and 31st December 2020 were identified from eight electronic databases. Inclusion required the study to meet the Cochrane guideline definition of a systematic review or meta-analysis. Reviews concerning neurofibromatosis type 2, spinal and pediatric meningiomas were excluded. The reporting and methodological quality of articles were assessed against the following modified guidelines: Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA), A MeaSurement Tool to Assess Systematic Reviews (AMSTAR2) and the Risk of Bias in Systematic Reviews (ROBIS) guidelines. RESULTS 117 systematic reviews were identified, 57 of which included meta-analysis (48.7%). The number of meningioma systematic reviews published each year has increased with 63 studies (53.9%) published between 01/2018 and 12/2020. A median of 17 studies (IQR 9–29) were included per review. Impact factor of journals publishing a systematic review with or without a meta-analysis was similar (median 2.3 vs 1.8, P=0.397). The mean PRISMA scores for systematic reviews with a meta-analysis was 21.11 (SD 4.1, 78% adherence) and without was 13.89 (SD 3.4, 63% adherence). Twenty-nine systematic reviews with meta-analysis (51%) and 11 without meta-analysis (18%) achieved greater than 80% adherence to PRISMA recommendations. Methodological quality assessment using AMSTAR2 revealed one study (0.9%) as high quality whilst 111 (94.8%) studies were graded as critically low. One hundred and two articles (87.2%) did not utilize a comprehensive search strategy as defined by the AMSTAR2 tool. Ninety-nine studies (84.6%) obtained a high level of concern for potential bias as per the ROBIS assessment. One hundred and eight articles (92.3%) failed to present information that a protocol had been established prior to study commencement and 76 articles (65.0%) did not conduct a risk of bias assessment. Across the three tools, domains relating to the establishment of a protocol prior to review commencement and conducting appropriate risk of bias assessments were frequently low scoring. CONCLUSION Overall reporting and methodological quality of meningioma systematic reviews was sub-optimal. Established critical appraisal tools and reporting guidelines should be utilized a priori to assist in producing high-quality systematic reviews.


2020 ◽  
Vol 2020 ◽  
pp. 1-14 ◽  
Author(s):  
Yu-Xi Li ◽  
Xi-li Xiao ◽  
Dong-Ling Zhong ◽  
Liao-Jun Luo ◽  
Han Yang ◽  
...  

Background. Migraine is a common neurological disease, which burdens individuals and society all over the world. Acupuncture, an important method in Traditional Chinese Medicine, is widely used in clinical practice as a treatment for migraine. Several systematic reviews (SRs) have investigated the effectiveness and safety of acupuncture for migraine. Objective. To summarize and critically assess the quality of relevant SRs and present an objective and comprehensive evidence on the effectiveness and safety of acupuncture for migraine. Data Sources. MEDLINE, Embase, Cochrane Library, PROSPERO database, Chinese National Knowledge Infrastructure (CNKI), Chinese Biological Medicine (CBM), China Science and Technology Journal (SCTJ), and WanFang database (WF) were searched from inception to December 2019 and grey literatures were manually searched. Selection Criteria. SRs which meet the criteria were independently selected by 2 reviewers according to a predetermined protocol. Data Extraction. Characteristics of included SRs were independently extracted by 2 reviewers following a predefined data extraction form. Review Appraisal. The methodological quality, risk of bias, and reporting quality of included SRs were assessed, respectively, by a Measurement Tool to Assess Systematic Reviews (AMSTAR) 2, the Risk of Bias in Systematic reviews (ROBIS) tool, and the Preferred Reporting Item for Systematic Review and Meta-analysis-Acupuncture (PRISMA-A) statement. The quality of outcomes was evaluated by the Grading of Recommendations Assessment, Development, and Evaluation (GRADE). Results. A total of 15 SRs were included. All the SRs were published between 2011–2019. Based on AMSTAR 2, 14 out of 15 SRs were rated critically low quality and 1 was rated low quality. According to ROBIS tool, 9 SRs (60%) were low risk of bias. With the PRISMA-A checklist, we found 11 out of 15 SRs were found adequately reported over 70%. With the GRADE tool, we found high quality of evidence indicated that the effective rate of acupuncture was superior to western medicine in treatment of migraine. Besides, acupuncture reduced more headache days and the times of using painkiller and was more effective in reducing the frequency and degree of headache than western medicine and sham acupuncture. Limitations. There might be some missing information. The accuracy of the conclusions may be decreased reduced since we were unable to synthesis all the evidence. Conclusions. Based on high quality of evidence, we concluded that acupuncture may be an effective and safe therapy for migraine. However, the quality of SRs in acupuncture for migraine still needs more improvement.


2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Ryan Chow ◽  
Eileen Huang ◽  
Allen Li ◽  
Sophie Li ◽  
Sarah Y. Fu ◽  
...  

Abstract Background Postpartum depression (PPD) is a highly prevalent mental health problem that affects parental health with implications for child health in infancy, childhood, adolescence and beyond. The primary aim of this study was to critically appraise available systematic reviews describing interventions for PPD. The secondary aim was to evaluate the methodological quality of the included systematic reviews and their conclusions. Methods An electronic database search of MEDLINE, Embase, and the Cochrane Library from 2000 to 2020 was conducted to identify systematic reviews that examined an intervention for PPD. A Measurement Tool to Assess Systematic Reviews was utilized to independently score each included systematic review which was then critically appraised to better define the most effective therapeutic options for PPD. Results Of the 842 studies identified, 83 met the a priori criteria for inclusion. Based on the systematic reviews with the highest methodological quality, we found that use of antidepressants and telemedicine were the most effective treatments for PPD. Symptoms of PPD were also improved by traditional herbal medicine and aromatherapy. Current evidence for physical exercise and cognitive behavioural therapy in treating PPD remains equivocal. A significant, but weak relationship between AMSTAR score and journal impact factor was observed (p = 0.03, r = 0.24; 95% CI, 0.02 to 0.43) whilst no relationship was found between the number of total citations (p = 0.27, r = 0.12; 95% CI, − 0.09 to 0.34), or source of funding (p = 0.19). Conclusion Overall the systematic reviews on interventions for PPD are of low-moderate quality and are not improving over time. Antidepressants and telemedicine were the most effective therapeutic interventions for PPD treatment.


BMJ Open ◽  
2021 ◽  
Vol 11 (4) ◽  
pp. e044472
Author(s):  
Saar Hommes ◽  
Ruben Vromans ◽  
Felix Clouth ◽  
Xander Verbeek ◽  
Ignace de Hingh ◽  
...  

ObjectivesTo assess the communicative quality of colorectal cancer patient decision aids (DAs) about treatment options, the current systematic review was conducted.DesignSystematic review.Data sourcesDAs (published between 2006 and 2019) were identified through academic literature (MEDLINE, Embase, CINAHL, Cochrane Library and PsycINFO) and online sources.Eligibility criteriaDAs were only included if they supported the decision-making process of patients with colon, rectal or colorectal cancer in stages I–III.Data extraction and synthesisAfter the search strategy was adapted from similar systematic reviews and checked by a colorectal cancer surgeon, two independent reviewers screened and selected the articles. After initial screening, disagreements were resolved with a third reviewer. The review was conducted in concordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. DAs were assessed using the International Patient Decision Aid Standards (IPDAS) and Communicative Aspects (CA) checklist.ResultsIn total, 18 DAs were selected. Both the IPDAS and CA checklist revealed that there was a lot of variation in the (communicative) quality of DAs. The findings highlight that (1) personalisation of treatment information in DAs is lacking, (2) outcome probability information is mostly communicated verbally and (3) information in DAs is generally biased towards a specific treatment. Additionally, (4) DAs about colorectal cancer are lengthy and (5) many DAs are not written in plain language.ConclusionsBoth instruments (IPDAS and CA) revealed great variation in the (communicative) quality of colorectal cancer DAs. Developers of patient DAs should focus on personalisation techniques and could use both the IPDAS and CA checklist in the developmental process to ensure personalised health communication and facilitate shared decision making in clinical practice.


2021 ◽  
Vol 10 (1) ◽  
Author(s):  
Emma Farrell ◽  
Marta Bustillo ◽  
Carel W. le Roux ◽  
Joe Nadglowski ◽  
Eva Hollmann ◽  
...  

Abstract Background Obesity is a prevalent, complex, progressive and relapsing chronic disease characterised by abnormal or excessive body fat that impairs health and quality of life. It affects more than 650 million adults worldwide and is associated with a range of health complications. Qualitative research plays a key role in understanding patient experiences and the factors that facilitate or hinder the effectiveness of health interventions. This review aims to systematically locate, assess and synthesise qualitative studies in order to develop a more comprehensive understanding of the lived experience of people with obesity. Methods This is a protocol for a qualitative evidence synthesis of the lived experience of people with obesity. A defined search strategy will be employed in conducting a comprehensive literature search of the following databases: PubMed, Embase, PsycInfo, PsycArticles and Dimensions (from 2011 onwards). Qualitative studies focusing on the lived experience of adults with obesity (BMI >30) will be included. Two reviewers will independently screen all citations, abstracts and full-text articles and abstract data. The quality of included studies will be appraised using the critical appraisal skills programme (CASP) criteria. Thematic synthesis will be conducted on all of the included studies. Confidence in the review findings will be assessed using GRADE CERQual. Discussion The findings from this synthesis will be used to inform the EU Innovative Medicines Initiative (IMI)-funded SOPHIA (Stratification of Obesity Phenotypes to Optimize Future Obesity Therapy) study. The objective of SOPHIA is to optimise future obesity treatment and stimulate a new narrative, understanding and vocabulary around obesity as a set of complex and chronic diseases. The findings will also be useful to health care providers and policy makers who seek to understand the experience of those with obesity. Systematic review registration PROSPERO CRD42020214560.


2021 ◽  
Author(s):  
Luísa Prada ◽  
Ana Prada ◽  
Miguel Antunes ◽  
Ricardo Fernandes ◽  
João Costa ◽  
...  

Abstract Introduction:Over the last years, the number of systematic reviews published is steadily increasing due to the global interest in this type of evidence synthesis. However, little is known about the characteristics of this research published in Portuguese medical journals. This study aims to evaluate the publication trends and overall quality of these systematic reviews.Material and Methods:Systematic reviews were identified through an electronic search up to August 2020, targeting Portuguese Medical journals indexed in MEDLINE. Systematic reviews selection and data extraction were done independently by three authors. The overall quality critical appraisal using the A MeaSurement Tool to Assess systematic Reviews (AMSTAR II) was independently assessed by three authors. Disagreements were solved by consensus.Results:Seventy systematic reviews published in 5 Portuguese medical journals were included. Most (n=57; 81,4%) were systematic reviews without meta-analysis. Until 2010, the number of systematic reviews per year increased. Since then, the number of reviews published has not remained stable and no less than 3 SRs were published per year. According to the systematic reviews’ typology, most have been predominantly conducted to assess the effectiveness of health interventions (n=28; 40,0%). General and Internal Medicine (n=26; 37,1%) was the most addressed field. Most systematic reviews (n=45; 64,3%) were rated as being of “critically low-quality”.Conclusions:There were consistent flaws in the methodological quality report of the systematic reviews included, particularly in establishing a prior protocol and not assessing the potential impact of the risk of bias on the results.Through the years, the number of systematic reviews published increased, yet their quality is suboptimal. There is a need to improve the reporting of systematic reviews in Portuguese medical journals, which can be achieved by better adherence to quality checklists/tools.Systematic review registration: INPLASY202090105


F1000Research ◽  
2019 ◽  
Vol 8 ◽  
pp. 221 ◽  
Author(s):  
Assem M. Khamis ◽  
Lara A. Kahale ◽  
Hector Pardo-Hernandez ◽  
Holger J. Schünemann ◽  
Elie A. Akl

Background: The living systematic review (LSR) is an emerging approach for improved evidence synthesis that uses continual updating to include relevant new evidence as soon as it is published. The objectives of this study are to: 1) assess the methods of conduct and reporting of living systematic reviews using a living study approach; and 2) describe the life cycle of living systematic reviews, i.e., describe the changes over time to their methods and findings. Methods: For objective 1, we will begin by conducting a cross-sectional survey and then update its findings every 6 months by including newly published LSRs. For objective 2, we will conduct a prospective longitudinal follow-up of the cohort of included LSRs. To identify LSRs, we will continually search the following electronic databases: Medline, EMBASE and the Cochrane library. We will also contact groups conducting LSRs to identify eligible studies that we might have missed. We will follow the standard systematic review methodology for study selection and data abstraction. For each LSR update, we will abstract information on the following: 1) general characteristics, 2) systematic review methodology, 3) living approach methodology, 4) results, and 5) editorial and publication processes. We will update the findings of both the surveys and the longitudinal follow-up of included LSRs every 6 months. In addition, we will identify articles addressing LSR methods to be included in an ‘LSR methods repository’. Conclusion: The proposed living methodological survey will allow us to monitor how the methods of conduct, and reporting as well as the findings of LSRs change over time. Ultimately this should help with ensuring the quality and transparency of LSRs.


Author(s):  
Christine Baldwin ◽  
Rosemary Smith ◽  
Michelle Gibbs ◽  
C Elizabeth Weekes ◽  
Peter W Emery

ABSTRACT There is considerable heterogeneity across the findings of systematic reviews of oral nutritional supplement (ONS) interventions, presenting difficulties for healthcare decision-makers and patients alike. It is not known whether heterogeneity arises from differences in patient populations or relates to methodological rigor. This overview aimed to collate and compare findings from systematic reviews of ONSs compared with routine care in adult patients who were malnourished or at risk of malnutrition with any clinical condition and to examine their methodological quality. Three electronic databases were searched to July 2019, supplemented with hand-searching. Data on all outcomes were extracted and review methodological quality assessed using A MeaSurement Tool for Assessment of systematic Reviews (AMSTAR). Twenty-two reviews were included, 11 in groups from mixed clinical backgrounds and 11 in specific clinical conditions. Ninety-one meta-analyses were identified for 12 different outcomes but there was discordance between results. Significant benefits of ONSs were reported in 4 of 4 analyses of energy intake, 7 of 11 analyses of body weight, 7 of 22 analyses of mortality, 10 of 17 analyses of complications (total and infectious), 1 of 3 analyses of muscle strength, 4 of 9 analyses of body composition/nutritional status, 2 of 14 analyses of length of stay, and 2 of 5 analyses of hospital readmissions. Ten reviews were high quality (AMSTAR scores 8–11), 9 moderate (AMSTAR scores 3–8), and 3 poor (AMSTAR scores 0–3). Methodological deficiencies were limitations to searches, poor reporting of heterogeneity, and failure to incorporate quality of evidence into any recommendations. Discordance between reviews was not markedly reduced when only high-quality reviews were considered. Evidence for the effects of ONS in malnourished patients or those who are at risk of malnutrition is uncertain, and discordance in results can arise from differences in clinical background of patients or the etiological basis of malnutrition.


2018 ◽  
Vol 34 (S1) ◽  
pp. 107-107
Author(s):  
Thomas Poder ◽  
Véronique Déry ◽  
Jean-Francois Fisette

Introduction:Speech recognition is increasingly used in medical reporting. The aim of this article is to identify in the literature the advantages and weaknesses of this technology, as well as barriers and facilitators to its implementation.Methods:A systematic review of systematic reviews has been conducted in PubMed, Scopus, Cochrane Library and Center for Reviews and Dissemination up to August 2017. The grey literature has also been consulted. The quality of systematic reviews has been assessed with the AMSTAR checklist. Inclusion criteria were to use speech recognition for medical reporting (front or back-end). A Survey has also been conducted in Quebec, Canada, to identify the dissemination of this technology in this province, as well as the factors of success or failure in its implementation.Results:Five systematic reviews were identified. These reviews indicated a high level of heterogeneity across studies. The quality of the studies reported was generally poor. Speech recognition is not as accurate as human transcription but can dramatically reduce the turnaround times for reporting. In front-end use, medical doctors need to spend more time for dictation and correction than with human transcription. With speech recognition, major errors can be up to three times more frequent. In back-end use, a potential increase in the productivity of transcriptionist is noted.Conclusions:Speech recognition offers some advantages for medical reporting, the main one being a reduction in turnaround times. However, these advantages are challenged by an increased burden for medical doctor and risks of additional errors in medical reports. It is also hard to identify for which medical specialties and which clinical activities the use of speech recognition will be the most beneficial.


2019 ◽  
Vol 2 (2) ◽  
pp. 50-57
Author(s):  
Amanda Yang Shen ◽  
Robert S Ware ◽  
Tom J O'Donohoe ◽  
Jason Wasiak

Background: An increasing number of systematic reviews are published on an annual basis. Although perusal of the full text of articles is preferable, abstracts are sometimes relied upon to guide clinical decisions. Despite this, the abstracts of systematic reviews have historically been poorly reported. We evaluated the reporting quality of systematic review abstracts within hand and wrist pathology literature. Methods: We searched MEDLINE®, EMBASE and Cochrane Library from inception to December 2017 for systematic reviews in hand and wrist pathology using the 12-item PRISMA-A checklist to assess abstract reporting quality. Results: A total of 114 abstracts were included. Most related to fracture (38%) or arthritis (17%) management. Forty-seven systematic reviews (41%) included meta-analysis. Mean PRISMA-A score was 3.6/12 with Cochrane reviews having the highest mean score and hand-specific journals having the lowest. Abstracts longer than 300 words (mean difference [MD]: 1.43, 95% CI [0.74, 2.13]; p <0.001) and systematic reviews with meta-analysis (MD: 0.64, 95% CI [0.05, 1.22]; p = 0.034) were associated with higher scores. Unstructured abstracts were associated with lower scores (MD: –0.65, 95% CI [–1.28, –0.02]; p = 0.044). A limitation of this study is the possible exclusion of relevant studies that were not published in the English language. Conclusion: Abstracts of systematic reviews pertaining to hand and wrist pathology have been suboptimally reported as assessed by the PRISMA-A checklist. Improvements in reporting quality could be achieved by endorsement of PRISMA-A guidelines by authors and journals, and reducing constraints on abstract length.


Sign in / Sign up

Export Citation Format

Share Document