scholarly journals Search for unpublished data by systematic reviewers: an audit

BMJ Open ◽  
2017 ◽  
Vol 7 (10) ◽  
pp. e017737 ◽  
Author(s):  
Hedyeh Ziai ◽  
Rujun Zhang ◽  
An-Wen Chan ◽  
Nav Persaud

ObjectivesWe audited a selection of systematic reviews published in 2013 and reported on the proportion of reviews that researched for unpublished data, included unpublished data in analysis and assessed for publication bias.DesignAudit of systematic reviews.Data sourcesWe searched PubMed and Ovid MEDLINE In-Process & Other Non-Indexed Citations between 1 January 2013 and 31 December 2013 for the following journals:Journal of the American Medical Association,The British Medical Journal,Lancet,Annals of Internal Medicineand theCochrane Database of Systematic Reviews. We also searched the Cochrane Library and included 100 randomly selected Cochrane reviews.Eligibility criteriaSystematic reviews published in 2013 in the selected journals were included. Methodological reviews were excluded.Data extraction and synthesisTwo reviewers independently reviewed each included systematic review. The following data were extracted: whether the review searched for grey literature or unpublished data, the sources searched, whether unpublished data were included in analysis, whether publication bias was assessed and whether there was evidence of publication bias.Main findings203 reviews were included for analysis. 36% (73/203) of studies did not describe any attempt to obtain unpublished studies or to search grey literature. 89% (116/130) of studies that sought unpublished data found them. 33% (68/203) of studies included an assessment of publication bias, and 40% (27/68) of these found evidence of publication bias.ConclusionA significant fraction of systematic reviews included in our study did not search for unpublished data. Publication bias may be present in almost half the published systematic reviews that assessed for it. Exclusion of unpublished data may lead to biased estimates of efficacy or safety in systematic reviews.

2006 ◽  
Vol 1 (4) ◽  
pp. 3 ◽  
Author(s):  
Li Zhang ◽  
Margaret Sampson ◽  
Jessie McGowan

Introduction - This study applied the principles of evidence based information practice to clarify the role of information specialists and librarians in the preparation of Cochrane systematic reviews and to determine whether information specialists impact the quality of searching in Cochrane systematic reviews. Objectives - This research project sought to determine how the contribution of the person responsible for searching in the preparation of Cochrane systematic reviews was reported; whether the contribution was recognized through authorship or acknowledgement; the qualifications of the searcher; and the association between the type of contributorship and characteristics of the search strategy, assessability, and the presence of certain types of errors. Methods - Data sources: The Cochrane Database of Systematic Reviews, The Cochrane Library 3 (2002). Inclusion criteria: The study included systematic reviews that met the following criteria: one or more sections of the Cochrane Highly Sensitive Search Strategy were utilised, primary studies were either randomised controlled trials (RCTs) or quasi-RCTs, and included and excluded studies were clearly identified. Data extraction: Two librarians assessed the searches for errors, establishing consensus on discordant ratings. Results - Of the 169 reviews screened for this project, 105 met all eligibility criteria. Authors fulfilled the searching role in 41.9% of reviews studied, acknowledged persons or groups in 13.3%, a combination in 9.5%, and the role was not reported in 35.2% of reviews. For the 78 reviews in which meta-analyses were performed, the positions of those responsible for statistical decisions were examined for comparative purposes. The statistical role was performed by an author in 47.4% of cases and unreported in the same number of cases. Insufficient analyzable data was obtained regarding professional qualifications (3/105 for searching, 2/78 for statistical decisions). Search quality was assessed for 66 searches across 74 reviews. In general, it was more possible to assess the search quality when the searcher role was reported. An association was found between the reporting of searcher role and the presence of a consequential error. There was no association between the number of consequential errors and how the contribution of the searcher was reported. Conclusions - Qualifications of the persons responsible for searching and statistical decision-making were poorly reported in Cochrane reviews, but more complete role reporting is associated with greater assessability of searches and fewer substantive errors in search strategies.


Author(s):  
Danah AlMubarak ◽  
Nikolaos Pandis ◽  
Martyn T Cobourne ◽  
Jadbinder Seehra

Summary Background This study aimed to assess the reporting of the methodological quality of search strategies undertaken in orthodontic quantitative systematic reviews (SRs) and hence their reproducibility. Materials and methods A search of a single electronic database (Medline via PubMed) was undertaken to identify interventional orthodontic SRs with meta-analysis published within a 10-year period. The Cochrane Library of Systematic Reviews was also sourced. Full articles were reviewed by two assessors against the eligibility criteria. The reporting quality of each search strategy was assessed using a previously validated checklist with a score of 1 or 2 given for each of the eight items. Cumulative totals were calculated. Guided by previous research, the authors agreed the following cut-offs to categorize the overall level of quality: 8–10 (poor), 10–12 (fair), and greater than 13 (good). Results A total of 127 SRs were analysed. The overall median quality score for the reporting of the search strategy was 14 [interquartile range (IQR): 13–15]. Cochrane SRs and those originating in Europe received higher aggregate scores, whereas no difference was evident based on Prospero registration. The continent of the corresponding author predicated the overall score. Non-Cochrane reviews achieved lower overall scores compared to Cochrane reviews (−1.0, 95% confidence interval: −1.65, −0.34, P = 0.003). The most frequently searched database was EMBASE (N = 93) and the median number of authors was 5 (IQR 4–6). Authors of 26.8% of SRs searched the grey literature. Language restrictions were applied to the search strategies of 88 (69.3%) SRs. Conclusions The reporting quality of search strategies undertaken in orthodontic SRs is at a good level but differences between Cochrane and non-Cochrane reviews currently exist. The reporting of searching of the grey literature and application of no language restrictions can be improved.


2019 ◽  
Vol 4 (1) ◽  
Author(s):  
Pauline A. J. Steegmans ◽  
Nicola Di Girolamo ◽  
Reint A. Meursinge Reynders

Abstract Background Titles and abstracts are the most read sections of biomedical papers. It is therefore important that abstracts transparently report both the beneficial and adverse effects of health care interventions and do not mislead the reader. Misleading reporting, interpretation, or extrapolation of study results is called “spin”. In this study, we will assess whether adverse effects of orthodontic interventions were reported or considered in the abstracts of both Cochrane and non-Cochrane reviews and whether spin was identified and what type of spin. Methods Eligibility criteria were defined for the type of study designs, participants, interventions, outcomes, and settings. We will include systematic reviews of clinical orthodontic interventions published in the five leading orthodontic journals and in the Cochrane Database. Empty reviews will be excluded. We will manually search eligible reviews published between 1 August 2009 and 31 July 2019. Data collection forms were developed a priori. All study selection and data extraction procedures will be conducted by two reviewers independently. Our main outcomes will be the prevalence of reported or considered adverse effects of orthodontic interventions in the abstract of systematic reviews and the prevalence of “spin” related to these adverse effects. We will also record the prevalence of three subtypes of spin, i.e., misleading reporting, misleading interpretation, and misleading extrapolation-related spin. All statistics will be calculated for the following groups: (1) all journals individually, (2) all journals together, and (3) the five leading orthodontic journals and the Cochrane Database of Systematic Reviews separately. Generalized linear models will be developed to compare the various groups. Discussion We expect that our results will raise the awareness of the importance of reporting and considering of adverse effects and the presence of the phenomenon of spin related to these effects in abstracts of systematic reviews of orthodontic interventions. This is important, because an incomplete and inadequate reporting, interpretation, or extrapolation of findings on adverse effects in abstracts of systematic reviews can mislead readers and could lead to inadequate clinical practice. Our findings could result in policy implications for making judgments about the acceptance for publication of systematic reviews of orthodontic interventions.


BMJ Open ◽  
2021 ◽  
Vol 11 (6) ◽  
pp. e041680
Author(s):  
Shu-Yue Pan ◽  
Rui-Juan Cheng ◽  
Zi-Jing Xia ◽  
Qiu-Ping Zhang ◽  
Yi Liu

ObjectivesGout, characterised by hyperuricaemia with monosodium urate crystal formation and inflammation, is the most common inflammatory arthritis in adults. Recent studies have found that elevated uric acid levels are related to the occurrence of dementia. We conducted a study to investigate the association between dementia and gout or hyperuricaemia.DesignSystematic review and meta-analysis of cohort studies.Data sourcesStudies were screened from inception to 28 June 2019 by searching Medline, Embase and the Cochrane Library databases.Eligibility criteriaCohort studies comparing the risk of dementia in patients with gout and hyperuricaemia versus non-gout and non-hyperuricaemia controls were enrolled.Data extraction and analysisTwo reviewers separately selected studies and extracted data using the Medical Subject Headings without restriction on languages or countries. The adjusted HRs were pooled using the DerSimonian and Laird random effects model. Sensitivity analyses were conducted to evaluate the stability of the results. Publication bias was evaluated using Egger’s and Begg’s tests. Quality assessment was performed according to the Newcastle-Ottawa Scale.ResultsFour cohort studies that met the inclusion criteria were included in our meta-analysis. We found that gout and hyperuricaemia did not increase the risk of dementia, with a pooled HR of 0.94 (95% CI 0.69 to 1.28), but might decrease the risk of Alzheimer’s disease (AD), with a pooled HR of 0.78 (95% CI 0.64 to 0.95). There was little evidence of publication bias. Quality assessment of the included studies was high (range: 6–8 points).ConclusionsOur study shows that gout and hyperuricaemia do not increase the risk of dementia. However, gout and hyperuricaemia might have a protective effect against AD. Due to the limited number of research articles, more investigations are needed to demonstrate the potential relationship between dementia and gout or hyperuricaemia.


2019 ◽  
Vol 8 (1) ◽  
Author(s):  
Rachel Perry ◽  
Verity Leach ◽  
Chris Penfold ◽  
Philippa Davies

Abstract Background Infantile colic is a distressing condition characterised by excessive crying in the first few months of life. The aim of this research was to update the synthesis of evidence of complementary and alternative medicine (CAM) research literature on infantile colic and establish what evidence is currently available. Methods Medline, Embase and AMED (via Ovid), Web of Science and Central via Cochrane library were searched from their inception to September 2018. Google Scholar and OpenGrey were searched for grey literature and PROSPERO for ongoing reviews. Published systematic reviews that included randomised controlled trials (RCTs) of infants aged up to 1 year, diagnosed with infantile colic using standard diagnostic criteria, were eligible. Reviews of RCTs that assessed the effectiveness of any individual CAM therapy were included. Three reviewers were involved in data extraction and quality assessment using the AMSTAR-2 scale and risk of bias using the ROBIS tool. Results Sixteen systematic reviews were identified. Probiotics, fennel extract and spinal manipulation show promise to alleviate symptoms of colic, although some concerns remain. Acupuncture and soy are currently not recommended. The majority of the reviews were assessed as having high or unclear risk of bias and low confidence in the findings. Conclusion There is clearly a need for larger and more methodologically sound RCTs to be conducted on the effectiveness of some CAM therapies for IC. Particular focus on probiotics in non-breastfed infants is pertinent. Systematic review registration PROSPERO: CRD42018092966.


BMJ Open ◽  
2021 ◽  
Vol 11 (9) ◽  
pp. e051839
Author(s):  
Lawrence Mbuagbaw ◽  
Anel Schoonees ◽  
Joy Oliver ◽  
Dachi Arikpo ◽  
Solange Durão ◽  
...  

IntroductionCochrane Africa (https://africa.cochrane.org/) aims to increase Cochrane reviews addressing high priority questions in sub-Saharan Africa (SSA). Researchers residing in SSA, despite often drawing on Cochrane methods, training or resources, conduct and publish systematic reviews outside of Cochrane. Our objective was to investigate the extent to which Cochrane authors from SSA publish Cochrane and non-Cochrane reviews.MethodsWe conducted a bibliometric study of systematic reviews and overviews of systematic reviews from SSA, first by identifying SSA Cochrane authors, then retrieving their first and last author systematic reviews and overviews from PubMed (2008 to April 2019) and using descriptive analyses to investigate the country of origin, types of reviews and trends in publishing Cochrane and non-Cochrane systematic reviews over time. To be eligible, a review had to have predetermined objectives, eligibility criteria, at least two databases searched, data extraction, quality assessment and a first or last author with a SSA affiliation.ResultsWe identified 657 Cochrane authors and 757 eligible systematic reviews. Most authors were from South Africa (n=332; 51%), followed by Nigeria (n=126; 19%). Three-quarters of the reviews (71%) were systematic reviews of interventions. The intervention reviews were more likely to be Cochrane reviews (60.3% vs 39.7%). Conversely, the overviews (23.8% vs 76.2%), qualitative reviews (14.8% vs 85.2%), diagnostic test accuracy reviews (16.1% vs 83.9%) and the ‘other’ reviews (11.1% vs 88.9%) were more likely to be non-Cochrane reviews. During the study period, the number of non-Cochrane reviews increased more than the number of Cochrane reviews. About a quarter of the reviews covered infectious disease topics.ConclusionCochrane authors from SSA are increasingly publishing a diverse variety of systematic reviews and overviews of systematic reviews, often opting for non-Cochrane journals.


BMJ Open ◽  
2018 ◽  
Vol 8 (3) ◽  
pp. e020869 ◽  
Author(s):  
Marius Goldkuhle ◽  
Vikram M Narayan ◽  
Aaron Weigl ◽  
Philipp Dahm ◽  
Nicole Skoetz

ObjectiveTo compare cancer-related systematic reviews (SRs) published in the Cochrane Database of SRs (CDSR) and high-impact journals, with respect to type, content, quality and citation rates.DesignMethodological SR with assessment and comparison of SRs and meta-analyses. Two authors independently assessed methodological quality using an Assessment of Multiple Systematic Reviews (AMSTAR)-based extraction form. Both authors independently screened search results, extracted content-relevant characteristics and retrieved citation numbers of the included reviews using the Clarivate Analytics Web of Science database.Data sourcesCancer-related SRs were retrieved from the CDSR, as well as from the 10 journals which publish oncological SRs and had the highest impact factors, using a comprehensive search in both the CDSR and MEDLINE.Eligibility criteria for selecting studiesWe included all cancer-related SRs and meta-analyses published from January 2011 to May 2016. Methodological SRs were excluded.ResultsWe included 346 applicable Cochrane reviews and 215 SRs from high-impact journals. Cochrane reviews consistently met more individual AMSTAR criteria, notably with regard to an a priori design (risk ratio (RR) 3.89; 95% CI 3.10 to 4.88), inclusion of the grey literature and trial registries (RR 3.52; 95% CI 2.84 to 4.37) in their searches, and the reporting of excluded studies (RR 8.80; 95% CI 6.06 to 12.78). Cochrane reviews were less likely to address questions of prognosis (RR 0.04; 95% CI 0.02 to 0.09), use individual patient data (RR 0.03; 95% CI 0.01 to 0.09) or be based on non-randomised controlled trials (RR 0.04; 95% CI 0.02 to 0.09). Citation rates of Cochrane reviews were notably lower than those for high-impact journals (Cochrane reviews: mean number of citations 6.52 (range 0–143); high-impact journal SRs: 74.45 (0–652)).ConclusionsWhen comparing cancer-related SRs published in the CDSR versus those published in high-impact medical journals, Cochrane reviews were consistently of higher methodological quality, but cited less frequently.


2002 ◽  
Vol 18 (4) ◽  
pp. 820-823 ◽  
Author(s):  
Susan Mallett ◽  
Mike Clarke

Objectives: To describe the number of trials and participants in a typical systematic review from The Cochrane Database of Systematic Reviews.Methods: The number of trials in 1,000 Cochrane systematic reviews in issue 1, 2001 of The Cochrane Database of Systematic Reviews was counted for three categories of trial: included trials, ongoing trials, and trials awaiting assessment for inclusion. (The term trial is used in this paper, although a small number of Cochrane reviews include studies that are not trials.) The total number of participants in included trials was extracted from a sample of reviews.Results: A total of 9,778 trials were included in the Cochrane reviews. There were a further 356 ongoing trials and 1,138 trials awaiting assessment for inclusion. A typical review contained six included trials. Forty percent of the reviews listed ongoing trials and/or trials awaiting assessment for inclusion. Based on a sample of 258 reviews, the median number of participants per review was 945 (interquartile range, 313 to 2,511) per review and 118 (interquartile range, 60 to 241) per trial.Conclusion: This report is a descriptive study of the number of trials and participants in a typical Cochrane review from The Cochrane Library, issue 1, 2001.


2019 ◽  
Author(s):  
Claudia Hacke ◽  
David Nunan

AbstractObjectiveTo explore factors underpinning discrepancies in reported pooled effect estimates from Cochrane and non-Cochrane systematic reviews answering the same question.Study Design and SettingWe observed discrepant pooled effects in 23 out of 24 pairs of meta-analyses from Cochrane and non-Cochrane systematic reviews answering the same question. Here we present the results of a systematic assessment of methodological quality and factors that explain the observed quantitative discrepancies. Methodological quality of each review was assessed using AMSTAR (Assessing the Methodological Quality of Systematic Reviews). Matched pairs were contrasted at the macro- (review methodology), meso- (application of methodology) and micro- (data extraction) level and reasons for differences were derived.ResultsAll Cochrane reviews had high methodological quality (AMSTAR 8-11), whereas the majority (87.5%) of non-Cochrane reviews were classified as moderate (AMSTAR 4-7). Only one pair included exactly the same studies for their respective meta-analyses but there was still a discrepancy in the pooled estimate due to differences in data extraction. One pair did not include any study of its match and for one pair the same effect estimates were reported despite inclusion of different studies. The remaining pairs included at least one study in their match. Due to insufficient reporting (predominantly affecting non-Cochrane reviews) we were only able to completely ascertain the reasons for discrepancies in all included studies for 9/24 (37.5%) pairs. Across all pairs, differences in pre-defined methods (macro-level) including search strategy, eligibility criteria and performance of dual screening could possibly explain mismatches in included studies. Study selection procedures (meso-level) including disagreements in the interpretation of pre-defined eligibility criteria (14 matches) were identified as reasons underpinning discrepant review findings. Comparison of data extraction from primary studies (micro-level) was not possible in 13/24 pairs as a result of the non-Cochrane review providing insufficient details of the studies included in their meta-analyses. Two out of 24 pairs completely agreed on the numerical data presented for the same studies in their respective meta-analysis. Both review types provided sufficient information to check the accuracy of data extraction for 8 pairs (45 studies) where there were discrepancies. An assessment of 50% (22 studies) of these showed that reasons for differences in extracted data could be identified in 15 studies. We found examples for both types of review where data presented were discrepant from that given in the source study without a plausible explanation.ConclusionMethodological and author judgements and performance are key aspects underpinning poor overlap of included studies and discrepancies in reported pooled effect estimates between topic-matched reviews. Though caution must be taken when extrapolating, our findings raise the question as to what extent the entire meta-analysis evidence-base accurately reflects the available primary research both in terms of volume and data. Reinforcing awareness of the application of guidelines for systematic reviews and meta-analyses may help mitigate some of the key issues identified in our analysis.What is new?Key findings Non-Cochrane reviews were of a lower overall methodological quality compared with Cochrane reviews. Discrepant results of meta-analyses on the same topic can be attributed to differences in included studies based on review author decision, judgements and performance at different stages of the review process.What this adds to what was known?This study provides the most robust analysis to date of the potential methodological factors underpinning discrepant review findings between matched meta-analyses answering the same question. Assessing differences between reviews at the macro-, meso-, and micro-levels is a useful method to identify reasons for discrepant meta-analyses at key stages of the review process.What is the implication and what should change now?There is a need for a standardised approach to performing matched-pair analysis of meta-analyses and systematic reviews answering the same question. Our paper provides a base for this that can be refined by replication and expert consensus.


BMJ Open ◽  
2021 ◽  
Vol 11 (4) ◽  
pp. e043091
Author(s):  
Rikke Siersbaek ◽  
John Alexander Ford ◽  
Sara Burke ◽  
Clíona Ní Cheallaigh ◽  
Steve Thomas

ObjectiveThe objective of this study was to identify and understand the health system contexts and mechanisms that allow for homeless populations to access appropriate healthcare when needed.DesignA realist review.Data sourcesOvid MEDLINE, embase.com, CINAHL, ASSIA and grey literature until April 2019.Eligibility criteria for selecting studiesThe purpose of the review was to identify health system patterns which enable access to healthcare for people who experience homelessness. Peer-reviewed articles were identified through a systematic search, grey literature search, citation tracking and expert recommendations. Studies meeting the inclusion criteria were assessed for rigour and relevance and coded to identify data relating to contexts, mechanisms and/or outcomes.AnalysisInductive and deductive coding was used to generate context–mechanism–outcome configurations, which were refined and then used to build several iterations of the overarching programme theory.ResultsSystematic searching identified 330 review articles, of which 24 were included. An additional 11 grey literature and primary sources were identified through citation tracking and expert recommendation. Additional purposive searching of grey literature yielded 50 records, of which 12 were included, for a total of 47 included sources. The analysis found that healthcare access for populations experiencing homelessness is improved when services are coordinated and delivered in a way that is organised around the person with a high degree of flexibility and a culture that rejects stigma, generating trusting relationships between patients and staff/practitioners. Health systems should provide long-term, dependable funding for services to ensure sustainability and staff retention.ConclusionsWith homelessness on the rise internationally, healthcare systems should focus on high-level factors such as funding stability, building inclusive cultures and setting goals which encourage and support staff to provide flexible, timely and connected services to improve access.


Sign in / Sign up

Export Citation Format

Share Document