scholarly journals The magnitude of small-study effects in the Cochrane Database of Systematic Reviews: an empirical study of nearly 30 000 meta-analyses

2019 ◽  
Vol 25 (1) ◽  
pp. 27-32 ◽  
Author(s):  
Lifeng Lin ◽  
Linyu Shi ◽  
Haitao Chu ◽  
Mohammad Hassan Murad
Keyword(s):  
Systematic Reviews ◽  
Statistical Tests ◽  
Small Study ◽  
P Values ◽  
Binary Decision ◽  
Empirical Distributions ◽  
Cochrane Database ◽  
Rules Of Thumb ◽  
Meta Analyses ◽  
Relative Change

Publication bias, more generally termed as small-study effect, is a major threat to the validity of meta-analyses. Most meta-analysts rely on the p values from statistical tests to make a binary decision about the presence or absence of small-study effects. Measures are available to quantify small-study effects’ magnitude, but the current literature lacks clear rules to help evidence users in judging whether such effects are minimal or substantial. This article aims to provide rules of thumb for interpreting the measures. We use six measures to evaluate small-study effects in 29 932 meta-analyses from the Cochrane Database of Systematic Reviews. They include Egger’s regression intercept and the skewness under both the fixed-effect and random-effects settings, the proportion of suppressed studies, and the relative change of the estimated overall result due to small-study effects. The cut-offs for different extents of small-study effects are determined based on the quantiles in these distributions. We present the empirical distributions of the six measures and propose a rough guide to interpret the measures’ magnitude. The proposed rules of thumb may help evidence users grade the certainty in evidence as impacted by small-study effects.

The clinical outcome of the Metha short hip stem: a systematic scoping review

2020 ◽  
pp. 112070002090371 ◽  
Author(s):  
Michael-Alexander Malahias ◽  
Purnachandra Tejaswi ◽  
Dimitrios Chytas ◽  
Vikram Kadu ◽  
Dimitrios Karanikas ◽  
...  
Keyword(s):  
Clinical Outcome ◽  
Systematic Reviews ◽  
Scoping Review ◽  
Hip Resurfacing ◽  
Long Term Results ◽  
Middle Aged ◽  
Cochrane Database ◽  
Pubmed Database ◽  
Meta Analyses ◽  
Age Range

Introduction: Short femoral stems were designed to bridge the gap between conventional straight design stems and hip resurfacing prostheses in total hip arthroplasty (THA). A number of clinical trials have been recently conducted to assess the clinical and safety profile of the cementless, colarless, tapered Metha short hip stem in young or active middle-aged individuals. Methods: A systematic scoping review was conducted according to Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines. 4 reviewers independently conducted the search using the MEDLINE/PubMed database and the Cochrane Database of Systematic Reviews. These databases were queried with the terms “short” AND “hip” AND “stem”. Results: From the initial 773 studies we finally chose 12 studies after applying our inclusion-exclusion criteria. The number of operated hips that were included in these studies was 5048 (mean BMI range: 22.7–35.2, mean age range: 44.4–60.4 years, mean follow-up range: 2–9 years). The mean modified Coleman methodology score was 52.3/100, while it ranged from 31/100 to 63/100. All mean clinical outcome scores that were used in the studies illustrated significant postoperative improvement when compared with the respective initial values. The revision rate of the Metha stem for component-related reasons was 2.5%, while the rate of major complications not requiring revision of the Metha stem was 2.8%. Conclusions: The Metha stem performs well in young or active middle-aged THA patients. Further studies are required for the assessment of the long-term results.

scholarly journals Systematic Reviews and Meta-Analysis in Spine Surgery—How Good Are They in Methodological Quality? A Systematic Review

2020 ◽  
pp. 219256822090681 ◽  
Author(s):  
Muthu Sathish ◽  
Ramakrishnan Eswar
Keyword(s):  
Systematic Review ◽  
Spine Surgery ◽  
Systematic Reviews ◽  
Methodological Quality ◽  
Meta Analysis ◽  
Eligibility Criteria ◽  
Cochrane Database ◽  
Critical Flaw ◽  
Meta Analyses

Study Design: Systematic review. Objectives: To assess the methodological quality of systematic reviews and meta-analyses in spine surgery over the past 2 decades. Materials and Methods: We conducted independent and in duplicate systematic review of the published systematic reviews and meta-analyses between 2000 and 2019 from PubMed Central and Cochrane Database pertaining to spine surgery involving surgical intervention. We searched bibliographies to identify additional relevant studies. Methodological quality was evaluated with AMSTAR score and graded with AMSTAR 2 criteria. Results: A total of 96 reviews met the eligibility criteria, with mean AMSTAR score of 7.51 (SD = 1.98). Based on AMSTAR 2 criteria, 13.5% (n = 13) and 18.7% (n = 18) of the studies had high and moderate level of confidence of results, respectively, without any critical flaws. A total of 29.1% (n = 28) of the studies had at least 1 critical flaw and 38.5% (n = 37) of the studies had more than 1 critical flaw, so that their results have low and critically low confidence, respectively. Failure to analyze the conflict of interest of authors of primary studies included in review and lack of list of excluded studies with justification were the most common critical flaw. Regression analysis demonstrated that studies with funding and studies published in recent years were significantly associated with higher methodological quality. Conclusion: Despite improvement in methodological quality of systematic reviews and meta-analyses in spine surgery in current decade, a substantial proportion continue to show critical flaws. With increasing number of review articles in spine surgery, stringent measures must be taken to adhere to methodological quality by following PRISMA and AMSTAR guidelines to attain higher standards of evidence in published literature.

scholarly journals Nutritional Supplementation during Pulmonary Rehabilitation in COPD: A Systematic Review

2019 ◽  
Author(s):  
Abdulelah M Aldhahir ◽  
Ahmed M Al Rajah ◽  
Yousef S Aldabayan ◽  
Salifu Drammeh ◽  
Vanitha Subbu ◽  
...  
Keyword(s):  
Systematic Reviews ◽  
Pulmonary Rehabilitation ◽  
Nutritional Supplementation ◽  
Systematic Search ◽  
Current Evidence ◽  
Chronic Obstructive ◽  
Obstructive Pulmonary Disease ◽  
Cochrane Database ◽  
Registration Number ◽  
Meta Analyses

ABSTRACTBackgroundUptake of nutritional supplementation during pulmonary rehabilitation (PR) for people with chronic obstructive pulmonary disease (COPD) has been limited by an absence of rigorous evidence-based studies supporting use. Our objective were to report and summarise the current evidence supporting use of nutritional supplementation to improve outcomes during pulmonary rehabilitation in stable COPD patients.MethodsA systematic search was conducted up to May 7th, 2019 (registration number CRD42018089142). The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were used. Six databases were included: Medical Literature Analysis and Retrieval System Online or MEDLARS Online (Medline), Allied and Complementary Medicine Database (AMED), the Cochrane Database of Systematic Reviews, Excerpta Medica dataBASE (Embase), Cumulative Index of Nursing and Allied Health Literature (CINAHL), and Web of Science.ResultsThis systematic search generated 580 initial matches, of which 24 studies (1035 COPD participants) met the pre-specified criteria and were included. Our analysis does not confirm an impact of nutritional supplementation during PR, but studies, supplements and PR programmes were heterogeneous in nature.ConclusionThere is currently insufficient evidence on the effect of nutritional supplementation on improving outcomes during PR in patients with COPD. Therefore, controversy remains and further research is needed.

scholarly journals Neutrophil counts and cancer prognosis: an umbrella review of systematic reviews and meta-analyses of observational studies

2018 ◽  
Author(s):  
Meghan A. Cupp ◽  
Margarita Cariolou ◽  
Ioanna Tzoulaki ◽  
Evangelou Evangelos ◽  
Antonio J. Berlanga-Taylor
Keyword(s):  
Systematic Reviews ◽  
Strong Evidence ◽  
Observational Studies ◽  
Cancer Prognosis ◽  
Small Study ◽  
List Type ◽  
Cancer Outcomes ◽  
Umbrella Review ◽  
Eligibility Criteria ◽  
Meta Analyses

ABSTRACTOBJECTIVETo evaluate the strength and validity of evidence on the association between the neutrophil to lymphocyte ratio (NLR) or tumour-associated neutrophils (TAN) and cancer prognosis.DESIGNUmbrella review of systematic reviews and meta-analyses of observational studies.DATA SOURCESMedline, EMBASE, and Cochrane Database of Systematic Reviews.ELIGIBILITY CRITERIASystematic reviews or meta-analyses of observational studies evaluating the association between NLR or TAN and specific cancer outcomes related to disease progression or survival.DATA SYNTHESISThe available evidence was graded as strong, highly suggestive, suggestive, or weak through the application of pre-set grading criteria. For each included meta-analysis, the grading criteria considered the significance of the random effects estimate, the significance of the largest included study, the number of studies and individuals included, the heterogeneity between included studies, the 95% prediction intervals, presence of small study effects, excess significance and credibility ceilings.RESULTS239 meta-analyses investigating the association between NLR or TAN and cancer outcomes were identified from 57 published studies meeting the eligibility criteria, with 81 meta-analyses from 36 studies meeting the criteria for inclusion. No meta-analyses found a hazard ratio (HR) in the opposite direction of effect (HR<1). When assessed for significance and bias related to heterogeneity and small study effects, only three (4%) associations between NLR and outcomes in gastrointestinal and nasopharyngeal cancers were supported by strong evidence.CONCLUSIONDespite many publications exploring the association between NLR and cancer prognosis, the evidence is limited by significant heterogeneity and small study effects. There is a lack of evidence on the association between TAN and cancer prognosis, with all nine associations identified arising from the same study. Further research is required to provide strong evidence for associations between both TAN and NLR and poor cancer prognosis.REGISTRATIONThis umbrella review is registered on PROSPERO (CRD42017069131)FUNDINGMedical Research CouncilCOPYRIGHTOpen access article under terms of CC BYSHORT TITLENeutrophils and cancer prognosis: an umbrella reviewKEY RESULTWhen assessed for significance and bias related to heterogeneity and small study effects, only three (4%) associations between NLR and overall survival and progression-free survival in gastrointestinal and nasopharyngeal cancers were supported by strong evidence.WHAT THIS PAPER ADDSWHAT IS ALREADY KNOWN ON THE TOPICNeutrophil counts have been linked to the progression of cancer due to their tumourigenic role in the cancer microenvironment.Numerous meta-analyses and individual studies have explored the association between neutrophil counts and cancer outcomes for a variety of cancer sites, leading to a large body of evidence with variable strength and validity.Uncertainty exists around the association between neutrophils and cancer outcomes, depending on the site, outcome and treatments considered.WHAT THIS STUDY ADDSAll meta-analyses included in this review indicated an association between high neutrophil counts and poor cancer prognosis.There is strong evidence supporting the association between the neutrophil to lymphocyte ratio and poor cancer prognosis in some respiratory and gastrointestinal cancers.Further research is required to strengthen the existing body of evidence, particularly for the association between tumour-associated neutrophils and cancer outcomes.

scholarly journals Risk factors for interpersonal violence: an umbrella review of meta-analyses

2018 ◽  
Vol 213 (4) ◽  
pp. 609-614 ◽  
Author(s):  
Seena Fazel ◽  
E. Naomi Smith ◽  
Zheng Chang ◽  
John Richard Geddes
Keyword(s):  
Risk Factors ◽  
Systematic Reviews ◽  
Interpersonal Violence ◽  
Research Evidence ◽  
Small Study ◽  
Modifiable Risk Factors ◽  
Umbrella Review ◽  
Population Effect ◽  
Health And Social Care ◽  
Meta Analyses

BackgroundInterpersonal violence is a leading cause of morbidity and mortality. The strength and population effect of modifiable risk factors for interpersonal violence, and the quality of the research evidence is not known.AimsWe aimed to examine the strength and population effect of modifiable risk factors for interpersonal violence, and the quality and reproducibility of the research evidence.MethodWe conducted an umbrella review of systematic reviews and meta-analyses of risk factors for interpersonal violence. A systematic search was conducted to identify systematic reviews and meta-analyses in general population samples. Effect sizes were extracted, converted into odds ratios and synthesised, and population attributable risk fractions (PAF) were calculated. Quality analyses were performed, including of small study effects, adjustment for confounders and heterogeneity. Secondary analyses for aggression, intimate partner violence and homicide were conducted, and systematic reviews (without meta-analyses) were summarised.ResultsWe identified 22 meta-analyses reporting on risk factors for interpersonal violence. Neuropsychiatric disorders were among the strongest in relative and absolute terms. The neuropsychiatric risk factor that had the largest effect at a population level were substance use disorders, with a PAF of 14.8% (95% CI 9.0–21.6%), and the most important historical factor was witnessing or being a victim of violence in childhood (PAF = 12.2%, 95% CI 6.5–17.4%). There was evidence of small study effects and large heterogeneity.ConclusionsNational strategies for the prevention of interpersonal violence may need to review policies concerning the identification and treatment of modifiable risk factors.Declarations of interestJ.R.G. is an NIHR Senior Investigator. The views expressed within this article are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care.

scholarly journals Review of Systematic Reviews on Acute Procedural Pain in Children in the Hospital Setting

2008 ◽  
Vol 13 (1) ◽  
pp. 51-57 ◽  
Author(s):  
Jennifer Stinson ◽  
Janet Yamada ◽  
Alison Dickson ◽  
Jasmine Lamba ◽  
Bonnie Stevens
Keyword(s):  
Systematic Reviews ◽  
Acute Pain ◽  
Clinical Decision Making ◽  
Hospital Setting ◽  
Clinical Decision ◽  
Future Research ◽  
Hospitalized Children ◽  
Assessment Measure ◽  
Cochrane Database ◽  
Meta Analyses

BACKGROUND: Acute pain is a common experience for hospitalized children. Despite mounting research on treatments for acute procedure-related pain, it remains inadequately treated.OBJECTIVE: To critically appraise all systematic reviews on the effectiveness of acute procedure-related pain management in hospitalized children.METHODS: Published systematic reviews and meta-analyses on pharmacological and nonpharmacological management of acute procedure-related pain in hospitalized children aged one to 18 years were evaluated. Electronic searches were conducted in the Cochrane Database of Systematic Reviews, Medline, EMBASE, the Cumulative Index to Nursing and Allied Health Literature and PsycINFO. Two reviewers independently selected articles for review and assessed their quality using a validated seven-point quality assessment measure. Any disagreements were resolved by a third reviewer.RESULTS: Of 1469 published articles on interventions for acute pain in hospitalized children, eight systematic reviews met the inclusion criteria and were included in the analysis. However, only five of these reviews were of high quality. Critical appraisal of pharmacological pain interventions indicated that amethocaine was superior to EMLA (AstraZeneca Canada Inc) for reducing needle pain. Distraction and hypnosis were nonpharmacological interventions effective for management of acute procedure-related pain in hospitalized children.CONCLUSIONS: There is growing evidence of rigorous evaluations of both pharmacological and nonpharmacological strategies for acute procedure-related pain in children; however, the evidence underlying some commonly used strategies is limited. The present review will enable the creation of a future research plan to facilitate clinical decision making and to develop clinical policy for managing acute procedure-related pain in children.

scholarly journals Increasing number of databases searched in systematic reviews and meta-analyses between 1994 and 2014

2017 ◽  
Vol 104 (4) ◽  
Author(s):  
Michael T. Lam, BMSc(C) ◽  
Mary McDiarmid, MISt
Keyword(s):  
Systematic Reviews ◽  
Median Number ◽  
Health Sciences ◽  
Bibliographic Databases ◽  
Ovid Medline ◽  
Cochrane Database ◽  
The Mean ◽  
Meta Analyses

Objectives: The purpose of this study was to determine whether the number of bibliographic databases used to search the health sciences literature in individual systematic reviews (SRs) and meta-analyses (MAs) changed over a twenty-year period related to the official 1995 launch of the Cochrane Database of Systematic Reviews (CDSR).Methods: Ovid MEDLINE was searched using a modified version of a strategy developed by the Scottish Intercollegiate Guidelines Network to identify SRs and MAs. Records from 3 milestone years were searched: the year immediately preceding (1994) and 1 (2004) and 2 (2014) decades following the CDSR launch. Records were sorted with randomization software. Abstracts or full texts of the records were examined to identify database usage until 100 relevant records were identified from each of the 3 years.Results: The mean and median number of bibliographic databases searched in 1994, 2004, and 2014 were 1.62 and 1, 3.34 and 3, and 3.73 and 4, respectively. Studies that searched only 1 database decreased over the 3 milestone years (60% in 1994, 28% in 2004, and 10% in 2014).Conclusions: The number of bibliographic databases searched in individual SRs and MAs increased from 1994 to 2014.

scholarly journals Impact of blinding on estimated treatment effects in randomised clinical trials: meta-epidemiological study

BMJ ◽  
2020 ◽  
pp. l6802 ◽  
Author(s):  
Helene Moustgaard ◽  
Gemma L Clayton ◽  
Hayley E Jones ◽  
Isabelle Boutron ◽  
Lars Jørgensen ◽  
...  
Keyword(s):  
Epidemiological Study ◽  
Systematic Reviews ◽  
Treatment Effects ◽  
Healthcare Providers ◽  
Eligibility Criteria ◽  
Double Blind ◽  
Cochrane Database ◽  
Patient Reported ◽  
Meta Analyses ◽  
The Impact

Abstract Objectives To study the impact of blinding on estimated treatment effects, and their variation between trials; differentiating between blinding of patients, healthcare providers, and observers; detection bias and performance bias; and types of outcome (the MetaBLIND study). Design Meta-epidemiological study. Data source Cochrane Database of Systematic Reviews (2013-14). Eligibility criteria for selecting studies Meta-analyses with both blinded and non-blinded trials on any topic. Review methods Blinding status was retrieved from trial publications and authors, and results retrieved automatically from the Cochrane Database of Systematic Reviews. Bayesian hierarchical models estimated the average ratio of odds ratios (ROR), and estimated the increases in heterogeneity between trials, for non-blinded trials (or of unclear status) versus blinded trials. Secondary analyses adjusted for adequacy of concealment of allocation, attrition, and trial size, and explored the association between outcome subjectivity (high, moderate, low) and average bias. An ROR lower than 1 indicated exaggerated effect estimates in trials without blinding. Results The study included 142 meta-analyses (1153 trials). The ROR for lack of blinding of patients was 0.91 (95% credible interval 0.61 to 1.34) in 18 meta-analyses with patient reported outcomes, and 0.98 (0.69 to 1.39) in 14 meta-analyses with outcomes reported by blinded observers. The ROR for lack of blinding of healthcare providers was 1.01 (0.84 to 1.19) in 29 meta-analyses with healthcare provider decision outcomes (eg, readmissions), and 0.97 (0.64 to 1.45) in 13 meta-analyses with outcomes reported by blinded patients or observers. The ROR for lack of blinding of observers was 1.01 (0.86 to 1.18) in 46 meta-analyses with subjective observer reported outcomes, with no clear impact of degree of subjectivity. Information was insufficient to determine whether lack of blinding was associated with increased heterogeneity between trials. The ROR for trials not reported as double blind versus those that were double blind was 1.02 (0.90 to 1.13) in 74 meta-analyses. Conclusion No evidence was found for an average difference in estimated treatment effect between trials with and without blinded patients, healthcare providers, or outcome assessors. These results could reflect that blinding is less important than often believed or meta-epidemiological study limitations, such as residual confounding or imprecision. At this stage, replication of this study is suggested and blinding should remain a methodological safeguard in trials.

scholarly journals Evidence-based umbrella review of cognitive effects of prefrontal tDCS

2020 ◽  
Author(s):  
Luis C Farhat ◽  
Andre F Carvalho ◽  
Marco Solmi ◽  
Andre R Brunoni
Keyword(s):  
Systematic Reviews ◽  
Meta Analysis ◽  
Measurement Tool ◽  
Affective Neuroscience ◽  
Umbrella Review ◽  
Quality Of Evidence ◽  
Neuroscience Research ◽  
Cochrane Database ◽  
Meta Analyses

Abstract Transcranial direct current stimulation (tDCS) is a non-invasive brain stimulation technique, which has been increasingly used as an investigational tool in neuroscience. In social and affective neuroscience research, the prefrontal cortex has been primarily targeted, since this brain region is critically involved in complex psychobiological processes subserving both ‘hot’ and ‘cold’ domains. Although several studies have suggested that prefrontal tDCS can enhance neuropsychological outcomes, meta-analyses have reported conflicting results. Therefore, we aimed to assess the available evidence by performing an umbrella review of meta-analyses. We evaluated the effects of prefrontal active vs sham tDCS on different domains of cognition among healthy and neuropsychiatric individuals. A MeaSurement Tool to Assess Systematic Reviews 2 was employed to evaluate the quality of meta-analyses, and the GRADE system was employed to grade the quality of evidence of every comparison from each meta-analysis. PubMed/MEDLINE, PsycINFO and the Cochrane Database of Systematic Reviews were searched, and 11 meta-analyses were included resulting in 55 comparisons. Only 16 comparisons reported significant effects favoring tDCS, but 13 of them had either very low or low quality of evidence. Of the remaining 39 comparisons which reported non-significant effects, 38 had either very low or low quality of evidence. Meta-analyses were rated as having critically low and low quality. Among several reasons to explain these findings, the lack of consensus and reproducibility in tDCS research is discussed.

Sign in / Sign up

Export Citation Format

Share Document