scholarly journals Addressing publication bias in meta-analysis: Empirical findings from community-augmented meta-analyses of infant language development

2019 ◽  
Author(s):  
Sho Tsuji ◽  
Alejandrina Cristia ◽  
Michael C. Frank ◽  
Christina Bergmann
Keyword(s):  
Publication Bias ◽  
Developmental Psychology ◽  
Meta Analysis ◽  
Grey Literature ◽  
Search Strategies ◽  
Lessons Learned ◽  
Infant Language Development ◽  
Meta Analyses ◽  
Infant Language ◽  
Open Question

Meta-analyses are an indispensable research synthesis tool for characterizing bodies of literature and advancing theories. One important open question concerns the inclusion of unpublished data into meta-analyses. Finding such studies can be effortful, but their exclusion potentially leads to consequential biases like overestimation of a literature’s mean effect. We address two key questions using MetaLab, a collection of community-augmented meta-analyses focused on developmental psychology. First, we assess to what extent these datasets include grey literature, and by what search strategies they are unearthed. An average of 11% of datapoints are from unpublished literature, and that standard search strategies like database searches, complemented with individualized approaches like including authors’ own data, contribute the majority of this literature. Second, we analyze the effect of including versus excluding unpublished literature on estimates of effect size and publication bias, and find this decision does not affect outcomes. We discuss lessons learned and implications.

scholarly journals Addressing Publication Bias in Meta-Analysis

2020 ◽  
Vol 228 (1) ◽  
pp. 50-61 ◽  
Author(s):  
Sho Tsuji ◽  
Alejandrina Cristia ◽  
Michael C. Frank ◽  
Christina Bergmann
Keyword(s):  
Publication Bias ◽  
Effect Size ◽  
Developmental Psychology ◽  
Meta Analysis ◽  
Research Synthesis ◽  
Search Strategies ◽  
Lessons Learned ◽  
Synthesis Tool ◽  
Meta Analyses ◽  
Open Question

Abstract. Meta-analyses are an indispensable research synthesis tool for characterizing bodies of literature and advancing theories. One important open question concerns the inclusion of unpublished data into meta-analyses. Finding such studies can be effortful, but their exclusion potentially leads to consequential biases like overestimation of a literature’s mean effect. We address two questions about unpublished data using MetaLab, a collection of community-augmented meta-analyses focused on developmental psychology. First, we assess to what extent MetaLab datasets include gray literature, and by what search strategies they are unearthed. We find that an average of 11% of datapoints are from unpublished literature; standard search strategies like database searches, complemented with individualized approaches like including authors’ own data, contribute the majority of this literature. Second, we analyze the effect of including versus excluding unpublished literature on estimates of effect size and publication bias, and find this decision does not affect outcomes. We discuss lessons learned and implications.

Is grey literature essential for a better control of publication bias in psychiatry? An example from three meta-analyses of schizophrenia

2005 ◽  
Vol 20 (8) ◽  
pp. 550-553 ◽  
Author(s):  
José Luis R. Martin ◽  
Víctor Pérez ◽  
Montse Sacristán ◽  
Enric Álvarez
Keyword(s):  
Publication Bias ◽  
Systematic Reviews ◽  
Meta Analysis ◽  
Grey Literature ◽  
Risk Of Bias ◽  
Medical Databases ◽  
Study Results ◽  
Heterogeneous Nature ◽  
Meta Analyses ◽  
Literature Material

AbstractSystematic reviews in mental health have become useful tools for health professionals in view of the massive amount and heterogeneous nature of biomedical information available today. In order to determine the risk of bias in the studies evaluated and to avoid bias in generalizing conclusions from the reviews it is therefore important to use a very strict methodology in systematic reviews. One bias which may affect the generalization of results is publication bias, which is determined by the nature and direction of the study results. To control or minimize this type of bias, the authors of systematic reviews undertake comprehensive searches of medical databases and expand on the findings, often undertaking searches of grey literature (material which is not formally published). This paper attempts to show the consequences (and risk) of generalizing the implications of grey literature in the control of publication bias, as was proposed in a recent systematic work. By repeating the analyses for the same outcome from three different systematic reviews that included both published and grey literature our results showed that confusion between grey literature and publication bias may affect the results of a concrete meta-analysis.

scholarly journals Estimating publication bias in meta-analyses of peer-reviewed studies: A meta-meta-analysis across disciplines and journal tiers

2019 ◽  
Author(s):  
Maya B Mathur ◽  
Tyler VanderWeele
Keyword(s):  
Publication Bias ◽  
Developmental Psychology ◽  
Meta Analysis ◽  
Statistical Significance ◽  
Open Data ◽  
Scientific Discipline ◽  
Point Estimate ◽  
Medical Journals ◽  
Lower Tier ◽  
Meta Analyses

Selective publication and reporting in individual papers compromise the scientific record, but are meta-analyses as compromised as their constituent studies? We systematically sampled 63 meta-analyses (each comprising at least 40 studies) in PLOS One, top medical journals, top psychology journals, and Metalab, an online, open-data database of developmental psychology meta-analyses. We empirically estimated publication bias in each. Across all meta-analyses, “statistically significant” results in the expected direction were only 1.17 times more likely to be published than “nonsignificant” results or those in the unexpected direction (95%CI: [0.94, 1.47]), with a confidence interval substantially overlapping the null. Comparable estimates were 0.83 for meta-analyses in PLOS One, 1.02 for top medical journals, 1.54 for top psychology journals, and 4.70 for Metalab. The severity of publication bias did differ across individual meta-analyses; in a small minority (10%; 95% CI: [2%, 21%]), publication bias appeared to favor "significant" results in the expected direction by more than 3-fold. We estimated that for 89% of meta-analyses, the amount of publication bias that would be required to attenuate the point estimate to the null exceeded the amount of publication estimated to be actually present in the vast majority of meta-analyses from the relevant scientific discipline (exceeding the 95th percentile of publication bias). Study-level measures (“statistical significance” with a point estimate in the expected direction and point estimate size) did not indicate more publication bias in higher-tier versus lower-tier journals, nor in the earliest studies published on a topic versus later studies. Overall, the mere act of performing a meta-analysis with a large number of studies (at least 40) and that includes non-headline results may largely mitigate publication bias in meta-analyses, suggesting optimism about the validity of meta-analytic results.

Identification of and Correction for Publication Bias: Comment

2019 ◽  
Author(s):  
Amanda Kvarven ◽  
Eirik Strømland ◽  
Magnus Johannesson
Keyword(s):  
Publication Bias ◽  
False Positive ◽  
Large Scale ◽  
Meta Analysis ◽  
False Positive Rate ◽  
Effect Sizes ◽  
Replication Studies ◽  
Moderate Reduction ◽  
Positive Rate ◽  
Meta Analyses

Andrews & Kasy (2019) propose an approach for adjusting effect sizes in meta-analysis for publication bias. We use the Andrews-Kasy estimator to adjust the result of 15 meta-analyses and compare the adjusted results to 15 large-scale multiple labs replication studies estimating the same effects. The pre-registered replications provide precisely estimated effect sizes, which do not suffer from publication bias. The Andrews-Kasy approach leads to a moderate reduction of the inflated effect sizes in the meta-analyses. However, the approach still overestimates effect sizes by a factor of about two or more and has an estimated false positive rate of between 57% and 100%.

scholarly journals Comparative efficacy and safety of interventions for treating head lice: a protocol for systematic review and network meta-analysis

2021 ◽  
Vol 5 (1) ◽  
pp. e001129
Author(s):  
Bill Stevenson ◽  
Wubshet Tesfaye ◽  
Julia Christenson ◽  
Cynthia Mathew ◽  
Solomon Abrha ◽  
...  
Keyword(s):  
Systematic Review ◽  
Randomised Controlled Trials ◽  
Meta Analysis ◽  
Grey Literature ◽  
Risk Of Bias ◽  
Controlled Trials ◽  
Head Lice ◽  
Efficacy And Safety ◽  
Randomised Controlled ◽  
Meta Analyses

BackgroundHead lice infestation is a major public health problem around the globe. Its treatment is challenging due to product failures resulting from rapidly emerging resistance to existing treatments, incorrect treatment applications and misdiagnosis. Various head lice treatments with different mechanism of action have been developed and explored over the years, with limited report on systematic assessments of their efficacy and safety. This work aims to present a robust evidence summarising the interventions used in head lice.MethodThis is a systematic review and network meta-analysis which will be reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-analyses statement for network meta-analyses. Selected databases, including PubMed, Embase, MEDLINE, Web of Science, CINAHL and Cochrane Central Register of Controlled Trials will be systematically searched for randomised controlled trials exploring head lice treatments. Searches will be limited to trials published in English from database inception till 2021. Grey literature will be identified through Open Grey, AHRQ, Grey Literature Report, Grey Matters, ClinicalTrials.gov, WHO International Clinical Trials Registry and International Standard Randomised Controlled Trials Number registry. Additional studies will be sought from reference lists of included studies. Study screening, selection, data extraction and assessment of methodological quality will be undertaken by two independent reviewers, with disagreements resolved via a third reviewer. The primary outcome measure is the relative risk of cure at 7 and 14 days postinitial treatment. Secondary outcome measures may include adverse drug events, ovicidal activity, treatment compliance and acceptability, and reinfestation. Information from direct and indirect evidence will be used to generate the effect sizes (relative risk) to compare the efficacy and safety of individual head lice treatments against a common comparator (placebo and/or permethrin). Risk of bias assessment will be undertaken by two independent reviewers using the Cochrane Risk of Bias tool and the certainty of evidence assessed using the Grading of Recommendations, Assessment, Development and Evaluations guideline for network meta-analysis. All quantitative analyses will be conducted using STATA V.16.DiscussionThe evidence generated from this systematic review and meta-analysis is intended for use in evidence-driven treatment of head lice infestations and will be instrumental in informing health professionals, public health practitioners and policy-makers.PROSPERO registration numberCRD42017073375.

scholarly journals Leishmanicidal effects of resveratrol and its derivatives: a systematic review and meta-Analysis

2020 ◽  
Author(s):  
Nasrin Amiri Dashatan ◽  
Marzieh Ashrafmansouri ◽  
Mehdi Koushki ◽  
Nayebali Ahmadi
Keyword(s):  
Systematic Review ◽  
Publication Bias ◽  
Data Extraction ◽  
Meta Analysis ◽  
Sensitivity Analyses ◽  
Protective Effects ◽  
Random Effects Models ◽  
Important Health ◽  
Mean Differences ◽  
Meta Analyses

Abstract Background Leishmaniasis is one of the most important health problems worldwide. The evidence has suggested that resveratrol and its derivatives have anti-leishmanial effects; however, the results are inconsistent and inconclusive. The aim of this study was to assess the effect of resveratrol and its derivatives on the Leishmania viability through a systematic review and meta-analysis of available relevant studies. Methods The electronic databases PubMed, ScienceDirect, Embase, Web of Science and Scopus were queried between October 2000 and April 2020 using a comprehensive search strategy. The eligible articles selected and data extraction conducted by two reviewers. Mean differences of IC50 (concentration leading to reduction of 50% of Leishmania) for each outcome was calculated using random-effects models. Sensitivity analyses and prespecified subgroup were conducted to evaluate potential heterogeneity and the stability of the pooled results. Publication bias was evaluated using the Egger’s and Begg’s tests. We also followed the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines for this review. Results Ten studies were included in the meta-analysis. We observed that RSV and its derivatives had significant reducing effects on Leishmania viability in promastigote [24.02 µg/ml; (95% CI 17.1, 30.8); P < 0.05; I2 = 99.8%; P heterogeneity = 0.00] and amastigote [18.3 µg/ml; (95% CI 13.5, 23.2); P < 0.05; I2 = 99.6%; P heterogeneity = 0.00] stages of Leishmania. A significant publication bias was observed in the meta-analysis. Sensitivity analyses showed a similar effect size while reducing the heterogeneity. Subgroup analysis indicated that the pooled effects of leishmanicidal of resveratrol and its derivatives were affected by type of stilbenes and Leishmania species. Conclusions Our findings clearly suggest that the strategies for the treatment of leishmaniasis should be focused on natural products such as RSV and its derivatives. Further study is needed to identify the mechanisms mediating this protective effects of RSV and its derivatives in leishmaniasis.

scholarly journals Comparing Bayesian Posterior Passing with Meta-analysis

2020 ◽  
Author(s):  
Joshua Pritsker
Keyword(s):  
Publication Bias ◽  
Fixed Effects ◽  
Meta Analysis ◽  
Future Research ◽  
Meta Analyses

Brand, von der Post, Ounsley, and Morgan (2019) introduced Bayesian posterior passing as an alternative to traditional meta-analyses. In this commentary I relate their procedure to traditional meta-analysis, showing that posterior passing is equivalent to fixed effects meta-analysis. To overcome the limitations of simple posterior passing, I introduce improved posterior passing methods to account for heterogeneity and publication bias. Additionally, practical limitations of posterior passing and the role that it can play in future research are discussed.

scholarly journals Global Burden of Anxiety and Depression among Cystic Fibrosis Patient: Systematic Review and Meta-Analysis

2021 ◽  
Vol 2021 ◽  
pp. 1-9
Author(s):  
Mistire Teshome Guta ◽  
Tiwabwork Tekalign ◽  
Nefsu Awoke ◽  
Robera Olana Fite ◽  
Getahun Dendir ◽  
...  
Keyword(s):  
Cystic Fibrosis ◽  
North America ◽  
Publication Bias ◽  
Meta Analysis ◽  
Systemic Review ◽  
Anxiety And Depression ◽  
Close Monitoring ◽  
The World ◽  
Trim And Fill ◽  
Meta Analyses

Aims. This systemic review and meta-analysis were aimed at determining the level of anxiety and depression among cystic fibrosis patients in the world. Methods. We conducted a systematic search of published studies from PubMed, EMBASE, MEDLINE, Cochrane, Scopus, Web of Science, CINAHL, and manually on Google Scholar. This meta-analysis follows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. The quality of studies was assessed by the modified Newcastle-Ottawa Scale (NOS). Meta-analysis was carried out using a random-effects method using the STATA™ Version 14 software. Trim and fill analysis was done to correct the presence of significant publication bias. Result. From 419,820 obtained studies, 26 studies from 2 different parts of the world including 9766. The overall global pooled prevalence of anxiety and depression after correction for publication bias by trim and fill analysis was found to be 24.91(95% CI: 20.8-28.9) for anxiety. The subgroup analyses revealed with the lowest prevalence, 23.59%, (95% CI: 8.08, 39.09)) in North America and the highest, 26.77%, (95% CI: 22.5, 31.04) seen in Europe for anxiety and with the highest prevalence, 18.67%, (95% CI: 9.82, 27.5) in North America and the lowest, 13.27%, (95% CI: -10.05, 16.5) seen in Europe for depression. Conclusion. The global prevalence of anxiety and depression among cystic fibrosis patients is common. Therefore, close monitoring of the patient, regularly screening for anxiety and depression, and appropriate prevention techniques is recommended.

scholarly journals Systematically Searching Empirical Literature in the Social Sciences: Results from Two Meta-Analyses Within the Domain of Education

2019 ◽  
Vol 15 (4) ◽  
pp. 245-265 ◽  
Author(s):  
David I. Pickup ◽  
Robert M. Bernard ◽  
Eugene Borokhovski ◽  
Anne C. Wade ◽  
Rana M. Tamim
Keyword(s):  
Information Retrieval ◽  
Meta Analysis ◽  
Grey Literature ◽  
Full Range ◽  
Empirical Literature ◽  
Bibliographic Databases ◽  
Campbell Collaboration ◽  
Manual Search ◽  
Review Team ◽  
Meta Analyses

Introduction. This paper provides an overview of the information retrieval strategy employed for two meta-analyses, conducted by a systematic review team at Concordia University (Montreal, QC, Canada). Both papers draw on standards first articulated by H.M. Cooper and further developed by the Campbell Collaboration, which promote a comprehensive approach to systematically searching an extensive array of resources (bibliographic databases, print resources, citation indices, etc.) in order to locate both published and unpublished research. The goal is to verify if searching comprehensively through multiple resources retrieves studies that are unique, and hence, improve the overall representativeness of a diverse body of literature. We also analyze the sensitivity and specificity of the results by data source. Methods. In order to determine the source sensitivity, we consider percentage of results from each source retrieved for full-text review. In order to determine the source specificity, we derive a percentage from the total number of studies included in the final meta-analysis compared against the overall number of initial results found. Results. Results demonstrate the need to search beyond the subject-specific databases of a particular discipline as unique results can be found in many places. Databases for related disciplines provided 129 unique includes to each meta-analysis, and multidisciplinary databases provided 44 and 99 unique includes for the two meta-analyses in question respectively. Manual search techniques were much more sensitive and specific than electronic searches of databases and yield a higher percentage of final includes. Discussion. The results demonstrate the utility of a comprehensive information retrieval methodology like that proposed by the Campbell Collaboration, which goes beyond the main subject databases to locate the full range of information sources, including grey literature.

Effects of the endocrine disruptor vinclozolin in male reproduction: a systematic review and meta-analysis†

2021 ◽  
Author(s):  
Mariana Feijó ◽  
Roberta V L Martins ◽  
Sílvia Socorro ◽  
Luísa Pereira ◽  
Sara Correia
Keyword(s):  
Systematic Review ◽  
Publication Bias ◽  
Meta Analysis ◽  
Endocrine Disrupting Chemicals ◽  
Reproductive Organs ◽  
Male Reproduction ◽  
Mean Differences ◽  
Trim And Fill ◽  
Meta Analyses ◽  
Q Statistic

Abstract Endocrine-disrupting chemicals have become an issue of scientific and public discussion. Vinclozolin (VNZ) is a fungicide that competitively antagonizes the binding of natural androgens to their receptor, disturbing the function of tissues that are sensitive to these hormones, as is the case of the male reproductive organs. A systematic review with meta-analyses of rodent studies was conducted to answer the following question: Does exposure to VNZ affect sperm parameters and testicular/epididymal weight? The methodology was prespecified according to the Cochrane Handbook for Systematic Reviews and PRISMA recommendations. Sixteen articles met the inclusion criteria, comprising a total of 1189 animals. The risk of publication bias was assessed using the Trim and Fill adjustment, funnel plot, and Egger regression test. Heterogeneity and inconsistency across the findings were tested using the Q-statistic and I2 of Higgins, respectively. Sensitivity was also analyzed. Statistical analysis was performed on Comprehensive Meta-Analysis software (Version 2.0), using random models and weighted mean differences along with a 95% confidence interval. Sperm motility, counts, daily sperm production (evidence of publication bias), and epididymis weight were decreased in VNZ-treated animals. Exposure length and dose, as well as the time point of exposure, influenced the obtained results. Despite the moderate/high heterogeneity observed, the sensitivity analysis overall demonstrated the robustness of the findings. The quality scores of the included studies were superior to 4 in a total of 9, then classified as good. The obtained data corroborate the capability of VNZ exposure to disrupt spermatogenic output and compromise male fertility.

Sign in / Sign up

Export Citation Format

Share Document