Publication and Related Biases

2013 ◽  
Author(s):  
Michael D. Jennions ◽  
Christopher J. Lortie ◽  
Michael S. Rosenberg ◽  
Hannah R. Rothstein
Keyword(s):  
Model Selection ◽  
Publication Bias ◽  
Effect Size ◽  
Statistical Tests ◽  
Meta Analysis ◽  
Narrow Sense ◽  
Funnel Plots ◽  
Trim And Fill ◽  
Fail Safe ◽  
Nonparametric Correlation

This chapter discusses the increased occurrence of publication bias in the scientific literature. Publication bias is associated with the inaccurate representation of the merit of a hypothesis or idea. A strict definition is that it occurs when the published literature reports results that systematically differ from those of all studies and statistical tests conducted; the result is that false conclusions are drawn. The chapter presents five main approaches used to either detect potential narrow sense publication bias or assess how sensitive the results of a meta-analysis are to the possible exclusion. These include funnel plots, tests for relationships between effect size and sample size using nonparametric correlation or regression, trim and fill method, fail-safe numbers, and model selection.

scholarly journals Foetal origins of depression? A systematic review and meta-analysis of low birth weight and later depression

2012 ◽  
Vol 43 (1) ◽  
pp. 1-12 ◽  
Author(s):  
W. Wojcik ◽  
W. Lee ◽  
I. Colman ◽  
R. Hardy ◽  
M. Hotopf
Keyword(s):  
Psychological Distress ◽  
Birth Weight ◽  
Low Birth Weight ◽  
Publication Bias ◽  
Effect Size ◽  
Meta Analysis ◽  
Severe Illness ◽  
Qualitative Synthesis ◽  
Normal Birth ◽  
Trim And Fill

BackgroundThe foetal origins hypothesis suggests an association between low birth weight and later depression, yet evidence supporting this association has been inconsistent.MethodWe systematically reviewed evidence for an association between low birth weight and adult depression or psychological distress in the general population by meta-analysis. We searched EMBASE, Medline, PsycINFO and ISI Web of Science for studies reporting observational data with low birth weight as the exposure and self- or clinician-rated depression or psychological distress measures as an outcome. Selective studies of exposures such as famine or outcomes such as severe illness only were excluded. Altogether,1454 studies were screened for relevance, 26 were included in the qualitative synthesis, 18 were included in the meta-analysis. A random effects meta-analysis method was used to obtain a pooled estimate of effect size.ResultsThe odds of depression or psychological distress was greater for those of low birth weight (<2500 g) compared to those of normal birth weight (>2500 g) or greater [odds ratio (OR) 1.15, 95% confidence intervals (CI) 1.00–1.32]. However, this association became non-significant after trim-and-fill correction for publication bias (OR 1.08, 95% CI 0.92–1.27). Using meta-regression, no differences in effect size were observed by gender, outcome measure of depression or psychological distress, or whether the effect size was adjusted for possible confounders.ConclusionsWe found evidence to support a weak association between low birth weight and later depression or psychological distress, which may be due to publication bias. It remains possible that the association may vary according to severity of symptoms or other factors.

Evidence-Based Pathology and the Pathologic Evaluation of Thymomas: Transcapsular Invasion Is Not a Significant Prognostic Feature

2008 ◽  
Vol 132 (6) ◽  
pp. 926-930 ◽  
Author(s):  
Ruta Gupta ◽  
Alberto M. Marchevsky ◽  
Robert J. McKenna ◽  
Mark Wick ◽  
Cesar Moran ◽  
...  
Keyword(s):  
Publication Bias ◽  
Statistical Tests ◽  
Case Reports ◽  
English Literature ◽  
Meta Analysis ◽  
Case Series ◽  
Stage I ◽  
Stage Ii ◽  
Evidence Based ◽  
Funnel Plots

Abstract Context.—Evaluation of transcapsular invasion is currently considered very important in the pathologic examination of thymomas. However, recent studies have questioned the prognostic value of stratifying thymoma patients into stage I and II disease. Evidence-based pathology promotes the use of systematic reviews of literature and meta-analysis of data to synthesize the results of multiple publications. Objective.—To analyze the data in the literature regarding the prognostic importance of transcapsular invasion in thymoma stage I and II. Design.—A systematic review of the English literature was carried out for “thymoma,” “stage,” and “prognoses.” Case reports, case series with fewer than 10 cases, and studies with follow-up periods shorter than 5 years were excluded. Twenty-one retrospective publications reporting the experience with 2451 thymomas were selected for review, including 1419 stage I and 1032 stage II patients. Meta-analysis was performed, and possible publication bias was studied with funnel plots of precision and various statistics. Results.—Meta-analysis yielded no significant differences in disease-free or overall survival rates in stage I and II thymoma patients. Funnel plots of precision and statistical tests such as the Egger regression intercept test showed no significant publication bias. Conclusions.—The lack of significant differences in the prognosis of patients with stages I and II thymoma suggests that evaluation of transcapsular invasion is of no clinical value in tumors that lack invasion of neighboring organs or the pleura. The data regarding the prognosis of stage II thymoma patients is somewhat heterogenous, with only some individuals having been treated with radiation therapy, suggesting the need for future randomized controlled trials.

Steep discounting of delayed monetary and food rewards in obesity: a meta-analysis

2016 ◽  
Vol 46 (11) ◽  
pp. 2423-2434 ◽  
Author(s):  
M. Amlung ◽  
T. Petker ◽  
J. Jackson ◽  
I. Balodis ◽  
J. MacKillop
Keyword(s):  
Publication Bias ◽  
Effect Size ◽  
Meta Analysis ◽  
Case Control ◽  
Medium Effect ◽  
Primary Analysis ◽  
Medium Effect Size ◽  
Meta Regression ◽  
Fail Safe ◽  
The Relationship

BackgroundAn increasing number of studies have investigated delay discounting (DD) in relation to obesity, but with mixed findings. This meta-analysis synthesized the literature on the relationship between monetary and food DD and obesity, with three objectives: (1) to characterize the relationship between DD and obesity in both case–control comparisons and continuous designs; (2) to examine potential moderators, including case–control v. continuous design, money v. food rewards, sample sex distribution, and sample age (<18 v. >18 years); and (3) to evaluate publication bias.MethodFrom 134 candidate articles, 39 independent investigations yielded 29 case–control and 30 continuous comparisons (total n = 10 278). Random-effects meta-analysis was conducted using Cohen's d as the effect size. Publication bias was evaluated using fail-safe N, Begg–Mazumdar and Egger tests, meta-regression of publication year and effect size, and imputation of missing studies.ResultsThe primary analysis revealed a medium effect size across studies that was highly statistically significant (d = 0.43, p < 10−14). None of the moderators examined yielded statistically significant differences, although notably larger effect sizes were found for studies with case–control designs, food rewards and child/adolescent samples. Limited evidence of publication bias was present, although the Begg–Mazumdar test and meta-regression suggested a slightly diminishing effect size over time.ConclusionsSteep DD of food and money appears to be a robust feature of obesity that is relatively consistent across the DD assessment methodologies and study designs examined. These findings are discussed in the context of research on DD in drug addiction, the neural bases of DD in obesity, and potential clinical applications.

scholarly journals Effects of instruction based on realistic mathematics education on mathematics achievement: A meta-analysis studyGerçekçi matematik eğitimi destekli öğretimin matematik başarısına etkisi: Bir meta-analiz çalışması

2015 ◽  
Vol 12 (2) ◽  
pp. 187 ◽  
Author(s):  
Abdullah Kaplan ◽  
Murat Duran ◽  
Muhammet Doruk ◽  
Mesut Öztürk
Keyword(s):  
Mathematics Education ◽  
Mathematics Achievement ◽  
Publication Bias ◽  
Effect Size ◽  
Meta Analysis ◽  
Statistical Significance ◽  
Realistic Mathematics Education ◽  
Realistic Mathematics ◽  
Study Process ◽  
Fail Safe

<p>The purpose of this research is to determine the overall effect of instruction based on realistic mathematics education on mathematics achievement. The Meta-Analysis method recently increased in popularity in the field of educational science is used to accomplish this goal. Some criterions have been put in the determination of the study included in the survey and 12 national thesis in accordance with these criterions constituted the sample of this study. Process effectiveness method of meta-analysis is used in the analysis of data and Hedges’s g is used in the calculation of effect size of the study. At the end of the study, realistic mathematics education has an overall, positive and moderate effect on the mathematics achievement (Q = 16.406; p = 0.127). Howewer, the distribution of the effect sizes of the studies included in the survey are found to be homogeneous. At the end of the Z test carried out for the purpose of statistical significance, the value of effect size is observed statistically significant (Z = 7.966; p &lt;0.05). The number of error protection is calculated 91 study to determine the publication bias of the study according to Orwin’s method. According to Orwin’s Fail Safe-N, the research could not have publication bias.</p><p> </p><p><strong>Özet</strong></p><p>Bu araştırmanın amacı gerçekçi matematik eğitiminin matematik başarısı üzerindeki genel etkisini belirlemektir. Bu amacı gerçekleştirmek için eğitim bilimleri alanında yakın zamanda popülerliği artan meta-analiz yöntemi kullanılmıştır. Araştırmaya dâhil edilecek çalışmaların belirlenmesinde bazı kriterler belirlenmiş ve bu kriterlere uygun olan 12 ulusal tez bu araştırmanın örneklemini oluşturmuştur. Verilerin analizinde işlem etkililiği meta analizi yöntemi kullanılmış olup araştırmaların etki büyüklüklerinin hesaplanmasında Hedges’s g kullanılmıştır. Araştırma sonucunda gerçekçi matematik eğitiminin matematik başarısı üzerinde genel olarak, pozitif yönde ve orta düzeyde (Q= 16.406; p= 0.127) etkiye sahip olduğu ve araştırmaya dâhil edilen çalışmaların etki büyüklükleri dağılımının homojen olduğu görülmüştür. İstatistiksel anlamlılık amacıyla yapılan Z testi sonunda etki büyüklüğü değerinin istatistiksel olarak anlamlı olduğu görülmüştür (Z=7.966; p&lt;0.05). Araştırmanın yayın yanlılığını test etmek amacıyla Orwin’in yöntemine göre hata koruma sayısı 91 hesaplanmıştır. Bulunan bu değere göre çalışmanın yayın yanlılığına sahip olmadığı söylenebilir.</p>

scholarly journals The Therapeutic Potential of the Stem Cell Secretome for Spinal Cord Repair: A Systematic Review and Meta-Analysis

2020 ◽  
Author(s):  
Catriona J Cunningham ◽  
◽  
Marc Vives Enrich ◽  
Molly M Pickford ◽  
William MacIntosh-Smith ◽  
...  
Keyword(s):  
Systematic Review ◽  
Spinal Cord ◽  
Spinal Cord Injury ◽  
Stem Cell ◽  
Publication Bias ◽  
Meta Analysis ◽  
Lesion Size ◽  
Cord Injury ◽  
Funnel Plots ◽  
Trim And Fill

There is currently no effective treatment for spinal cord injury leaving around 90% of patients with permanent disabilities. Stem cell therapies are showing promise in preclinical studies of central nervous system injury and there is increasing evidence suggesting the improvements in functional recovery are mediated by paracrine actions. In this systematic review and meta-analysis, we aimed to determine the overall efficacy of stem cell secretome therapies in promoting recovery in preclinical models of spinal cord injury. We searched PubMed and Embase to identify relevant studies. A random effects meta-analysis was conducted using the restricted maximum likelihood estimator. We assessed risk of bias using a modified CAMARADES checklist. Publication bias was then assessed using funnel plots and trim-and-fill analysis. We identified 26 studies that met our inclusion criteria. Overall, stem cell secretome therapies conferred improvement in locomotor score (SMD: 2.30, 95% CI: 1.68-2.91), reduction in lesion size (SMD: 3.27, 95% CI: 2.06-4.48) and increased axonal profiles in the lesion (SMD: 2.36, 95% CI: 1.02-3.71). We found there was significant asymmetry in the funnel plots for all three outcome measures, suggesting publication bias. Trim-and-fill analysis estimated 19 and 3 unpublished studies in the locomotor score and axonal profiles datasets respectively. The median score on the modified CAMARADES checklist was 4 (IQR 4-5). Reporting of power calculations and allocation concealment was absent. The stem cell secretome is showing great potential as a therapy for spinal cord injury. As the vast majority of studies began treatment acutely and favoured reduction in lesion size, we argue neuroprotection is likely the key mechanism of action. Future studies should focus on exploring the contribution of other mechanisms, the mediators involved and effect of treatment at a chronic stage of injury.

scholarly journals The effects of gender on problem solving skills of nursing students in Turkey: a meta-analysis studyTürkiye’deki hemşirelik öğrencilerinin problem çözme düzeylerine cinsiyetin etkisi: Bir meta analiz çalışması

2019 ◽  
Vol 16 (1) ◽  
pp. 134
Author(s):  
Filiz Kantek ◽  
Hande Yeşilbaş
Keyword(s):  
Nursing Students ◽  
Problem Solving ◽  
Publication Bias ◽  
Effect Size ◽  
Nursing Student ◽  
Meta Analysis ◽  
English Summary ◽  
Student Nurse ◽  
Problem Solving Skills ◽  
Fail Safe

The aim of this study is to the effects of gender on problem solving skills of nursing student using meta-analysis method in Turkey. Online databases were scanned to reach studies to be included in the survey. In scanning 'problem solving', 'student nurse', 'Turkey', 'problem solving skill' keywords were used in Turkish and English. Six studies that meet the criteria were included in the study. The CMA statistical program was used in the analysis of the data. Q and I2 values to test heterogeneity and, Orwin's Fail-Safe N and Tau coefficients to publication bias of the included studies were calculated. A random effects model was used to examine the effect size. The six studies' total sample size was 2079 nursing students, 436 male and 1643 female. Heterogeneity and publication bias test showed that studies were heterogeneous and there was not publication bias. The mean effect size on gender problem solving was calculated as 0.079. In conclusion, it was identified that the effect of gender on problem solving level of student nurses was insignificant for women. For future studies, it will be useful to conduct meta-analysis studies on the different factors that affect the problem solving level of student nurses.Extended English summary is in the end of Full Text PDF (TURKISH) file. ÖzetBu çalışmanın amacı, meta analiz yöntemi kullanarak Türkiye’deki hemşirelik öğrencilerinin problem çözme beceri düzeylerine cinsiyetin etkisini belirlemektir. Online very tabanları kullanılarak araştırmaya dahil edilecek çalışmalara ulaşıldı. Taramada ‘problem çözme-problem solving’, ‘öğrenci hemşire- student nurse’’, ‘Türkiye-Turkey’, ‘problem çözme becerisi- problem-solving skill’ anahtar kelimeleri kullanıldı. Dahil edilme kriterlerine uygun olan altı çalışma araştırmaya dahil edildi. CMA istatistik programını kullanarak veri analizi sağlandı. Meta analize dahil edilen çalışmaların heterojenliğini ve yayın yanlılığını test etmek için Q ve I2 değerleri, Orwin’s Fail-Safe N ve Tau katsayısı hesaplandı. Etki büyüklüğünün incelenmesinde rastgele etkiler modeli kullanıldı. Araştırmaya dahil edilen altı çalışmanın toplam örneklem büyüklüğü 2079 hemşirelik öğrencisi olup, 436’sı erkek ve 1643’ü kadındı. Heterojenlik ve yayın yanlılığı testi çalışmaların heterojen olduğunu ve yayın yanlılığının olmadığını gösterdi. Cinsiyetin problem çözme üzerine ortalama etki büyüklüğü 0.079 olarak hesaplandı. Sonuç olarak öğrenci hemşirelerin problem çözme düzeyine cinsiyetin etkisinin kadınlar lehine önemsiz düzeyde olduğu belirlendi. Gelecek çalışmalar için öğrenci hemşirelerin problem çözme düzeyini etkileyen farklı faktörlere ilişkin meta analiz çalışmalarının yapılması yararlı olacaktır.

scholarly journals Assessing and adjusting for publication bias in the relationship between anxiety and the error-related negativity

2018 ◽  
Author(s):  
Blair Saunders ◽  
Michael Inzlicht
Keyword(s):  
Publication Bias ◽  
Effect Size ◽  
Statistical Power ◽  
Meta Analysis ◽  
Event Related Potential ◽  
Medium Effect ◽  
Future Research ◽  
Error Related Negativity ◽  
Trim And Fill ◽  
Neural Reactivity

Many clinical neuroscience investigations have suggested that trait anxiety is associated with increased neural reactivity to mistakes in the form of an event-related potential called the error-related negativity (ERN). Several recent meta-analyses indicated that the anxiety-ERN association was of a small-to-medium effect size, however, these prior investigations did not comprehensively adjust effect sizes for publication bias. Here, in an updated meta-analysis (k=58, N=3819), we found support for an uncorrected effect size of r =-.19, and applied a range of methods to test for and correct publication bias (trim-and-fill, PET, PEESE, Peters’ test, three-parameter selection model). The majority of bias-correction methods suggested that the correlation between anxiety and the ERN is non-zero, but smaller than the uncorrected effect size (average adjusted effect size: r =-.12, range: r =-.05 to -.18). Moderation analyses also revealed more robust effects for clinical anxiety and anxious samples characterised by worry, however, it should be noted that these larger effects were also associated with elevated indicators of publication bias relative to the overall analysis. Mixed anxiety and sub-clinical anxiety were not associated with the amplitude of the ERN. Our results suggest that the anxiety-ERN relationship survives multiple corrections for publication bias, albeit not among all sub-types and populations of anxiety. Nevertheless, only 50% of the studies included in our analysis reported significant results, indicating that future research exploring the anxiety-ERN relationship would benefit from increased statistical power.

Acupuncture combined with swallowing training for poststroke dysphagia: a meta-analysis of randomised controlled trials

2019 ◽  
Vol 37 (2) ◽  
pp. 81-90
Author(s):  
Ling Xin Li ◽  
Kai Deng
Keyword(s):  
Publication Bias ◽  
Meta Analysis ◽  
Analysis Data ◽  
Effective Rate ◽  
China National Knowledge Infrastructure ◽  
Swallowing Function ◽  
Standard Mean Difference ◽  
Trim And Fill ◽  
Randomised Controlled ◽  
Fail Safe

Objective: This meta-analysis aimed to assess the efficacy and safety of the combination of acupuncture with swallowing training for poststroke dysphagia. Method: Nine electronic databases (including PubMed and China National Knowledge Infrastructure) were searched from their inception through June 2016. Seventeen studies were included in this meta-analysis. Data on 1479 eligible patients were extracted, and the relative risk (RR) and standard mean difference (SMD) with 95% CI for the effective rate (ER), swallowing function assessment (SFA), individual activity (IA), eg., modified Barthel Index (MBI) and quality of life (QOL) were evaluated. Results: The pooled ER (RR 1.26, 95% CI 1.19 to 1.34, P<0.001, 14 studies) and SFA (SMD 1.06, 95% CI 0.79 to 1.32, P<0.001, five studies) suggested that combination therapy yielded a significantly higher ER and improved the SFA scores to a greater degree than swallowing training alone in patients with poststroke dysphagia. The pooled QOL score (SMD 1.06; 95% CI −0.04 to 2.17, P=0.06, two studies) did not differ between groups. The MBI data (SMD 1.47, 95% CI 1.07 to 1.87, P<0.001, one study) showed significant improvement in IA. Some evidence of publication bias was observed for the ER, although the trim-and-fill analysis and fail-safe number indicated no influence of publication bias on its pooled effect size. There was no evidence of publication bias of any other outcome measures. Conclusion: This study showed that acupuncture combined with swallowing training may improve the ER, swallowing function and activities of daily life of patients with poststroke dysphagia compared with conventional swallowing training alone.

scholarly journals Therapeutic potential of extracellular vesicles in preclinical stroke models: a systematic review and meta-analysis

2020 ◽  
Vol 4 (1) ◽  
pp. e100047 ◽  
Author(s):  
Josephine M Thomas ◽  
Catriona J Cunningham ◽  
Catherine B Lawrence ◽  
Emmanuel Pinteaux ◽  
Stuart M Allan
Keyword(s):  
Systematic Review ◽  
Publication Bias ◽  
Extracellular Vesicles ◽  
Subgroup Analysis ◽  
Meta Analysis ◽  
Lesion Volume ◽  
Neurological Score ◽  
Stroke Models ◽  
Funnel Plots ◽  
Trim And Fill

ObjectivesCurrently there is a paucity of clinically available regenerative therapies for stroke. Extracellular vesicles (EV) have been investigated for their potential as modulators of regeneration in the poststroke brain. This systematic review and meta-analysis aims to provide a summary of the efficacy of therapeutic EVs in preclinical stroke models, to inform future research in this emerging field.MethodsStudies were identified by a comprehensive literature search of two online sources and subsequent screening. Studies using lesion volume or neurological score as outcome measures were included. Standardised mean difference (SMD) and 95% CIs were calculated using a restricted maximum likelihood random effects model. Publication bias was assessed with Egger’s regression and presented as funnel plots with trim and fill analysis. Subgroup analysis was performed to assess the effects of different study variables. Study quality and risk of bias were assessed using the CAMARADES checklist.ResultsA total of 20 publications were included in the systematic review, of which 19 were assessed in the meta-analysis (43 comparisons). Overall, EV interventions improved lesion volume (SMD: −1.95, 95% CI −2.72 to 1.18) and neurological scores (SMD: −1.26, 95% CI −1.64 to 0.87) compared with control groups. Funnel plots were asymmetrical suggesting publication bias, and trim and fill analysis predicted seven missing studies for lesion volume. Subgroup analysis suggested administration at 0–23 hours after stroke was the most effective timepoint for EV treatment. The median score on the CAMARADES checklist was 7 (IQR: 5–8).ConclusionsEVs may offer a promising new avenue for stroke therapies, as EV-based interventions had positive impacts on lesion volume and neurological score in preclinical stroke models.PROSPERO registration numberCRD42019134925.

scholarly journals Hybrid test for publication bias in meta-analysis

2020 ◽  
Vol 29 (10) ◽  
pp. 2881-2899 ◽  
Author(s):  
Lifeng Lin
Keyword(s):  
Publication Bias ◽  
Real World ◽  
Statistical Tests ◽  
Meta Analysis ◽  
Superior Performance ◽  
Simulation Studies ◽  
Rank Tests ◽  
Hybrid Test ◽  
Trim And Fill ◽  
Meta Analyses

Publication bias frequently appears in meta-analyses when the included studies’ results (e.g., p-values) influence the studies’ publication processes. Some unfavorable studies may be suppressed from publication, so the meta-analytic results may be biased toward an artificially favorable direction. Many statistical tests have been proposed to detect publication bias in recent two decades. However, they often make dramatically different assumptions about the cause of publication bias; therefore, they are usually powerful only in certain cases that support their particular assumptions, while their powers may be fairly low in many other cases. Although several simulation studies have been carried out to compare different tests’ powers under various situations, it is typically infeasible to justify the exact mechanism of publication bias in a real-world meta-analysis and thus select the corresponding optimal publication bias test. We introduce a hybrid test for publication bias by synthesizing various tests and incorporating their benefits, so that it maintains relatively high powers across various mechanisms of publication bias. The superior performance of the proposed hybrid test is illustrated using simulation studies and three real-world meta-analyses with different effect sizes. It is compared with many existing methods, including the commonly used regression and rank tests, and the trim-and-fill method.

Sign in / Sign up

Export Citation Format

Share Document