scholarly journals Correcting for bias in psychology: A comparison of meta-analytic methods

2017 ◽  
Author(s):  
Evan C Carter ◽  
Felix D. Schönbrodt ◽  
Will M Gervais ◽  
Joseph Hilgard
Keyword(s):  
Publication Bias ◽  
Large Scale ◽  
Meta Analysis ◽  
Sensitivity Analyses ◽  
Simulation Studies ◽  
Research Practices ◽  
Simulation Code ◽  
Questionable Research Practices ◽  
Analytic Methods ◽  
Primary Literature

Publication bias and questionable research practices in primary research can lead to badly overestimated effects in meta-analysis. Methodologists have proposed a variety of statistical approaches to correct for such overestimation. However, much of this work has not been tailored specifically to psychology, so it is not clear which methods work best for data typically seen in our field. Here, we present a comprehensive simulation study to examine how some of the most promising meta-analytic methods perform on data that might realistically be produced by research in psychology. We created such scenarios by simulating several levels of questionable research practices, publication bias, heterogeneity, and using study sample sizes empirically derived from the literature. Our results clearly indicated that no single meta-analytic method consistently outperformed all others. Therefore, we recommend that meta-analysts in psychology focus on sensitivity analyses—that is, report on a variety of methods, consider the conditions under which these methods fail (as indicated by simulation studies such as ours), and then report how conclusions might change based on which conditions are most plausible. Moreover, given the dependence of meta-analytic methods on untestable assumptions, we strongly recommend that researchers in psychology continue their efforts on improving the primary literature and conducting large-scale, pre-registered replications. We provide detailed results and simulation code at https://osf.io/rf3ys and interactive figures at http://www.shinyapps.org/apps/metaExplorer/.

scholarly journals Correcting for Bias in Psychology: A Comparison of Meta-Analytic Methods

2019 ◽  
Vol 2 (2) ◽  
pp. 115-144 ◽  
Author(s):  
Evan C. Carter ◽  
Felix D. Schönbrodt ◽  
Will M. Gervais ◽  
Joseph Hilgard
Keyword(s):  
Publication Bias ◽  
Large Scale ◽  
Meta Analysis ◽  
Sensitivity Analyses ◽  
Simulation Studies ◽  
Research Practices ◽  
Simulation Code ◽  
Questionable Research Practices ◽  
Analytic Methods ◽  
Primary Literature

Publication bias and questionable research practices in primary research can lead to badly overestimated effects in meta-analysis. Methodologists have proposed a variety of statistical approaches to correct for such overestimation. However, it is not clear which methods work best for data typically seen in psychology. Here, we present a comprehensive simulation study in which we examined how some of the most promising meta-analytic methods perform on data that might realistically be produced by research in psychology. We simulated several levels of questionable research practices, publication bias, and heterogeneity, and used study sample sizes empirically derived from the literature. Our results clearly indicated that no single meta-analytic method consistently outperformed all the others. Therefore, we recommend that meta-analysts in psychology focus on sensitivity analyses—that is, report on a variety of methods, consider the conditions under which these methods fail (as indicated by simulation studies such as ours), and then report how conclusions might change depending on which conditions are most plausible. Moreover, given the dependence of meta-analytic methods on untestable assumptions, we strongly recommend that researchers in psychology continue their efforts to improve the primary literature and conduct large-scale, preregistered replications. We provide detailed results and simulation code at https://osf.io/rf3ys and interactive figures at http://www.shinyapps.org/apps/metaExplorer/ .

scholarly journals Meta-Analysis on the Association of Neuropeptide Y rs16139 Variant With the Risk of Alcoholism

2021 ◽  
Vol 12 ◽  
Author(s):  
Biqing Chen ◽  
Manish Yadav ◽  
Madhubala Mulkalwar ◽  
Lakkakula Saikrishna ◽  
Henu Verma ◽  
...  
Keyword(s):  
Neuropeptide Y ◽  
Publication Bias ◽  
Large Scale ◽  
Meta Analysis ◽  
Conclusive Evidence ◽  
Sensitivity Analyses ◽  
Significant Heterogeneity ◽  
Web Tool ◽  
Alcoholism Risk ◽  
The Relationship

Introduction: The neuropeptide-Y (NPY) is involved in the development of alcoholism through NPY receptors. A T>C mutation causes substitution of leucine to proline at codon 7 (L7P; rs16139) in the signal peptide of neuropeptide Y is known to cause a 42% increase in plasma NPY levels. Studies that analyzed the association between NPY rs16139 and alcoholism risk did not demonstrate conclusive evidence for this relationship. The present study aims to evaluate the association between NPY gene rs16139 variant and alcohol dependence.Method: An electronic search of databases including PubMed and Google Scholar was performed to retrieve studies investigating the association between NPY rs16139 and alcoholism. The pooled odds ratio (OR) with 95% confidence interval (CI) was calculated in allelic and dominant genetic models. Sensitivity analyses and publication bias were assessed in our meta-analysis. The meta-analysis was conducted using the MetaGenyo web tool.Result: Significant heterogeneity was observed across studies (p < 0.001). Our results have shown that there is no significant association between NPY rs16139 variant and the risk of alcoholism in allelic (OR = 0.98, 95% CI 0.70–1.38, p = 0.921) and dominant models (OR = 0.98, 95% CI 0.69–1.40, p = 0.919). Begg's funnel plot and Egger's test have not shown publication bias (p = 0.332).Conclusion: To the best of our knowledge, this is the first meta-analysis that evaluates the relationship between the NPY rs16139 polymorphism and the risk of alcoholism. Our large-scale meta-analysis suggests that NPY rs16139 polymorphism is not associated with alcoholism. However, further studies are needed to increase our understanding of the relationship between NPY variants in alcoholism.

Identification of and Correction for Publication Bias: Comment

2019 ◽  
Author(s):  
Amanda Kvarven ◽  
Eirik Strømland ◽  
Magnus Johannesson
Keyword(s):  
Publication Bias ◽  
False Positive ◽  
Large Scale ◽  
Meta Analysis ◽  
False Positive Rate ◽  
Effect Sizes ◽  
Replication Studies ◽  
Moderate Reduction ◽  
Positive Rate ◽  
Meta Analyses

Andrews & Kasy (2019) propose an approach for adjusting effect sizes in meta-analysis for publication bias. We use the Andrews-Kasy estimator to adjust the result of 15 meta-analyses and compare the adjusted results to 15 large-scale multiple labs replication studies estimating the same effects. The pre-registered replications provide precisely estimated effect sizes, which do not suffer from publication bias. The Andrews-Kasy approach leads to a moderate reduction of the inflated effect sizes in the meta-analyses. However, the approach still overestimates effect sizes by a factor of about two or more and has an estimated false positive rate of between 57% and 100%.

scholarly journals Risk of dementia in gout and hyperuricaemia: a meta-analysis of cohort studies

BMJ Open ◽  
2021 ◽  
Vol 11 (6) ◽  
pp. e041680
Author(s):  
Shu-Yue Pan ◽  
Rui-Juan Cheng ◽  
Zi-Jing Xia ◽  
Qiu-Ping Zhang ◽  
Yi Liu
Keyword(s):  
Cohort Studies ◽  
Quality Assessment ◽  
Publication Bias ◽  
Data Extraction ◽  
Meta Analysis ◽  
Sensitivity Analyses ◽  
Cochrane Library ◽  
Crystal Formation ◽  
Medical Subject Headings ◽  
Eligibility Criteria

ObjectivesGout, characterised by hyperuricaemia with monosodium urate crystal formation and inflammation, is the most common inflammatory arthritis in adults. Recent studies have found that elevated uric acid levels are related to the occurrence of dementia. We conducted a study to investigate the association between dementia and gout or hyperuricaemia.DesignSystematic review and meta-analysis of cohort studies.Data sourcesStudies were screened from inception to 28 June 2019 by searching Medline, Embase and the Cochrane Library databases.Eligibility criteriaCohort studies comparing the risk of dementia in patients with gout and hyperuricaemia versus non-gout and non-hyperuricaemia controls were enrolled.Data extraction and analysisTwo reviewers separately selected studies and extracted data using the Medical Subject Headings without restriction on languages or countries. The adjusted HRs were pooled using the DerSimonian and Laird random effects model. Sensitivity analyses were conducted to evaluate the stability of the results. Publication bias was evaluated using Egger’s and Begg’s tests. Quality assessment was performed according to the Newcastle-Ottawa Scale.ResultsFour cohort studies that met the inclusion criteria were included in our meta-analysis. We found that gout and hyperuricaemia did not increase the risk of dementia, with a pooled HR of 0.94 (95% CI 0.69 to 1.28), but might decrease the risk of Alzheimer’s disease (AD), with a pooled HR of 0.78 (95% CI 0.64 to 0.95). There was little evidence of publication bias. Quality assessment of the included studies was high (range: 6–8 points).ConclusionsOur study shows that gout and hyperuricaemia do not increase the risk of dementia. However, gout and hyperuricaemia might have a protective effect against AD. Due to the limited number of research articles, more investigations are needed to demonstrate the potential relationship between dementia and gout or hyperuricaemia.

Use of Statins and Breast Cancer: A Meta-Analysis of Seven Randomized Clinical Trials and Nine Observational Studies

2005 ◽  
Vol 23 (34) ◽  
pp. 8606-8612 ◽  
Author(s):  
Stefanos Bonovas ◽  
Kalitsa Filioussi ◽  
Nikolaos Tsavaris ◽  
Nikolaos M. Sitaras
Keyword(s):  
Breast Cancer ◽  
Breast Cancer Risk ◽  
Cancer Risk ◽  
Publication Bias ◽  
Observational Studies ◽  
Fixed Effects ◽  
Randomized Clinical Trials ◽  
Meta Analysis ◽  
Sensitivity Analyses ◽  
Fixed Effects Model

Purpose A growing body of evidence suggests that statins may have chemopreventive potential against breast cancer. Laboratory studies demonstrate that statins induce apoptosis and reduce cell invasiveness in various cell lines, including breast carcinoma cells. However, the clinical relevance of these data remains unclear. The nonconclusive nature of the epidemiologic data prompted us to conduct a detailed meta-analysis of the studies published on the subject in peer-reviewed literature. Patients and Methods A comprehensive search for articles published up until 2005 was performed; reviews of each study were conducted; and data were abstracted. Before meta-analysis, the studies were evaluated for publication bias and heterogeneity. Pooled relative risk (RR) estimates and 95% CIs were calculated using the random and the fixed-effects models. Subgroup and sensitivity analyses were also performed. Results Seven large randomized trials and nine observational studies (five case-control and four cohort studies) contributed to the analysis. We found no evidence of publication bias or heterogeneity among the studies. Statin use did not significantly affect breast cancer risk (fixed effects model: RR = 1.03; 95% CI, 0.93 to 1.14; random effects model: RR = 1.02; 95% CI, 0.89 to 1.18). When the analyses were stratified into subgroups, there was no evidence that study design substantially influenced the estimate of effects. Furthermore, the sensitivity analysis confirmed the stability of our results. Conclusion Our meta-analysis findings do not support a protective effect of statins against breast cancer. However, this conclusion is limited by the relatively short follow-up times of the studies analyzed. Further studies are required to investigate the potential decrease in breast cancer risk among long-term statin users.

scholarly journals Leishmanicidal effects of resveratrol and its derivatives: a systematic review and meta-Analysis

2020 ◽  
Author(s):  
Nasrin Amiri Dashatan ◽  
Marzieh Ashrafmansouri ◽  
Mehdi Koushki ◽  
Nayebali Ahmadi
Keyword(s):  
Systematic Review ◽  
Publication Bias ◽  
Data Extraction ◽  
Meta Analysis ◽  
Sensitivity Analyses ◽  
Protective Effects ◽  
Random Effects Models ◽  
Important Health ◽  
Mean Differences ◽  
Meta Analyses

Abstract Background Leishmaniasis is one of the most important health problems worldwide. The evidence has suggested that resveratrol and its derivatives have anti-leishmanial effects; however, the results are inconsistent and inconclusive. The aim of this study was to assess the effect of resveratrol and its derivatives on the Leishmania viability through a systematic review and meta-analysis of available relevant studies. Methods The electronic databases PubMed, ScienceDirect, Embase, Web of Science and Scopus were queried between October 2000 and April 2020 using a comprehensive search strategy. The eligible articles selected and data extraction conducted by two reviewers. Mean differences of IC50 (concentration leading to reduction of 50% of Leishmania) for each outcome was calculated using random-effects models. Sensitivity analyses and prespecified subgroup were conducted to evaluate potential heterogeneity and the stability of the pooled results. Publication bias was evaluated using the Egger’s and Begg’s tests. We also followed the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines for this review. Results Ten studies were included in the meta-analysis. We observed that RSV and its derivatives had significant reducing effects on Leishmania viability in promastigote [24.02 µg/ml; (95% CI 17.1, 30.8); P < 0.05; I2 = 99.8%; P heterogeneity = 0.00] and amastigote [18.3 µg/ml; (95% CI 13.5, 23.2); P < 0.05; I2 = 99.6%; P heterogeneity = 0.00] stages of Leishmania. A significant publication bias was observed in the meta-analysis. Sensitivity analyses showed a similar effect size while reducing the heterogeneity. Subgroup analysis indicated that the pooled effects of leishmanicidal of resveratrol and its derivatives were affected by type of stilbenes and Leishmania species. Conclusions Our findings clearly suggest that the strategies for the treatment of leishmaniasis should be focused on natural products such as RSV and its derivatives. Further study is needed to identify the mechanisms mediating this protective effects of RSV and its derivatives in leishmaniasis.

scholarly journals Excess success in “Don’t count calorie labeling out: Calorie counts on the left side of menu items lead to lower calorie food choices”

2019 ◽  
Author(s):  
Gregory Francis ◽  
Evelina Thunell
Keyword(s):  
Success Rate ◽  
Publication Bias ◽  
Food Choices ◽  
Effect Sizes ◽  
Sample Sizes ◽  
Research Practices ◽  
Policy Makers ◽  
Questionable Research Practices ◽  
Calorie Labeling

Based on findings from six experiments, Dallas, Liu &amp; Ubel (2019) concluded that placing calorie labels to the left of menu items influences consumers to choose lower calorie food options. Contrary to previously reported findings, they suggested that calorie labels do influence food choices, but only when placed to the left because they are in this case read first. If true, these findings have important implications for the design of menus and may help address the obesity pandemic. However, an analysis of the reported results indicates that they seem too good to be true. We show that if the effect sizes in Dallas et al. (2019) are representative of the populations, a replication of the six studies (with the same sample sizes) has a probability of only 0.014 of producing uniformly significant outcomes. Such a low success rate suggests that the original findings might be the result of questionable research practices or publication bias. We therefore caution readers and policy makers to be skeptical about the results and conclusions reported by Dallas et al. (2019).

scholarly journals Using a Two-Sample Mendelian Randomization Method in Assessing the Causal Relationships Between Human Blood Metabolites and Heart Failure

2021 ◽  
Vol 8 ◽  
Author(s):  
Zixian Wang ◽  
Shiyu Chen ◽  
Qian Zhu ◽  
Yonglin Wu ◽  
Guifeng Xu ◽  
...  
Keyword(s):  
Heart Failure ◽  
Human Blood ◽  
Large Scale ◽  
Genome Wide Association Study ◽  
Mendelian Randomization ◽  
Causal Effect ◽  
Meta Analysis ◽  
Sensitivity Analyses ◽  
Blood Metabolites ◽  
Causal Relationships

Background: Heart failure (HF) is the main cause of morbidity and mortality worldwide, and metabolic dysfunction is an important factor related to HF pathogenesis and development. However, the causal effect of blood metabolites on HF remains unclear.Objectives: Our chief aim is to investigate the causal relationships between human blood metabolites and HF risk.Methods: We used an unbiased two-sample Mendelian randomization (MR) approach to assess the causal relationships between 486 human blood metabolites and HF risk. Exposure information was obtained from Sample 1, which is the largest metabolome-based genome-wide association study (mGWAS) data containing 7,824 Europeans. Outcome information was obtained from Sample 2, which is based on the results of a large-scale GWAS meta-analysis of HF and contains 47,309 cases and 930,014 controls of Europeans. The inverse variance weighted (IVW) model was used as the primary two-sample MR analysis method and followed the sensitivity analyses, including heterogeneity test, horizontal pleiotropy test, and leave-one-out analysis.Results: We observed that 11 known metabolites were potentially related to the risk of HF after using the IVW method (P &lt; 0.05). After adding another four MR models and performing sensitivity analyses, we found a 1-SD increase in the xenobiotics 4-vinylphenol sulfate was associated with ~22% higher risk of HF (OR [95%CI], 1.22 [1.07–1.38]).Conclusions: We revealed that the 4-vinylphenol sulfate may nominally increase the risk of HF by 22% after using a two-sample MR approach. Our findings may provide novel insights into the pathogenesis underlying HF and novel strategies for HF prevention.

Surgery in stage IV melanoma patients: Results from a single institution.

2012 ◽  
Vol 30 (15_suppl) ◽  
pp. e19035-e19035
Author(s):  
Elisabetta Pennacchioli ◽  
Sara Gandini ◽  
Francesco Verrecchia ◽  
Giulio Tosti ◽  
Federica Baldini ◽  
...  
Keyword(s):  
Systematic Review ◽  
Publication Bias ◽  
Meta Analysis ◽  
Stage Iv ◽  
Sensitivity Analyses ◽  
Summary Estimate ◽  
Visceral Metastases ◽  
Melanoma Patients ◽  
Improvement In Survival

e19035 Background: There is no consensus regarding the appropriate management of melanoma patients at stage IV and resectable metastases. Methods: We conducted a systematic review and meta-analysis to examine the effect of surgery on overall survival (OS) in patients with resectable metastases. The systematic review was performed by searching MEDLINE, EMBASE, Cochrane and ISI Web of Science. The meta-analysis was performed using time-to-event data from which hazard ratios (HRs) and 95% confidence intervals (CIs) of OS were estimated and summary estimates were obtain through random effects model. Heterogeneity and publication bias were investigated by sensitivity analyses and funnel plot regression Results: Thirty-three retrospective studies evaluating survival from resected and unresected metastases were found from 1978 until 2010. In resected patients median follow-up is 1.5 years and 2-year OS is 27% whereas in unresected the median follow-up is 6 months and 2-year OS is 12%. Meta-analysis was carried out on the seven studies reporting information on visceral metastases and including 1,165 patients. We included also 93 patients at stage IV treated at the European Institute of Oncology, Milan, Italy. Surgery has shown an improvement in survival of patients managed with palliative resection of their visceral metastases: the summary estimate suggests a significant decrease in the risk of mortality of 40% for patients who undergo surgery, with no indication of heterogeneity (HR=0.6 (0.45-0.78); I2=0). However we found indication for publication bias (P=0.03) suggesting that small studies published only mainly if they found significant results. Conclusions: Our results suggest the benefit of surgical resection for advanced-stage melanoma. Patients with limited sites and numbers of metastases should be considered for curative resection regardless of the location of the disease.

Sign in / Sign up

Export Citation Format

Share Document