scholarly journals Role of Meta-analysis in Interpreting the Scientific Literature

2013 ◽  
Author(s):  
Michael D. Jennions ◽  
Christopher J. Lortie ◽  
Julia Koricheva
Keyword(s):  
Statistical Power ◽  
Scientific Literature ◽  
Meta Analysis ◽  
Effect Sizes ◽  
P Values ◽  
Single Study ◽  
Broad Scale

This chapter begins with a brief review of why effect sizes and their variances are more informative than P-values. It then discusses how meta-analysis promotes “effective thinking” that can change approaches to several commonplace problems. Specifically, it addresses the issues of (1) exemplar studies versus average trends, (2) resolving “conflict” between specific studies, (3) presenting results, (4) deciding on the level at which to replicate studies, (5) understanding the constraints imposed by low statistical power, and (6) asking broad-scale questions that cannot be resolved in a single study. The chapter focuses on estimating effect sizes as a key outcome of meta-analysis, but acknowledges that other outcomes might be of more interest in other situations.

EDITORIAL: Systematic reviews and meta-analysis

1997 ◽  
Vol 27 (1) ◽  
pp. 3-7 ◽  
Author(s):  
G. LEWIS ◽  
R. CHURCHILL ◽  
M. HOTOPF
Keyword(s):  
Clinical Practice ◽  
Systematic Reviews ◽  
Statistical Power ◽  
Scientific Literature ◽  
Meta Analysis ◽  
Medical Interventions ◽  
Single Study ◽  
Causes Of Disease ◽  
Limited Statistical Power

Conclusions about medical interventions or the causes of disease are based upon reviews of the scientific literature. Single studies usually have limited statistical power or may be difficult to interpret or generalize and so the findings from a single study can rarely justify a change in clinical practice or in an aetiological theory. Even when planning larger studies or mega-trials (Yusuf et al. 1984), a thorough review of existing literature is needed and the results of the study need to be placed in that context, though single studies can exert an important and powerful influence.

Effectiveness of psychological interventions to improve emotion regulation in youth - a meta-analysis

2019 ◽  
Author(s):  
Bettina Moltrecht ◽  
Jessica Deighton ◽  
Praveetha Patalay ◽  
Julian Childs
Keyword(s):  
Emotion Regulation ◽  
Emotion Dysregulation ◽  
Psychological Intervention ◽  
Meta Analysis ◽  
Age Groups ◽  
Effect Sizes ◽  
What Works ◽  
Large Heterogeneity ◽  
Larger Sample

Background: Research investigating the role of emotion regulation (ER) in the development and treatment of psychopathology has increased in recent years. Evidence suggests that an increased focus on ER in treatment can improve existing interventions. Most ER research has neglected young people, therefore the present meta-analysis summarizes the evidence for existing psychosocial intervention and their effectiveness to improve ER in youth. Methods: A systematic review and meta-analysis was conducted according to the PRISMA guidelines. Twenty-one randomized-control-trials (RCTs) assessed changes in ER following a psychological intervention in youth exhibiting various psychopathological symptoms.Results: We found moderate effect sizes for current interventions to decrease emotion dysregulation in youth (g=-.46) and small effect sizes to improve emotion regulation (g=0.36). Significant differences between studies including intervention components, ER measures and populations studied resulted in large heterogeneity. Conclusion: This is the first meta-analysis that summarizes the effectiveness for existing interventions to improve ER in youth. The results suggest that interventions can enhance ER in youth, and that these improvements correlate with improvements in psychopathology. More RCTs including larger sample sizes, different age groups and psychopathologies are needed to increase our understanding of what works for who and when.

scholarly journals Association Between Diabetes Medications and the Risk of Parkinson's Disease: A Systematic Review and Meta-Analysis

2021 ◽  
Vol 12 ◽  
Author(s):  
Xiaocui Qin ◽  
Xia Zhang ◽  
Pinyu Li ◽  
Min Wang ◽  
Li Yan ◽  
...  
Keyword(s):  
Parkinson’S Disease ◽  
Parkinson's Disease ◽  
Meta Analysis ◽  
Limited Data ◽  
Dipeptidyl Peptidase 4 ◽  
Single Study ◽  
Dipeptidyl Peptidase 4 Inhibitors ◽  
Glucagon Like Peptide 1

Background: Diabetes mellitus (DM) increases the risk of Parkinson's disease (PD). However, whether DM medications play a part on that increased PD risk is unclear. We designed this meta-analysis to assess the influence of different oral DM medications on the PD risk in patients with DM.Methods: We searched PubMed, Embase, and CENTRAL databases for relevant studies up until January 2021. We pooled adjusted outcomes to assess the PD risk in patients using different DM medications including sulfonylurea, metformin, glitazones (GTZ), dipeptidyl peptidase-4 inhibitors (DPP4i), and glucagon-like peptide-1 agonists (GLP1a).Results: We included 10 studies in our analysis. Our results indicate a lack of significant association between the PD risk and the use of sulfonylureas (three studies; HR, 1.26; 95% CI, 0.95 to 1.66; I2, 70%; p = 0.11), DPP4i (three studies; HR, 0.69; 95% CI, 0.35 to 1.38; I2, 88%; p = 0.30), metformin (five studies; HR, 1.23; 95% CI, 0.98 to 1.78; I2, 84%; p = 0.13), and GTZ (six studies; HR, 0.88; 95% CI, 0.66 to 1.16; I2, 92%; p = 0.35). After exclusion of a single study in the GTZ analysis, our results indicate a significantly reduced PD risk with GTZ use (HR, 0.78; 95% CI, 0.65 to 0.93; I2, 59%; p = 0.06). Similarly, after the exclusion of a single study, our results indicate a significantly increased PD risk with the use of metformin (HR, 1.50; 95% CI, 1.11 to 2.02; I2, 80%; p = 0.008). We also found a significantly reduced PD risk with the use of GLP1a (two studies; HR, 0.41; 95% CI, 0.19 to 0.87; I2, 0%; p = 0.02).Conclusion: The role of different DM medications on the PD risk remains unclear, and the quality of studies is low. While our analysis suggests a lack of association between the use of metformin, GTZ, DPP4i, and sulfonylureas and the PD risk, metformin (to a higher degree) and GTZ may still increase the risk. Limited data suggest a protective effect of GLP1a on the PD risk.

scholarly journals The earth is flat (p>0.05): Significance thresholds and the crisis of unreplicable research

2017 ◽  
Author(s):  
Valentin Amrhein ◽  
Fränzi Korner-Nievergelt ◽  
Tobias Roth
Keyword(s):  
Publication Bias ◽  
Null Hypothesis ◽  
Statistical Power ◽  
Alternative Hypothesis ◽  
Statistical Significance ◽  
Practical Importance ◽  
Decision Rules ◽  
Effect Sizes ◽  
P Values ◽  
True Effect

The widespread use of 'statistical significance' as a license for making a claim of a scientific finding leads to considerable distortion of the scientific process (American Statistical Association, Wasserstein & Lazar 2016). We review why degrading p-values into 'significant' and 'nonsignificant' contributes to making studies irreproducible, or to making them seem irreproducible. A major problem is that we tend to take small p-values at face value, but mistrust results with larger p-values. In either case, p-values can tell little about reliability of research, because they are hardly replicable even if an alternative hypothesis is true. Also significance (p≤0.05) is hardly replicable: at a realistic statistical power of 40%, given that there is a true effect, only one in six studies will significantly replicate the significant result of another study. Even at a good power of 80%, results from two studies will be conflicting, in terms of significance, in one third of the cases if there is a true effect. This means that a replication cannot be interpreted as having failed only because it is nonsignificant. Many apparent replication failures may thus reflect faulty judgement based on significance thresholds rather than a crisis of unreplicable research. Reliable conclusions on replicability and practical importance of a finding can only be drawn using cumulative evidence from multiple independent studies. However, applying significance thresholds makes cumulative knowledge unreliable. One reason is that with anything but ideal statistical power, significant effect sizes will be biased upwards. Interpreting inflated significant results while ignoring nonsignificant results will thus lead to wrong conclusions. But current incentives to hunt for significance lead to publication bias against nonsignificant findings. Data dredging, p-hacking and publication bias should be addressed by removing fixed significance thresholds. Consistent with the recommendations of the late Ronald Fisher, p-values should be interpreted as graded measures of the strength of evidence against the null hypothesis. Also larger p-values offer some evidence against the null hypothesis, and they cannot be interpreted as supporting the null hypothesis, falsely concluding that 'there is no effect'. Information on possible true effect sizes that are compatible with the data must be obtained from the observed effect size, e.g., from a sample average, and from a measure of uncertainty, such as a confidence interval. We review how confusion about interpretation of larger p-values can be traced back to historical disputes among the founders of modern statistics. We further discuss potential arguments against removing significance thresholds, such as 'we need more stringent decision rules', 'sample sizes will decrease' or 'we need to get rid of p-values'.

Linking Moral Identity With Moral Emotions: A Meta-Analysis

2019 ◽  
Vol 23 (4) ◽  
pp. 444-457 ◽  
Author(s):  
Jean Paul Lefebvre ◽  
Tobias Krettenauer
Keyword(s):  
Positive Emotions ◽  
Moral Identity ◽  
Meta Analysis ◽  
Moral Emotions ◽  
Effect Sizes ◽  
Moral Functioning ◽  
Moral Anger ◽  
Other Regarding ◽  
The Relationship

This meta-analysis examined the relationship between moral identity and moral emotions drawing on 57 independent studies. Moral identity was significantly associated with moral emotions, r = .32, p < .01, 95% confidence interval [CI: .27, .36]. Effect sizes were moderated by the type of moral emotion. Studies reporting other-regarding emotions (sympathy, empathy, and compassion) had the largest effect sizes ( r = .41), while negative other-evaluative emotions (moral anger, contempt, and disgust) had the smallest ( r = .16). Self-evaluative and other-evaluative positive emotions had intermediate effect sizes ( r values between .29 and .32). The type of emotion measure also was a significant moderator, with trait measures of emotion ( r = .38) correlating more strongly with moral identity than state measures ( r = .24). Effect sizes did not differ for the type of moral identity measure being used, publication status, or cultural origin of the study sample. The results of this meta-analysis demonstrate a robust empirical connection between moral identity and moral emotions, which confirms the multifaceted role of moral identity in moral functioning.

scholarly journals 10 easy steps to conducting transparent, reproducible meta-analyses for infant researchers

2021 ◽  
Author(s):  
Loretta Gasparini ◽  
Sho Tsuji ◽  
Christina Bergmann
Keyword(s):  
Developmental Psychology ◽  
Evidence Synthesis ◽  
Meta Analysis ◽  
Research Question ◽  
The Body ◽  
Effect Sizes ◽  
Robust Methods ◽  
Meta Analyses ◽  
Analytic Models

Meta-analyses provide researchers with an overview of the body of evidence in a topic, with quantified estimates of effect sizes and the role of moderators, and weighting studies according to their precision. We provide a guide for conducting a transparent and reproducible meta-analysis in the field of developmental psychology within the framework of the MetaLab platform, in 10 steps: 1) Choose a topic for your meta-analysis, 2) Formulate your research question and specify inclusion criteria, 3) Preregister and carefully document all stages of your meta-analysis, 4) Conduct the literature search, 5) Collect and screen records, 6) Extract data from eligible studies, 7) Read the data into analysis software and compute effect sizes, 8) Create meta-analytic models to assess the strength of the effect and investigate possible moderators, 9) Visualize your data, 10) Write up and promote your meta-analysis. Meta-analyses can inform future studies, through power calculations, by identifying robust methods and exposing research gaps. By adding a new meta-analysis to MetaLab, datasets across multiple topics of developmental psychology can be synthesized, and the dataset can be maintained as a living, community-augmented meta-analysis to which researchers add new data, allowing for a cumulative approach to evidence synthesis.

scholarly journals The Role of Public Defenders Within the American Justice System

2020 ◽  
Vol 8 (2) ◽  
Author(s):  
Brianna Fisher ◽  
Patti Thompson
Keyword(s):  
Work Ethic ◽  
Meta Analysis ◽  
Effect Sizes ◽  
Weighted Mean ◽  
Public Defenders ◽  
Weighted Means ◽  
Negative Effect ◽  
American Cities ◽  
Positive Effect

This paper examines the inner workings of the legal system in connection with public defenders and their duties. The factors of total caseload, plea deals, waivers, and bench verdicts were implemented to create a comprehensive means of measuring work ethic, due to their significant presence in the everyday workings of public defenders. Through a meta-analysis of the five most densely populated African American cities within the state of Michigan, the combined and individual weighted mean effect sizes were calculated to determine if there was a strong positive or negative effect of the four factors mentioned above on work ethic. Additionally, both the upper and lower credibility levels were calculated to analyze their proximity to the weighted means. For all four factors tested, the weighted mean effect sizes relate a positive effect of the factors on work ethic, with all means falling closer to their upper credibility levels. These results confirm that the factors of total caseload, plea deals, waivers, and bench verdicts affect how cases are being handled and how clients are being treated.

scholarly journals Making our “meta-hypotheses” clear: heterogeneity and the role of direct replications in science

2021 ◽  
Vol 11 (2) ◽  
Author(s):  
Eirik Strømland
Keyword(s):  
Early Phase ◽  
Scientific Literature ◽  
Replication Study ◽  
Effect Sizes ◽  
Mature Stage ◽  
Initial Finding ◽  
Replication Studies ◽  
Homogeneity Assumption ◽  
Scientific Status

AbstractThis paper argues that some of the discussion around meta-scientific issues can be viewed as an argument over different “meta-hypotheses” – assumptions made about how different hypotheses in a scientific literature relate to each other. I argue that, currently, such meta-hypotheses are typically left unstated except in methodological papers and that the consequence of this practice is that it is hard to determine what can be learned from a direct replication study. I argue in favor of a procedure dubbed the “limited homogeneity assumption” – assuming very little heterogeneity of effect sizes when a literature is initiated but switching to an assumption of heterogeneity once an initial finding has been successfully replicated in a direct replication study. Until that has happened, we do not allow the literature to proceed to a mature stage. This procedure will elevate the scientific status of direct replication studies in science. Following this procedure, a well-designed direct replication study is a means of falsifying an overall claim in an early phase of a literature and thus sets up a hurdle against the canonization of false facts in the behavioral sciences.

Sign in / Sign up

Export Citation Format

Share Document