scholarly journals Critical Essay: Meta-analysis: A critical realist critique and alternative

2016 ◽  
Vol 70 (1) ◽  
pp. 11-39 ◽  
Author(s):  
Matthew J Brannan ◽  
Steve Fleetwood ◽  
Joe O’Mahoney ◽  
Steve Vincent
Keyword(s):  
Quantitative Research ◽  
Meta Analysis ◽  
Explanatory Power ◽  
Organization Studies ◽  
Critical Realist ◽  
Quantitative Studies ◽  
Potential Alternative ◽  
The Individual ◽  
Meta Analyses ◽  
Meta Theory

Meta-analysis has proved increasingly popular in management and organization studies as a way of combining existing empirical quantitative research to generate a statistical estimate of how strongly variables are associated. Whilst a number of studies identify technical, procedural and practical limitations of meta-analyses, none have yet tackled the meta-theoretical flaws in this approach. We deploy critical realist meta-theory to argue that the individual quantitative studies, upon which meta-analysis relies, lack explanatory power because they are rooted in quasi-empiricist meta-theory. This problem, we argue, is carried over in meta-analyses. We then propose a ‘critical realist synthesis’ as a potential alternative to the use of meta-analysis in organization studies and social science more widely.

The Orchard Plot: Cultivating a Forest Plot for Use in Ecology, Evolution and Beyond

2019 ◽  
Author(s):  
Shinichi Nakagawa ◽  
Malgorzata Lagisz ◽  
Rose E O'Dea ◽  
Joanna Rutkowska ◽  
Yefeng Yang ◽  
...  
Keyword(s):  
Meta Analysis ◽  
R Package ◽  
Effect Sizes ◽  
Forest Plot ◽  
Point Estimates ◽  
Aggregate Effect ◽  
The Individual ◽  
Meta Analyses ◽  
Heterogeneous Effect ◽  
Intuitive Interpretation

‘Classic’ forest plots show the effect sizes from individual studies and the aggregate effect from a meta-analysis. However, in ecology and evolution meta-analyses routinely contain over 100 effect sizes, making the classic forest plot of limited use. We surveyed 102 meta-analyses in ecology and evolution, finding that only 11% use the classic forest plot. Instead, most used a ‘forest-like plot’, showing point estimates (with 95% confidence intervals; CIs) from a series of subgroups or categories in a meta-regression. We propose a modification of the forest-like plot, which we name the ‘orchard plot’. Orchard plots, in addition to showing overall mean effects and CIs from meta-analyses/regressions, also includes 95% prediction intervals (PIs), and the individual effect sizes scaled by their precision. The PI allows the user and reader to see the range in which an effect size from a future study may be expected to fall. The PI, therefore, provides an intuitive interpretation of any heterogeneity in the data. Supplementing the PI, the inclusion of underlying effect sizes also allows the user to see any influential or outlying effect sizes. We showcase the orchard plot with example datasets from ecology and evolution, using the R package, orchard, including several functions for visualizing meta-analytic data using forest-plot derivatives. We consider the orchard plot as a variant on the classic forest plot, cultivated to the needs of meta-analysts in ecology and evolution. Hopefully, the orchard plot will prove fruitful for visualizing large collections of heterogeneous effect sizes regardless of the field of study.

What do we know about meta-analyses in accounting, auditing, and corporate governance?

2019 ◽  
Vol 27 (1) ◽  
pp. 17-43 ◽  
Author(s):  
Patrick Velte
Keyword(s):  
Corporate Governance ◽  
Literature Review ◽  
Quantitative Research ◽  
Management Accounting ◽  
Meta Analysis ◽  
Financial Accounting ◽  
Content Type ◽  
Moderating Variables ◽  
Independent Variable ◽  
Meta Analyses

PurposeThe purpose of this paper is to summarize the key findings of meta-analyses in accounting, auditing and corporate governance research.Design/methodology/approachA structured literature review on 63 meta-analyses related to financial accounting, management accounting, auditing, and corporate governance is conducted.FindingsMost of the previous meta-analyses have been focused on the topic of corporate governance with a clear focus on board composition as an independent variable and by integrating research- and country-specific moderating variables. Meta-analyses in management accounting are rare; furthermore, no meta-analysis in tax accounting has been published yet.Originality/valueMeta-analyses on these topics are expected to become increasingly important in the following years because of the increased number of empirical quantitative research. This literature review gives useful insights for research, practice and regulation.

scholarly journals The Effect of Bilingualism on Older Adults’ Inhibitory Control: A Meta-Analysis

2019 ◽  
Author(s):  
Bonnie A Armstrong ◽  
Natalie Ein ◽  
Brenda I Wong ◽  
Sara N Gallant ◽  
Lingqian Li
Keyword(s):  
Older Adults ◽  
Second Language Acquisition ◽  
Inhibitory Control ◽  
Language Proficiency ◽  
Meta Analysis ◽  
Individual Characteristics ◽  
Bilingual Advantage ◽  
The Individual ◽  
Meta Analyses ◽  
Design And Methods

AbstractBackground and ObjectivesThe effect bilingualism has on older adults’ inhibitory control has been extensively investigated, yet there is continued controversy regarding whether older adult bilinguals show superior inhibitory control compared with monolinguals. The objective of the current meta-analysis was to examine the reliability and magnitude of the bilingualism effect on older adults’ inhibitory control as measured by the Simon and Stroop tasks. In addition, we examined whether individual characteristics moderate the bilingual advantage in inhibition, including age (young–old vs old–old), age of second language acquisition, immigrant status, language proficiency, and frequency of language use.Research Design and MethodsA total of 22 samples for the Simon task and 14 samples for the Stroop task were derived from 28 published and unpublished articles (32 independent samples, with 4 of these samples using more than 1 task) and were analyzed in 2 separate meta-analyses.ResultsAnalyses revealed a reliable effect of bilingualism on older adults’ performance on the Simon (g = 0.60) and Stroop (g = 0.27) tasks. Interestingly, individual characteristics did not moderate the association between bilingualism and older adults’ inhibitory control.Discussion and ImplicationsThe results suggest there is a bilingual advantage in inhibitory control for older bilinguals compared with older monolinguals, regardless of the individual characteristics previously thought to moderate this effect. Based on these findings, bilingualism may protect inhibitory control from normal cognitive decline with age.

scholarly journals A Meta-Analysis of 46 Studies Identified by the FDA Demonstrates that Soy Protein Decreases Circulating LDL and Total Cholesterol Concentrations in Adults

2019 ◽  
Vol 149 (6) ◽  
pp. 968-981 ◽  
Author(s):  
Sonia Blanco Mejia ◽  
Mark Messina ◽  
Siying S Li ◽  
Effie Viguiliouk ◽  
Laura Chiavaroli ◽  
...  
Keyword(s):  
Total Cholesterol ◽  
Soy Protein ◽  
Ldl Cholesterol ◽  
Meta Analysis ◽  
Controlled Trials ◽  
Heart Health ◽  
Health Claim ◽  
The Individual ◽  
Dose Response Effect ◽  
Meta Analyses

ABSTRACT Background Certain plant foods (nuts and soy protein) and food components (viscous fibers and plant sterols) have been permitted by the FDA to carry a heart health claim based on their cholesterol-lowering ability. The FDA is currently considering revoking the heart health claim for soy protein due to a perceived lack of consistent LDL cholesterol reduction in randomized controlled trials. Objective We performed a meta-analysis of the 46 controlled trials on which the FDA will base its decision to revoke the heart health claim for soy protein. Methods We included the 46 trials on adult men and women, with baseline circulating LDL cholesterol concentrations ranging from 110 to 201 mg/dL, as identified by the FDA, that studied the effects of soy protein on LDL cholesterol and total cholesterol (TC) compared with non-soy protein. Two independent reviewers extracted relevant data. Data were pooled by the generic inverse variance method with a random effects model and expressed as mean differences with 95% CI. Heterogeneity was assessed and quantified. Results Of the 46 trials identified by the FDA, 43 provided data for meta-analyses. Of these, 41 provided data for LDL cholesterol, and all 43 provided data for TC. Soy protein at a median dose of 25 g/d during a median follow-up of 6 wk decreased LDL cholesterol by 4.76 mg/dL (95% CI: −6.71, −2.80 mg/dL, P < 0.0001; I2 = 55%, P < 0.0001) and decreased TC by 6.41 mg/dL (95% CI: −9.30, −3.52 mg/dL, P < 0.0001; I2 = 74%, P < 0.0001) compared with non-soy protein controls. There was no dose–response effect or evidence of publication bias for either outcome. Inspection of the individual trial estimates indicated most trials (∼75%) showed a reduction in LDL cholesterol (range: −0.77 to −58.60 mg/dL), although only a minority of these were individually statistically significant. Conclusions Soy protein significantly reduced LDL cholesterol by approximately 3–4% in adults. Our data support the advice given to the general public internationally to increase plant protein intake. This trial was registered at clinicaltrials.gov as NCT03468127.

scholarly journals Individual Predictors of the Radical Right-Wing Vote in Europe: A Meta-Analysis of Articles in Peer-Reviewed Journals (1995–2016)

2018 ◽  
Vol 53 (3) ◽  
pp. 569-593 ◽  
Author(s):  
Daniel Stockemer ◽  
Tobias Lentz ◽  
Danielle Mayer
Keyword(s):  
Meta Analysis ◽  
Radical Right ◽  
Political Regime ◽  
Right Wing ◽  
Quantitative Methodology ◽  
Success Rates ◽  
Quantitative Studies ◽  
Individual Vote ◽  
Individual Variables ◽  
The Individual

In this article, we summarize the individual demand-level factors explaining the radical right-wing vote in European countries. To do so, we first review 46 quantitative peer-reviewed articles featuring the individual vote choice in favour of a radical right-wing party as the dependent variable. To identify relevant articles, we use Kai Arzheimer’s bibliography on the radical right and employ the following inclusion criterion: the articles must be written in English, they must use the individual vote for a radical right-wing party as the dependent variable, they must use a quantitative methodology and they must include some type of regression analysis. Using this strategy, we conduct a meta-analysis of 329 relevant models and find that over 20 individual variables are tested. Because many variables such as attitudes towards immigration, employment, age, education and gender only show moderate success rates in attempting to explain an individual’s propensity to vote for the radical right, we complement the review of quantitative studies with an analysis of 14 qualitative publications. The review of these qualitative works shows that the processes through which somebody becomes a voter, supporter or activist of the radical right are often more complex than the commonly used surveys can portray them. Frequently, feelings of relative economic deprivation and dissatisfaction with the political regime trigger an awakening that makes individuals seek engagement. However, the processes behind this awakening are complex and can only be partially captured by quantitative studies.

scholarly journals Are Community Gardening and Horticultural Interventions Beneficial for Psychosocial Well-Being? A Meta-Analysis

2020 ◽  
Vol 17 (10) ◽  
pp. 3584
Author(s):  
Giuseppina Spano ◽  
Marina D’Este ◽  
Vincenzo Giannico ◽  
Giuseppe Carrus ◽  
Mario Elia ◽  
...  
Keyword(s):  
Human Health ◽  
Effect Size ◽  
Meta Analysis ◽  
Well Being ◽  
City Management ◽  
Community Gardening ◽  
Quantitative Studies ◽  
Bibliographic Records ◽  
Study Heterogeneity ◽  
Meta Analyses

Recent literature has revealed the positive effect of gardening on human health; however, empirical evidence on the effects of gardening-based programs on psychosocial well-being is scant. This meta-analysis aims to examine the scientific literature on the effect of community gardening or horticultural interventions on a variety of outcomes related to psychosocial well-being, such as social cohesion, networking, social support, and trust. From 383 bibliographic records retrieved (from 1975 to 2019), seven studies with a total of 22 effect sizes were selected on the basis of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Meta-analytic findings on 11 comparisons indicate a positive and moderate effect of horticultural or gardening interventions on psychosocial well-being. Moderation analysis shows a greater effect size in individualistic than collectivistic cultures. A greater effect size was also observed in studies involving community gardening compared to horticultural intervention. Nevertheless, an effect of publication bias and study heterogeneity has been detected. Despite the presence of a large number of qualitative studies on the effect of horticulture/gardening on psychosocial well-being, quantitative studies are lacking. There is a strong need to advance into further high-quality studies on this research topic given that gardening has promising applied implications for human health, the community, and sustainable city management.

scholarly journals Methods used to meta-analyse results from interrupted time series studies: A methodological systematic review protocol

F1000Research ◽  
2020 ◽  
Vol 9 ◽  
pp. 110
Author(s):  
Elizabeth Korevaar ◽  
Amalia Karahalios ◽  
Andrew B. Forbes ◽  
Simon L. Turner ◽  
Steve McDonald ◽  
...  
Keyword(s):  
Public Health ◽  
Time Series ◽  
Systematic Reviews ◽  
Data Extraction ◽  
Meta Analysis ◽  
Interrupted Time Series ◽  
Analysis Methods ◽  
Study Selection ◽  
The Individual ◽  
Meta Analyses

Background: Systematic reviews are used to inform healthcare decision making. In reviews that aim to examine the effects of organisational, policy change or public health interventions, or exposures, evidence from interrupted time series (ITS) studies may be included. A core component of many systematic reviews is meta-analysis, which is the statistical synthesis of results across studies. There is currently a lack of guidance informing the choice of meta-analysis methods for combining results from ITS studies, and there have been no studies examining the meta-analysis methods used in practice. This study therefore aims to describe current meta-analysis methods used in a cohort of reviews of ITS studies. Methods: We will identify the 100 most recent reviews (published between 1 January 2000 and 11 October 2019) that include meta-analyses of ITS studies from a search of eight electronic databases covering several disciplines (public health, psychology, education, economics). Study selection will be undertaken independently by two authors. Data extraction will be undertaken by one author, and for a random sample of the reviews, two authors. From eligible reviews we will extract details at the review level including discipline, type of interruption and any tools used to assess the risk of bias / methodological quality of included ITS studies; at the meta-analytic level we will extract type of outcome, effect measure(s), meta-analytic methods, and any methods used to re-analyse the individual ITS studies. Descriptive statistics will be used to summarise the data. Conclusions: This review will describe the methods used to meta-analyse results from ITS studies. Results from this review will inform future methods research examining how different meta-analysis methods perform, and ultimately, the development of guidance.

scholarly journals Invited Commentary: The Disillusionment of Developmental Origins of Health and Disease (DOHaD) Epidemiology

2019 ◽  
Vol 189 (1) ◽  
pp. 1-5 ◽  
Author(s):  
Stephen E Gilman ◽  
Mady Hornig
Keyword(s):  
Observational Studies ◽  
Meta Analysis ◽  
Autism Spectrum ◽  
Developmental Origins ◽  
Short Term ◽  
Hyperactivity Disorder ◽  
Health And Disease ◽  
The Individual ◽  
Meta Analyses

Abstract The developmental origins of health and disease (DOHaD) model promises a greater understanding of early development but has left unresolved the balance of risks and benefits to offspring of medication use during pregnancy. Masarwa et al. (Am J Epidemiol. 2018;187(8):1817–1827) conducted a meta-analysis of the association between in utero acetaminophen exposure and risks of attention deficit hyperactivity disorder (ADHD) and autism spectrum disorder (ASD). A challenge of meta-analyzing results from observational studies is that summary measures of risk do not correspond to well-defined interventions when the individual studies adjusted for different covariate sets, which was the case here. This challenge limits the usefulness of observational meta-analyses for inferences about etiology and treatment planning. With that limitation understood, Masarwa et al. reported a 20%–30% higher risk of ADHD and ASD following prenatal acetaminophen exposure. Surprisingly, most of the original studies did not report diagnoses of ADHD or ASD. As a result, their summary estimates of risk are not informative about children’s likelihood of ADHD and ASD diagnoses. The long-term promise of DOHaD remains hopeful, but more effort is needed in the short-term to critically evaluate observational studies suggesting risks associated with medications used to treat conditions during pregnancy that might have adverse consequences for a developing fetus.

Statistical Models for the Meta-analysis of Nonindependent Data

2013 ◽  
Author(s):  
Kerrie Mengersen ◽  
Michael D. Jennions ◽  
Christopher H. Schmid
Keyword(s):  
Statistical Models ◽  
Single Case ◽  
Meta Analysis ◽  
Dry Weight ◽  
Multiple Treatments ◽  
The Individual ◽  
Meta Analyses ◽  
Size Estimates ◽  
Over Time

In many meta-analyses, independence is questionable because there are several effect estimates per study and/or some of the individual studies included in the meta-analysis might not provide independent estimates of the effect. Within-study nonindependence can arise due to multiple measures of the same effect on the same experimental units being made over time, multiple treatments being compared to the same set of control individuals, or different measures being taken (e.g., plant height, dry weight, and photosynthesis rate) from the same experimental units to generate several different effect size estimates. This chapter discusses nonindependence among effect sizes both within and among studies. It focuses on four commonplace situations where nonindependence can occur in ecology and evolution meta-analyses. Each of these four situations is illustrated with a single case study.

Meta-Analysis in Single-Subject Evaluation Research

2021 ◽  
pp. 143-154
Author(s):  
Charles Auerbach
Keyword(s):  
Meta Analysis ◽  
Evaluation Research ◽  
Effect Sizes ◽  
Single Subject ◽  
Building Research ◽  
Single Subject Research ◽  
Research Designs ◽  
The Individual ◽  
Meta Analyses ◽  
Intervening Variables

Meta-analytic techniques can be used to aggregate evaluation results across studies. In the case of single-subject research designs, we could combine findings from evaluations with 5, 10 or 20 clients to determine, on average, how effective an intervention is. This is a more complex and sophisticated way of understanding differences across studies than reporting those changes qualitatively or simply reporting the individual effect sizes for each study. In this chapter, the authors discuss why meta-analysis is important to consider in single-subject research, particularly in the context of building research evidence. They then demonstrate how to do this using SSD for R functions. Building upon effect sizes, introduced in Chapter 4, the authors illustrate the conditions under which it is appropriate to use traditional effect sizes to conduct meta-analyses, how to introduce intervening variables, and how to evaluate statistical output. Additionally, the authors discuss and illustrate the computation and interpretation of a mean Non-Overlap of All Pairs in situations which traditional effect sizes cannot be used.

Sign in / Sign up

Export Citation Format

Share Document