A Review and Evaluation of Meta-Analysis Practices in Management Research

2008 ◽  
Vol 35 (2) ◽  
pp. 393-419 ◽  
Author(s):  
Inge Geyskens ◽  
Rekha Krishnan ◽  
Jan-Benedict E. M. Steenkamp ◽  
Paulo V. Cunha
Keyword(s):  
Empirical Evidence ◽  
Meta Analysis ◽  
Management Research ◽  
Research Findings ◽  
Meta Analyses ◽  
Integrate Research ◽  
Made In

Meta-analysis has become increasingly popular in management research to quantitatively integrate research findings across a large number of studies. In an effort to help shape future applications of meta-analysis in management, this study chronicles and evaluates the decisions that management researchers made in 69 meta-analytic studies published between 1980 and 2007 in 14 management journals. It performs four meta-analyses of relationships that have been studied with varying frequency in management research, to provide empirical evidence that meta-analytical decisions influence results. The implications of the findings are discussed with a focus on the changes that seem appropriate.

An Empirical Review of Research and Reporting Practices in Psychological Meta-Analyses

2020 ◽  
Vol 24 (3) ◽  
pp. 195-209
Author(s):  
Richard E. Hohn ◽  
Kathleen L. Slaney ◽  
Donna Tafreshi
Keyword(s):  
Meta Analysis ◽  
Reporting Standards ◽  
Exclusion Criteria ◽  
Research Practices ◽  
Reporting Practices ◽  
Reporting Decisions ◽  
Research Findings ◽  
State Of Research ◽  
Meta Analyses ◽  
Made In

As meta-analytic studies have come to occupy a sizable contingent of published work in the psychological sciences, clarity in the research and reporting practices of such work is crucial to the interpretability and reproducibility of research findings. The present study examines the state of research and reporting practices within a random sample of 384 published psychological meta-analyses across several important dimensions (e.g., search methods, exclusion criteria, statistical techniques). In addition, we surveyed the first authors of the meta-analyses in our sample to ask them directly about the research practices employed and reporting decisions made in their studies, including the assessments and procedures they conducted and the guidelines or materials they relied on. Upon cross-validating the first author responses with what was reported in their published meta-analyses, we identified numerous potential gaps in reporting and research practices. In addition to providing a survey of recent reporting practices, our findings suggest that (a) there are several research practices conducted by meta-analysts that are ultimately not reported; (b) some aspects of meta-analysis research appear to be conducted at disappointingly low rates; and (c) the adoption of the reporting standards, including the Meta-Analytic Reporting Standards (MARS), has been slow to nonexistent within psychological meta-analytic research.

118 The Use of Meta-analysis in Companion Animal Research

2021 ◽  
Vol 99 (Supplement_3) ◽  
pp. 62-62
Author(s):  
Emma N Bermingham
Keyword(s):  
Body Size ◽  
Statistical Analysis ◽  
Meta Analysis ◽  
Energy Requirements ◽  
Specific Question ◽  
Research Findings ◽  
Adult Cat ◽  
Number Of Publications ◽  
Meta Analyses ◽  
The Three Rs

Abstract In a world of the “Three Rs” (replace, reduce and refine), combined with more research published via open access research journals, there is increasing interest in the statistical analysis of existing literature. Meta-analysis – the combination of multiple studies, can be used to get better oversight into a specific question of interest. Additionally, it can be used to identify gaps in knowledge. For example, while there are a number of publications investigating energy requirements in adult cat and dog, few studies assess older animals. Similarly, in the dog, there is a lack of literature around dogs at the extremes of body size (i.e. giant and toy breeds). Herein, we describe several published examples that have been used to determine energy requirements of cats and dogs, and more recently, the impacts of diet on the microbiome of the cat and dog. This includes the use of Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines, research findings and general findings related to research design and quality.

Het kwantitatief integreren van empirische studies: de methode van meta-analyse

2006 ◽  
Vol 19 (3) ◽  
Author(s):  
Marise Ph. Born ◽  
Stefan T. Mol
Keyword(s):  
Meta Analysis ◽  
Research Question ◽  
Empirical Studies ◽  
Dominant Position ◽  
The Social ◽  
Research Findings ◽  
The Many ◽  
Quantitative Integration ◽  
Meta Analyses ◽  
Research Domain

Quantitatively integrating empirical studies: The method of meta-analysis Quantitatively integrating empirical studies: The method of meta-analysis Marise Ph. Born & Stefan T. Mol, Gedrag & Organisatie, Volume 19, September 2006, nr. 3, pp. 251-271 Meta-analysis is a quantitative integration of results of a series of empirical studies into a specific research question. The method of meta-analysis has obtained a dominant position in the social sciences and beyond, as it may help in obtaining an overview of the explosively increased number of research publications. This contribution discusses the basics and consecutive steps in performing a meta-analysis. A meta-analysis that we conducted on expatriates serves as an illustration. Next to the many points in favor of meta-analyses, such as having a better overview of a research domain and shifting the traditional focus on significances of effects to sizes of effects, several important controversies remain. One of these is the issue of waving away a specific cause of variance in research findings as a methodological artifact, or interpreting it as a meaningful case of variance. We maintain that every social or industrial- and organizational psychologist who wants to stay up-to-date scientifically should be able to interpret meta-analyses.

Meta-Analyses of Built Environment Effects on Travel: No New Platinum Standard

2019 ◽  
pp. 0739456X1985642 ◽  
Author(s):  
Petter Næss
Keyword(s):  
Land Use ◽  
Meta Analysis ◽  
Model Specification ◽  
Specification Error ◽  
Environment Effects ◽  
Alternative Approach ◽  
Poor Understanding ◽  
Research Findings ◽  
Meta Analyses ◽  
Transportation Studies

This commentary presents a critique of a particular, strictly quantitative way of reviewing research findings within the field of land use and transportation studies, so-called meta-analyses. Beyond criticism raised earlier, the article draws attention to serious bias resulting when meta-analysis include studies encumbered with model specification error due to poor understanding of causal mechanisms. The article also discusses underestimated limitations due to neglect of differences between geographical contexts and inconsistent measurement of variables across studies. An example of an alternative approach is offered at the end of the article.

scholarly journals An Updated Systematic Review and Meta-Analysis of the Obstetric Consequences of Female Genital Mutilation/Cutting

2014 ◽  
Vol 2014 ◽  
pp. 1-8 ◽  
Author(s):  
R. C. Berg ◽  
J. Odgaard-Jensen ◽  
A. Fretheim ◽  
V. Underland ◽  
G. Vist
Keyword(s):  
Systematic Review ◽  
Prospective Studies ◽  
Retrospective Studies ◽  
Meta Analysis ◽  
Statistical Significance ◽  
Previous Systematic Review ◽  
Genital Mutilation ◽  
Meta Analyses ◽  
Female Genital ◽  
Made In

In our recent systematic review inObstetrics and Gynecology Internationalof the association between FGM/C and obstetric harm we concluded that FGM/C significantly increases the risk of delivery complications. The findings were based on unadjusted effect estimates from both prospective and retrospective studies. To accommodate requests by critics, we aimed to validate these results through additional analyses based on adjusted estimates from prospective studies. We judged that 7 of the 28 studies included in our original systematic review were prospective. Statistical adjustments for measured confounding factors were made in eight studies, including three prospective studies. The adjusted confounders differed across studies in number and type. Results from meta-analyses based on adjusted estimates, with or without data from retrospective studies, consistently pointed in the same direction as our earlier findings. There were only small differences in the sizes or the level of statistical significance. Using GRADE, we assessed that our confidence in the effect estimates was very low or low for all outcomes. The adjusted estimates generally show similar obstetric harms from FGM/C as unadjusted estimates do. Thus, the current analyses confirm the findings from our previous systematic review. There are sufficient grounds to conclude that FGM/C, with respect to obstetric circumstances, involves harm.

scholarly journals Systematic review and meta-analysis of the relationship between the heartbeat-evoked potential and interoception.

2020 ◽  
Author(s):  
Michel-Pierre Coll ◽  
Hannah Hobson ◽  
Jennifer Murphy
Keyword(s):  
Systematic Review ◽  
Empirical Evidence ◽  
Evoked Potential ◽  
Meta Analysis ◽  
Clinical Status ◽  
The Body ◽  
Indirect Measures ◽  
Heartbeat Evoked Potential ◽  
Meta Analyses ◽  
The Relationship

The Heartbeat Evoked Potential (HEP) has been proposed as a neurophysiological marker of interoceptive processing. Despite its use to validate interoceptive measures and to assess interoceptive functioning in clinical groups, the empirical evidence for a relationship between HEP amplitude and interoceptive processing, including measures of such processing, is scattered across several studies with varied designs. The aim of this systematic review and meta-analysis was to examine the body of HEP-interoception research, and consider the associations the HEP shows with various direct and indirect measures of interoception, and how it is affected by manipulations of interoceptive processing. Specifically, we assessed the effect on HEP amplitude of manipulating attention to the heartbeat; manipulating participants’ arousal; the association between the HEP and behavioural measures of cardiac interoception; and comparisons between healthy and clinical groups. Following database searches and screening, 45 studies were included in the systematic review and 42 in the meta-analyses. We noted variations in the ways individual studies have attempted to address key confounds, particularly the cardiac field artefact. Meta-analytic summaries indicated there were moderate to large effects of attention, arousal, and clinical status on the HEP, and a moderate association between HEP amplitude and behavioural measures of interoception. Problematically, the reliability of the meta-analytic effects documented here remain unknown, given the lack of standardised protocols for measuring the HEP. Thus, it is possible effects are driven by confounds such as cardiac factors or somatosensory effects.

scholarly journals The puzzling relationship between multi-lab replications and meta-analyses of the rest of the literature

2020 ◽  
Author(s):  
Molly Lewis ◽  
Maya B Mathur ◽  
Tyler VanderWeele ◽  
Michael C. Frank
Keyword(s):  
Empirical Evidence ◽  
Effect Size ◽  
Meta Analysis ◽  
Scientific Research ◽  
Psychological Phenomena ◽  
Analytic Estimate ◽  
Meta Analyses ◽  
The Relationship ◽  
Size Estimates

What is the best way to estimate the size of important effects? Should we aggregate across disparate findings using statistical meta-analysis, or instead run large, multi-lab replications (MLR)? A recent paper by Kvarven, Strømland, and Johannesson (2020) compared effect size estimates derived from these two different methods for 15 different psychological phenomena. The authors report that, for the same phenomenon, the meta-analytic estimate tends to be about three times larger than the MLR estimate. These results pose an important puzzle: What is the relationship between these two estimates? Kvarven et al. suggest that their results undermine the value of meta-analysis. In contrast, we argue that both meta-analysis and MLR are informative, and that the discrepancy between estimates obtained via the two methods is in fact still unexplained. Informed by re-analyses of Kvarven et al.’s data and by other empirical evidence, we discuss possible sources of this discrepancy and argue that understanding the relationship between estimates obtained from these two methods is an important puzzle for future meta-scientific research.

scholarly journals Assessing robustness against potential publication bias in coordinate based fMRI meta-analyses using the Fail-Safe N

2017 ◽  
Author(s):  
Freya Acar ◽  
Ruth Seurinck ◽  
Simon B. Eickhoff ◽  
Beatrijs Moerkerke
Keyword(s):  
Publication Bias ◽  
Meta Analysis ◽  
Popular Approach ◽  
Analytical Results ◽  
Research Findings ◽  
Meta Analyses ◽  
Fail Safe ◽  
File Drawer ◽  
Cluster Level ◽  
File Drawer Effect

AbstractThe importance of integrating research findings is incontrovertible and coordinate based meta-analyses have become a popular approach to combine results of fMRI studies when only peaks of activation are reported. Similar to classical meta-analyses, coordinate based meta-analyses may be subject to different forms of publication bias which impacts results and possibly invalidates findings. We develop a tool that assesses the robustness to potential publication bias on cluster level. We investigate the possible influence of the file-drawer effect, where studies that do not report certain results fail to get published, by determining the number of noise studies that can be added to an existing fMRI meta-analysis before the results are no longer statistically significant. In this paper we illustrate this tool through an example and test the effect of several parameters through extensive simulations. We provide an algorithm for which code is freely available to generate noise studies and enables users to determine the robustness of meta-analytical results.

scholarly journals Advancing Meta-Analysis With Knowledge-Management Platforms: Using metaBUS in Psychology

2019 ◽  
Vol 3 (1) ◽  
pp. 124-137 ◽  
Author(s):  
Frank A. Bosco ◽  
James G. Field ◽  
Kai R. Larsen ◽  
Yingyi Chang ◽  
Krista L. Uggerslev
Keyword(s):  
Knowledge Management ◽  
Meta Analysis ◽  
Applied Psychology ◽  
Human Judgment ◽  
Psychological Science ◽  
Research Findings ◽  
Advance Research ◽  
Research And Education ◽  
Meta Analyses

In this article, we provide a review of research-curation and knowledge-management efforts that may be leveraged to advance research and education in psychological science. After reviewing the approaches and content of other efforts, we focus on the metaBUS project’s platform, the most comprehensive effort to date. The metaBUS platform uses standards-based protocols in combination with human judgment to organize and make readily accessible a database of research findings, currently numbering more than 1 million. It allows users to conduct rudimentary, instant meta-analyses, and capacities for visualization and communication of meta-analytic findings have recently been added. We conclude by discussing challenges, opportunities, and recommendations for expanding the project beyond applied psychology.

Peer-Based Interventions to Influence Health-Related Behaviors and Attitudes: A Meta-Analysis

1999 ◽  
Vol 85 (3_suppl) ◽  
pp. 1179-1194 ◽  
Author(s):  
Emil J. Posavac ◽  
Kristienne R. Kattapong ◽  
Dennis E. Dew
Keyword(s):  
Health Education ◽  
Meta Analysis ◽  
Preventive Interventions ◽  
Health Issues ◽  
Education Programs ◽  
Health Related ◽  
Research Findings ◽  
The Mean ◽  
Meta Analyses ◽  
The Cost

The effects of 47 peer-based health education programs described in 36 published studies were examined. The over-all effect size was small: the mean d was .190 when controls received no program and .020 when controls received an alternative program. Programs were divided into those focusing on preventing or reducing smoking and programs on other health issues; the latter were further divided into primary prevention and secondary prevention programs. Differences among studies suggested several biases which were likely to have influenced the effect sizes. Preventive interventions that produce only small effects can be valuable because many participants would not have developed the problem even without the program. This review suggested that, when health education programs are studied, (a) detailed statistical information should be provided to facilitate using the research findings in meta-analyses and (b) the costs of innovative programs should be presented to judge whether the results are worth the cost.

Sign in / Sign up

Export Citation Format

Share Document