Meta-analyses and mega-mistakes: calling time on meta-analysis of the species richness–productivity relationship

Ecology ◽  
2010 ◽  
Vol 91 (9) ◽  
pp. 2522-2533 ◽  
Author(s):  
Robert J. Whittaker
Keyword(s):  
Species Richness ◽  
Meta Analysis ◽  
Meta Analyses

scholarly journals How to make a meta-analyst happy – what to report in your studies and how

2019 ◽  
Vol 2 ◽  
Author(s):  
Pavel Saska
Keyword(s):  
Species Richness ◽  
Data Extraction ◽  
Meta Analysis ◽  
Temporal Structure ◽  
Large Data ◽  
Carabid Beetles ◽  
Mean Values ◽  
Spatio Temporal ◽  
Meta Analyses ◽  
The Impact

Meta-analysis represents an approach of synthesizing many independent data sets, and is useful in situations when abundant literature provides no conclusive evidence. Besides the quality of the research itself, the value of an individual study for meta-analysis depends to the large extent also on the quality of data presentation. The literature based on carabid beetles (Coleoptera: Carabidae) as the study is enormous, therefore there is a great potential for the use in meta-analyses. In this paper I put together some notes which arose during my work on meta-analysis focused on the effects of field and crop management on populations of carabid beetles inside the crop fields of Europe and America north of Mexico. The aim of this contribution is to provide a set of recommendations which may potentially improve the use of each individual paper in future meta-analyses, and thus increase the impact of the original paper as well as the generality of conclusions drawn from future meta-analyses, hence based on larger sample size. Be accurate in describing the treatments. For example, “low” and “high intensity of management” is not enough. Mention also details that are constant across treatments, but may still provide useful information. E.g. “practice usual for the area” is not enough. Be precise with describing spatio-temporal structure in the study. Provide redundant information so everyone can check if he/she understood well the hierarchy of the experiment and the number of replications associated with each stratum. A scheme may be useful. Report the grand totals as well as treatment totals for both “abundance” and species richness. Text, tables or supplementary materials is preferred. If using mean values, always make it clear what is the number of replicates and provide standard errors. But, remember that total or treatment species richness cannot be reconstructed from the mean! Be explicit in stating what the means represent, also in figures. Expressions like “Mean abundance” are not enough. Provide species lists with the greatest resolution possible. Most journals allow for supplementary materials where this information can be provided. Remember that data can also be extracted from figures. Provide high resolution and accurate figures. For example, large data points on a line make data extraction difficult. Be accurate in describing the treatments. For example, “low” and “high intensity of management” is not enough. Mention also details that are constant across treatments, but may still provide useful information. E.g. “practice usual for the area” is not enough. Be precise with describing spatio-temporal structure in the study. Provide redundant information so everyone can check if he/she understood well the hierarchy of the experiment and the number of replications associated with each stratum. A scheme may be useful. Report the grand totals as well as treatment totals for both “abundance” and species richness. Text, tables or supplementary materials is preferred. If using mean values, always make it clear what is the number of replicates and provide standard errors. But, remember that total or treatment species richness cannot be reconstructed from the mean! Be explicit in stating what the means represent, also in figures. Expressions like “Mean abundance” are not enough. Provide species lists with the greatest resolution possible. Most journals allow for supplementary materials where this information can be provided. Remember that data can also be extracted from figures. Provide high resolution and accurate figures. For example, large data points on a line make data extraction difficult. With little extra effort during the preparation phase, the impact of your paper and the use of your data may considerably increase in the future.

A Meta-Analysis of Interviews and Cognitive Ability

2013 ◽  
Vol 12 (4) ◽  
pp. 157-169 ◽  
Author(s):  
Philip L. Roth ◽  
Allen I. Huffcutt
Keyword(s):  
Cognitive Ability ◽  
Meta Analysis ◽  
Incremental Validity ◽  
Employment Interviews ◽  
Moderator Analysis ◽  
The Future ◽  
Research Questions ◽  
Meta Analyses ◽  
The Relationship

The topic of what interviews measure has received a great deal of attention over the years. One line of research has investigated the relationship between interviews and the construct of cognitive ability. A previous meta-analysis reported an overall corrected correlation of .40 ( Huffcutt, Roth, & McDaniel, 1996 ). A more recent meta-analysis reported a noticeably lower corrected correlation of .27 ( Berry, Sackett, & Landers, 2007 ). After reviewing both meta-analyses, it appears that the two studies posed different research questions. Further, there were a number of coding judgments in Berry et al. that merit review, and there was no moderator analysis for educational versus employment interviews. As a result, we reanalyzed the work by Berry et al. and found a corrected correlation of .42 for employment interviews (.15 higher than Berry et al., a 56% increase). Further, educational interviews were associated with a corrected correlation of .21, supporting their influence as a moderator. We suggest a better estimate of the correlation between employment interviews and cognitive ability is .42, and this takes us “back to the future” in that the better overall estimate of the employment interviews – cognitive ability relationship is roughly .40. This difference has implications for what is being measured by interviews and their incremental validity.

Power-Enhanced Funnel Plots for Meta-Analysis

2020 ◽  
Vol 228 (1) ◽  
pp. 43-49 ◽  
Author(s):  
Michael Kossmeier ◽  
Ulrich S. Tran ◽  
Martin Voracek
Keyword(s):  
Statistical Power ◽  
Meta Analysis ◽  
Relevant Information ◽  
Selective Reporting ◽  
Graphical Display ◽  
Funnel Plot ◽  
Evidential Value ◽  
Graphical Displays ◽  
Funnel Plots ◽  
Meta Analyses

Abstract. Currently, dedicated graphical displays to depict study-level statistical power in the context of meta-analysis are unavailable. Here, we introduce the sunset (power-enhanced) funnel plot to visualize this relevant information for assessing the credibility, or evidential value, of a set of studies. The sunset funnel plot highlights the statistical power of primary studies to detect an underlying true effect of interest in the well-known funnel display with color-coded power regions and a second power axis. This graphical display allows meta-analysts to incorporate power considerations into classic funnel plot assessments of small-study effects. Nominally significant, but low-powered, studies might be seen as less credible and as more likely being affected by selective reporting. We exemplify the application of the sunset funnel plot with two published meta-analyses from medicine and psychology. Software to create this variation of the funnel plot is provided via a tailored R function. In conclusion, the sunset (power-enhanced) funnel plot is a novel and useful graphical display to critically examine and to present study-level power in the context of meta-analysis.

Which Data to Meta-Analyze, and How?

2019 ◽  
Vol 227 (1) ◽  
pp. 64-82 ◽  
Author(s):  
Martin Voracek ◽  
Michael Kossmeier ◽  
Ulrich S. Tran
Keyword(s):  
Empirical Research ◽  
Meta Analysis ◽  
Curve Analysis ◽  
Research Progress ◽  
Primary Data ◽  
Statistical Procedures ◽  
Graphical Displays ◽  
Primary Data Analysis ◽  
Meta Analyses ◽  
Data Analytic

Abstract. Which data to analyze, and how, are fundamental questions of all empirical research. As there are always numerous flexibilities in data-analytic decisions (a “garden of forking paths”), this poses perennial problems to all empirical research. Specification-curve analysis and multiverse analysis have recently been proposed as solutions to these issues. Building on the structural analogies between primary data analysis and meta-analysis, we transform and adapt these approaches to the meta-analytic level, in tandem with combinatorial meta-analysis. We explain the rationale of this idea, suggest descriptive and inferential statistical procedures, as well as graphical displays, provide code for meta-analytic practitioners to generate and use these, and present a fully worked real example from digit ratio (2D:4D) research, totaling 1,592 meta-analytic specifications. Specification-curve and multiverse meta-analysis holds promise to resolve conflicting meta-analyses, contested evidence, controversial empirical literatures, and polarized research, and to mitigate the associated detrimental effects of these phenomena on research progress.

The Orchard Plot: Cultivating a Forest Plot for Use in Ecology, Evolution and Beyond

2019 ◽  
Author(s):  
Shinichi Nakagawa ◽  
Malgorzata Lagisz ◽  
Rose E O'Dea ◽  
Joanna Rutkowska ◽  
Yefeng Yang ◽  
...  
Keyword(s):  
Meta Analysis ◽  
R Package ◽  
Effect Sizes ◽  
Forest Plot ◽  
Point Estimates ◽  
Aggregate Effect ◽  
The Individual ◽  
Meta Analyses ◽  
Heterogeneous Effect ◽  
Intuitive Interpretation

‘Classic’ forest plots show the effect sizes from individual studies and the aggregate effect from a meta-analysis. However, in ecology and evolution meta-analyses routinely contain over 100 effect sizes, making the classic forest plot of limited use. We surveyed 102 meta-analyses in ecology and evolution, finding that only 11% use the classic forest plot. Instead, most used a ‘forest-like plot’, showing point estimates (with 95% confidence intervals; CIs) from a series of subgroups or categories in a meta-regression. We propose a modification of the forest-like plot, which we name the ‘orchard plot’. Orchard plots, in addition to showing overall mean effects and CIs from meta-analyses/regressions, also includes 95% prediction intervals (PIs), and the individual effect sizes scaled by their precision. The PI allows the user and reader to see the range in which an effect size from a future study may be expected to fall. The PI, therefore, provides an intuitive interpretation of any heterogeneity in the data. Supplementing the PI, the inclusion of underlying effect sizes also allows the user to see any influential or outlying effect sizes. We showcase the orchard plot with example datasets from ecology and evolution, using the R package, orchard, including several functions for visualizing meta-analytic data using forest-plot derivatives. We consider the orchard plot as a variant on the classic forest plot, cultivated to the needs of meta-analysts in ecology and evolution. Hopefully, the orchard plot will prove fruitful for visualizing large collections of heterogeneous effect sizes regardless of the field of study.

Laser Influence on Dental Sensitivity Compared to Other Light Sources Used During In-office Dental Bleaching: Systematic Review and Meta-analysis

2020 ◽  
Vol 45 (6) ◽  
pp. 589-597
Author(s):  
BGS Casado ◽  
EP Pellizzer ◽  
JR Souto Maior ◽  
CAA Lemos ◽  
BCE Vasconcelos ◽  
...  
Keyword(s):  
Randomized Clinical Trials ◽  
Color Change ◽  
Meta Analysis ◽  
Present Review ◽  
Cochrane Library ◽  
Light Sources ◽  
Eligibility Criteria ◽  
Tooth Color ◽  
Significant Difference ◽  
Meta Analyses

Clinical Relevance The use of laser light during bleaching will not reduce the incidence or severity of sensitivity and will not increase the degree of color change compared with nonlaser light sources. SUMMARY Objective: To evaluate whether the use of laser during in-office bleaching promotes a reduction in dental sensitivity after bleaching compared with other light sources. Methods: The present review was conducted following the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) and is registered with PROSPERO (CDR42018096591). Searches were conducted in the PubMed/Medline, Web of Science, and Cochrane Library databases for relevant articles published up to August 2018. Only randomized clinical trials among adults that compared the use of laser during in-office whitening and other light sources were considered eligible. Results: After analysis of the texts retrieved during the database search, six articles met the eligibility criteria and were selected for the present review. For the outcome dental sensitivity, no significant difference was found favoring any type of light either for intensity (mean difference [MD]: −1.60; confidence interval [CI]: −3.42 to 0.22; p=0.09) or incidence (MD: 1.00; CI: 0.755 to 1.33; p=1.00). Regarding change in tooth color, no significant differences were found between the use of the laser and other light sources (MD: −2.22; CI: −6.36 to 1.93; p=0.29). Conclusions: Within the limitations of the present study, laser exerts no influence on tooth sensitivity compared with other light sources when used during in-office bleaching. The included studies demonstrated that laser use during in-office bleaching may have no influence on tooth color change.

Identification of and Correction for Publication Bias: Comment

2019 ◽  
Author(s):  
Amanda Kvarven ◽  
Eirik Strømland ◽  
Magnus Johannesson
Keyword(s):  
Publication Bias ◽  
False Positive ◽  
Large Scale ◽  
Meta Analysis ◽  
False Positive Rate ◽  
Effect Sizes ◽  
Replication Studies ◽  
Moderate Reduction ◽  
Positive Rate ◽  
Meta Analyses

Andrews & Kasy (2019) propose an approach for adjusting effect sizes in meta-analysis for publication bias. We use the Andrews-Kasy estimator to adjust the result of 15 meta-analyses and compare the adjusted results to 15 large-scale multiple labs replication studies estimating the same effects. The pre-registered replications provide precisely estimated effect sizes, which do not suffer from publication bias. The Andrews-Kasy approach leads to a moderate reduction of the inflated effect sizes in the meta-analyses. However, the approach still overestimates effect sizes by a factor of about two or more and has an estimated false positive rate of between 57% and 100%.

scholarly journals Application of the Subspace-Based Methods in Health Monitoring of Civil Structures: A Systematic Review and Meta-Analysis

2020 ◽  
Vol 10 (10) ◽  
pp. 3607
Author(s):  
Hoofar Shokravi ◽  
Hooman Shokravi ◽  
Norhisham Bakhary ◽  
Mahshid Heidarrezaei ◽  
Seyed Saeid Rahimian Koloor ◽  
...  
Keyword(s):  
System Identification ◽  
Systematic Reviews ◽  
Health Monitoring ◽  
Meta Analysis ◽  
Civil Structures ◽  
Literature Reviews ◽  
Research Gap ◽  
Meta Analyses ◽  
Subspace System Identification ◽  
Insight Into

A large number of research studies in structural health monitoring (SHM) have presented, extended, and used subspace system identification. However, there is a lack of research on systematic literature reviews and surveys of studies in this field. Therefore, the current study is undertaken to systematically review the literature published on the development and application of subspace system identification methods. In this regard, major databases in SHM, including Scopus, Google Scholar, and Web of Science, have been selected and preferred reporting items for systematic reviews and meta-analyses (PRISMA) has been applied to ensure complete and transparent reporting of systematic reviews. Along this line, the presented review addresses the available studies that employed subspace-based techniques in the vibration-based damage detection (VDD) of civil structures. The selected papers in this review were categorized into authors, publication year, name of journal, applied techniques, research objectives, research gap, proposed solutions and models, and findings. This study can assist practitioners and academicians for better condition assessment of structures and to gain insight into the literature.

scholarly journals Impact of Chemotherapy in the Adjuvant Setting of Early Stage Uterine Leiomyosarcoma: A Systematic Review and Updated Meta-Analysis

Cancers ◽  
2020 ◽  
Vol 12 (7) ◽  
pp. 1899
Author(s):  
Alessandro Rizzo ◽  
Margherita Nannini ◽  
Annalisa Astolfi ◽  
Valentina Indio ◽  
Pierandrea De Iaco ◽  
...  
Keyword(s):  
Systematic Review ◽  
Recurrence Rate ◽  
Locoregional Recurrence ◽  
Early Stage ◽  
Meta Analysis ◽  
Distant Recurrence ◽  
Uterine Leiomyosarcoma ◽  
Primary Surgery ◽  
Locoregional Recurrence Rate ◽  
Meta Analyses

Background: Although the use of adjuvant chemotherapy (AC) appears to be increasing over the past few years, several clinical trials and previous meta-analyses failed to determine whether AC could improve clinical outcomes in uterine leiomyosarcoma (uLMS). The aim of this systematic review and meta-analysis was to compare AC (with or without radiotherapy) versus observation (obs) after primary surgery in early stage uLMS. Materials and Methods: Randomized controlled (RCTs) and non-randomized studies (NRSs) were retrieved. Outcomes of interest were as follows: distant recurrence rate, locoregional recurrence rate and overall recurrence rate. Results about distant recurrence rate, locoregional recurrence rate and overall recurrence rate were compared by calculating odds ratios (ORs) with 95% confidence intervals (CIs); ORs were combined with Mantel–Haenszel method. Results: Nine studies were included in the analysis, involving 545 patients (AC: 252, obs: 293). Compared with obs, AC did not reduce locoregional and distant recurrence rate, with a pooled OR of 1.36 and 0.63, respectively. Similarly, administration of AC did not decrease overall recurrence rate in comparison to obs. Conclusion: According to our results, AC (with or without radiotherapy) did not decrease recurrence rate in early stage uLMS; thus, the role of AC in this setting remains unclear.

scholarly journals Systematic Review: Allogenic Use of Stromal Vascular Fraction (SVF) and Decellularized Extracellular Matrices (ECM) as Advanced Therapy Medicinal Products (ATMP) in Tissue Regeneration

2020 ◽  
Vol 21 (14) ◽  
pp. 4982 ◽  
Author(s):  
Pietro Gentile ◽  
Aris Sterodimas ◽  
Jacopo Pizzicannella ◽  
Laura Dionisi ◽  
Domenico De Fazio ◽  
...  
Keyword(s):  
Systematic Review ◽  
Meta Analysis ◽  
Stromal Vascular Fraction ◽  
Lateral Epicondylitis ◽  
Medicinal Products ◽  
Advanced Therapy Medicinal Products ◽  
Advanced Therapy ◽  
Meta Analyses ◽  
Clinical Experiments

Stromal vascular fraction (SVF) containing adipose stem cells (ASCs) has been used for many years in regenerative plastic surgery for autologous applications, without any focus on their potential allogenic role. Allogenic SVF transplants could be based on the possibility to use decellularized extracellular matrix (ECM) as a scaffold from a donor then re-cellularized by ASCs of the recipient, in order to develop the advanced therapy medicinal products (ATMP) in fully personalized clinical approaches. A systematic review of this field has been realized in accordance with the Preferred Reporting for Items for Systematic Reviews and Meta-Analyses-Protocols (PRISMA-P) guidelines. Multistep research of the PubMed, Embase, MEDLINE, Pre-MEDLINE, PsycINFO, CINAHL, Clinicaltrials.gov, Scopus database, and Cochrane databases has been conducted to identify articles and investigations on human allogenic ASCs transplant for clinical use. Of the 341 articles identified, 313 were initially assessed for eligibility on the basis of the abstract. Of these, only 29 met all the predetermined criteria for inclusion according to the PICOS (patients, intervention, comparator, outcomes, and study design) approach, and 19 have been included in quantitative synthesis (meta-analysis). Ninety-one percent of the studies previously screened (284 papers) were focused on the in vitro results and pre-clinical experiments. The allogenic use regarded the treatment of perianal fistulas, diabetic foot ulcers, knee osteoarthritis, acute respiratory distress syndrome, refractory rheumatoid arthritis, pediatrics disease, fecal incontinence, ischemic heart disease, autoimmune encephalomyelitis, lateral epicondylitis, and soft tissue defects. The information analyzed suggested the safety and efficacy of allogenic ASCs and ECM transplants without major side effects.

Sign in / Sign up

Export Citation Format

Share Document