scholarly journals Applying meta-analysis to research on bilingualism: An introduction

2021 ◽  
pp. 1-6
Author(s):  
Luke Plonsky ◽  
Ekaterina Sudina ◽  
Yuhang Hu
Keyword(s):  
Meta Analysis ◽  
Full Length ◽  
Analytic Approach ◽  
Coding Scheme ◽  
Domain Definition ◽  
Scientific Disciplines ◽  
Literature Reviews ◽  
Meta Analyses

Abstract Meta-analysis overcomes a number of the limitations of traditional literature reviews (Norris & Ortega, 2006). Consequently, the use of meta-analysis as a synthetic technique has been applied across a range of scientific disciplines in recent decades. This paper seeks to formally introduce the potential of meta-analysis to the field of bilingualism. In doing so, we first describe a number of advantages to the meta-analytic approach such as greater systematicity, objectivity, and transparency relative to narrative reviews. We also outline the major stages in conducting a meta-analysis, highlighting critical considerations encountered at each stage. These include (a) domain definition, (b) coding scheme development and implementation, (c) analysis, and (d) interpretation. The focus, however, is on providing a conceptual introduction rather than a full-length tutorial. Meta-analyses in bilingualism and nearby fields are referred to throughout in order to illustrate the points being made.

scholarly journals Application of the Subspace-Based Methods in Health Monitoring of Civil Structures: A Systematic Review and Meta-Analysis

2020 ◽  
Vol 10 (10) ◽  
pp. 3607
Author(s):  
Hoofar Shokravi ◽  
Hooman Shokravi ◽  
Norhisham Bakhary ◽  
Mahshid Heidarrezaei ◽  
Seyed Saeid Rahimian Koloor ◽  
...  
Keyword(s):  
System Identification ◽  
Systematic Reviews ◽  
Health Monitoring ◽  
Meta Analysis ◽  
Civil Structures ◽  
Literature Reviews ◽  
Research Gap ◽  
Meta Analyses ◽  
Subspace System Identification ◽  
Insight Into

A large number of research studies in structural health monitoring (SHM) have presented, extended, and used subspace system identification. However, there is a lack of research on systematic literature reviews and surveys of studies in this field. Therefore, the current study is undertaken to systematically review the literature published on the development and application of subspace system identification methods. In this regard, major databases in SHM, including Scopus, Google Scholar, and Web of Science, have been selected and preferred reporting items for systematic reviews and meta-analyses (PRISMA) has been applied to ensure complete and transparent reporting of systematic reviews. Along this line, the presented review addresses the available studies that employed subspace-based techniques in the vibration-based damage detection (VDD) of civil structures. The selected papers in this review were categorized into authors, publication year, name of journal, applied techniques, research objectives, research gap, proposed solutions and models, and findings. This study can assist practitioners and academicians for better condition assessment of structures and to gain insight into the literature.

Domain definition and search techniques in meta-analyses of L2 research (Or why 18 meta-analyses of feedback have different results)

2014 ◽  
Vol 31 (2) ◽  
pp. 267-278 ◽  
Author(s):  
Luke Plonsky ◽  
Dan Brown
Keyword(s):  
Quantitative Research ◽  
Meta Analysis ◽  
Critical Role ◽  
Corrective Feedback ◽  
Search Techniques ◽  
Domain Definition ◽  
Analytic Process ◽  
Unpublished Studies ◽  
Unique Approach ◽  
Meta Analyses

Applied linguists have turned increasingly in recent years to meta-analysis as the preferred means of synthesizing quantitative research. The first step in the meta-analytic process involves defining a domain of interest. Despite its apparent simplicity, this step involves a great deal of subjectivity on the part of the meta-analyst. This article problematizes the importance of clearly defining and operationalizing meta-analytic domains. Toward that end, we present a critical review of one particular domain, corrective feedback, which has been subject to 18 unique meta-analyses. Specifically, we examine the unique approach each study has taken in defining their domain of interest. In order to demonstrate the critical role of this stage in the meta-analytic process, we also examine variability in summary effects as a function of the unique subdomains in the sample. Because different techniques used to identify candidate studies carry assumptions about the type of research that falls within the domain of interest (e.g. published vs. unpublished), we also include a brief review of search techniques employed in a set of 81 meta-analyses of second language research. Building on the work of In’nami and Koizumi (2010) and Oswald and Plonsky (2010), the results for this phase of the analysis show that L2 meta-analysts generally rely on a stable but very limited set of search strategies, none of which is likely to yield unpublished studies. Based on our findings related both to domain definition and search techniques employed by L2 researchers, we make specific recommendations for future meta-analytic practice in the field.

scholarly journals The median and the mode as robust meta-analysis methods in the presence of small study effects

2018 ◽  
Author(s):  
Fernando Pires Hartwig ◽  
George Davey Smith ◽  
Amand Floriaan Schmidt ◽  
Jack Bowden
Keyword(s):  
Meta Analysis ◽  
Small Study ◽  
Sensitivity Analyses ◽  
Literature Reviews ◽  
Detailed Simulation ◽  
The Mean ◽  
Modelling Assumptions ◽  
Meta Analyses ◽  
Based Medicine

AbstractMeta-analyses based on systematic literature reviews are commonly used to obtain a quantitative summary of the available evidence on a given topic. Despite its attractive simplicity, and its established position at the summit of the evidence-based medicine hierarchy, the reliability of any meta-analysis is largely constrained by the quality of its constituent studies. One major limitation is small study effects, whose presence can often easily be detected, but not so easily adjusted for. Here, robust methods of estimation based on the median and mode are proposed as tools to increase the reliability of findings in a meta-analysis. By re-examining data from published meta-analyses, and by conducting a detailed simulation study, we show that these two simple methods offer notable robustness to a range of plausible bias mechanisms, without making any explicit modelling assumptions. In conclusion, when performing a meta-analysis with suspected small study effects, we recommend reporting the mean, median and modal pooled estimates as a simple but informative sensitivity analyses.

A Meta-Analysis of Interviews and Cognitive Ability

2013 ◽  
Vol 12 (4) ◽  
pp. 157-169 ◽  
Author(s):  
Philip L. Roth ◽  
Allen I. Huffcutt
Keyword(s):  
Cognitive Ability ◽  
Meta Analysis ◽  
Incremental Validity ◽  
Employment Interviews ◽  
Moderator Analysis ◽  
The Future ◽  
Research Questions ◽  
Meta Analyses ◽  
The Relationship

The topic of what interviews measure has received a great deal of attention over the years. One line of research has investigated the relationship between interviews and the construct of cognitive ability. A previous meta-analysis reported an overall corrected correlation of .40 ( Huffcutt, Roth, & McDaniel, 1996 ). A more recent meta-analysis reported a noticeably lower corrected correlation of .27 ( Berry, Sackett, & Landers, 2007 ). After reviewing both meta-analyses, it appears that the two studies posed different research questions. Further, there were a number of coding judgments in Berry et al. that merit review, and there was no moderator analysis for educational versus employment interviews. As a result, we reanalyzed the work by Berry et al. and found a corrected correlation of .42 for employment interviews (.15 higher than Berry et al., a 56% increase). Further, educational interviews were associated with a corrected correlation of .21, supporting their influence as a moderator. We suggest a better estimate of the correlation between employment interviews and cognitive ability is .42, and this takes us “back to the future” in that the better overall estimate of the employment interviews – cognitive ability relationship is roughly .40. This difference has implications for what is being measured by interviews and their incremental validity.

Power-Enhanced Funnel Plots for Meta-Analysis

2020 ◽  
Vol 228 (1) ◽  
pp. 43-49 ◽  
Author(s):  
Michael Kossmeier ◽  
Ulrich S. Tran ◽  
Martin Voracek
Keyword(s):  
Statistical Power ◽  
Meta Analysis ◽  
Relevant Information ◽  
Selective Reporting ◽  
Graphical Display ◽  
Funnel Plot ◽  
Evidential Value ◽  
Graphical Displays ◽  
Funnel Plots ◽  
Meta Analyses

Abstract. Currently, dedicated graphical displays to depict study-level statistical power in the context of meta-analysis are unavailable. Here, we introduce the sunset (power-enhanced) funnel plot to visualize this relevant information for assessing the credibility, or evidential value, of a set of studies. The sunset funnel plot highlights the statistical power of primary studies to detect an underlying true effect of interest in the well-known funnel display with color-coded power regions and a second power axis. This graphical display allows meta-analysts to incorporate power considerations into classic funnel plot assessments of small-study effects. Nominally significant, but low-powered, studies might be seen as less credible and as more likely being affected by selective reporting. We exemplify the application of the sunset funnel plot with two published meta-analyses from medicine and psychology. Software to create this variation of the funnel plot is provided via a tailored R function. In conclusion, the sunset (power-enhanced) funnel plot is a novel and useful graphical display to critically examine and to present study-level power in the context of meta-analysis.

Which Data to Meta-Analyze, and How?

2019 ◽  
Vol 227 (1) ◽  
pp. 64-82 ◽  
Author(s):  
Martin Voracek ◽  
Michael Kossmeier ◽  
Ulrich S. Tran
Keyword(s):  
Empirical Research ◽  
Meta Analysis ◽  
Curve Analysis ◽  
Research Progress ◽  
Primary Data ◽  
Statistical Procedures ◽  
Graphical Displays ◽  
Primary Data Analysis ◽  
Meta Analyses ◽  
Data Analytic

Abstract. Which data to analyze, and how, are fundamental questions of all empirical research. As there are always numerous flexibilities in data-analytic decisions (a “garden of forking paths”), this poses perennial problems to all empirical research. Specification-curve analysis and multiverse analysis have recently been proposed as solutions to these issues. Building on the structural analogies between primary data analysis and meta-analysis, we transform and adapt these approaches to the meta-analytic level, in tandem with combinatorial meta-analysis. We explain the rationale of this idea, suggest descriptive and inferential statistical procedures, as well as graphical displays, provide code for meta-analytic practitioners to generate and use these, and present a fully worked real example from digit ratio (2D:4D) research, totaling 1,592 meta-analytic specifications. Specification-curve and multiverse meta-analysis holds promise to resolve conflicting meta-analyses, contested evidence, controversial empirical literatures, and polarized research, and to mitigate the associated detrimental effects of these phenomena on research progress.

The Orchard Plot: Cultivating a Forest Plot for Use in Ecology, Evolution and Beyond

2019 ◽  
Author(s):  
Shinichi Nakagawa ◽  
Malgorzata Lagisz ◽  
Rose E O'Dea ◽  
Joanna Rutkowska ◽  
Yefeng Yang ◽  
...  
Keyword(s):  
Meta Analysis ◽  
R Package ◽  
Effect Sizes ◽  
Forest Plot ◽  
Point Estimates ◽  
Aggregate Effect ◽  
The Individual ◽  
Meta Analyses ◽  
Heterogeneous Effect ◽  
Intuitive Interpretation

‘Classic’ forest plots show the effect sizes from individual studies and the aggregate effect from a meta-analysis. However, in ecology and evolution meta-analyses routinely contain over 100 effect sizes, making the classic forest plot of limited use. We surveyed 102 meta-analyses in ecology and evolution, finding that only 11% use the classic forest plot. Instead, most used a ‘forest-like plot’, showing point estimates (with 95% confidence intervals; CIs) from a series of subgroups or categories in a meta-regression. We propose a modification of the forest-like plot, which we name the ‘orchard plot’. Orchard plots, in addition to showing overall mean effects and CIs from meta-analyses/regressions, also includes 95% prediction intervals (PIs), and the individual effect sizes scaled by their precision. The PI allows the user and reader to see the range in which an effect size from a future study may be expected to fall. The PI, therefore, provides an intuitive interpretation of any heterogeneity in the data. Supplementing the PI, the inclusion of underlying effect sizes also allows the user to see any influential or outlying effect sizes. We showcase the orchard plot with example datasets from ecology and evolution, using the R package, orchard, including several functions for visualizing meta-analytic data using forest-plot derivatives. We consider the orchard plot as a variant on the classic forest plot, cultivated to the needs of meta-analysts in ecology and evolution. Hopefully, the orchard plot will prove fruitful for visualizing large collections of heterogeneous effect sizes regardless of the field of study.

Laser Influence on Dental Sensitivity Compared to Other Light Sources Used During In-office Dental Bleaching: Systematic Review and Meta-analysis

2020 ◽  
Vol 45 (6) ◽  
pp. 589-597
Author(s):  
BGS Casado ◽  
EP Pellizzer ◽  
JR Souto Maior ◽  
CAA Lemos ◽  
BCE Vasconcelos ◽  
...  
Keyword(s):  
Randomized Clinical Trials ◽  
Color Change ◽  
Meta Analysis ◽  
Present Review ◽  
Cochrane Library ◽  
Light Sources ◽  
Eligibility Criteria ◽  
Tooth Color ◽  
Significant Difference ◽  
Meta Analyses

Clinical Relevance The use of laser light during bleaching will not reduce the incidence or severity of sensitivity and will not increase the degree of color change compared with nonlaser light sources. SUMMARY Objective: To evaluate whether the use of laser during in-office bleaching promotes a reduction in dental sensitivity after bleaching compared with other light sources. Methods: The present review was conducted following the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) and is registered with PROSPERO (CDR42018096591). Searches were conducted in the PubMed/Medline, Web of Science, and Cochrane Library databases for relevant articles published up to August 2018. Only randomized clinical trials among adults that compared the use of laser during in-office whitening and other light sources were considered eligible. Results: After analysis of the texts retrieved during the database search, six articles met the eligibility criteria and were selected for the present review. For the outcome dental sensitivity, no significant difference was found favoring any type of light either for intensity (mean difference [MD]: −1.60; confidence interval [CI]: −3.42 to 0.22; p=0.09) or incidence (MD: 1.00; CI: 0.755 to 1.33; p=1.00). Regarding change in tooth color, no significant differences were found between the use of the laser and other light sources (MD: −2.22; CI: −6.36 to 1.93; p=0.29). Conclusions: Within the limitations of the present study, laser exerts no influence on tooth sensitivity compared with other light sources when used during in-office bleaching. The included studies demonstrated that laser use during in-office bleaching may have no influence on tooth color change.

Identification of and Correction for Publication Bias: Comment

2019 ◽  
Author(s):  
Amanda Kvarven ◽  
Eirik Strømland ◽  
Magnus Johannesson
Keyword(s):  
Publication Bias ◽  
False Positive ◽  
Large Scale ◽  
Meta Analysis ◽  
False Positive Rate ◽  
Effect Sizes ◽  
Replication Studies ◽  
Moderate Reduction ◽  
Positive Rate ◽  
Meta Analyses

Andrews & Kasy (2019) propose an approach for adjusting effect sizes in meta-analysis for publication bias. We use the Andrews-Kasy estimator to adjust the result of 15 meta-analyses and compare the adjusted results to 15 large-scale multiple labs replication studies estimating the same effects. The pre-registered replications provide precisely estimated effect sizes, which do not suffer from publication bias. The Andrews-Kasy approach leads to a moderate reduction of the inflated effect sizes in the meta-analyses. However, the approach still overestimates effect sizes by a factor of about two or more and has an estimated false positive rate of between 57% and 100%.

scholarly journals Impact of Chemotherapy in the Adjuvant Setting of Early Stage Uterine Leiomyosarcoma: A Systematic Review and Updated Meta-Analysis

Cancers ◽  
2020 ◽  
Vol 12 (7) ◽  
pp. 1899
Author(s):  
Alessandro Rizzo ◽  
Margherita Nannini ◽  
Annalisa Astolfi ◽  
Valentina Indio ◽  
Pierandrea De Iaco ◽  
...  
Keyword(s):  
Systematic Review ◽  
Recurrence Rate ◽  
Locoregional Recurrence ◽  
Early Stage ◽  
Meta Analysis ◽  
Distant Recurrence ◽  
Uterine Leiomyosarcoma ◽  
Primary Surgery ◽  
Locoregional Recurrence Rate ◽  
Meta Analyses

Background: Although the use of adjuvant chemotherapy (AC) appears to be increasing over the past few years, several clinical trials and previous meta-analyses failed to determine whether AC could improve clinical outcomes in uterine leiomyosarcoma (uLMS). The aim of this systematic review and meta-analysis was to compare AC (with or without radiotherapy) versus observation (obs) after primary surgery in early stage uLMS. Materials and Methods: Randomized controlled (RCTs) and non-randomized studies (NRSs) were retrieved. Outcomes of interest were as follows: distant recurrence rate, locoregional recurrence rate and overall recurrence rate. Results about distant recurrence rate, locoregional recurrence rate and overall recurrence rate were compared by calculating odds ratios (ORs) with 95% confidence intervals (CIs); ORs were combined with Mantel–Haenszel method. Results: Nine studies were included in the analysis, involving 545 patients (AC: 252, obs: 293). Compared with obs, AC did not reduce locoregional and distant recurrence rate, with a pooled OR of 1.36 and 0.63, respectively. Similarly, administration of AC did not decrease overall recurrence rate in comparison to obs. Conclusion: According to our results, AC (with or without radiotherapy) did not decrease recurrence rate in early stage uLMS; thus, the role of AC in this setting remains unclear.

Sign in / Sign up

Export Citation Format

Share Document