scholarly journals Meta-Analysis with Robust Variance Estimation: Expanding the Range of Working Models

2020 ◽  
Author(s):  
James E Pustejovsky ◽  
Elizabeth Tipton
Keyword(s):  
Variance Estimation ◽  
Meta Analysis ◽  
Estimation Methods ◽  
Single Type ◽  
Dependence Structure ◽  
Dependent Effect ◽  
Working Models ◽  
Robust Variance ◽  
Robust Variance Estimation ◽  
Meta Regression

In prevention science and related fields, large meta-analyses are common, and these analyses often involve dependent effect size estimates. Robust variance estimation (RVE) methods provide a way to include all dependent effect sizes in a single meta-regression model, even when the nature of the dependence is unknown. RVE uses a working model of the dependence structure, but the two currently available working models are limited to each describing a single type of dependence. Drawing on flexible tools from multivariate meta-analysis, this paper describes an expanded range of working models, along with accompanying estimation methods, which offer benefits in terms of better capturing the types of data structures that occur in practice and improving the efficiency of meta-regression estimates. We describe how the methods can be implemented using existing software (the ‘metafor’ and ‘clubSandwich’ packages for R) and illustrate the approach in a meta-analysis of randomized trials examining the effects of brief alcohol interventions for adolescents and young adults.

Listening Ears or Reading Eyes: A Meta-Analysis of Reading and Listening Comprehension Comparisons

2021 ◽  
pp. 003465432110608
Author(s):  
Virginia Clinton-Lisell
Keyword(s):  
Listening Comprehension ◽  
Variance Estimation ◽  
Meta Analysis ◽  
Age Groups ◽  
Robust Variance ◽  
Reading Condition ◽  
P 16 ◽  
Robust Variance Estimation ◽  
Literal Comprehension

In this study, a meta-analysis of reading and listening comprehension comparisons across age groups was conducted. Based on robust variance estimation (46 studies; N = 4,687), the overall difference between reading and listening comprehension was not reliably different (g = 0.07, p = .23). Reading was beneficial over listening when the reading condition was self-paced (g = 0.13, p = .049) rather than experimenter-paced (g = −0.32, p = .16). Reading also had a benefit when inferential and general comprehension rather than literal comprehension was assessed (g = 0.36, p = .02; g = .15, p = .05; g = −0.01, p = .93, respectively). There was some indication that reading and listening were more similar in languages with transparent orthographies than opaque orthographies (g = 0.001, p = .99; g = 0.10, p = .19, respectively). The findings may be used to inform theories of comprehension about modality influences in that both lower-level skill and affordances vary comparisons of reading and listening comprehension. Moreover, the findings may guide choices of modality; however, both audio and written options are needed for accessible instruction.

No Evidence of Creative Benefit Accompanying Dyslexia: A Meta-Analysis

2021 ◽  
pp. 002221942110103
Author(s):  
Florina Erbeli ◽  
Peng Peng ◽  
Marianne Rice
Keyword(s):  
Random Effects ◽  
Variance Estimation ◽  
Meta Analysis ◽  
Robust Variance ◽  
Summary Effect ◽  
Significant Performance ◽  
Mean And Variance ◽  
Robust Variance Estimation ◽  
The Mean ◽  
Creativity Task

Research on the question of creative benefit accompanying dyslexia has produced conflicting findings. In this meta-analysis, we determined summary effects of mean and variance differences in creativity between groups with and without dyslexia. Twenty studies were included ( n = 770 individuals with dyslexia, n = 1,671 controls). A random-effects robust variance estimation (RVE) analysis indicated no mean ( g = −0.02, p = .84) or variance differences ( g = −0.0004, p = .99) in creativity between groups. The mean summary effect was moderated by age, gender, and creativity domain. Compared with adolescents, adults with dyslexia showed an advantage over nondyslexic adults in creativity. In addition, a higher proportion of males in the dyslexia group was associated with poorer performance compared with the controls. Finally, the dyslexia group showed a significant performance disadvantage in verbal versus figural creativity. Regarding variance differences, they varied across age and creativity domains. Compared with adults, adolescents showed smaller variability in the dyslexia group. If the creativity task measured verbal versus figural or combined creativity, then the dyslexia group exhibited smaller variability. Altogether, our results suggest that individuals with dyslexia as a group are no more creative or show greater variability in creativity than peers without dyslexia.

scholarly journals Multi-level meta-analysis of single-case experimental designs using robust variance estimation

2021 ◽  
Author(s):  
Man Chen ◽  
James E Pustejovsky
Keyword(s):  
Effect Size ◽  
Variance Estimation ◽  
Single Case ◽  
Meta Analysis ◽  
Data Series ◽  
Robust Variance ◽  
Auto Correlation ◽  
Robust Variance Estimation ◽  
Multi Level ◽  
Size Estimates

Single-case experimental designs (SCEDs) are used to study the effects of interventions on the behavior of individual cases, by making comparisons between repeated measurements of an outcome under different conditions. In research areas where SCEDs are prevalent, there is a need for methods to synthesize results across multiple studies. One approach to synthesis uses a multi-level meta-analysis (MLMA) model to describe the distribution of effect sizes across studies and across cases within studies. However, MLMA relies on having accurate sampling variances of effect size estimates for each case, which may not be possible due to auto-correlation in the raw data series. One possible solution is to combine MLMA with robust variance estimation (RVE), which provides valid assessments of uncertainty even if the sampling variances of effect size estimates are inaccurate. Another possible solution is to forgo MLMA and use simpler, ordinary least squares (OLS) methods, with RVE. This study evaluates the performance of effect size estimators and methods of synthesizing SCEDs in the presence of auto-correlation, for several different effect size metrics, via a Monte Carlo simulation designed to emulate the features of real data series. Results demonstrate that the MLMA model with RVE performs properly in terms of bias, accuracy, and confidence interval coverage for estimating overall average log response ratios. The OLS estimator corrected with RVE performs the best in estimating overall average Tau effect sizes. None of the available methods perform adequately for meta-analysis of within-case standardized mean differences.

Sign in / Sign up

Export Citation Format

Share Document