scholarly journals Interval estimation of the overall treatment effect in random-effects meta-analyses: Recommendations from a simulation study comparing frequentist, Bayesian, and bootstrap methods

2020 ◽  
Author(s):  
Frank Weber ◽  
Guido Knapp ◽  
Anne Glass ◽  
Günther Kundt ◽  
Katja Ickstadt
Keyword(s):  
Literature Review ◽  
Random Effects ◽  
Simulation Study ◽  
Treatment Effect ◽  
Profile Likelihood ◽  
Meta Analysis ◽  
Artery Bypass ◽  
Interval Length ◽  
Interval Estimators ◽  
Meta Analyses

There exists a variety of interval estimators for the overall treatment effect in a random-effects meta-analysis. A recent literature review summarizing existing methods suggested that in most situations, the Hartung-Knapp/Sidik-Jonkman (HKSJ) method was preferable. However, a quantitative comparison of those methods in a common simulation study is still lacking. Thus, we conduct such a simulation study for continuous and binary outcomes, focusing on the medical field for application.Based on the literature review and some new theoretical considerations, a practicable number of interval estimators is selected for this comparison: the classical normal-approximation interval using the DerSimonian-Laird heterogeneity estimator, the HKSJ interval using either the Paule-Mandel or the Sidik-Jonkman heterogeneity estimator, the Skovgaard higher-order profile likelihood interval, a parametric bootstrap interval, and a Bayesian interval using different priors. We evaluate the performance measures (coverage and interval length) at specific points in the parameter space, i.e. not averaging over a prior distribution. In this sense, our study is conducted from a frequentist point of view.We confirm the main finding of the literature review, the general recommendation of the HKSJ method (here with the Sidik-Jonkman heterogeneity estimator). For meta-analyses including only 2 studies, the high length of the HKSJ interval limits its practical usage. In this case, the Bayesian interval using a weakly informative prior for the heterogeneity may help. Our recommendations are illustrated using a real-world meta-analysis dealing with the efficacy of an intramyocardial bone marrow stem cell transplantation during coronary artery bypass grafting.

USING PREDICTION INTERVALS FROM RANDOM-EFFECTS META-ANALYSES IN AN ECONOMIC MODEL

2014 ◽  
Vol 30 (1) ◽  
pp. 44-49 ◽  
Author(s):  
Conor Teljeur ◽  
Michelle O'Neill ◽  
Patrick Moran ◽  
Linda Murphy ◽  
Patricia Harrington ◽  
...  
Keyword(s):  
Random Effects ◽  
Treatment Effect ◽  
Meta Analysis ◽  
Clinical Effectiveness ◽  
Cost Utility ◽  
Prediction Intervals ◽  
Confidence Bounds ◽  
Robot Assisted ◽  
Meta Analyses ◽  
Robot Assisted Surgery

Objectives: When incorporating treatment effect estimates derived from a random-effect meta-analysis it is tempting to use the confidence bounds to determine the potential range of treatment effect. However, prediction intervals reflect the potential effect of a technology rather than the more narrowly defined average treatment effect. Using a case study of robot-assisted radical prostatectomy, this study investigates the impact on a cost-utility analysis of using clinical effectiveness derived from random-effects meta-analyses presented as confidence bounds and prediction intervals, respectively.Methods: To determine the cost-utility of robot-assisted prostatectomy, an economic model was developed. The clinical effectiveness of robot-assisted surgery compared with open and conventional laparoscopic surgery was estimated using meta-analysis of peer-reviewed publications. Assuming treatment effect would vary across studies due to both sampling variability and differences between surgical teams, random-effects meta-analysis was used to pool effect estimates.Results: Using the confidence bounds approach the mean and median ICER was €24,193 and €26,731/QALY (95%CI: €13,752 to €68,861/QALY), respectively. The prediction interval approach produced an equivalent mean and median ICER of €26,920 and €26,643/QALY (95%CI: -€135,244 to €239,166/QALY), respectively. Using prediction intervals, there is a probability of 0.042 that robot-assisted surgery will result in a net reduction in QALYs.Conclusions: Using prediction intervals rather than confidence bounds does not affect the point estimate of the treatment effect. In meta-analyses with significant heterogeneity, the use of prediction intervals will produce wider ranges of treatment effect, and hence result in greater uncertainty, but a better reflection of the effect of the technology.

scholarly journals Simulation study of estimating between-study variance and overall effect in meta-analyses of log-response-ratio for normal data

2020 ◽  
Author(s):  
Ilyas Bakbergenuly ◽  
David C. Hoaglin ◽  
Elena Kulinskaya
Keyword(s):  
Profile Likelihood ◽  
Meta Analysis ◽  
Response Ratio ◽  
Normal Distributions ◽  
Lognormal Distributions ◽  
Effect Measure ◽  
Log Response Ratio ◽  
Interval Estimators ◽  
Point Estimators ◽  
Meta Analyses

Methods for random-effects meta-analysis require an estimate of the between-study variance, $\tau^2$. The performance of estimators of $\tau^2$ (measured by bias and coverage) affects their usefulness in assessing heterogeneity of study-level effects, and also the performance of related estimators of the overall effect. For the effect measure log-response-ratio (LRR, also known as the logarithm of the ratio of means, RoM), we review four point estimators of $\tau^2$ (the popular methods of DerSimonian-Laird (DL), restricted maximum likelihood, and Mandel and Paule (MP), and the less-familiar method of Jackson), four interval estimators for $\tau^2$ (profile likelihood, Q-profile, Biggerstaff and Jackson, and Jackson), five point estimators of the overall effect (the four related to the point estimators of $\tau^2$ and an estimator whose weights use only study-level sample sizes), and seven interval estimators for the overall effect (four based on the point estimators for $\tau^2$, the Hartung-Knapp-Sidik-Jonkman (HKSJ) interval, a modification of HKSJ that uses the MP estimator of $\tau^2$ instead of the DL estimator, and an interval based on the sample-size-weighted estimator). We obtain empirical evidence from extensive simulations of data from normal distributions. Simulations from lognormal distributions are in a separate report Bakbergenuly et al. 2019b.

An Exact Test for Meta-analysis with Binary Endpoints

2007 ◽  
Vol 46 (06) ◽  
pp. 662-668 ◽  
Author(s):  
C. Gromann ◽  
O. Kuss
Keyword(s):  
Random Effects ◽  
Fixed Effects ◽  
Meta Analysis ◽  
Artery Bypass ◽  
Sparse Data ◽  
Random Effects Model ◽  
Exact Test ◽  
Standard Methods ◽  
Meta Analyses ◽  
Empirical Size

Summary Objectives : We reintroduce an exact Mantel-Haenszel (MH) procedure for meta-analysis with binary endpoints which is expected to workespeciallywell i sparse data, e.g., in meta-analyses of safety or adverse events. Methods : The performance of the exact MH procedure in terms of empirical size and power is compared to the asymptotic MH and to the two standard procedures (fixed effects and random effects model) in a simulation study. We illustrate the methods with a metaanalysis of postoperative stroke occurrence after offpump or on-pump surgery in coronary artery bypass grafting. Results : We find that in almost all situations the asymptotic MH procedure outperforms its competitors; especially the standard methods yield poor results in terms of power and size. Conclusions : There is no need to use the reintroduced exact MH procedure; the asymptotic MH procedure will be sufficient in most practical situations. The standard methods (fixed effects and random effects model) should not be used in the sparse data situation.

scholarly journals Beta-binomial models for meta-analysis with binary outcomes: Variations, extensions, and additional insights from econometrics

2021 ◽  
pp. 263208432199622
Author(s):  
Tim Mathes ◽  
Oliver Kuss
Keyword(s):  
Simulation Study ◽  
Count Data ◽  
Negative Binomial ◽  
Meta Analysis ◽  
Negative Binomial Regression ◽  
Binary Outcomes ◽  
Small Scale ◽  
Panel Count Data ◽  
Count Data Models ◽  
Meta Analyses

Background Meta-analysis of systematically reviewed studies on interventions is the cornerstone of evidence based medicine. In the following, we will introduce the common-beta beta-binomial (BB) model for meta-analysis with binary outcomes and elucidate its equivalence to panel count data models. Methods We present a variation of the standard “common-rho” BB (BBST model) for meta-analysis, namely a “common-beta” BB model. This model has an interesting connection to fixed-effect negative binomial regression models (FE-NegBin) for panel count data. Using this equivalence, it is possible to estimate an extension of the FE-NegBin with an additional multiplicative overdispersion term (RE-NegBin), while preserving a closed form likelihood. An advantage due to the connection to econometric models is, that the models can be easily implemented because “standard” statistical software for panel count data can be used. We illustrate the methods with two real-world example datasets. Furthermore, we show the results of a small-scale simulation study that compares the new models to the BBST. The input parameters of the simulation were informed by actually performed meta-analysis. Results In both example data sets, the NegBin, in particular the RE-NegBin showed a smaller effect and had narrower 95%-confidence intervals. In our simulation study, median bias was negligible for all methods, but the upper quartile for median bias suggested that BBST is most affected by positive bias. Regarding coverage probability, BBST and the RE-NegBin model outperformed the FE-NegBin model. Conclusion For meta-analyses with binary outcomes, the considered common-beta BB models may be valuable extensions to the family of BB models.

scholarly journals When does the use of individual patient data in network meta-analysis make a difference? A simulation study

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Steve Kanters ◽  
Mohammad Ehsanul Karim ◽  
Kristian Thorlund ◽  
Aslam Anis ◽  
Nick Bansback
Keyword(s):  
Simulation Study ◽  
Treatment Effect ◽  
Mean Squared Error ◽  
Individual Patient Data ◽  
Meta Analysis ◽  
Model Performance ◽  
Patient Data ◽  
Meta Regression ◽  
Treatment Comparisons ◽  
The Impact

Abstract Background The use of individual patient data (IPD) in network meta-analyses (NMA) is rapidly growing. This study aimed to determine, through simulations, the impact of select factors on the validity and precision of NMA estimates when combining IPD and aggregate data (AgD) relative to using AgD only. Methods Three analysis strategies were compared via simulations: 1) AgD NMA without adjustments (AgD-NMA); 2) AgD NMA with meta-regression (AgD-NMA-MR); and 3) IPD-AgD NMA with meta-regression (IPD-NMA). We compared 108 parameter permutations: number of network nodes (3, 5 or 10); proportion of treatment comparisons informed by IPD (low, medium or high); equal size trials (2-armed with 200 patients per arm) or larger IPD trials (500 patients per arm); sparse or well-populated networks; and type of effect-modification (none, constant across treatment comparisons, or exchangeable). Data were generated over 200 simulations for each combination of parameters, each using linear regression with Normal distributions. To assess model performance and estimate validity, the mean squared error (MSE) and bias of treatment-effect and covariate estimates were collected. Standard errors (SE) and percentiles were used to compare estimate precision. Results Overall, IPD-NMA performed best in terms of validity and precision. The median MSE was lower in the IPD-NMA in 88 of 108 scenarios (similar results otherwise). On average, the IPD-NMA median MSE was 0.54 times the median using AgD-NMA-MR. Similarly, the SEs of the IPD-NMA treatment-effect estimates were 1/5 the size of AgD-NMA-MR SEs. The magnitude of superior validity and precision of using IPD-NMA varied across scenarios and was associated with the amount of IPD. Using IPD in small or sparse networks consistently led to improved validity and precision; however, in large/dense networks IPD tended to have negligible impact if too few IPD were included. Similar results also apply to the meta-regression coefficient estimates. Conclusions Our simulation study suggests that the use of IPD in NMA will considerably improve the validity and precision of estimates of treatment effect and regression coefficients in the most NMA IPD data-scenarios. However, IPD may not add meaningful validity and precision to NMAs of large and dense treatment networks when negligible IPD are used.

Meta-analysis of effect sizes reported at multiple time points: A multivariate approach

2012 ◽  
Vol 9 (5) ◽  
pp. 610-620 ◽  
Author(s):  
Thomas A Trikalinos ◽  
Ingram Olkin
Keyword(s):  
Random Effects ◽  
Treatment Effects ◽  
Multivariate Analyses ◽  
Meta Analysis ◽  
Effect Sizes ◽  
Multiple Time ◽  
Time Points ◽  
Fixed And Random Effects ◽  
Multiple Time Points ◽  
Meta Analyses

Background Many comparative studies report results at multiple time points. Such data are correlated because they pertain to the same patients, but are typically meta-analyzed as separate quantitative syntheses at each time point, ignoring the correlations between time points. Purpose To develop a meta-analytic approach that estimates treatment effects at successive time points and takes account of the stochastic dependencies of those effects. Methods We present both fixed and random effects methods for multivariate meta-analysis of effect sizes reported at multiple time points. We provide formulas for calculating the covariance (and correlations) of the effect sizes at successive time points for four common metrics (log odds ratio, log risk ratio, risk difference, and arcsine difference) based on data reported in the primary studies. We work through an example of a meta-analysis of 17 randomized trials of radiotherapy and chemotherapy versus radiotherapy alone for the postoperative treatment of patients with malignant gliomas, where in each trial survival is assessed at 6, 12, 18, and 24 months post randomization. We also provide software code for the main analyses described in the article. Results We discuss the estimation of fixed and random effects models and explore five options for the structure of the covariance matrix of the random effects. In the example, we compare separate (univariate) meta-analyses at each of the four time points with joint analyses across all four time points using the proposed methods. Although results of univariate and multivariate analyses are generally similar in the example, there are small differences in the magnitude of the effect sizes and the corresponding standard errors. We also discuss conditional multivariate analyses where one compares treatment effects at later time points given observed data at earlier time points. Limitations Simulation and empirical studies are needed to clarify the gains of multivariate analyses compared with separate meta-analyses under a variety of conditions. Conclusions Data reported at multiple time points are multivariate in nature and are efficiently analyzed using multivariate methods. The latter are an attractive alternative or complement to performing separate meta-analyses.

Alternative Tendon Coaptations to the Pulvertaft Weave Technique: A Systematic Review and Meta-Analysis of Biomechanical Studies

Hand ◽  
2021 ◽  
pp. 155894472110432
Author(s):  
Emily M. Graham ◽  
Jeremie D. Oliver ◽  
Russell Hendrycks ◽  
Dino Maglic ◽  
Shaun D. Mendenhall
Keyword(s):  
Systematic Review ◽  
Yield Strength ◽  
Random Effects ◽  
Meta Analysis ◽  
Dichotomous Data ◽  
Inverse Variance ◽  
Significant Difference ◽  
Alternative Techniques ◽  
Meta Analyses

Background The Pulvertaft weave technique (PT) is frequently used during tendon repairs and transfers. However, this technique is associated with limitations. In this systematic review and meta-analysis, quantitative and qualitative analyses were performed on in vitro, biomechanical studies that compared the PT with alternative techniques. Methods Articles included for qualitative and/or qualitative analysis were identified following Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Studies included in the meta-analysis were analyzed either as continuous data with inverse variance and random effects or as dichotomous data using a Mantel-Haenszel analysis assuming random effects to calculate an odds ratio. Results A comprehensive electronic search yielded 8 studies meeting inclusion criteria for meta-analysis. Two studies with a total of 65 tendon coaptations demonstrated no significant difference in strength between the PT and traditional side-to-side (STS) techniques ( P = .92). Two studies with a total of 43 tendon coaptations showed that the STS with 1 weave has a higher yield strength than the PT ( P = .03). Two studies with a total of 62 tendon repairs demonstrated no significant difference in strength between the PT and the step-cut (SC) techniques ( P = .70). The final 2 studies included 46 tendon repairs and demonstrated that the wrap around (WA) technique has a higher yield strength than the PT ( P < .001). Conclusions The STS, SC, and WA techniques are preferred for improving tendon form. The STS and WA techniques have superior yield strengths than the PT, and the SC technique withstands similar stress to failure as the PT.

What do we know about meta-analyses in accounting, auditing, and corporate governance?

2019 ◽  
Vol 27 (1) ◽  
pp. 17-43 ◽  
Author(s):  
Patrick Velte
Keyword(s):  
Corporate Governance ◽  
Literature Review ◽  
Quantitative Research ◽  
Management Accounting ◽  
Meta Analysis ◽  
Financial Accounting ◽  
Content Type ◽  
Moderating Variables ◽  
Independent Variable ◽  
Meta Analyses

PurposeThe purpose of this paper is to summarize the key findings of meta-analyses in accounting, auditing and corporate governance research.Design/methodology/approachA structured literature review on 63 meta-analyses related to financial accounting, management accounting, auditing, and corporate governance is conducted.FindingsMost of the previous meta-analyses have been focused on the topic of corporate governance with a clear focus on board composition as an independent variable and by integrating research- and country-specific moderating variables. Meta-analyses in management accounting are rare; furthermore, no meta-analysis in tax accounting has been published yet.Originality/valueMeta-analyses on these topics are expected to become increasingly important in the following years because of the increased number of empirical quantitative research. This literature review gives useful insights for research, practice and regulation.

Meta-Analysis of Meta-Analyses in Communication: Comparing Fixed Effects and Random Effects Analysis Models

2010 ◽  
Vol 58 (3) ◽  
pp. 257-278 ◽  
Author(s):  
Ashley Anker ◽  
Amber Marie Reinhart ◽  
Thomas Hugh Feeley
Keyword(s):  
Random Effects ◽  
Fixed Effects ◽  
Meta Analysis ◽  
Meta Analyses ◽  
Analysis Models
Sign in / Sign up

Export Citation Format

Share Document