scholarly journals Simulation study of estimating between-study variance and overall effect in meta-analyses of log-response-ratio for normal data

2020 ◽  
Author(s):  
Ilyas Bakbergenuly ◽  
David C. Hoaglin ◽  
Elena Kulinskaya
Keyword(s):  
Profile Likelihood ◽  
Meta Analysis ◽  
Response Ratio ◽  
Normal Distributions ◽  
Lognormal Distributions ◽  
Effect Measure ◽  
Log Response Ratio ◽  
Interval Estimators ◽  
Point Estimators ◽  
Meta Analyses

Methods for random-effects meta-analysis require an estimate of the between-study variance, $\tau^2$. The performance of estimators of $\tau^2$ (measured by bias and coverage) affects their usefulness in assessing heterogeneity of study-level effects, and also the performance of related estimators of the overall effect. For the effect measure log-response-ratio (LRR, also known as the logarithm of the ratio of means, RoM), we review four point estimators of $\tau^2$ (the popular methods of DerSimonian-Laird (DL), restricted maximum likelihood, and Mandel and Paule (MP), and the less-familiar method of Jackson), four interval estimators for $\tau^2$ (profile likelihood, Q-profile, Biggerstaff and Jackson, and Jackson), five point estimators of the overall effect (the four related to the point estimators of $\tau^2$ and an estimator whose weights use only study-level sample sizes), and seven interval estimators for the overall effect (four based on the point estimators for $\tau^2$, the Hartung-Knapp-Sidik-Jonkman (HKSJ) interval, a modification of HKSJ that uses the MP estimator of $\tau^2$ instead of the DL estimator, and an interval based on the sample-size-weighted estimator). We obtain empirical evidence from extensive simulations of data from normal distributions. Simulations from lognormal distributions are in a separate report Bakbergenuly et al. 2019b.

scholarly journals Estimation in meta-analyses of response ratios

2020 ◽  
Vol 20 (1) ◽  
Author(s):  
Ilyas Bakbergenuly ◽  
David C. Hoaglin ◽  
Elena Kulinskaya
Keyword(s):  
Meta Analysis ◽  
Weighted Average ◽  
Effective Sample Size ◽  
Logarithmic Scale ◽  
Response Ratio ◽  
Effect Measure ◽  
Interval Estimators ◽  
Point Estimators ◽  
Meta Analyses ◽  
And Control

Abstract Background For outcomes that studies report as the means in the treatment and control groups, some medical applications and nearly half of meta-analyses in ecology express the effect as the ratio of means (RoM), also called the response ratio (RR), analyzed in the logarithmic scale as the log-response-ratio, LRR. Methods In random-effects meta-analysis of LRR, with normal and lognormal data, we studied the performance of estimators of the between-study variance, τ2, (measured by bias and coverage) in assessing heterogeneity of study-level effects, and also the performance of related estimators of the overall effect in the log scale, λ. We obtained additional empirical evidence from two examples. Results The results of our extensive simulations showed several challenges in using LRR as an effect measure. Point estimators of τ2 had considerable bias or were unreliable, and interval estimators of τ2 seldom had the intended 95% coverage for small to moderate-sized samples (n<40). Results for estimating λ differed between lognormal and normal data. Conclusions For lognormal data, we can recommend only SSW, a weighted average in which a study’s weight is proportional to its effective sample size, (when n≥40) and its companion interval (when n≥10). Normal data posed greater challenges. When the means were far enough from 0 (more than one standard deviation, 4 in our simulations), SSW was practically unbiased, and its companion interval was the only option.

scholarly journals Interval estimation of the overall treatment effect in random-effects meta-analyses: Recommendations from a simulation study comparing frequentist, Bayesian, and bootstrap methods

2020 ◽  
Author(s):  
Frank Weber ◽  
Guido Knapp ◽  
Anne Glass ◽  
Günther Kundt ◽  
Katja Ickstadt
Keyword(s):  
Literature Review ◽  
Random Effects ◽  
Simulation Study ◽  
Treatment Effect ◽  
Profile Likelihood ◽  
Meta Analysis ◽  
Artery Bypass ◽  
Interval Length ◽  
Interval Estimators ◽  
Meta Analyses

There exists a variety of interval estimators for the overall treatment effect in a random-effects meta-analysis. A recent literature review summarizing existing methods suggested that in most situations, the Hartung-Knapp/Sidik-Jonkman (HKSJ) method was preferable. However, a quantitative comparison of those methods in a common simulation study is still lacking. Thus, we conduct such a simulation study for continuous and binary outcomes, focusing on the medical field for application.Based on the literature review and some new theoretical considerations, a practicable number of interval estimators is selected for this comparison: the classical normal-approximation interval using the DerSimonian-Laird heterogeneity estimator, the HKSJ interval using either the Paule-Mandel or the Sidik-Jonkman heterogeneity estimator, the Skovgaard higher-order profile likelihood interval, a parametric bootstrap interval, and a Bayesian interval using different priors. We evaluate the performance measures (coverage and interval length) at specific points in the parameter space, i.e. not averaging over a prior distribution. In this sense, our study is conducted from a frequentist point of view.We confirm the main finding of the literature review, the general recommendation of the HKSJ method (here with the Sidik-Jonkman heterogeneity estimator). For meta-analyses including only 2 studies, the high length of the HKSJ interval limits its practical usage. In this case, the Bayesian interval using a weakly informative prior for the heterogeneity may help. Our recommendations are illustrated using a real-world meta-analysis dealing with the efficacy of an intramyocardial bone marrow stem cell transplantation during coronary artery bypass grafting.

scholarly journals Vehicle Exposure and Spinal Musculature Fatigue in Military Warfighters: A Meta-Analysis

2016 ◽  
Vol 51 (11) ◽  
pp. 981-990 ◽  
Author(s):  
Roger O. Kollock ◽  
Kenneth E. Games ◽  
Alan E. Wilson ◽  
JoEllen M. Sefton
Keyword(s):  
Confidence Interval ◽  
Exposure Time ◽  
Sports Medicine ◽  
Occupational Safety ◽  
Data Extraction ◽  
Meta Analysis ◽  
Technical Information ◽  
Lumbar Region ◽  
Response Ratio ◽  
Log Response Ratio

Context: Spinal musculature fatigue from vehicle exposure may place warfighters at risk for spinal injuries and pain. Research on the relationship between vehicle exposure and spinal musculature fatigue is conflicting. A better understanding of the effect of military duty on musculoskeletal function is needed before sports medicine teams can develop injury-prevention programs. Objective: To determine if the literature supports a definite effect of vehicle exposure on spinal musculature fatigue. Data Sources: We searched the MEDLINE, Military & Government Collection (EBSCO), National Institute for Occupational Safety and Health Technical Information Center, PubMed, and Web of Science databases for articles published between January 1990 and September 2015. Study Selection: To be included, a study required a clear sampling method, preexposure and postexposure assessments of fatigue, a defined objective measurement of fatigue, a defined exposure time, and a study goal of exposing participants to forces related to vehicle exposure. Data Extraction: Sample size, mean preexposure and postexposure measures of fatigue, vehicle type, and exposure time. Data Synthesis: Six studies met the inclusion criteria. We used the Scottish Intercollegiate Guidelines Network algorithm to determine the appropriate tool for quality appraisal of each article. Unweighted random-effects model meta-analyses were conducted, and a natural log response ratio was used as the effect metric. The overall meta-analysis demonstrated that vehicle exposure increased fatigue of the spinal musculature (P = .03; natural log response ratio = −0.22, 95% confidence interval = −0.42, −0.02). Using the spinal region as a moderator, we observed that vehicle ride exposure significantly increased fatigue at the lumbar musculature (P = .02; natural log response ratio = −0.27, 95% confidence interval = −0.50, −0.04) but not at the cervical or thoracic region. Conclusions: Vehicle exposure increased fatigue at the lumbar region.

Participant Characteristics Predicting Communication Outcomes in AAC Implementation for Individuals with ASD and IDD: A Systematic Review and Meta-analysis

2021 ◽  
Author(s):  
Jay Ganz ◽  
James E Pustejovsky ◽  
Joe Reichle ◽  
Kimberly Vannest ◽  
Margaret Foster ◽  
...  
Keyword(s):  
Social Communication ◽  
Meta Analysis ◽  
Autism Spectrum ◽  
Response Ratio ◽  
Average Effect ◽  
Design Quality ◽  
Participant Characteristics ◽  
Log Response Ratio ◽  
High Degree ◽  
Degree Of Heterogeneity

This meta-analysis examined social communication outcomes in augmentative and alternative communication (AAC) interventions, or those that involved aided (e.g., speech generating devices, picture point systems) or unaided AAC (e.g., gestures, manual sign language) as a component of intervention, and the extent to which communication outcomes were predicted by participant characteristics. Variables of interest included chronological age, communication mode used prior to intervention, number of words produced and imitation skills of participants prior to intervention. Investigators identified 117 primary studies that implemented AAC interventions with school-aged individuals (up to 22 years) with autism spectrum disorder and/or intellectual disability associated with complex communication needs and assessed social-communication outcomes. All included studies involved single-case experimental designs and met basic study design quality standards. We synthesized findings across studies using two complementary effect size indices, Tau(AB) and the log response ratio, and multi-level meta-analysis with robust variance estimation. With Tau(AB), the overall average effect across 338 participants was 0.72, 95% CI [0.67, 0.76], with a high degree of heterogeneity across studies. With the log response ratio, the overall average effect corresponded to a 538% increase from baseline levels of responding, 95% CI [388%, 733%], with a high degree of heterogeneity across studies and contrasts. Moderator analyses detected few differences in effectiveness when comparing across diagnoses, ages, the number and type of communication modes the participants used prior to intervention, the number of words used by the participants prior to intervention, and imitation use prior to intervention.

scholarly journals Bias and correction for the log response ratio in ecological meta-analysis

Ecology ◽  
2015 ◽  
Vol 96 (8) ◽  
pp. 2056-2063 ◽  
Author(s):  
Marc J. Lajeunesse
Keyword(s):  
Meta Analysis ◽  
Response Ratio ◽  
Log Response Ratio

scholarly journals Selecting Communicative Interventions Targets for School-Aged participants with ASD and ID: A Single-Case Experimental Design Meta-analysis

2021 ◽  
Author(s):  
Jay Ganz ◽  
James E Pustejovsky ◽  
Joe Reichle ◽  
Kimberly Vannest ◽  
Margaret Foster ◽  
...  
Keyword(s):  
Effect Size ◽  
Augmentative And Alternative Communication ◽  
Meta Analysis ◽  
Autism Spectrum ◽  
Original Research ◽  
Response Ratio ◽  
Alternative Communication ◽  
Complex Communication Needs ◽  
Log Response Ratio ◽  
Communication Needs

Objective: This meta-analysis reviews the literature on communication modes, communicative functions, and types of augmentative and alternative communication (AAC) interventions for school-age participants with autism spectrum disorders and/or intellectual disabilities who experience complex communication needs. Considering potential differences related to outcomes that were targeted for intervention could help identify the most effective means of individualizing AAC interventions. Methods: We performed a systematic literature search using Academic Search Ultimate, ERIC, PsycINFO, Web of Science, and Proquest Dissertations &amp; Theses Global to retrieve research conducted between 1978 and the beginning of 2020. Studies included in the synthesis are (a) in English; (b) has one or more participants with an intellectual delay, developmental disability(ies); (c) reported the results of an augmentative and alternative communication (AAC) intervention to supplement or replace conventional speech for people with complex communication needs; (d) was a SCED; (e) measured social-communicative outcomes. We synthesized results across studies using multi-level meta-analyses of two case-level effect size metrics, Tau and log response ratio. We conducted moderator analyses using meta-regression with robust variance estimation.Results: Across 114 included studies with 330 participants and 767 effect size, overall Tau effects were moderate, Tau = 0.72, 95% CI [0.67, 0.77], and heterogeneous. For the subset of data series where log response ratio could be estimated, the overall average effect was LRR = 1.86, 95% CI [1.58, 2.13], and effects were highly heterogeneous. There were few statistically significant differences found between moderator categories, which included communication mode, communicative function, and type of AAC implemented.Conclusions: This meta-analysis highlights the potential differences related to outcomes that were targeted for AAC interventions for individuals with ASD and IDD. AAC intervention has been shown to improve communication outcomes in this population. However, there was a lack of sufficient data to analyze for some potential moderators such as insufficient descriptive information on participant characteristics. This is likely due to the heterogeneity of the participants and implementation factors; however, these factors were frequently underreported by original study authors which disallowed systematic analysis. That said, there is a need for more detailed participant characteristic descriptions in original research reports to support future aggregation across the literature. Sponsorship: We received funding for the review from the Institute of Education Sciences.Protocol: The review protocol was registered in the PROSPERO system (CRD42018112428).

scholarly journals Effect of Statins on Helicobacter pylori Eradication Rate: A Systematic Review and Meta-Analysis

2021 ◽  
pp. 1-9
Author(s):  
Mohsen Rajabnia ◽  
Amir Sadeghi ◽  
Saeed Abdi ◽  
Mihnea-Alexandru Găman ◽  
Mohammad Reza Zali ◽  
...  
Keyword(s):  
Helicobacter Pylori ◽  
Random Effects ◽  
Eradication Rate ◽  
Standard Treatment ◽  
Meta Analysis ◽  
Treatment Regimens ◽  
Randomized Controlled ◽  
Effect Measure ◽  
H Pylori ◽  
Meta Analyses

Statins have been used as adjuvants to standard treatment in order to increase the eradication rates<i></i>of<i> Helicobacter pylori</i> infection. This study aimed to summarize the results of the efficacy of adding statins to standard treatments used for the eradication<i></i>of<i> H. pylori</i> infection. We conducted a systematic search using a comprehensive combination of keywords in PubMed/MEDLINE, Web of Science, and Scopus to retrieve relevant studies from 1990 to 2020. The estimate of pooled relative risk (RR), as the effect measure, was calculated using random effects meta-analyses in Stata 14. We finally included 5 studies (all of them were randomized controlled trials). The meta-analysis of all studies showed that the pooled RR (95% confidence interval) was 1.03 (0.64–1.68) in the random effects model, which was not statistically significant. In other words, based on our meta-analysis, the addition of statins as an adjuvant therapy to the standard treatment regimens does not increase the rate of <i>H. pylori</i> eradication. However, further evidence is needed to confirm this result as the number of available studies was small.

scholarly journals Is OR “portable” in meta-analysis? Time to consider bivariate generalized linear mixed model

2020 ◽  
Author(s):  
Mengli Xiao ◽  
Yong Chen ◽  
Stephen Cole ◽  
Richard MacLehose ◽  
David Richardson ◽  
...  
Keyword(s):  
Clinical Trials ◽  
Mixed Model ◽  
Linear Mixed Model ◽  
Meta Analysis ◽  
Generalized Linear Mixed Model ◽  
Baseline Risk ◽  
Marginal Effect ◽  
List Type ◽  
Effect Measure ◽  
Meta Analyses

AbstractObjectivesA recent paper by Doi et al. advocated completely replacing the relative risk (RR) with the odds ratio (OR) as the effect measure used to report the association between a treatment and a binary outcome in clinical trials and meta-analyses. Besides some practical advantages of RR over OR and the well-known issue of the OR being non-collapsible, Doi et al.’s key assumption that the OR is “portable” in the meta-analysis, i.e., study-specific ORs are likely not correlated with baseline risks, was not well justified.Study designs and settingsWe summarized the Spearman’s rank correlation coefficient between study-specific OR and the baseline risk in 40,243 meta-analyses from the Cochrane Database of Systematic Reviews (CDSR).ResultsStudy-specific ORs are negatively correlated with baseline risk of disease (i.e., higher ORs tend to be observed in studies with lower baseline risks of disease) for most meta-analyses in CDSR. Using a meta-analysis comparing the effect of oral sumatriptan (100 mg) versus placebo on mitigating the acute headache at 2 hours after drug administration, we demonstrate that there is a strong negative correlation between OR (RR or RD) with the baseline risk and the conditional effects notably vary with baseline risks.ConclusionsReplacing RR or RD with OR is currently unadvisable in clinical trials and meta-analyses. It is possible that no effect measure is “portable” in a meta-analysis. In cases where portability of the effect measure is challenging to satisfy, we suggest presenting the conditional effect based on the baseline risk using a bivariate generalized linear mixed model. The bivariate generalized linear mixed model can be used to account for correlation between the effect measure and baseline disease risk. Furthermore, in addition to the overall (or marginal) effect, we recommend that investigators also report the effects conditioning on the baseline risk.What is New?Key findingsIn most meta-analyses in Cochrane Database of Systematic Reviews, there is notable negative correlation between ORs and baseline risks.When such a correlation is not negligible, the OR is not “portable” across studies with different baseline risks.When an effect measure is not “portable”, one may derive the effects conditioning on the baseline risk from a bivariate generalized linear mixed model.What this study adds to what was knownThe recommendation to replace the RR with the OR in clinical trials and meta-analyses is misguided.The OR is not a better effect summary than RR and RD in a single study or in meta-analyses; the noncollapsibility of OR can lead to misleading results in a single study and the OR is generally not portable in the meta-analysis.In addition to reporting effect measures such as the OR, RR or RD, it is also important to present the baseline risk.What is the implication and what should change now?When none of the effects are “portable” in a meta-analysis, in addition to report the overall (or marginal) effect, one should also report the effects conditioning on the baseline risk, regardless of the measure of choice.

A Meta-Analysis of Interviews and Cognitive Ability

2013 ◽  
Vol 12 (4) ◽  
pp. 157-169 ◽  
Author(s):  
Philip L. Roth ◽  
Allen I. Huffcutt
Keyword(s):  
Cognitive Ability ◽  
Meta Analysis ◽  
Incremental Validity ◽  
Employment Interviews ◽  
Moderator Analysis ◽  
The Future ◽  
Research Questions ◽  
Meta Analyses ◽  
The Relationship

The topic of what interviews measure has received a great deal of attention over the years. One line of research has investigated the relationship between interviews and the construct of cognitive ability. A previous meta-analysis reported an overall corrected correlation of .40 ( Huffcutt, Roth, & McDaniel, 1996 ). A more recent meta-analysis reported a noticeably lower corrected correlation of .27 ( Berry, Sackett, & Landers, 2007 ). After reviewing both meta-analyses, it appears that the two studies posed different research questions. Further, there were a number of coding judgments in Berry et al. that merit review, and there was no moderator analysis for educational versus employment interviews. As a result, we reanalyzed the work by Berry et al. and found a corrected correlation of .42 for employment interviews (.15 higher than Berry et al., a 56% increase). Further, educational interviews were associated with a corrected correlation of .21, supporting their influence as a moderator. We suggest a better estimate of the correlation between employment interviews and cognitive ability is .42, and this takes us “back to the future” in that the better overall estimate of the employment interviews – cognitive ability relationship is roughly .40. This difference has implications for what is being measured by interviews and their incremental validity.

Power-Enhanced Funnel Plots for Meta-Analysis

2020 ◽  
Vol 228 (1) ◽  
pp. 43-49 ◽  
Author(s):  
Michael Kossmeier ◽  
Ulrich S. Tran ◽  
Martin Voracek
Keyword(s):  
Statistical Power ◽  
Meta Analysis ◽  
Relevant Information ◽  
Selective Reporting ◽  
Graphical Display ◽  
Funnel Plot ◽  
Evidential Value ◽  
Graphical Displays ◽  
Funnel Plots ◽  
Meta Analyses

Abstract. Currently, dedicated graphical displays to depict study-level statistical power in the context of meta-analysis are unavailable. Here, we introduce the sunset (power-enhanced) funnel plot to visualize this relevant information for assessing the credibility, or evidential value, of a set of studies. The sunset funnel plot highlights the statistical power of primary studies to detect an underlying true effect of interest in the well-known funnel display with color-coded power regions and a second power axis. This graphical display allows meta-analysts to incorporate power considerations into classic funnel plot assessments of small-study effects. Nominally significant, but low-powered, studies might be seen as less credible and as more likely being affected by selective reporting. We exemplify the application of the sunset funnel plot with two published meta-analyses from medicine and psychology. Software to create this variation of the funnel plot is provided via a tailored R function. In conclusion, the sunset (power-enhanced) funnel plot is a novel and useful graphical display to critically examine and to present study-level power in the context of meta-analysis.

Sign in / Sign up

Export Citation Format

Share Document