scholarly journals Correcting for outcome reporting bias in a meta-analysis: A meta-regression approach

2021 ◽  
Author(s):  
Robbie Cornelis Maria van Aert ◽  
Jelte M. Wicherts
Keyword(s):  
Effect Size ◽  
Meta Analysis ◽  
New Method ◽  
Reporting Bias ◽  
Average Effect ◽  
Outcome Reporting Bias ◽  
Monte Carlo Simulation Study ◽  
Average Effect Size ◽  
Outcome Reporting ◽  
Meta Regression

Outcome reporting bias (ORB) refers to the biasing effect caused by researchers selectively reporting outcomes based on their statistical significance. ORB leads to inflated average effect size estimates in a meta-analysis if only the outcome with the largest effect size is reported due to ORB. We propose a new method (CORB) to correct for ORB that includes an estimate of the variability of the outcomes' effect size as a moderator in a meta-regression model. An estimate of the variability of the outcomes' effect size can be computed by assuming a correlation among the outcomes. Results of a Monte-Carlo simulation study showed that effect size in meta-analyses may be severely overestimated without any correction for ORB. The CORB method accurately estimates effect size when overestimation caused by ORB is the largest. Applying the new method to a meta-analysis on the effect of playing violent video games on aggressive cognition showed that the average effect size estimate decreased when correcting for ORB. We recommend to routinely apply methods to correct for ORB in any meta-analysis. We provide annotated R code and functions to facilitate researchers to apply the CORB method.

scholarly journals Cross-linguistic influence in simultaneous and early sequential bilingual children: a meta-analysis

2021 ◽  
pp. 1-33
Author(s):  
Chantal VAN DIJK ◽  
Elise VAN WONDEREN ◽  
Elly KOUTAMANIS ◽  
Gerrit Jan KOOTSTRA ◽  
Ton DIJKSTRA ◽  
...  
Keyword(s):  
Effect Size ◽  
Meta Analysis ◽  
Experimental Studies ◽  
Uniform Method ◽  
Bilingual Children ◽  
Average Effect ◽  
Language Dominance ◽  
Average Effect Size ◽  
Comprehensive Theory ◽  
Bilingual Development

Abstract Although cross-linguistic influence at the level of morphosyntax is one of the most intensively studied topics in child bilingualism, the circumstances under which it occurs remain unclear. In this meta-analysis, we measured the effect size of cross-linguistic influence and systematically assessed its predictors in 750 simultaneous and early sequential bilingual children in 17 unique language combinations across 26 experimental studies. We found a significant small to moderate average effect size of cross-linguistic influence, indicating that cross-linguistic influence is part and parcel of bilingual development. Language dominance, operationalized as societal language, was a significant predictor of cross-linguistic influence, whereas surface overlap, language domain and age were not. Perhaps an even more important finding was that definitions and operationalisations of cross-linguistic influence and its predictors varied considerably between studies. This could explain the absence of a comprehensive theory in the field. To solve this issue, we argue for a more uniform method of studying cross-linguistic influence.

scholarly journals A model-based correction for outcome reporting bias in meta-analysis

Biostatistics ◽  
2013 ◽  
Vol 15 (2) ◽  
pp. 370-383 ◽  
Author(s):  
J. Copas ◽  
K. Dwan ◽  
J. Kirkham ◽  
P. Williamson
Keyword(s):  
Meta Analysis ◽  
Reporting Bias ◽  
Outcome Reporting Bias ◽  
Outcome Reporting ◽  
Model Based

Model-based sensitivity analysis for outcome reporting bias in the meta analysis of benefit and harm outcomes

2017 ◽  
Vol 28 (3) ◽  
pp. 889-903 ◽  
Author(s):  
John Copas ◽  
Anthony Marson ◽  
Paula Williamson ◽  
Jamie Kirkham
Keyword(s):  
Meta Analysis ◽  
Partial Epilepsy ◽  
Risk Assessments ◽  
Reporting Bias ◽  
Drug Resistant ◽  
Cochrane Systematic Review ◽  
Outcome Reporting Bias ◽  
Outcome Reporting ◽  
Study Results ◽  
Test Treatment

Outcome reporting bias occurs when outcomes in research studies are selectively reported, the selection being influenced by the study results. For benefit outcomes, we have shown how risk assessments using the Outcome Reporting Bias in Trials risk classification scale can be used to calculate bias-adjusted treatment effect estimates. This paper presents a new and simpler version of the benefits method, and shows how it can be extended to cover the partial reporting and non-reporting of harm outcomes. Our motivating example is a Cochrane systematic review of 12 studies of Topiramate add-on therapy for drug-resistant partial epilepsy. Bias adjustments for partially reported or unreported outcomes suggest that the review has overestimated the benefits and underestimated the harms of the test treatment.

scholarly journals EFEKTIVITAS MODEL PEMBELAJARAN TERHADAP PENINGKATAN PEMAHAMAN KONSEP FISIKA SISWA SMA: META-ANALISIS

2021 ◽  
Vol 7 (2) ◽  
pp. 238
Author(s):  
M. Ifdal Hafiz Chan ◽  
Edja Annisa Septia ◽  
Kurnia Febrianti ◽  
Desnita Desnita
Keyword(s):  
High School Students ◽  
Direct Instruction ◽  
Effect Size ◽  
Meta Analysis ◽  
Problem Based Learning ◽  
Learning Model ◽  
Discovery Learning ◽  
Learning Models ◽  
Average Effect ◽  
Average Effect Size

ABSTRAKTujuan dari penelitian ini adalah untuk melihat efektivitas dari beberapa model pembelajaran dalam meningkatkan pemahaman konsep fisika siswa SMA. Penelitian ini merupakan penelitian meta-analisis yang menggambarkan effect size dari penelitian-penelitian pendidikan mengenai pengaruh model-model pembelajaran terhadap pemahaman konsep fisika siswa SMA. Model pembelajaran yang di bandingkan dalam penelitian meta-analisis ini adalah model pembelajaran Discovery Learning, Inkuiri, Kooperatif, Direct Instruction, dan Problem Based Learning. Model pembelajaran yang memiliki efektivitas tertinggi adalah model pembelajaran Langsung (Direct Instruction) dengan nilai rata-rata effect size 1.43 yang termasuk dalam kategori tinggi. Model pembelajaran lain yang juga memiliki nilai rata-rata effect size yang termasuk dalam kategori tinggi adalah model pembelajaran Inkuiri dengan nilai rata-rata effect size 1.39, model pembelajaran Kooperatif dengan nilai rata-rata effect size 1.11, model pembelajaran Discovery Learning dengan nilai rata-rata effect size 0.96, serta model pembelajaran Problem Based Learning dengan nilai rata-rata effect size 0.92. Kata kunci: Meta-Analisis; Discovery Learning; Inkuiri; Kooperatif; Direct Instruction; Problem Based Learning; Pemahaman Konsep; Fisika ABSTRACT The purpose of this study was to see the effectiveness of several learning models in improving the understanding of physics concepts for high school students. This research is a meta-analysis that describes the effect size of educational studies regarding the influence of learning models on the understanding of physics concepts for high school students. The learning models compared in this meta-analysis are the Discovery Learning, Inquiry, Cooperative, Direct Instruction, and Problem Based Learning learning models. The learning model that has the highest effectiveness is the Direct Instruction model with an average effect size value of 1.43 which is included in the high category. Other learning models that also have an average effect size value that is included in the high category are the Inquiry learning model with an average effect size value of 1.39, the Cooperative learning model with an average effect size value of 1.11, and the Discovery Learning learning model with an average value. the average effect size is 0.96, as well as the Problem Based Learning model with an average effect size value of 0.92. Keywords: Meta-Analysis; Discovery Learning; Inquiry; Cooperative; Direct Instruction; Problem Based Learning; Concept Understanding; Physics.

Learner-Control Effects: A Review of Reviews and a Meta-Analysis

1996 ◽  
Vol 15 (2) ◽  
pp. 157-174 ◽  
Author(s):  
Richard P. Niemiec ◽  
Christian Sikorski ◽  
Herbert J. Walberg
Keyword(s):  
Effect Size ◽  
Learner Control ◽  
Meta Analysis ◽  
Computer Assisted Instruction ◽  
Computer Assisted ◽  
Optimal Conditions ◽  
Average Effect ◽  
Average Effect Size ◽  
Review Of Reviews ◽  
Assisted Instruction

This article concerns the effects of learner control in computer-assisted instruction (CAI). After reviewing previous reviews of research on the topic, twenty-four studies of learner control were subjected to meta-analysis. The results of both the review and meta-analysis are equivocal. Several reviews indicate that learner control works less well with younger, less able students. Other reviews indicate that, given optimal conditions, learner control can work with any students. The meta-analysis, however, yielded an average effect size that was small and negative suggesting that the average student would be slightly better off without it. Although learner control has theoretical appeal, its effects on learning seem neither powerful nor consistent.

No Strong Evidence of Stereotype Threat in Females: A Reassessment of the Meta-Analysis

2021 ◽  
pp. 1932202X2110615
Author(s):  
Russell T. Warne
Keyword(s):  
Stereotype Threat ◽  
Strong Evidence ◽  
Effect Size ◽  
Meta Analysis ◽  
Compelling Evidence ◽  
Apparent Effect ◽  
Average Effect ◽  
Population Effect ◽  
Average Effect Size ◽  
Real Phenomenon

Recently, Picho-Kiroga (2021) published a meta-analysis on the effect of stereotype threat on females. Their conclusion was that the average effect size for stereotype threat studies was d = .28, but that effects are overstated because the majority of studies on stereotype threat in females include methodological characteristics that inflate the apparent effect size. In this response, I show that Picho-Kiroga et al. (2021) committed fundamental errors in their meta-analysis that undermine confidence in the article and warrant major corrections. But even if the data were not flawed, the conclusion that Picho-Kiroga et al. (2021) should have reached is that their results are most consistent with a population effect size of zero. There is no compelling evidence that stereotype threat is a real phenomenon in females.

scholarly journals A Meta–Analytic and Conceptual Update on the Associations between Procrastination and Multidimensional Perfectionism

2017 ◽  
Vol 31 (2) ◽  
pp. 137-159 ◽  
Author(s):  
Fuschia M. Sirois ◽  
Danielle S. Molnar ◽  
Jameson K. Hirsch ◽  
Mitja Back
Keyword(s):  
Effect Size ◽  
Meta Analysis ◽  
Self Regulation ◽  
Average Effect ◽  
Personality Psychology ◽  
Average Effect Size ◽  
Partial Correlations ◽  
Unidimensional Construct ◽  
Meta Analyses ◽  
Behavioural Dynamics

The equivocal and debated findings from a 2007 meta–analysis, which viewed perfectionism as a unidimensional construct, suggested that perfectionism was unrelated to procrastination. The present meta–analysis aimed to provide a conceptual update and reanalysis of the procrastination–perfectionism association guided by both a multidimensional view of perfectionism and self–regulation theory. The random–effects meta–analyses revealed a small to medium positive average effect size ( r = .23; k = 43, N = 10 000; 95% confidence interval (95% CI) [0.19, 0.27]) for trait procrastination and perfectionistic concerns and a small to medium negative average effect size ( r = −.22; k = 38, N = 9544; 95% CI [−0.26, −0.18]) for procrastination and perfectionistic strivings. The average correlations remained significant after statistically accounting for the joint variance between the two perfectionism dimensions via semi–partial correlations. For perfectionistic concerns, but not perfectionistic strivings, the effects depended on the perfectionism measure used. All effects did not vary by the trait procrastination measure used or the respondent's sex. Our findings confirm that from a multidimensional perspective, trait procrastination is both positively and negatively associated with higher–order perfectionism dimensions and further highlights the value of a self–regulation perspective for understanding the cognitive, affective and behavioural dynamics that characterise these traits. Copyright © 2017 European Association of Personality Psychology

Sign in / Sign up

Export Citation Format

Share Document