scholarly journals Does Every Study? Implementing Ordinal Constraint in Meta-Analysis

2020 ◽  
Author(s):  
Julia M. Haaf ◽  
Jeffrey N. Rouder
Keyword(s):  
Effect Size ◽  
Model Comparison ◽  
Meta Analysis ◽  
Underlying Mechanism ◽  
Average Effect ◽  
True Effect Size ◽  
Unconstrained Model ◽  
Comparison Approach ◽  
Major Implication ◽  
Surface Statistics

The most prominent goal when conducting a meta-analysis is to estimate the true effect size across a set of studies. This approach is problematic whenever the analyzed studies are inconsistent, i.e. some studies show an effect in the predicted direction while others show no effect and still others show an effect in the opposite direction. In case of such an inconsistency, the average effect may be a product of a mixture of mechanisms. The first question in any meta-analysis should therefore be whether all studies show an effect in the same direction. To tackle this question a model with multiple ordinal constraints is proposed---one constraint for each study in the set. This "every study" model is compared to a set of alternative models, such as an unconstrained model that predicts effects in both directions. If the ordinal constraints hold, one underlying mechanism may suffice to explain the results from all studies. A major implication is then that average effects become interpretable. We illustrate the model-comparison approach using Carbajal et al.'s (2020) meta-analysis on the familiar-word-recognition effect, show how predictor analyses can be incorporated in the approach, and provide R-code for interested researchers. As common in meta-analysis, only surface statistics (such as effect size and sample size) are provided from each study, and the modeling approach can be adapted to suit these conditions.

scholarly journals Cross-linguistic influence in simultaneous and early sequential bilingual children: a meta-analysis

2021 ◽  
pp. 1-33
Author(s):  
Chantal VAN DIJK ◽  
Elise VAN WONDEREN ◽  
Elly KOUTAMANIS ◽  
Gerrit Jan KOOTSTRA ◽  
Ton DIJKSTRA ◽  
...  
Keyword(s):  
Effect Size ◽  
Meta Analysis ◽  
Experimental Studies ◽  
Uniform Method ◽  
Bilingual Children ◽  
Average Effect ◽  
Language Dominance ◽  
Average Effect Size ◽  
Comprehensive Theory ◽  
Bilingual Development

Abstract Although cross-linguistic influence at the level of morphosyntax is one of the most intensively studied topics in child bilingualism, the circumstances under which it occurs remain unclear. In this meta-analysis, we measured the effect size of cross-linguistic influence and systematically assessed its predictors in 750 simultaneous and early sequential bilingual children in 17 unique language combinations across 26 experimental studies. We found a significant small to moderate average effect size of cross-linguistic influence, indicating that cross-linguistic influence is part and parcel of bilingual development. Language dominance, operationalized as societal language, was a significant predictor of cross-linguistic influence, whereas surface overlap, language domain and age were not. Perhaps an even more important finding was that definitions and operationalisations of cross-linguistic influence and its predictors varied considerably between studies. This could explain the absence of a comprehensive theory in the field. To solve this issue, we argue for a more uniform method of studying cross-linguistic influence.

Abstract 3789: Nessun Dorma : Have the Risks of Rosiglitazone been Exaggerated?

Circulation ◽  
2007 ◽  
Vol 116 (suppl_16) ◽  
Author(s):  
George A Diamond ◽  
Sanjay Kaul
Keyword(s):  
Confidence Intervals ◽  
Effect Size ◽  
Fixed Effects ◽  
Meta Analysis ◽  
Cardiovascular Death ◽  
Sensitivity Analyses ◽  
Odds Ratios ◽  
True Effect Size ◽  
Index Study ◽  
The Stability

Background A highly publicized meta-analysis of 42 clinical trials comprising 27,844 diabetics ignited a firestorm of controversy by charging that treatment with rosiglitazone was associated with a “…worrisome…” 43% greater risk of myocardial infarction ( p =0.03) and a 64% greater risk of cardiovascular death ( p =0.06). Objective The investigators excluded 4 trials from the infarction analysis and 19 trials from the mortality analysis in which no events were observed. We sought to determine if these exclusions biased the results. Methods We compared the index study to a Bayesian meta-analysis of the entire 42 trials (using odds ratio as the measure of effect size) and to fixed-effects and random-effects analyses with and without a continuity correction that adjusts for values of zero. Results The odds ratios and confidence intervals for the analyses are summarized in the Table . Odds ratios for infarction ranged from 1.43 to 1.22 and for death from 1.64 to 1.13. Corrected models resulted in substantially smaller odds ratios and narrower confidence intervals than did uncorrected models. Although corrected risks remain elevated, none are statistically significant (*p<0.05). Conclusions Given the fragility of the effect sizes and confidence intervals, the charge that roziglitazone increases the risk of adverse events is not supported by these additional analyses. The exaggerated values observed in the index study are likely the result of excluding the zero-event trials from analysis. Continuity adjustments mitigate this error and provide more consistent and reliable assessments of true effect size. Transparent sensitivity analyses should therefore be performed over a realistic range of the operative assumptions to verify the stability of such assessments especially when outcome events are rare. Given the relatively wide confidence intervals, additional data will be required to adjudicate these inconclusive results.

scholarly journals META ANALISIS PENGARUH MODEL DISCOVERY LEARNING TERHADAP PENGUASAAN KONSEP DAN PENGETAHUAN PESERTA DIDIK FISIKA SMA

2021 ◽  
Vol 7 (2) ◽  
pp. 305
Author(s):  
Elvi Suryanti ◽  
Asrizal Asrizal ◽  
Fatni Mufit
Keyword(s):  
High School ◽  
Effect Size ◽  
Meta Analysis ◽  
Learning Model ◽  
Discovery Learning ◽  
Average Effect ◽  
High School Physics ◽  
High Effect ◽  
Physics Concepts ◽  
Very High

ABSTRAKTujuan penelitian ini adalah untuk mengetahui pengaruh Model discovery learning terhadap penguasaan konsep dan pengetahuan fisika SMA. Metode yang digunakan dalam penelitian ini adalah meta-analisis. Teknik analisis data dalam penelitian ini menggunakan perhitungan nilai effect size untuk setiap artikel. Berdasarkan meta analisis yang dilakukan, dapat dinyatakan bahwa hasil penelitian ini yaitu : 1) pengaruh model discovery learning terhadap penguasaan konsep dan pengetahuan  fisika SMA ditinjau dari tahun terbit artikel memberikan efek yang Sangat Tinggi terdapat pada tahun 2017,2018, dan 2020 dengan masing-masing rata-rata effect size yaitu 1,6; 1,15; dan 1,62. 2) pengaruh model discovery learning terhadap penguasaan konsep dan pengetahuan  fisika SMA ditinjau dari tingkatan kelas memberikan efek Sangat Tinggi terdapat pada kelas XI dengan rata-rata effect size 1,25. 3) pengaruh discovery learning terhadap penguasaan konsep fisika SMA memberikan efek Sangat Tinggi  terdapat pada artikel keenam (J6) dengan  nilai effect size 2,38. 4) pengaruh discovery learning terhadap pengetahuan  fisika SMA memberikan efek Sangat Tinggi  terdapat pada artikel kesepuluh (J10) dengan  nilai effect size 3,95. Hal ini menunjukkan bahwa terdapat pengaruh model discovery learning terhadap penguasaan konsep dan pengetahuan  fisika SMA. Kata kunci: Discovery Learning; Penguasaan Konsep; Pengetahuan. ABSTRACTThe purpose of this study was to determine the effect of the discovery learning model on mastery of high school physics concepts and knowledge. The method used in this research is meta-analysis. The data analysis technique in this study uses the calculation of the effect size value for each article. Based on the meta-analysis carried out, it can be stated that the results of this study are: 1) the effect of the discovery learning model on mastery of concepts and knowledge of high school physics in terms of the year the article was published gave a Very High effect in 2017,2018, and 2020 with each the average effect size is 1.6; 1.15; and 1.62. 2) the influence of the discovery learning model on the mastery of concepts and knowledge of high school physics in terms of grade level gives a very high effect in class XI with an average effect size of 1.25. 3) the effect of discovery learning on mastery of high school physics concepts gives a Very High effect found in the sixth article (J6) with an effect size value of 2.38. 4) the effect of discovery learning on high school physics knowledge gives a very high effect in the tenth article (J10) with an effect size value of 3.95. This shows that there is an influence of the discovery learning model on the mastery of high school physics concepts and knowledge. Keywords: Discovery Learning; Concept Mastery; Knowledge.

scholarly journals EFEKTIVITAS MODEL PEMBELAJARAN TERHADAP PENINGKATAN PEMAHAMAN KONSEP FISIKA SISWA SMA: META-ANALISIS

2021 ◽  
Vol 7 (2) ◽  
pp. 238
Author(s):  
M. Ifdal Hafiz Chan ◽  
Edja Annisa Septia ◽  
Kurnia Febrianti ◽  
Desnita Desnita
Keyword(s):  
High School Students ◽  
Direct Instruction ◽  
Effect Size ◽  
Meta Analysis ◽  
Problem Based Learning ◽  
Learning Model ◽  
Discovery Learning ◽  
Learning Models ◽  
Average Effect ◽  
Average Effect Size

ABSTRAKTujuan dari penelitian ini adalah untuk melihat efektivitas dari beberapa model pembelajaran dalam meningkatkan pemahaman konsep fisika siswa SMA. Penelitian ini merupakan penelitian meta-analisis yang menggambarkan effect size dari penelitian-penelitian pendidikan mengenai pengaruh model-model pembelajaran terhadap pemahaman konsep fisika siswa SMA. Model pembelajaran yang di bandingkan dalam penelitian meta-analisis ini adalah model pembelajaran Discovery Learning, Inkuiri, Kooperatif, Direct Instruction, dan Problem Based Learning. Model pembelajaran yang memiliki efektivitas tertinggi adalah model pembelajaran Langsung (Direct Instruction) dengan nilai rata-rata effect size 1.43 yang termasuk dalam kategori tinggi. Model pembelajaran lain yang juga memiliki nilai rata-rata effect size yang termasuk dalam kategori tinggi adalah model pembelajaran Inkuiri dengan nilai rata-rata effect size 1.39, model pembelajaran Kooperatif dengan nilai rata-rata effect size 1.11, model pembelajaran Discovery Learning dengan nilai rata-rata effect size 0.96, serta model pembelajaran Problem Based Learning dengan nilai rata-rata effect size 0.92. Kata kunci: Meta-Analisis; Discovery Learning; Inkuiri; Kooperatif; Direct Instruction; Problem Based Learning; Pemahaman Konsep; Fisika ABSTRACT The purpose of this study was to see the effectiveness of several learning models in improving the understanding of physics concepts for high school students. This research is a meta-analysis that describes the effect size of educational studies regarding the influence of learning models on the understanding of physics concepts for high school students. The learning models compared in this meta-analysis are the Discovery Learning, Inquiry, Cooperative, Direct Instruction, and Problem Based Learning learning models. The learning model that has the highest effectiveness is the Direct Instruction model with an average effect size value of 1.43 which is included in the high category. Other learning models that also have an average effect size value that is included in the high category are the Inquiry learning model with an average effect size value of 1.39, the Cooperative learning model with an average effect size value of 1.11, and the Discovery Learning learning model with an average value. the average effect size is 0.96, as well as the Problem Based Learning model with an average effect size value of 0.92. Keywords: Meta-Analysis; Discovery Learning; Inquiry; Cooperative; Direct Instruction; Problem Based Learning; Concept Understanding; Physics.

Learner-Control Effects: A Review of Reviews and a Meta-Analysis

1996 ◽  
Vol 15 (2) ◽  
pp. 157-174 ◽  
Author(s):  
Richard P. Niemiec ◽  
Christian Sikorski ◽  
Herbert J. Walberg
Keyword(s):  
Effect Size ◽  
Learner Control ◽  
Meta Analysis ◽  
Computer Assisted Instruction ◽  
Computer Assisted ◽  
Optimal Conditions ◽  
Average Effect ◽  
Average Effect Size ◽  
Review Of Reviews ◽  
Assisted Instruction

This article concerns the effects of learner control in computer-assisted instruction (CAI). After reviewing previous reviews of research on the topic, twenty-four studies of learner control were subjected to meta-analysis. The results of both the review and meta-analysis are equivocal. Several reviews indicate that learner control works less well with younger, less able students. Other reviews indicate that, given optimal conditions, learner control can work with any students. The meta-analysis, however, yielded an average effect size that was small and negative suggesting that the average student would be slightly better off without it. Although learner control has theoretical appeal, its effects on learning seem neither powerful nor consistent.

No Strong Evidence of Stereotype Threat in Females: A Reassessment of the Meta-Analysis

2021 ◽  
pp. 1932202X2110615
Author(s):  
Russell T. Warne
Keyword(s):  
Stereotype Threat ◽  
Strong Evidence ◽  
Effect Size ◽  
Meta Analysis ◽  
Compelling Evidence ◽  
Apparent Effect ◽  
Average Effect ◽  
Population Effect ◽  
Average Effect Size ◽  
Real Phenomenon

Recently, Picho-Kiroga (2021) published a meta-analysis on the effect of stereotype threat on females. Their conclusion was that the average effect size for stereotype threat studies was d = .28, but that effects are overstated because the majority of studies on stereotype threat in females include methodological characteristics that inflate the apparent effect size. In this response, I show that Picho-Kiroga et al. (2021) committed fundamental errors in their meta-analysis that undermine confidence in the article and warrant major corrections. But even if the data were not flawed, the conclusion that Picho-Kiroga et al. (2021) should have reached is that their results are most consistent with a population effect size of zero. There is no compelling evidence that stereotype threat is a real phenomenon in females.

scholarly journals A Meta–Analytic and Conceptual Update on the Associations between Procrastination and Multidimensional Perfectionism

2017 ◽  
Vol 31 (2) ◽  
pp. 137-159 ◽  
Author(s):  
Fuschia M. Sirois ◽  
Danielle S. Molnar ◽  
Jameson K. Hirsch ◽  
Mitja Back
Keyword(s):  
Effect Size ◽  
Meta Analysis ◽  
Self Regulation ◽  
Average Effect ◽  
Personality Psychology ◽  
Average Effect Size ◽  
Partial Correlations ◽  
Unidimensional Construct ◽  
Meta Analyses ◽  
Behavioural Dynamics

The equivocal and debated findings from a 2007 meta–analysis, which viewed perfectionism as a unidimensional construct, suggested that perfectionism was unrelated to procrastination. The present meta–analysis aimed to provide a conceptual update and reanalysis of the procrastination–perfectionism association guided by both a multidimensional view of perfectionism and self–regulation theory. The random–effects meta–analyses revealed a small to medium positive average effect size ( r = .23; k = 43, N = 10 000; 95% confidence interval (95% CI) [0.19, 0.27]) for trait procrastination and perfectionistic concerns and a small to medium negative average effect size ( r = −.22; k = 38, N = 9544; 95% CI [−0.26, −0.18]) for procrastination and perfectionistic strivings. The average correlations remained significant after statistically accounting for the joint variance between the two perfectionism dimensions via semi–partial correlations. For perfectionistic concerns, but not perfectionistic strivings, the effects depended on the perfectionism measure used. All effects did not vary by the trait procrastination measure used or the respondent's sex. Our findings confirm that from a multidimensional perspective, trait procrastination is both positively and negatively associated with higher–order perfectionism dimensions and further highlights the value of a self–regulation perspective for understanding the cognitive, affective and behavioural dynamics that characterise these traits. Copyright © 2017 European Association of Personality Psychology

Bias in emerging biomarkers for bipolar disorder

2016 ◽  
Vol 46 (11) ◽  
pp. 2287-2297 ◽  
Author(s):  
A. F. Carvalho ◽  
C. A. Köhler ◽  
B. S. Fernandes ◽  
J. Quevedo ◽  
K. W. Miskowiak ◽  
...  
Keyword(s):  
Bipolar Disorder ◽  
Effect Size ◽  
Comprehensive Evaluation ◽  
Meta Analysis ◽  
Selective Reporting ◽  
True Effect Size ◽  
Genetic Biomarkers ◽  
Morning Cortisol ◽  
Meta Analyses ◽  
Positive Results

BackgroundTo date no comprehensive evaluation has appraised the likelihood of bias or the strength of the evidence of peripheral biomarkers for bipolar disorder (BD). Here we performed an umbrella review of meta-analyses of peripheral non-genetic biomarkers for BD.MethodThe Pubmed/Medline, EMBASE and PsycInfo electronic databases were searched up to May 2015. Two independent authors conducted searches, examined references for eligibility, and extracted data. Meta-analyses in any language examining peripheral non-genetic biomarkers in participants with BD (across different mood states) compared to unaffected controls were included.ResultsSix references, which examined 13 biomarkers across 20 meta-analyses (5474 BD cases and 4823 healthy controls) met inclusion criteria. Evidence for excess of significance bias (i.e. bias favoring publication of ‘positive’ nominally significant results) was observed in 11 meta-analyses. Heterogeneity was high for (I2 ⩾ 50%) 16 meta-analyses. Only two biomarkers met criteria for suggestive evidence namely the soluble IL-2 receptor and morning cortisol. The median power of included studies, using the effect size of the largest dataset as the plausible true effect size of each meta-analysis, was 15.3%.ConclusionsOur findings suggest that there is an excess of statistically significant results in the literature of peripheral biomarkers for BD. Selective publication of ‘positive’ results and selective reporting of outcomes are possible mechanisms.

Meta-Analysis: Literature Synthesis or Effect-Size Surface Estimation?

1992 ◽  
Vol 17 (4) ◽  
pp. 363-374 ◽  
Author(s):  
Donald B. Rubin
Keyword(s):  
Effect Size ◽  
Meta Analysis ◽  
Effect Sizes ◽  
Standard Errors ◽  
Statistical Techniques ◽  
Average Effect ◽  
Statistical Tools ◽  
Surface Estimation ◽  
Literature Synthesis ◽  
Relevant Factors

A traditional meta-analysis can be thought of as a literature synthesis, in which a collection of observed studies is analyzed to obtain summary judgments about overall significance and size of effects. Many aspects of the current set of statistical tools for meta-analysis are highly useful—for example, the development of clear and concise effect-size indicators with associated standard errors. I am less happy, however, with more esoteric statistical techniques and their implied objects of estimation (i.e., their estimands) which are tied to the conceptualization of average effect sizes, weighted or otherwise, in a population of studies. In contrast to these average effect sizes of literature synthesis, I believe that the proper estimand is an effect-size surface, which is a function only of scientifically relevant factors, and which can only be estimated by extrapolating a response surface of observed effect sizes to a region of ideal studies. This effect-size surface perspective is presented and contrasted with the literature synthesis perspective. The presentation is entirely conceptual. Moreover, it is designed to be provocative, thereby prodding researchers to rethink traditional meta-analysis and ideally stimulating meta-analysts to attempt effect-surface estimations.

scholarly journals Low statistical power in biomedical science: a review of three human research domains

2017 ◽  
Vol 4 (2) ◽  
pp. 160254 ◽  
Author(s):  
Estelle Dumas-Mallet ◽  
Katherine S. Button ◽  
Thomas Boraud ◽  
Francois Gonon ◽  
Marcus R. Munafò
Keyword(s):  
Effect Size ◽  
Statistical Power ◽  
Meta Analysis ◽  
Statistical Significance ◽  
Average Power ◽  
Biomedical Science ◽  
Significant Finding ◽  
Biomedical Sciences ◽  
True Effect Size ◽  
Meta Analyses

Studies with low statistical power increase the likelihood that a statistically significant finding represents a false positive result. We conducted a review of meta-analyses of studies investigating the association of biological, environmental or cognitive parameters with neurological, psychiatric and somatic diseases, excluding treatment studies, in order to estimate the average statistical power across these domains. Taking the effect size indicated by a meta-analysis as the best estimate of the likely true effect size, and assuming a threshold for declaring statistical significance of 5%, we found that approximately 50% of studies have statistical power in the 0–10% or 11–20% range, well below the minimum of 80% that is often considered conventional. Studies with low statistical power appear to be common in the biomedical sciences, at least in the specific subject areas captured by our search strategy. However, we also observe evidence that this depends in part on research methodology, with candidate gene studies showing very low average power and studies using cognitive/behavioural measures showing high average power. This warrants further investigation.

Sign in / Sign up

Export Citation Format

Share Document