Experimental and Quasi-Experimental Studies of Inquiry-Based Science Teaching

2012 ◽  
Vol 82 (3) ◽  
pp. 300-329 ◽  
Author(s):  
Erin Marie Furtak ◽  
Tina Seidel ◽  
Heidi Iverson ◽  
Derek C. Briggs
Keyword(s):  
Science Education ◽  
Education Reform ◽  
Science Teaching ◽  
Effect Size ◽  
Meta Analysis ◽  
Experimental Studies ◽  
Effect Sizes ◽  
Science Education Reform ◽  
Quasi Experimental ◽  
Meta Analyses

Although previous meta-analyses have indicated a connection between inquiry-based teaching and improved student learning, the type of instruction characterized as inquiry based has varied greatly, and few have focused on the extent to which activities are led by the teacher or student. This meta-analysis introduces a framework for inquiry-based teaching that distinguishes between cognitive features of the activity and degree of guidance given to students. This framework is used to code 37 experimental and quasi-experimental studies published between 1996 and 2006, a decade during which inquiry was the main focus of science education reform. The overall mean effect size is .50. Studies that contrasted epistemic activities or the combination of procedural, epistemic, and social activities had the highest mean effect sizes. Furthermore, studies involving teacher-led activities had mean effect sizes about .40 larger than those with student-led conditions. The importance of establishing the validity of the treatment construct in meta-analyses is also discussed.

Systematic Reviews and Meta-Analyses of Crime Prevention Evaluations

2018 ◽  
Author(s):  
Anthony Petrosino ◽  
Claire Morgan ◽  
Trevor Fronius
Keyword(s):  
Systematic Review ◽  
Systematic Reviews ◽  
Effect Size ◽  
Meta Analysis ◽  
Experimental Studies ◽  
Small Sample ◽  
Effect Sizes ◽  
What Works ◽  
Management Procedures ◽  
Meta Analyses

Systematic reviews and meta-analyses have become a focal point of evidence-based policy in criminology. Systematic reviews use explicit and transparent processes to identify, retrieve, code, analyze, and report on existing research studies bearing on a question of policy or practice. Meta-analysis can combine the results from the most rigorous evaluations identified in a systematic review to provide policymakers with the best evidence on what works for a variety of interventions relevant to reducing crime and making the justice system fairer and more effective. The steps of a systematic review using meta-analysis include specifying the topic area, developing management procedures, specifying the search strategy, developing eligibility criteria, extracting data from the studies, computing effect sizes, developing an analysis strategy, and interpreting and reporting the results. In a systematic review using meta-analysis, after identifying and coding eligible studies, the researchers create a measure of effect size for each experimental versus control contrast of interest in the study. Most commonly, reviewers do this by standardizing the difference between scores of the experimental and control groups, placing outcomes that are conceptually similar but measured differently (e.g., such as re-arrest or reconviction) on the same common scale or metric. Though these are different indices, they do measure a program’s effect on some construct (e.g., criminality). These effect sizes are usually averaged across all similar studies to provide a summary of program impact. The effect sizes also represent the dependent variable in the meta-analysis, and more advanced syntheses explore the role of potential moderating variables, such as sample size or other characteristics related to effect size. When done well and with full integrity, a systematic review using meta-analysis can provide the most comprehensive assessment of the available evaluative literature addressing the research question, as well as the most reliable statement about what works. Drawing from a larger body of research increases statistical power by reducing standard error; individual studies often use small sample sizes, which can result in large margins of error. In addition, conducting meta-analysis can be faster and less resource-intensive than replicating experimental studies. Using meta-analysis instead of relying on an individual program evaluation can help ensure that policy is guided by the totality of evidence, drawing upon a solid basis for generalizing outcomes.

scholarly journals A Statistical Method for Synthesizing Meta-Analyses

2013 ◽  
Vol 2013 ◽  
pp. 1-9 ◽  
Author(s):  
Liansheng Larry Tang ◽  
Michael Caudy ◽  
Faye Taxman
Keyword(s):  
Statistical Method ◽  
Effect Size ◽  
Meta Analysis ◽  
Effect Sizes ◽  
Weighted Mean ◽  
Summary Effect ◽  
Different Types ◽  
Meta Analyses ◽  
Size Estimates ◽  
Similar Search

Multiple meta-analyses may use similar search criteria and focus on the same topic of interest, but they may yield different or sometimes discordant results. The lack of statistical methods for synthesizing these findings makes it challenging to properly interpret the results from multiple meta-analyses, especially when their results are conflicting. In this paper, we first introduce a method to synthesize the meta-analytic results when multiple meta-analyses use the same type of summary effect estimates. When meta-analyses use different types of effect sizes, the meta-analysis results cannot be directly combined. We propose a two-step frequentist procedure to first convert the effect size estimates to the same metric and then summarize them with a weighted mean estimate. Our proposed method offers several advantages over existing methods by Hemming et al. (2012). First, different types of summary effect sizes are considered. Second, our method provides the same overall effect size as conducting a meta-analysis on all individual studies from multiple meta-analyses. We illustrate the application of the proposed methods in two examples and discuss their implications for the field of meta-analysis.

Can Reliance be Placed on a Single Meta-Analysis?

1990 ◽  
Vol 24 (3) ◽  
pp. 405-415 ◽  
Author(s):  
Nathaniel McConaghy
Keyword(s):  
Literature Review ◽  
Effect Size ◽  
Meta Analysis ◽  
Statistical Significance ◽  
Effect Sizes ◽  
Control Groups ◽  
Consistent Finding ◽  
Placebo Controls ◽  
Effect Of Treatment ◽  
Meta Analyses

Meta-analysis replaced statistical significance with effect size in the hope of resolving controversy concerning evaluation of treatment effects. Statistical significance measured reliability of the effect of treatment, not its efficacy. It was strongly influenced by the number of subjects investigated. Effect size as assessed originally, eliminated this influence but by standardizing the size of the treatment effect could distort it. Meta-analyses which combine the results of studies which employ different subject types, outcome measures, treatment aims, no-treatment rather than placebo controls or therapists with varying experience can be misleading. To ensure discussion of these variables meta-analyses should be used as an aid rather than a substitute for literature review. While meta-analyses produce contradictory findings, it seems unwise to rely on the conclusions of an individual analysis. Their consistent finding that placebo treatments obtain markedly higher effect sizes than no treatment hopefully will render the use of untreated control groups obsolete.

scholarly journals Differences Between Traditional and Distance Education Academic Performances: A Meta-Analytic Approach

2003 ◽  
Vol 4 (2) ◽  
Author(s):  
Mickey Shachar ◽  
Yoram Neumann
Keyword(s):  
Distance Education ◽  
Effect Size ◽  
Meta Analysis ◽  
Experimental Studies ◽  
Traditional Group ◽  
Course Grades ◽  
Quasi Experimental ◽  
The Mean ◽  
Education Academic ◽  
Education Courses

<P class=abstract>This meta-analysis research estimated and compared the differences between the academic performance of students enrolled in distance education courses relative to those enrolled in traditional settings, as demonstrated by their final course grades/ scores within the 1990-2002 period.<BR> <BR> Eighty-six experimental and quasi-experimental studies met the established inclusion criteria for the meta-analysis (including data from over 15,000 participating students), and provided effect sizes, clearly demonstrating that: (1) in two thirds of the cases, students taking courses by distance education outperformed their student counterparts enrolled in traditionally instructed courses; (2) the overall effect size d+ was calculated as 0.37 standard deviation units (0.33 < 95% confidence interval < 0.40); and (3) this effect size of 0.37 indicates the mean percentile standing of the DE group is at the 65th percentile of the traditional group (mean defined as the 50th percentile). </P>

scholarly journals Comparison of the Effectiveness of Whole Body Vibration in Stroke Patients: A Meta-Analysis

2018 ◽  
Vol 2018 ◽  
pp. 1-10 ◽  
Author(s):  
Yoo Jung Park ◽  
Sun Wook Park ◽  
Han Suk Lee
Keyword(s):  
Muscle Strength ◽  
Effect Size ◽  
Meta Analysis ◽  
Whole Body Vibration ◽  
Treatment Method ◽  
Effect Sizes ◽  
Whole Body ◽  
Safe Alternative ◽  
Body Vibration ◽  
Meta Analyses

Objectives. The goals of this study were to assess the effectiveness of WBV (whole body vibration) training through an analysis of effect sizes, identify advantages of WBV training, and suggest other effective treatment methods. Methods. Four databases, namely, EMBASE, PubMed, EBSCO, and Web of Science, were used to collect articles on vibration. Keywords such as “vibration” and “stroke” were used in the search for published articles. Consequently, eleven studies were selected in the second screening using meta-analyses. Results. The total effect size of patients with dementia in the studies was 0.25, which was small. The effect size of spasticity was the greatest at 1.24 (high), followed by metabolism at 0.99 (high), balance, muscle strength, gait, and circulation in the decreasing order of effect size. Conclusions. The effect sizes for muscle strength and balance and gait function, all of which play an important role in performance of daily activities, were small. In contrast, effect sizes for bone metabolism and spasticity were moderate. This suggests that WBV training may provide a safe, alternative treatment method for improving the symptoms of stroke in patients.

scholarly journals The Effects of Menstrual Cycle Phase on Exercise Performance in Eumenorrheic Women: A Systematic Review and Meta-Analysis

2020 ◽  
Vol 50 (10) ◽  
pp. 1813-1827 ◽  
Author(s):  
Kelly Lee McNulty ◽  
Kirsty Jayne Elliott-Sale ◽  
Eimear Dolan ◽  
Paul Alan Swinton ◽  
Paul Ansdell ◽  
...  
Keyword(s):  
Systematic Review ◽  
Menstrual Cycle ◽  
Effect Size ◽  
Exercise Performance ◽  
Meta Analysis ◽  
Experimental Studies ◽  
Follicular Phase ◽  
Effect Sizes ◽  
Cycle Phase ◽  
Early Follicular Phase

Abstract Background Concentrations of endogenous sex hormones fluctuate across the menstrual cycle (MC), which could have implications for exercise performance in women. At present, data are conflicting, with no consensus on whether exercise performance is affected by MC phase. Objective To determine the effects of the MC on exercise performance and provide evidence-based, practical, performance recommendations to eumenorrheic women. Methods This review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Four databases were searched for published experimental studies that investigated the effects of the MC on exercise performance, which included at least one outcome measure taken in two or more defined MC phases. All data were meta-analysed using multilevel models grounded in Bayesian principles. The initial meta-analysis pooled pairwise effect sizes comparing exercise performance during the early follicular phase with all other phases (late follicular, ovulation, early luteal, mid-luteal and late luteal) amalgamated. A more comprehensive analysis was then conducted, comparing exercise performance between all phases with direct and indirect pairwise effect sizes through a network meta-analysis. Results from the network meta-analysis were summarised by calculating the Surface Under the Cumulative Ranking curve (SUCRA). Study quality was assessed using a modified Downs and Black checklist and a strategy based on the recommendations of the Grading of Recommendations Assessment Development and Evaluation (GRADE) working group. Results Of the 78 included studies, data from 51 studies were eligible for inclusion in the initial pairwise meta-analysis. The three-level hierarchical model indicated a trivial effect for both endurance- and strength-based outcomes, with reduced exercise performance observed in the early follicular phase of the MC, based on the median pooled effect size (ES0.5 = − 0.06 [95% credible interval (CrI): − 0.16 to 0.04]). Seventy-three studies had enough data to be included in the network meta-analysis. The largest effect was identified between the early follicular and the late follicular phases of the MC (ES0.5 = − 0.14 [95% CrI: − 0.26 to − 0.03]). The lowest SUCRA value, which represents the likelihood that exercise performance is poor, or among the poorest, relative to other MC phases, was obtained for the early follicular phase (30%), with values for all other phases ranging between 53 and 55%. The quality of evidence for this review was classified as “low” (42%). Conclusion The results from this systematic review and meta-analysis indicate that exercise performance might be trivially reduced during the early follicular phase of the MC, compared to all other phases. Due to the trivial effect size, the large between-study variation and the number of poor-quality studies included in this review, general guidelines on exercise performance across the MC cannot be formed; rather, it is recommended that a personalised approach should be taken based on each individual's response to exercise performance across the MC.

scholarly journals Effect Sizes, Power, and Biases in Intelligence Research: A Meta-Meta-Analysis

2018 ◽  
Author(s):  
Michele B. Nuijten ◽  
Marcel A. L. M. van Assen ◽  
Hilde Augusteijn ◽  
Elise Anne Victoire Crompvoets ◽  
Jelte M. Wicherts
Keyword(s):  
Publication Bias ◽  
Effect Size ◽  
Meta Analysis ◽  
Convincing Evidence ◽  
Small Study ◽  
Effect Sizes ◽  
Medium Effect ◽  
Average Effect ◽  
Average Effect Size ◽  
Meta Analyses

In this meta-study, we analyzed 2,442 effect sizes from 131 meta-analyses in intelligence research, published from 1984 to 2014, to estimate the average effect size, median power, and evidence for bias. We found that the average effect size in intelligence research was a Pearson’s correlation of .26, and the median sample size was 60. Furthermore, across primary studies, we found a median power of 11.9% to detect a small effect, 54.5% to detect a medium effect, and 93.9% to detect a large effect. We documented differences in average effect size and median estimated power between different types of in intelligence studies (correlational studies, studies of group differences, experiments, toxicology, and behavior genetics). On average, across all meta-analyses (but not in every meta-analysis), we found evidence for small study effects, potentially indicating publication bias and overestimated effects. We found no differences in small study effects between different study types. We also found no convincing evidence for the decline effect, US effect, or citation bias across meta-analyses. We conclude that intelligence research does show signs of low power and publication bias, but that these problems seem less severe than in many other scientific fields.

Unresolved Heterogeneity in Meta-Analysis: Combined Construct Invalidity, Confounding, and Other Challenges to Understanding Mean Effect Sizes

2020 ◽  
Vol 46 (2-3) ◽  
pp. 343-354 ◽  
Author(s):  
Timothy R Levine ◽  
René Weber
Keyword(s):  
Content Analysis ◽  
Publication Bias ◽  
Effect Size ◽  
Meta Analysis ◽  
Effect Sizes ◽  
Size Distributions ◽  
Questionable Research Practices ◽  
Graphical Displays ◽  
Meta Analyses ◽  
Effect Heterogeneity

Abstract We examined the interplay between how communication researchers use meta-analyses to make claims and the prevalence, causes, and implications of unresolved heterogeneous findings. Heterogeneous findings can result from substantive moderators, methodological artifacts, and combined construct invalidity. An informal content analysis of meta-analyses published in four elite communication journals revealed that unresolved between-study effect heterogeneity was ubiquitous. Communication researchers mainly focus on computing mean effect sizes, to the exclusion of how effect sizes in primary studies are distributed and of what might be driving effect size distributions. We offer four recommendations for future meta-analyses. Researchers are advised to be more diligent and sophisticated in testing for heterogeneity. We encourage greater description of how effects are distributed, coupled with greater reliance on graphical displays. We council greater recognition of combined construct invalidity and advocate for content expertise. Finally, we endorse greater awareness and improved tests for publication bias and questionable research practices.

Intimacy, Concreteness, and the “Self-Reference Effect”

2002 ◽  
Vol 49 (1) ◽  
pp. 73-79 ◽  
Author(s):  
Uwe Czienskowski ◽  
Stefanie Giljohann
Keyword(s):  
Incidental Learning ◽  
Recall Performance ◽  
Meta Analysis ◽  
Experimental Studies ◽  
Theoretical Explanation ◽  
The Self ◽  
Current Paper ◽  
Effect Sizes ◽  
Meta Analyses ◽  
Self Reference

Abstract. Reference to oneself during incidental learning of words frequently results in better recall performance than reference to other persons. However, this effect occurs under different conditions with differing strength, and sometimes it is even reversed. Meta-analyses and further experimental studies suggest that increased recall performance under a self-referential encoding task occurs only if it is compared with a nonintimate other person and if abstract material is presented, irrespective of the type of previously presented words (adjectives or nouns). In the current paper, two experiments are reported which support the assumption that this intimacy effect on memory only occurs if no pictorial or concrete features of the material (nouns) to be learned can be exploited for an improvement in encoding or remembering the material. All results agree with predicted effect sizes, which were drawn from a meta-analysis and subsequently conducted experimental studies. This suggests that a recall advantage of referring to oneself compared to other persons is subordinate to the effects of concreteness or imageability. Moreover, the current results offer a theoretical explanation of some previously reported but nevertheless puzzling results from imagery instructions, which indicate decreased recall performance for self-reference compared to other-reference.

Auditory Hallucinations and Reduced Language Lateralization in Schizophrenia: A Meta-analysis of Dichotic Listening Studies

2013 ◽  
Vol 19 (4) ◽  
pp. 410-418 ◽  
Author(s):  
Sebastian Ocklenburg ◽  
René Westerhausen ◽  
Marco Hirnstein ◽  
Kenneth Hugdahl
Keyword(s):  
Effect Size ◽  
Dichotic Listening ◽  
Meta Analysis ◽  
Auditory Hallucinations ◽  
Effect Sizes ◽  
Healthy Controls ◽  
Language Lateralization ◽  
Trait Marker ◽  
Listening Studies ◽  
Meta Analyses

AbstractReduced left-hemispheric language lateralization has been proposed to be a trait marker for schizophrenia, but the empirical evidence is ambiguous. Recent studies suggest that auditory hallucinations are critical for whether a patient shows reduced language lateralization. Therefore, the aim of the study was to statistically integrate studies investigating language lateralization in schizophrenia patients using dichotic listening. To this end, two meta-analyses were conducted, one comparing schizophrenia patients with healthy controls (n = 1407), the other comparing schizophrenia patients experiencing auditory hallucinations with non-hallucinating controls (n = 407). Schizophrenia patients showed weaker language lateralization than healthy controls but the effect size was small (g = −0.26). When patients with auditory hallucinations were compared to non-hallucinating controls, the effect size was substantially larger (g = −0.45). These effect sizes suggest that reduced language lateralization is a weak trait marker for schizophrenia as such and a strong trait marker for the experience of auditory hallucinations within the schizophrenia population. (JINS, 2013, 19, 1–9.)

Sign in / Sign up

Export Citation Format

Share Document