Test Frequency, Stakes, and Feedback in Student Achievement: A Meta-Analysis

2019 ◽  
Vol 43 (3-4) ◽  
pp. 111-151 ◽  
Author(s):  
Richard P. Phelps
Keyword(s):  
Student Achievement ◽  
Early Childhood ◽  
Effect Size ◽  
English Language ◽  
Variance Estimation ◽  
Meta Analysis ◽  
The United States ◽  
Research Literature ◽  
Effect Sizes ◽  
Test Frequency

Background: Test frequency, stakes associated with educational tests, and feedback from test results have been identified in the research literature as relevant factors in student achievement. Objectives: Summarize the separate and joint contribution to student achievement of these three treatments and their interactions via multivariable meta-analytic techniques using a database of English-language studies spanning a century (1910–2010), comprising 149 studies and 509 effect size estimates. Research design: Analysis employed robust variance estimation. Considered as potential moderators were hundreds of study features comprising various test designs and test administration, demographic, and source document characteristics. Subjects: Subjects were students at all levels, from early childhood to adult, mostly from the United States but also eight other countries. Results: We find a summary effect size of 0.84 for the three treatments collectively. Further analysis suggests benefits accrue to the incremental addition of combinations of testing and feedback or stakes and feedback. Moderator analysis shows higher effect sizes associated with the following study characteristics: more recent year of publication, summative (rather than formative) testing, constructed (rather than selected) item response formats, alignment of subject matter between pre- and posttests, and recognition/recall (rather than core subjects, art, or physical education). Conversely, lower effect sizes are associated with postsecondary students (rather than early childhood–upper secondary), special education population, larger study population, random assignment (rather than another sampling method), use of shadow test as outcome measure, designation of individuals (rather than groups) as units of analysis, and academic (rather than corporate or government) research.

scholarly journals Pharmacist-Led Interventions to Improve Medication Adherence in Older Adults: A Meta-Analysis

2021 ◽  
Vol 5 (Supplement_1) ◽  
pp. 335-335
Author(s):  
Zachary Marcum ◽  
Shangqing Jiang ◽  
Jennifer Bacci ◽  
Todd Ruppar
Keyword(s):  
Older Adults ◽  
Medication Adherence ◽  
Effect Size ◽  
English Language ◽  
Randomized Clinical Trials ◽  
Meta Analysis ◽  
Effect Sizes ◽  
Analysis Model ◽  
Improve Medication Adherence ◽  
The Mean

Abstract As pharmacists work to ensure reimbursement for chronic disease management services on the national (e.g., Medicare) level, summative evidence of their impact on important health metrics, such as medication adherence, is needed. The objective of this study was to assess the effectiveness of pharmacist-led interventions on medication adherence in older adults. In April 2020, a comprehensive search was conducted in six databases for publications of randomized clinical trials of pharmacist-led interventions to improve medication adherence in older adults. English-language studies with codable data on medication adherence and diverse adherence-promoting interventions targeting older adults (age 65+) were eligible. A standardized mean difference effect size (intervention vs. control) was calculated for the medication adherence outcome in each study. Study effect sizes were pooled using a random-effects meta-analysis model. Moderator analyses were then conducted to explore for differences in effect size due to intervention, sample, and study characteristics. The primary outcome was medication adherence using any method of measurement. This meta-analysis included 40 unique randomized trials of pharmacist-led interventions with data from 8,822 unique patients (mean age, range: 65 to 85 years). The mean effect size was 0.57 (95% Confidence Interval [CI]: 0.38-0.76). When two outlier studies were excluded from the analysis, the mean effect size decreased to 0.41 (95% CI: 0.27-0.54). Moderator analyses showed larger effect sizes for interventions containing medication education and when interventions had components delivered at least partly in patients’ homes. In conclusion, this meta-analysis found a significant improvement in medication adherence among older adults receiving pharmacist-led interventions.

A Systematic Review and Meta-Analysis of the Relation Between Engagement and Achievement in Early Childhood Research

2021 ◽  
pp. 027112142110327
Author(s):  
Esther R. Lindström ◽  
Jason C. Chow ◽  
Kathleen N. Zimmerman ◽  
Hongyang Zhao ◽  
Elise Settanni ◽  
...  
Keyword(s):  
Early Childhood ◽  
Academic Engagement ◽  
Literature Search ◽  
Meta Analysis ◽  
Effect Sizes ◽  
Data Sets ◽  
Systematic Literature Search ◽  
Random Effects Models ◽  
Unique Data ◽  
Early Childhood Research

Engagement in early childhood has been linked with later achievement, but the relation between these variables and how they are measured in early childhood requires examination. We estimated the overall association between academic engagement and achievement in children prior to kindergarten entry. Our systematic literature search yielded 13,521 reports for structured eligibility screening; from this pool of studies, we identified 21 unique data sets, with 199 effect sizes for analysis. We coded eligible studies, extracted effect sizes, accounted for effect size dependency, and used random-effects models to synthesize findings. The overall correlation between academic engagement and achievement was r = .24 (range: −.08 to −.71), and moderator analyses did not significantly predict the relation between the two constructs. This study aligns with previous research on this topic and examines issues related to these measures, their constraints, and applications as they pertain to early childhood research.

scholarly journals Pulmonary Health Effects of Indoor Volatile Organic Compounds—A Meta-Analysis

2021 ◽  
Vol 18 (4) ◽  
pp. 1578 ◽  
Author(s):  
Kyle L. Alford ◽  
Naresh Kumar
Keyword(s):  
Volatile Organic Compounds ◽  
Organic Compounds ◽  
Effect Size ◽  
Meta Analysis ◽  
The United States ◽  
High Income ◽  
Pulmonary Diseases ◽  
Health Concern ◽  
Pubmed Database ◽  
Volatile Organic

Volatile organic compounds (VOCs) are commonly found in consumer products, including furniture, sealants and paints. Thus, indoor VOCs have become a public health concern, especially in high-income countries (HICs), where people spend most of their time indoors, and indoor and outdoor air exchange is minimal due to a lack of ventilation. VOCs produce high levels of reaction with the airway epithelium and mucosa membrane and is linked with pulmonary diseases. This paper takes a stock of the literature to assess the strength of association (measured by effect size) between VOCs and pulmonary diseases with the focus on asthma and its related symptoms by conducting a meta-analysis. The literature was searched using the PubMed database. A total of 49 studies that measured VOCs or VOC types and pulmonary health outcomes were included in the analysis. The results of these studies were tabulated, and standard effect size of each study was computed. Most studies were conducted in high-income countries, including France (n = 7), Japan (n = 7) and the United States (n = 6). Our analysis suggests that VOCs have a medium-sized effect on pulmonary diseases, including the onset of asthma (effect size (or Cohen’s d) ~0.37; 95% confidence interval (CI) = 0.25–0.49; n = 23) and wheezing (effective size ~0.26; 95% CI = 0.10–0.42; n = 10). The effect size also varied by country, age and disease type. Multiple stakeholders must be engaged in strategies to mitigate and manage VOC exposure and its associated pulmonary disease burden.

scholarly journals Morning Report: Can an Established Medical Education Tradition Be Validated?

2013 ◽  
Vol 5 (3) ◽  
pp. 374-384 ◽  
Author(s):  
Matthew McNeill ◽  
Sayed K. Ali ◽  
Daniel E. Banks ◽  
Ishak A. Mansi
Keyword(s):  
Medical Education ◽  
Patient Care ◽  
Outcome Measures ◽  
English Language ◽  
Meta Analysis ◽  
The United States ◽  
Cochrane Library ◽  
Clinical Effects ◽  
Morning Report ◽  
Residency Programs

Abstract Background Morning report is accepted as an essential component of residency education throughout different parts of the world. Objective To review the evidence of the educational value, purpose, methods, and outcomes of morning report. Methods A literature search of PubMed, Ovid, and the Cochrane Library for English-language studies published between January 1, 1966, and October 31, 2011, was performed. We searched for keywords and Medical Subject Heading terms related to medical education, methods, attitudes, and outcomes in regard to “morning report.” Title and abstract review, followed by a full-text review by 3 authors, was performed to identify all pertinent articles. Results We identified 71 citations; 40 articles were original studies and 31 were commentaries, editorials, or review articles; 56 studies (79%) originated from internal medicine residency programs; 6 studies (8%) focused on ambulatory morning report; and 63 (89%) originated from the United States. Identified studies varied in objectives, methods, and outcome measures, and were not suitable for meta-analysis. Main outcome measures were resident satisfaction, faculty satisfaction, preparation for professional examinations, use of evidence-based medicine, clinical effects on patient care, adverse event detection, and utilization of a curriculum in case selection. Conclusions Morning report has heterogeneous purposes, methods, and settings. As an educational tool, morning report is challenging to define, its outcome is difficult to measure, and this precludes firm conclusions about its contribution to resident education or patient care. Residency programs should tailor morning report to meet their own unique educational objectives and needs.

scholarly journals A Statistical Method for Synthesizing Meta-Analyses

2013 ◽  
Vol 2013 ◽  
pp. 1-9 ◽  
Author(s):  
Liansheng Larry Tang ◽  
Michael Caudy ◽  
Faye Taxman
Keyword(s):  
Statistical Method ◽  
Effect Size ◽  
Meta Analysis ◽  
Effect Sizes ◽  
Weighted Mean ◽  
Summary Effect ◽  
Different Types ◽  
Meta Analyses ◽  
Size Estimates ◽  
Similar Search

Multiple meta-analyses may use similar search criteria and focus on the same topic of interest, but they may yield different or sometimes discordant results. The lack of statistical methods for synthesizing these findings makes it challenging to properly interpret the results from multiple meta-analyses, especially when their results are conflicting. In this paper, we first introduce a method to synthesize the meta-analytic results when multiple meta-analyses use the same type of summary effect estimates. When meta-analyses use different types of effect sizes, the meta-analysis results cannot be directly combined. We propose a two-step frequentist procedure to first convert the effect size estimates to the same metric and then summarize them with a weighted mean estimate. Our proposed method offers several advantages over existing methods by Hemming et al. (2012). First, different types of summary effect sizes are considered. Second, our method provides the same overall effect size as conducting a meta-analysis on all individual studies from multiple meta-analyses. We illustrate the application of the proposed methods in two examples and discuss their implications for the field of meta-analysis.

scholarly journals The use of effect size indices to determine practical significance

2006 ◽  
Vol 25 (3) ◽  
Author(s):  
H. S. Styn ◽  
S. M. Ellis
Keyword(s):  
Effect Size ◽  
Statistical Significance ◽  
Empirical Studies ◽  
Research Literature ◽  
Effect Sizes ◽  
Practical Significance ◽  
Significance Tests ◽  
Statistical Application ◽  
Significant Difference

The determination of significance of differences in means and of relationships between variables is of importance in many empirical studies. Usually only statistical significance is reported, which does not necessarily indicate an important (practically significant) difference or relationship. With studies based on probability samples, effect size indices should be reported in addition to statistical significance tests in order to comment on practical significance. Where complete populations or convenience samples are worked with, the determination of statistical significance is strictly speaking no longer relevant, while the effect size indices can be used as a basis to judge significance. In this article attention is paid to the use of effect size indices in order to establish practical significance. It is also shown how these indices are utilized in a few fields of statistical application and how it receives attention in statistical literature and computer packages. The use of effect sizes is illustrated by a few examples from the research literature.

scholarly journals Changes of Ankle Dorsiflexion Using Compression Tissue Flossing: A Systematic Review and Meta-Analysis

2020 ◽  
pp. 1-9
Author(s):  
Devin S. Kielur ◽  
Cameron J. Powden
Keyword(s):  
Confidence Interval ◽  
Range Of Motion ◽  
Effect Size ◽  
Evidence Synthesis ◽  
Meta Analysis ◽  
Effect Sizes ◽  
Lower Extremity Injury ◽  
Level Of Evidence ◽  
Physically Active ◽  
Evidence Database

Context: Impaired dorsiflexion range of motion (DFROM) has been established as a predictor of lower-extremity injury. Compression tissue flossing (CTF) may address tissue restrictions associated with impaired DFROM; however, a consensus is yet to support these effects. Objectives: To summarize the available literature regarding CTF on DFROM in physically active individuals. Evidence Acquisition: PubMed and EBSCOhost (CINAHL, MEDLINE, and SPORTDiscus) were searched from 1965 to July 2019 for related articles using combination terms related to CTF and DRFOM. Articles were included if they measured the immediate effects of CTF on DFROM. Methodological quality was assessed using the Physiotherapy Evidence Database scale. The level of evidence was assessed using the Strength of Recommendation Taxonomy. The magnitude of CTF effects from pre-CTF to post-CTF and compared with a control of range of motion activities only were examined using Hedges g effect sizes and 95% confidence intervals. Randomeffects meta-analysis was performed to synthesize DFROM changes. Evidence Synthesis: A total of 6 studies were included in the analysis. The average Physiotherapy Evidence Database score was 60% (range = 30%–80%) with 4 out of 6 studies considered high quality and 2 as low quality. Meta-analysis indicated no DFROM improvements for CTF compared with range of motion activities only (effect size = 0.124; 95% confidence interval, −0.137 to 0.384; P = .352) and moderate improvements from pre-CTF to post-CTF (effect size = 0.455; 95% confidence interval, 0.022 to 0.889; P = .040). Conclusions: There is grade B evidence to suggest CTF may have no effect on DFROM when compared with a control of range of motion activities only and results in moderate improvements from pre-CTF to post-CTF. This suggests that DFROM improvements were most likely due to exercises completed rather than the band application.

Effect Sizes: Conventional Choices and Calculations

2013 ◽  
Author(s):  
Michael S. Rosenberg ◽  
Hannah R. Rothstein ◽  
Jessica Gurevitch
Keyword(s):  
Critical Point ◽  
Effect Size ◽  
Evolutionary Biology ◽  
Meta Analysis ◽  
Statistical Parameter ◽  
Effect Sizes ◽  
Common Effect ◽  
Inference Methods ◽  
Conventional Effect ◽  
Estimate Effect Size

One of the fundamental concepts in meta-analysis is that of the effect size. An effect size is a statistical parameter that can be used to compare, on the same scale, the results of different studies in which a common effect of interest has been measured. This chapter describes the conventional effect sizes most commonly encountered in ecology and evolutionary biology, and the types of data associated with them. While choice of a specific measure of effect size may influence the interpretation of results, it does not influence the actual inference methods of meta-analysis. One critical point to remember is that one cannot combine different measures of effect size in a single meta-analysis: once you have chosen how you are going to estimate effect size, you need to use it for all of the studies to be analyzed.

M. Meta-Analysis

2021 ◽  
pp. 163-165
Author(s):  
Noémie Laurens
Keyword(s):  
Literature Review ◽  
Empirical Research ◽  
Effect Size ◽  
Meta Analysis ◽  
Research Synthesis ◽  
Effect Sizes ◽  
Primary Research ◽  
Unit Of Analysis ◽  
Research Studies

This chapter illustrates meta-analysis, which is a specific type of literature review, and more precisely a type of research synthesis, alongside traditional narrative reviews. Unlike in primary research, the unit of analysis of a meta-analysis is the results of individual studies. And unlike traditional reviews, meta-analysis only applies to: empirical research studies with quantitative findings hat are conceptually comparable and configured in similar statistical forms. What further distinguishes meta-analysis from other research syntheses is the method of synthesizing the results of studies — i.e. the use of statistics and, in particular, of effect sizes. An effect size represents the degree to which the phenomenon under study exists.

scholarly journals A-17 Category Test performance in individuals with alcohol versus methamphetamine dependence

2019 ◽  
Vol 34 (6) ◽  
pp. 876-876
Author(s):  
A Walker ◽  
A Hauson ◽  
S Sarkissians ◽  
A Pollard ◽  
C Flora-Tostado ◽  
...  
Keyword(s):  
Effect Size ◽  
Test Performance ◽  
Neuropsychological Functioning ◽  
Meta Analysis ◽  
Effect Sizes ◽  
Significant Heterogeneity ◽  
Category Test ◽  
Methamphetamine Dependence ◽  
Significant Difference ◽  
Axis I

Abstract Objective The Category Test (CT) has consistently been found to be sensitive at detecting the effects of alcohol on the brain. However, this test has not been as widely used in examining the effects of methamphetamine. The current meta-analysis compared effect sizes of studies that have examined performance on the CT in alcohol versus methamphetamine dependent participants. Data selection Three researchers independently searched nine databases (e.g., PsycINFO, Pubmed, ProceedingsFirst), extracted required data, and calculated effect sizes. Inclusion criteria identified studies that had (a) compared alcohol or methamphetamine dependent groups to healthy controls and (b) matched groups on either age, education, or IQ (at least 2 out of 3). Studies were excluded if participants were reported to have Axis I diagnoses (other than alcohol or methamphetamine dependence) or comorbidities known to impact neuropsychological functioning. Sixteen articles were coded and analyzed for the current study. Data synthesis Alcohol studies showed a large effect size (g = 0.745, p < 0.001) while methamphetamine studies evidenced a moderate effect size (g = 0.406, p = 0.001); both without statistically significant heterogeneity (I2 = 0). Subgroup analysis revealed a statistically significant difference between the effect sizes from alcohol versus methamphetamine studies (Q-between = 5.647, p = 0.017). Conclusions The CT is sensitive to the effects of both alcohol and methamphetamine and should be considered when examining dependent patients who might exhibit problem solving, concept formation, and set loss difficulties in everyday living.

Sign in / Sign up

Export Citation Format

Share Document