The Threshold and Inclusive Approaches to Determining “Best Available Evidence”

2016 ◽  
Vol 38 (4) ◽  
pp. 471-492 ◽  
Author(s):  
Jean Stockard ◽  
Timothy W. Wood
Keyword(s):  
Empirical Studies ◽  
Evidence Based ◽  
Policy Recommendations ◽  
Reading Curriculum ◽  
Literacy Interventions ◽  
What Works ◽  
Inclusive Approach ◽  
What Works Clearinghouse ◽  
Randomized Control ◽  
Program Data

Most evaluators have embraced the goal of evidence-based practice (EBP). Yet, many have criticized EBP review systems that prioritize randomized control trials and use various criteria to limit the studies examined. They suggest this could produce policy recommendations based on small, unrepresentative segments of the literature and recommend a more traditional, inclusive approach. This article reports two empirical studies assessing this criticism, focusing on the What Works Clearinghouse (WWC). An examination of outcomes of 252 WWC reports on literacy interventions found that 6% or fewer of the available studies were selected for review. Half of all intervention reports were based on only one study of a program. Data from 131 studies of a reading curriculum were used to compare conclusions using WWC procedures and more inclusive procedures. Effect estimates from the inclusive approach were more precise and closer to those of other reviews. Implications are discussed.

What Works Best: A Systematic Review of Actual Learning in Marketing and Management Education Research

2021 ◽  
pp. 027347532110039
Author(s):  
Donald R. Bacon ◽  
Kim A. Stewart
Keyword(s):  
Systematic Review ◽  
Education Research ◽  
Teaching Methods ◽  
Management Education ◽  
Empirical Studies ◽  
Evidence Based ◽  
Pedagogical Practice ◽  
What Works ◽  
Pedagogical Research ◽  
Actual Learning

After decades of research in marketing and management education, what do we now know about what educational practices work best for improving actual learning? We answer this question with a qualitative systematic literature review of every contribution published in five marketing and management education journals from inception through May 2020 that provides recommendations for evidence-based best pedagogical practice. Contributions were screened to identify empirical studies that employed measures of actual learning in an appropriate experimental design and reported analyses that met certain statistical standards. Of the 4,160 articles examined, 55 studies met our criteria. Based on the studies’ results, we developed a model for understanding the teaching methods that are most effective for achieving actual learning in marketing and management education. We provide evidence-based pedagogical recommendations for faculty and recommendations for additional research in key areas and for increasing the rigor of pedagogical research.

scholarly journals Relevance of the IES/NSF Protocol to Identification of Evidence-Based Practices

2018 ◽  
Vol 8 (1) ◽  
Author(s):  
Barbara R. Schirmer ◽  
Alison S. Lockman ◽  
Todd N. Schirmer
Keyword(s):  
Reciprocal Teaching ◽  
Book Reading ◽  
Shared Book Reading ◽  
Evidence Based ◽  
Instructional Approach ◽  
Theoretical Frameworks ◽  
Evidence Based Practices ◽  
What Works ◽  
What Works Clearinghouse ◽  
Improving Student Learning

The Institute of Education Sciences (IES) and the National Science Foundation (NSF) issued common guidelines that provide a protocol by which the use of particular methodological designs in a line of research inquiry provides evidence for each successive step in the process of bringing any given instructional intervention into practice. Our purpose was to determine if research on two widely used literacy instruction approaches has been conducted at each methodological stage in the IES/NSF protocol and is relevant to identifying the approach as an evidence-based practice. We applied the IES/NSF pipeline-of-evidence guidelines to assess whether practices touted as having a research base for effectiveness have emerged from an accumulation of empirical evidence and identification of conceptual or theoretical frameworks. In mapping the six steps of the IES/NSF protocol onto the shared book reading and reciprocal teaching studies that had met What Works Clearinghouse evidence standards, we found that only reciprocal teaching involved research at each stage in the protocol and only reciprocal teaching was identified as an evidence-based instructional approach by the What Works Clearinghouse. Our results indicate that the IES/NSF pipeline-of-evidence protocol offers a productive approach to identifying evidence-based practices. The different trajectories of research on reciprocal teaching and shared book reading indicates that research at each stage in the protocol is important to the development of an instructional approach that ultimately demonstrates effectiveness in improving student learning outcomes.

A Quantitative Synthesis of Intervention Research Published in Flagship EBD Journals: 2010 to 2019

2020 ◽  
pp. 019874292096134
Author(s):  
Justin D. Garwood ◽  
Corey Peltier ◽  
Tracy Sinclair ◽  
Heather Eisel ◽  
John W. McKenna ◽  
...  
Keyword(s):  
Student Outcomes ◽  
Emotional And Behavioral Disorders ◽  
Empirical Studies ◽  
Intervention Research ◽  
Future Directions ◽  
What Works ◽  
Effective Interventions ◽  
Quantitative Synthesis ◽  
What Works Clearinghouse ◽  
Standardized Mean Difference

Students with emotional and behavioral disorders (EBDs) could be considered some of the most challenging students to serve in schools. The need for effective interventions for these students is ever-present. To design and implement empirical studies to better inform the field of EBDs, researchers must have a firm understanding of the most up-to-date intervention literature. The purpose of this targeted quantitative synthesis is to create such a knowledge base for the field of EBDs. Results from 55 studies indicate a declining focus on intervention research for students with EBDs in the last 10 years (2010–2019). Of the intervention research available, the quality (40% did not meet What Works Clearinghouse standards) and effectiveness on student outcomes across academic, behavioral, and social skills domains was variable (between-case standardized mean difference [BC-SMD] = 0.13–8.26, Hedges’ g = −0.30 to 1.29). Future directions for the field of EBDs are included.

Synthesizing Evidence in Public Policy Contexts

2016 ◽  
Vol 41 (1) ◽  
pp. 3-26 ◽  
Author(s):  
Jeffrey C. Valentine ◽  
Sandra Jo Wilson ◽  
David Rindskopf ◽  
Timothy S. Lau ◽  
Emily E. Tanner-Smith ◽  
...  
Keyword(s):  
Meta Analysis ◽  
Research Question ◽  
Information Environment ◽  
Evidence Based ◽  
Limited Information ◽  
Governmental Agencies ◽  
What Works ◽  
Inverse Variance ◽  
What Works Clearinghouse ◽  
The U.S

For a variety of reasons, researchers and evidence-based clearinghouses synthesizing the results of multiple studies often have very few studies that are eligible for any given research question. This situation is less than optimal for meta-analysis as it is usually practiced, that is, by employing inverse variance weights, which allows more informative studies to contribute relatively more to the analysis. This article outlines the choices available for synthesis when there are few studies to synthesize. As background, we review the synthesis practices used in several projects done at the behest of governmental agencies and private foundations. We then discuss the strengths and limitations of different approaches to meta-analysis in a limited information environment. Using examples from the U.S. Department of Education’s What Works Clearinghouse as case studies, we conclude with a discussion of Bayesian meta-analysis as a potential solution to the challenges encountered when attempting to draw inferences about the effectiveness of interventions from a small number of studies.

An empirical review of gambling expansion and gambling-related harm

2018 ◽  
Vol 64 (5-6) ◽  
pp. 295-306 ◽  
Author(s):  
Debi A. LaPlante ◽  
Heather M. Gray ◽  
Pat M. Williams ◽  
Sarah E. Nelson
Keyword(s):  
Literature Review ◽  
Peer Review ◽  
Empirical Data ◽  
Empirical Studies ◽  
Expansion Method ◽  
Responsible Gambling ◽  
Review Literature ◽  
Evidence Based ◽  
Empirical Comparison ◽  
Related Harm

Abstract. Aims: To discuss and review the latest research related to gambling expansion. Method: We completed a literature review and empirical comparison of peer reviewed findings related to gambling expansion and subsequent gambling-related changes among the population. Results: Although gambling expansion is associated with changes in gambling and gambling-related problems, empirical studies suggest that these effects are mixed and the available literature is limited. For example, the peer review literature suggests that most post-expansion gambling outcomes (i. e., 22 of 34 possible expansion outcomes; 64.7 %) indicate no observable change or a decrease in gambling outcomes, and a minority (i. e., 12 of 34 possible expansion outcomes; 35.3 %) indicate an increase in gambling outcomes. Conclusions: Empirical data related to gambling expansion suggests that its effects are more complex than frequently considered; however, evidence-based intervention might help prepare jurisdictions to deal with potential consequences. Jurisdictions can develop and evaluate responsible gambling programs to try to mitigate the impacts of expanded gambling.

“State standards and IEP goals. A lot of TPT products.” What Early Childhood Educators Report Using to Plan Mathematics Instruction

2020 ◽  
Author(s):  
Corey Peltier ◽  
Tiffany K Peltier ◽  
Taylor Werthen ◽  
Andy Heuer
Keyword(s):  
Special Education ◽  
Early Childhood ◽  
Special Education Teachers ◽  
Mathematics Instruction ◽  
State Standards ◽  
Early Childhood Educators ◽  
Future Research ◽  
Exceptional Children ◽  
What Works ◽  
What Works Clearinghouse

Access to high-quality resources is integral for educators to provide research-aligned mathematics instruction. Identifying the supplemental resources educators use to plan mathematics instruction can inform the ways researchers and organizations disseminate research-based practices. The goal of this study was to identify the frequency in which early childhood educators (i.e., pre-Kindergarten through third grade) reported using various resources to plan for mathematics instruction. Furthermore, we investigated whether differences were observed based on teacher factors (i.e., general or special education, route to certification, years of experience) and locale (i.e., rural, urban, suburban). We retained data from 917 teachers for data analysis. The three most frequently reported resources by educators were colleagues, Teachers Pay Teachers, and Google/Yahoo. The three least frequently reported resources were the typical outlets researchers use to reach teachers: What Works Clearinghouse, Teaching Exceptional Children, and Teaching Children Mathematics. General and special education teachers differed on their self-reported usage of five resources: colleagues, Google/Yahoo, teaching blogs, Teaching Exceptional Children, and the What Works Clearinghouse. Rural educators self-reported that they were less likely than suburban educators to use colleagues or specialists at the district to plan instruction. Implications for future research and practice are discussed.

Sign in / Sign up

Export Citation Format

Share Document