Predictors of teachers' appreciation of evidence-based practice and educational research findings

Abstract. Teachers' reception of educational research is considered important for improving teaching and student learning. Yet, it is a challenging task requiring teachers to have access to scientific sources, the skill and time to find and exhaust such resources, and the capacity to interpret retrieved information. If such essential conditions are not met, teachers have hardly any chance to engage in research reception and, consequently, may question the value and relevance of research findings to their practice. Prior research has suggested that teachers are indeed critical of educational research findings and rarely refer to them. Based on data from the field trial ( N = 674) and main study ( N = 2,549) of a national extension study of the Programme for International Student Assessment (PISA) 2012 in Germany, this study explored the role of (a) teachers' access to scientific sources, (b) perceived lack of skill and time to search for research findings, and (c) their familiarity with research methods/statistics as potential predictors of their appreciation of evidence-based practice, and perceived irrelevance of educational research findings. Structural equation models demonstrated that perceived lack of skill and time to find research findings, in particular, substantially affected participants' irrelevance perceptions. The more participants assessed their sourcing skill and time to be too constrained to engage in research reception, the more they judged research findings to be irrelevant to their practice. Though source access and familiarity with research methods/statistics indicated only small or even no effects, they strongly correlated with participants' perceived lack of sourcing skill and time. Better source access and greater familiarity were associated with less concern about one's skill and time resources to search for relevant research findings. These findings potentially underline the relevance of strengthening both teachers' access to scientific sources and individual capacities to understanding research contents.

Download Full-text

Variable that influence achievement of indonesian students in the program international student assessment (PISA) 2015 using structural equation modelling (SEM)

Journal of Physics Conference Series ◽

10.1088/1742-6596/1521/4/042041 ◽

2020 ◽

Vol 1521 ◽

pp. 042041

Author(s):

V Eminita ◽

K A Notodiputro ◽

B Sartono

Keyword(s):

Structural Equation Modelling ◽

Structural Equation ◽

International Student ◽

Student Assessment ◽

International Student Assessment ◽

Equation Modelling

Download Full-text

The Dimensionality of Reading Self-Concept: Examining Its Stability Using Local Structural Equation Models

Assessment ◽

10.1177/10731911211069675 ◽

2022 ◽

pp. 107319112110696

Author(s):

Geetanjali Basarkod ◽

Herbert W. Marsh ◽

Baljinder K. Sahdra ◽

Philip D. Parker ◽

Jiesi Guo ◽

...

Keyword(s):

Reading Achievement ◽

Structural Equation ◽

Large Scale ◽

Structural Equation Models ◽

Student Assessment ◽

Cognitive Testing ◽

Self Concept ◽

International Student Assessment ◽

High Competence ◽

Achievement Levels

For results from large-scale surveys to inform policy and practice appropriately, all participants must interpret and respond to items similarly. While organizers of surveys assessing student outcomes often ensure this for achievement measures, doing so for psychological questionnaires is also critical. We demonstrate this by examining the dimensionality of reading self-concept—a crucial psychological construct for several outcomes—across reading achievement levels. We use Programme for International Student Assessment 2018 data ( N = 529,966) and local structural equation models (LSEMs) to do so. Results reveal that reading self-concept dimensions (assessed through reading competence and difficulty) vary across reading achievement levels. Students with low reading achievement show differentiated responses to the two item sets (high competence–high difficulty). In contrast, students with high reading achievement have reconciled responses (high competence–low difficulty). Our results highlight the value of LSEMs in examining factor structure generalizability of constructs in large-scale surveys and call for greater cognitive testing during item development.

Download Full-text

The impact of mathematics anxiety on self-regulated learning and mathematical literacy

Australian Journal of Education ◽

10.1177/0004944120947881 ◽

2020 ◽

Vol 64 (3) ◽

pp. 227-242

Author(s):

Florence Gabriel ◽

Sarah Buckley ◽

Abhinava Barthakur

Keyword(s):

Mathematics Anxiety ◽

Structural Equation ◽

International Student ◽

Mathematics Learning ◽

Student Assessment ◽

Mathematical Literacy ◽

Self Regulated Learning ◽

International Student Assessment ◽

Regulated Learning ◽

The Impact

Self-regulated learning has been shown to have a positive and long-lasting impact on students’ academic development, employability and career progression. Emotions, motivation and metacognition play an important role in students’ ability to monitor and regulate their learning, particularly when studying and engaging with Science, Technology, Engineering and Mathematics content. In this study, we investigated motivational, emotional and cognitive factors involved in self-regulated learning and their role in mathematics learning. Specifically, we analysed the impact of mathematics anxiety and self-regulated learning on mathematical literacy using the Australian subset of Programme for International Student Assessment 2012. Mathematics anxiety is a barrier to mathematical learning and is thought to hinder students’ engagement and the efficiency of their metacognitive processes. Using structural equation modelling, we showed that instrumental motivation and self-concept affect mathematics anxiety, which in turn negatively impacts mathematical literacy by affecting perseverance and self-efficacy. We consider the practical implications of our results and discuss how interventions to reduce students’ mathematics anxiety will allow for the development and/or improvement of self-regulated learning skills in mathematics.

Download Full-text

What Does Evidence-Based Practice Mean to You? A Follow-Up Study Examining School-Based Speech-Language Pathologists' Perspectives on Evidence-Based Practice

American Journal of Speech-Language Pathology ◽

10.1044/2019_ajslp-19-00171 ◽

2020 ◽

Vol 29 (2) ◽

pp. 688-704

Author(s):

Katrina Fulcher-Rood ◽

Anny Castilla-Earls ◽

Jeff Higginbotham

Keyword(s):

Clinical Practice ◽

Research Evidence ◽

Evidence Based Practice ◽

Evidence Based ◽

Clinical Expertise ◽

Speech Language Pathologists ◽

School Based ◽

Research Findings ◽

Ebp Implementation

Purpose The current investigation is a follow-up from a previous study examining child language diagnostic decision making in school-based speech-language pathologists (SLPs). The purpose of this study was to examine the SLPs' perspectives regarding the use of evidence-based practice (EBP) in their clinical work. Method Semistructured phone interviews were conducted with 25 school-based SLPs who previously participated in an earlier study by Fulcher-Rood et al. 2018). SLPs were asked questions regarding their definition of EBP, the value of research evidence, contexts in which they implement scientific literature in clinical practice, and the barriers to implementing EBP. Results SLPs' definitions of EBP differed from current definitions, in that SLPs only included the use of research findings. SLPs seem to discuss EBP as it relates to treatment and not assessment. Reported barriers to EBP implementation were insufficient time, limited funding, and restrictions from their employment setting. SLPs found it difficult to translate research findings to clinical practice. SLPs implemented external research evidence when they did not have enough clinical expertise regarding a specific client or when they needed scientific evidence to support a strategy they used. Conclusions SLPs appear to use EBP for specific reasons and not for every clinical decision they make. In addition, SLPs rely on EBP for treatment decisions and not for assessment decisions. Educational systems potentially present other challenges that need to be considered for EBP implementation. Considerations for implementation science and the research-to-practice gap are discussed.

Download Full-text

Three Paradoxical Effects on Academic Self-Concept Across Countries, Schools, and Students

European Psychologist ◽

10.1027/1016-9040/a000332 ◽

2019 ◽

Vol 24 (3) ◽

pp. 231-242 ◽

Cited By ~ 5

Author(s):

Herbert W. Marsh ◽

Philip D. Parker ◽

Reinhard Pekrun

Keyword(s):

International Student ◽

Student Assessment ◽

Cross Cultural ◽

Frame Of Reference ◽

School Level ◽

Self Concept ◽

Positive Effects ◽

International Student Assessment ◽

Country Level ◽

Starting School

Abstract. We simultaneously resolve three paradoxes in academic self-concept research with a single unifying meta-theoretical model based on frame-of-reference effects across 68 countries, 18,292 schools, and 485,490 15-year-old students. Paradoxically, but consistent with predictions, effects on math self-concepts were negative for: • being from countries where country-average achievement was high; explaining the paradoxical cross-cultural self-concept effect; • attending schools where school-average achievement was high; demonstrating big-fish-little-pond-effects (BFLPE) that generalized over 68 countries, Organisation for Economic Co-operation and Development (OECD)/non-OECD countries, high/low achieving schools, and high/low achieving students; • year-in-school relative to age; unifying different research literatures for associated negative effects for starting school at a younger age and acceleration/skipping grades, and positive effects for starting school at an older age (“academic red shirting”) and, paradoxically, even for repeating a grade. Contextual effects matter, resulting in significant and meaningful effects on self-beliefs, not only at the student (year in school) and local school level (BFLPE), but remarkably even at the macro-contextual country-level. Finally, we juxtapose cross-cultural generalizability based on Programme for International Student Assessment (PISA) data used here with generalizability based on meta-analyses, arguing that although the two approaches are similar in many ways, the generalizability shown here is stronger in terms of support for the universality of the frame-of-reference effects.

Download Full-text

Are There Test Administrator Effects in Large-Scale Educational Assessments?

Methodology ◽

10.1027/1614-2241.3.4.149 ◽

2007 ◽

Vol 3 (4) ◽

pp. 149-159 ◽

Cited By ~ 9

Author(s):

Oliver Lüdtke ◽

Alexander Robitzsch ◽

Ulrich Trautwein ◽

Frauke Kreuter ◽

Jan Marten Ihme

Keyword(s):

Large Scale ◽

International Student ◽

Student Assessment ◽

Student Groups ◽

Mathematics Scores ◽

International Student Assessment ◽

Sample Attrition ◽

Educational Assessments ◽

Test Administrator ◽

Practical Implications

Abstract. In large-scale educational assessments such as the Third International Mathematics and Sciences Study (TIMSS) or the Program for International Student Assessment (PISA), sizeable numbers of test administrators (TAs) are needed to conduct the assessment sessions in the participating schools. TA training sessions are run and administration manuals are compiled with the aim of ensuring standardized, comparable, assessment situations in all student groups. To date, however, there has been no empirical investigation of the effectiveness of these standardizing efforts. In the present article, we probe for systematic TA effects on mathematics achievement and sample attrition in a student achievement study. Multilevel analyses for cross-classified data using Markov Chain Monte Carlo (MCMC) procedures were performed to separate the variance that can be attributed to differences between schools from the variance associated with TAs. After controlling for school effects, only a very small, nonsignificant proportion of the variance in mathematics scores and response behavior was attributable to the TAs (< 1%). We discuss practical implications of these findings for the deployment of TAs in educational assessments.

Download Full-text