Data Literacy Training for K–12 Teachers: A Meta-Analysis of the Effects on Teacher Outcomes

This meta-analysis explores training teachers in the use of data, defined as any quantifiable information that helps teachers know more about their students for instructional decision-making. The questions addressed are as follows: (a) What are the features of data literacy training for kindergarten through 12th-grade teachers? (b) What are the effects of data literacy training on kindergarten through 12th-grade teacher outcomes? and (c) Do training characteristics moderate the effects of training? A comprehensive search of research conducted between 1975 and 2019 yielded 33 studies with 163 effect sizes that met inclusion criteria. Using a random effects model, findings demonstrated significant positive effects on knowledge and skills, g = .67, 95% confidence interval (CI) = [0.40, 0.93], and beliefs, g = .48, 95% CI = [0.17, 0.79]. A collaborative training format significantly and positively moderated effects. Implications for teacher trainings and the design of future research are discussed.

Game-Based Learning Analytics for Supporting Adolescents’ Reflection

Reflection is critical for adolescents’ problem solving and learning in game-based learning environments (GBLEs). Yet challenges exist in the literature because most studies lack a theoretical perspective and clear operational definition to inform how and when reflection should be scaffolded during game-based learning. In this paper, we address these issues by studying the quantity and quality of 120 adolescents’ written reflections and their relation to their learning and problem solving with Crystal Island, a GBLE. Specifically, we (1) define reflection and how it relates to skill and knowledge acquisition; (2) review studies examining reflection and its relation to problem solving and learning with emerging technologies; and (3) provide direction for building reflection prompts into GBLEs that are aligned with the learning goals built into the learning session (e.g., learn about microbiology versus successfully solve a problem) to maximize adolescents’ reflection, learning, and performance. Overall, our findings emphasize how important it is to examine not only the quantity of reflection but also the depth of written reflection as it relates to specific learning goals. We discuss the implications of using game-learning analytics to guide instructional decision making in the classroom.

A Collaborative Approach to Sharing Learner Event Data

This paper describes a collaboration organized around exchanging data between two technological systems to support teachers’ instructional decision-making. The goals of the collaboration among researchers, technology developers, and practitioners were not only to support teachers’ instructional decision-making but also to document the challenges and opportunities associated with bringing together data from instruction- and assessment-focused technologies. The approach described in this paper illustrates the potential importance of anchoring data products that combine data between two systems in the needs of teachers as well as aligning the content that students learn and are assessed on between systems. The increasing presence of data standards has made sharing complex data increasingly more feasible. The example collaboration described in this paper demonstrates the role that non-technical activities can play in supporting the exchange and use of learner event data.

Validation of a Learning Trajectory-Based Diagnostic Mathematics Assessment System as a Trading Zone

This study reports how a validation argument for a learning trajectory (LT) is constituted from test design, empirical recovery, and data use through a collaborative process, described as a “trading zone” among learning scientists, psychometricians, and practitioners. The validation argument is tied to a learning theory about learning trajectories and a framework (LT-based data-driven decision-making, or LT-DDDM) to guide instructional modifications. A validation study was conducted on a middle school LT on “Relations and Functions” using a Rasch model and stepwise regression. Of five potentially non-conforming items, three were adjusted, one retained to collect more data, and one was flagged as a discussion item. One LT level description was revised. A linear logistic test model (LLTM) revealed that LT level and item type explained substantial variance in item difficulty. Using the LT-DDDM framework, a hypothesized teacher analysis of a class report led to three conjectures for interventions, demonstrating the LT assessment’s potential to inform instructional decision-making.

The Assessment Evaluation Rubric: Promoting Learning and Learner-Centered Teaching through Assessment in Face-to-Face or Distanced Higher Education

It is common to create courses for the higher education context that accomplish content-driven teaching goals and then develop assessments (quizzes and exams) based on the target content. However, content-driven assessment can tend to support teaching- or teacher-centered instruction. Adult learning and educational psychology theories suggest that instead, assessment should be aligned with learning, not teaching, objectives. To support the alignment of assessments with instruction in higher education, the Assessment Evaluation Rubric (AER) was developed. The AER can be utilized to guide the development and evaluation/revision of assessments that are already used. The AER describes, or permits the evaluation of, four features of an assessment: its general alignment with learning goal(s), whether the assessment is intended to/effective as formative or summative, whether some systematic approach to cognitive complexity is reflected, and whether the assessment (instructions as well as results) itself is clearly interpretable. Each dimension (alignment, utility, complexity, and clarity) has four questions that can be rated as present/absent. Other rating methods can also be conceptualized for the AER’s 16 questions, depending on the user’s intent. Any instructor can use the AER to evaluate their own assessments and ensure that they—or new assessments in development—will promote learning and learner-centered teaching. As instructors shift from face-to-face toward virtual or hybrid teaching models, or as they shift online instruction (back) to face-to-face teaching, it creates an ideal opportunity to ensure that assessment is optimizing learning and is valid for instructional decision-making.

Precision of Single-Skill Mathematics CBM: Group Versus Individual Administration

The current availability of research examining the precision of single-skill mathematics (SSM) curriculum-based measurements (CBMs) for progress monitoring is limited. Given the observed variance in administration conditions across current practice and research use, we examined potential differences between student responding and precision of slope when SSM-CBMs were administered individually and in group (classroom) conditions. No differences in student performance or measure precision were observed between conditions, indicating flexibility in the practical and research use of SSM-CBMs across administration conditions. In addition, findings contributed to the literature examining the stability of SSM-CBMs slopes of progress when used for instructional decision-making. Implications for the administration and interpretation of SSM-CBMs in practice are discussed.

International Perspectives on Understanding and Improving Teachers’ Data-Based Instruction and Decision Making: Introduction to the Special Series

This article serves as an introduction to the special series, Data-Based Instruction and Decision-Making: An International Perspective. In this series, we bring together international researchers from both special and general education to address teachers’ use (or non-use) of data for instructional decision making. Via this special series, we aim to increase understanding of the challenges involved in teachers’ data-based instructional decision making for students with or at-risk for learning disabilities, and to further the development of approaches for improving teachers’ ability to plan, adjust, and adapt instruction in response to data.

Bringing Data-Based Individualization to Scale: A Call for the Next-Generation Technology of Teacher Supports

The purpose of this narrative synthesis of the curriculum-based measure (CBM) instructional utility literature is to deepen insight into the supports required to enrich teachers’ instructional decision-making within curriculum-based measure –data-based individualization (CBM-DBI) in ways that enhance the learning outcomes of students with intensive intervention needs, including students with learning disabilities. We begin by summarizing a recent meta-analysis of CBM-DBI studies focused on academic outcomes. We then reconsider studies from that meta-analysis to further explore the supports required to enrich teachers’ instructional decision-making within CBM-DBI and improve student learning. We next draw conclusions and propose a renewed program of instructional utility CBM-DBI research for capitalizing on technology’s potential to enhance productive instructional decision-making for students who require intensive intervention, fulfill DBI’s potential, and bring CBM-DBI to scale.

Steps for Success: Making Instructional Decisions for Students’ Essay Writing

The process of analyzing student data to determine an appropriate instructional decision is crucial for student academic growth. This article details how teachers can make data-driven decisions to carefully design writing instruction. Steps are presented for teachers to follow throughout the data driven decision-making process in order to meet students’ specific needs when they are writing an essay. A case study is provided throughout the article to illustrate how students and teachers may navigate through this process while using a technology-based graphic organizer.

Teacher beliefs about instructional approaches: Interrogating the notion of teaching methods

World language teachers have historically relied on the notion of teaching methods to inform elements of design and procedure in their instructional practice. Teacher beliefs about teaching methods, however, have been shown to be significantly influenced by their context, including their institution and their learners. This phenomenon has led some scholars to identify a postmethod condition, where teachers prioritize making responsive, principled decisions about instruction based on their context. This qualitative study investigated the patterns and realities of the postmethod condition in practice through the lens of teacher beliefs about teaching methods, focusing on ten secondary-level world language teachers of French and Spanish in the USA. Data sources included a survey about teaching methods, in-depth interviews, and classroom observations. Data analysis included descriptive statistics, multiple phases of coding, and integrating analysis of the three sources. Findings indicated that teachers in this group largely identified as adhering to one main teaching approach, with eight of the ten self-identifying as using primarily comprehensible input and/or TPRS (Teaching Proficiency through Reading and Storytelling) methods. However, through investigating their beliefs about grammar and accuracy; the four skills of reading, writing, listening, and speaking; the importance of input and output; and instructional flow, we found that the teachers examined and reexamined their teaching methods regularly, largely due to the influences of their learners and their institution. The relationship between the teachers’ beliefs and practices was mediated by context-driven instructional decision-making, indicating the presence of a postmethod condition.