Assessing the Effects of a School-Wide Data-Based Decision-Making Intervention on Student Achievement Growth in Primary Schools

AbstractThe COVID-19 pandemic produced widespread disruption to schooling, impacting 90% of the world’s students and moving entire school systems to remote and online learning. In the state of New South Wales, Australia, most students engaged in learning from home for at least eight weeks, with subsequent individual and intermittent school closures. However, while numerous claims have circulated in the popular media and in think tank reports, internationally, about the negative impacts on learning, there is limited empirical evidence of decreased student achievement. Drawing on data from more than 4800 Year 3 and 4 students from 113 NSW government schools, this paper compares student achievement during 2019 and 2020 in a sample of matched schools to examine the effects of the system-wide disruption. Somewhat surprisingly, our analysis found no significant differences between 2019 and 2020 in student achievement growth as measured by progressive achievement tests in mathematics or reading. A more nuanced picture emerges when the sample is examined by dis/advantage (ICSEA) and Year level. The Year 3 cohort in the least advantaged schools (ICSEA < 950) achieved 2 months less growth in mathematics, while the Year 3 students in mid-ICSEA schools (950–1050) achieved 2 months’ additional growth. No significant differences were identified for Indigenous students or students located in regional locations. These results provide an important counter-narrative to widespread speculation about alarming levels of ‘learning loss’ for all students. While the lower achievement growth in mathematics for Year 3 students in lower ICSEA schools must be addressed as a matter of urgency to avoid further inequities, most students are, academically, where they are expected to be. Our findings are a testament to the dedicated work of teachers during the 2020 pandemic to ensure that learning for most students was not compromised, despite unusually trying circumstances.

Download Full-text

Effects of a data-based decision making intervention on student achievement

Studies In Educational Evaluation ◽

10.1016/j.stueduc.2017.07.002 ◽

2017 ◽

Vol 55 ◽

pp. 58-67 ◽

Cited By ~ 11

Author(s):

L. (Laura) Staman ◽

A.C. (Anneke) Timmermans ◽

A.J. (Adrie) Visscher

Keyword(s):

Decision Making ◽

Student Achievement ◽

Data Based Decision Making

Download Full-text

Data-based decision making in primary schools in Ethiopia

Journal of Professional Capital and Community ◽

10.1108/jpcc-11-2018-0031 ◽

2019 ◽

Vol 4 (3) ◽

pp. 232-259

Author(s):

Ahmed Yibrie Ahmed

Keyword(s):

Decision Making ◽

School Improvement ◽

Primary Schools ◽

Data Use ◽

Monitoring And Evaluation ◽

Organizational Factors ◽

Structured Interviews ◽

Improvement Program ◽

Content Type ◽

Data Based Decision Making

Purpose The Ethiopian educational system has witnessed considerable structural and curricular changes aimed to address access, equity and relevance. At the same time, there are serious concerns about educational quality as a consequence of these changes. Data use can be an important approach for changing the planning, execution, monitoring and evaluation of activities having the purpose of improving teaching and learning. The purpose of this paper is, therefore, to investigate data use in primary education in Ethiopia. Design/methodology/approach Using a mixed methods approach, surveys and semi-structured interviews were conducted to collect data from a cluster random sample of eight primary schools representing four different levels of effectiveness in implementing a mandated school improvement program in Ethiopia. Findings The availability of wider ranges of input, process, outcome and context data per se does not ensure actual use. A complex combination of data, user and organizational factors influences data use in schools, with organizational factors appearing to be most influential. Unrealistic accountability pressures and lack of targeted supervision support seemed to cause unintended data use, such as abuse of data. Practical implications Schools need more systematic professional development in data use, with explicit attention to school leadership. Moreover, it is important to make educational inspection processes more responsive to the demands of the school improvement process by adding aspects of the school improvement tradition, such as data-based decision making. Originality/value This study contributes to understanding of the nature, characteristics and processes of data use in a developing country context, in which competing accountability mandates often shape policy and practice.

Download Full-text

The Transformation of Schools’ Social Networks during a Data-Based Decision Making Reform

Teachers College Record ◽

10.1177/016146811611800908 ◽

2016 ◽

Vol 118 (9) ◽

pp. 1-33

Author(s):

Trynke Keuning ◽

Marieke Van Geel ◽

Adrie Visscher ◽

Jean-Paul Fox ◽

Nienke M. Moolenaar

Keyword(s):

Decision Making ◽

Social Networks ◽

Student Achievement ◽

Teacher Collaboration ◽

Achievement Goals ◽

Collaborative Culture ◽

Staff Members ◽

Achievement Data ◽

Effective Teaching Strategies ◽

Data Based Decision Making

Context Collaboration within school teams is considered to be important to build the capacity school teams need to work in a data-based way. In a school characterized by a strong collaborative culture, teachers may have more access to the knowledge and skills for analyzing data, teachers have more opportunity to discuss the performance goals to be set, and they also can share effective teaching strategies to achieve those goals. Although many studies on data-based decision making (DBDM) foreground the importance of teacher collaboration, our knowledge of what such collaboration looks like and how such collaboration may change during a DBDM reform remains limited. Objective The current study uses a social network perspective to explore how collaboration in 32 elementary schools in the Netherlands takes shape in the interactions among teachers as they engage in a DBDM reform project. Research Design Schools’ social networks were examined at the start of the intervention and after having participated 1 year in the DBDM reform. Social networks regarding three DBDM topics are examined: (1) discussing student achievement; (2) discussing achievement goals; (3) and discussing instructional strategies. The density, reciprocity, and centralization of these networks were calculated, and multivariate multiple regression analysis was used to analyze changes over time. Conclusion Findings suggest that teachers’ DBDM related networks transform during the intervention, especially regarding the discussion of student achievement data: although the number of relationships remains stable, more reciprocal relationships are formed, and this network becomes less centralized around one or a few influential staff members.

Download Full-text

Embracing School Counselors’ Situatedness: Data-Based Decision Making as Fulfillment of a Complex Identity

Professional School Counseling ◽

10.1177/2156759x211011922 ◽

2021 ◽

Vol 24 (1_part_3) ◽

pp. 2156759X2110119

Author(s):

Brett Zyromski ◽

Catherine Griffith ◽

Jihyeon Choi

Keyword(s):

Decision Making ◽

School Counselors ◽

School Counselor ◽

Use Of Data ◽

Counselor Identity ◽

Using Data ◽

Definition Of ◽

Support Students ◽

Students Success ◽

Data Based Decision Making

Since at least the 1930s, school counselors have used data to inform school counseling programming. However, the evolving complexity of school counselors’ identity calls for an updated understanding of the use of data. We offer an expanded definition of data-based decision making that reflects the purpose of using data in educational settings and an appreciation of the complexity of the school counselor identity. We discuss implications for applying the data-based decision-making process using a multifaceted school counselor identity lens to support students’ success.

Download Full-text

Teacher Effects on Chilean Children’s Achievement Growth: A Cross-Classified Multiple Membership Accelerated Growth Curve Model

Educational Evaluation and Policy Analysis ◽

10.3102/0162373718781960 ◽

2018 ◽

Vol 40 (3) ◽

pp. 473-501 ◽

Cited By ~ 3

Author(s):

Lorena Ortega ◽

Lars-Erik Malmberg ◽

Pam Sammons

Keyword(s):

Student Achievement ◽

Growth Curve ◽

Growth Models ◽

Teacher Effects ◽

Combined Use ◽

Achievement Growth ◽

Children's Achievement ◽

Accelerated Growth ◽

Multiple Membership ◽

Achievement Trajectories

We investigated teacher effects (magnitude, predictors, and cumulativeness) on primary students’ achievement trajectories in Chile, using multilevel cross-classified (accelerated) growth models (four overlapping cohorts, spanning Grades 3 to 8; n = 19,704 students, and 851 language and 812 mathematics teachers, in 156 schools). It was found that teacher effects on achievement growth are large, exceeding school effects. Also, the contribution of teachers to student achievement growth was found to accumulate over time. The study advances the field by exploring teacher effects in the context of an emerging economy, contributing further evidence on the properties of teacher effects on student achievement growth and demonstrating the combined use of accelerated longitudinal designs, growth curve approaches, and cross-classified and multiple membership models.

Download Full-text

Using Evidence-Based Practice and Data-Based Decision Making in Inclusive Education

Education Sciences ◽

10.3390/educsci11030129 ◽

2021 ◽

Vol 11 (3) ◽

pp. 129

Author(s):

Gabrielle Wilcox ◽

Cristina Fernandez Conde ◽

Amy Kowbel

Keyword(s):

Decision Making ◽

Inclusive Education ◽

Evidence Based ◽

Teacher Capacity ◽

Evidence Based Practices ◽

Full Inclusion ◽

Current State ◽

Inclusive Settings ◽

Using Data ◽

Data Based Decision Making

There are longstanding calls for inclusive education for all regardless of student need or teacher capacity to meet those needs. Unfortunately, there are little empirical data to support full inclusion for all students and even less information on the role of data-based decision making in inclusive education specifically, even though there is extensive research on the effectiveness of data-based decision making. In this article, we reviewed what data-based decision making is and its role in education, the current state of evidence related to inclusive education, and how data-based decision making can be used to support decisions for students with reading disabilities and those with intellectual disabilities transitioning to adulthood. What is known about evidence-based practices in supporting reading and transition are reviewed in relationship to the realities of implementing these practices in inclusive education settings. Finally, implications for using data-based decisions in inclusive settings are discussed.

Download Full-text