Why Must Everything Be So Complicated? Demands and Challenges on Methods for Analyzing School Improvement Processes

In the recent years, awareness has risen by an increasing number of researchers that we need studies that appropriately model the complexity of school improvement if we want to increase our knowledge about school improvement substantially and to close the identified research gaps within this field (Feldhoff T, Radisch F, Klieme E, J Educ Admin 52(5):565–736, 2014; Hallinger P, Heck RH, School Effect School Improv 22(2):149–173, 2011; Sammons P, Davis S, Day C, Gu Q, J Educ Admin 52(5):565–589, 2014). So far, respective quantitative studies, that appropriately consider those complexities, have hardly been realized because of the high efforts of current methods and costs involved (Feldhoff T, Radisch F, Bischof LM, J Educ Admin 2(54):209–240, 2016). It is, therefore, apparent to look for new, innovative methods that can adequately take into account the complexity of school improvement. For a reasonable search and selection of innovative methods, it is necessary to describe the systematic complexity of school improvement and the resulting requirements for the methods in more detail. This is a central goal in this chapter. For this reason, we present our framework of complexity. We then formulate questions that will prompt the reader to reflect critically upon the methods in general and especially in this volume.

Designing and Piloting a Leadership Daily Practice Log: Using Logs to Study the Practice of Leadership

This article aims to validate the Leadership Daily Practice (LDP) log, an instrument for conducting research on leadership in schools. Using a combination of data sources—namely, a daily practice log, observations, and open-ended cognitive interviews—the authors evaluate the validity of the LDP log. Formal and informal leaders were asked to complete the LDP log for 2 weeks; observers shadowed a subsample of leaders in each school, 1 day per week. Using the three sources of data, the authors analyzed interview responses (specifically, the participants’ interpretations of the log); they matched log entries with observer recordings; and they compared (a) the characteristics of the social interactions that were entered into the log with (b) the overall sample of interactions that occurred while observers shadowed participants. The study shows that LDP log entries capture school leadership interactions as recorded by independent observers; it also demonstrates that study participants, with some exceptions, were not biased toward reporting certain types of interactions over others. Still, some log terminologies were problematic for participants, as was the limited sampling period of 2 weeks. The authors propose ways to (a) change the LDP log to reflect the concerns raised by participants in the cognitive interviews and (b) alter the sampling scheme to capture leadership around the school year. The LDP log is less costly and time-consuming than in-depth ethnographic studies, and it is an important tool for researchers who aim to collect data in schools, one that reaches beyond surveys.

Introduction

Schools are continuously confronted with various forms of change, including changes in students’ demographics, large-scale educational reforms, and accountability policies aimed at improving the quality of education. On the part of the schools, this requires sustained adaptation to, and co-development with, such changes to maintain or improve educational quality. As schools are multilevel, complex, and dynamic organizations, many conditions, factors, actors, and practices, as well as the (loosely coupled) interplay between them, can be involved therein (e.g. professional learning communities, accountability systems, leadership, instruction, stakeholders, etc.). School improvement can thus be understood through theories that are based on knowledge of systematic mechanisms that lead to effective schooling in combination with knowledge of context and path dependencies in individual school improvement journeys. Moreover, because theory-building, measuring, and analysing co-develop, fully understanding the school improvement process requires basic knowledge of the latest methodological and analytical developments and corresponding conceptualizations, as well as a continuous discourse on the link between theory and methodology. The complexity places high demands on the designs and methodologies from those who are tasked with empirically assessing and fostering improvements (e.g. educational researchers, quality care departments, and educational inspectorates).

Reframing Educational Leadership Research in the Twenty-First Century

For the last three decades, reviews on instructional leadership have pointed to studies that predominantly adopted conventional social science research methodologies, specifically analytical tools, such as descriptive, causal factor, correlational and advanced modelling. These methods have constraints and limitations, which include that variable-based linear models measures are treated as ‘rigorously real’ measures of social reality, that individuals use rational deduction (ignoring the value premise of decision-making), and that individuals are treated as independent and individualized. This paper proposes and illustrates how research approaches of complexity science can be applied within the social system to address complex instructional leadership questions. Consequently, reframing instructional leadership research through the lens of complexity science provides the most viable approach to understand the adaptive processes and the dynamic system of schools.

The Structure of Leadership Language: Rhetorical and Linguistic Methods for Studying School Improvement

In this chapter, I discuss the justification for a linguistic turn in the study of school improvement with an emphasis on the language of leadership, and in particular persuasion, in the implementation of reform. In addition to exploring the ways in which discourse analysis can be used more generally to understand the nature of school improvement, I also focus on the particular method of rhetorical analysis as it can be leveraged to understand how the structure of language can be in and of itself an improvement strategy for educational leaders. After discussing the methodological approach, I share examples of studies of principals’ talk in the context of reform and the findings that emerged. I then consider the methodological implications of this rhetorical and linguistic turn, before ending with implications for future research and practice about the role of language in improvement efforts.

Regulation Activities of Teachers in Secondary Schools: Development of a Theoretical Framework and Exploratory Analyses in Four Secondary Schools Based on Time Sampling Data

Previous research has revealed that teachers’ regulation activities in schools are most relevant for sustainable school improvement. However, previous studies have severe methodological and theoretical shortcomings. This paper presents the results of a mixed-method case study at four lower secondary schools, in which we developed a framework for understanding regulation activities and processes in schools and analyzed teachers’ regulation activities by using time sampling data of teachers’ performance-related and situation-specific day-to-day activities over 3 weeks. Our results revealed that teachers engage in regulation activities only relatively seldom. Significant differences between teachers were found that are systematically related to the teachers’ specific roles in the school. Teachers rated their regulation activities as especially beneficial for teaching, student learning, and teachers’ learning but as less beneficial for team and school development. Small differences between schools were identified. Further, the results revealed significant correlations between teachers’ perceived benefit of the daily activities and teachers’ daily satisfaction. Based on the results, it can be concluded that the newly developed method appears to be suitable for recording teachers’ daily regulation activities in a (relatively) valid manner and, consequently, for use as a complement to existing instruments. Limitations are discussed, and the need for further research is described.

Concept and Design Developments in School Improvement Research: General Discussion and Outlook for Further Research

This book aimed to present innovative designs, measurement instruments, and analysis methods by way of illustrative studies. Through these methodology and design developments, the complexity of school improvement in the context of new governance and accountability measures can be better depicted in future research projects. In this concluding chapter, we discuss what strengths the presented methodologies and designs have and to what extent they do better justice to the multilevel, complex, and dynamic nature of school improvement than previous approaches. In addition, we outline some needs for future research in order to gain new perspectives for future studies.

Taking Composition and Similarity Effects into Account: Theoretical and Methodological Suggestions for Analyses of Nested School Data in School Improvement Research

School improvement research is faced with a school teaching staff, which is not a simple homogeneous entity. The compositional attributes of the teaching staff – such as diversity – can have a crucial influence on school processes. Whether the teaching staff is highly fractured, consists of sharply dissociated subgroups, or has shared beliefs, affects the adoption of school improvement programs differently. However, school improvement research has not yet taken into account what different compositions of the teaching staff mean from a methodological viewpoint. It is true that the use of multilevel analysis is standard in school improvement research and that it considers nested school data. However, this method alone only takes averaged measures of teaching staffs into consideration but not their different compositions. In this contribution, we argue that school improvement research has to consider, theoretically and methodologically, how compositional attributes of the teaching staff can be conceptualized. We first discuss some advancements in the conceptualization of group composition from research on small groups and organizations. We then incorporate suggestions for different diversity typologies from small group research to describe the compositional attributes of the teaching staff. Additionally, we address how the composition of the teaching staff influences each teacher differently, depending on the specific position a teacher has within the teaching staff. We further suggest incorporating the Group Actor-Partner Interdependence Model (GAPIM; Kenny, DA, Garcia RL, Small Group Res 43:468–496, 2012) as a methodological approach for assessing these compositional influences. In addition to classic multilevel analysis, the GAPIM also considers the effects of the other teachers on staff and the similarity and dissimilarity of a teacher to the other members of the teaching staff. Finally, we illustrate the possibilities of the theoretical and methodological endorsements discussed by applying the GAPIM to a data set of 37 German upper secondary schools by way of example. We show that a teacher’s job satisfaction is not only influenced by their individual and collective teacher self-efficacy but also by positional effects: The similarity of a teacher to the other teachers on staff and the similarity among the other members of the teaching staff have additional influences on job satisfaction.

Recurrence Quantification Analysis as a Methodological Innovation for School Improvement Research

School improvement and educational change can be facilitated by learning through reflection, as this allows teachers to discover ways to develop and adapt to change. Higher levels of engagement in reflection have been found to be beneficial, but it is unclear from which everyday routine in engagement in reflection higher levels arise, and thus whether occasions to make knowledge explicit should be organized with a certain constancy. In this study, we therefore used a conceptualization of teacher learning through reflection as a situated and dynamic process in which available environmental information, learning activities, and professional practices are interconnected, and co-develop. Seventeen Dutch Vocational Education and Training teachers participated over a period of 5 months. We explored the use of daily and monthly logs as measurement instruments and Recurrence Quantification Analysis (RQA) as the analysis technique applied to the time-series generated from the daily logs. The findings indicated that teachers who make information from their working environment explicit more are also able to make new insights explicit more. The routine with which teachers make information explicit was found to be mostly unrelated to making new insights explicit. To reach their levels of engagement in reflection, some teachers organized opportunities to reflect with determined intervals, others seemed to recognize those opportunities as the working environment provided them, and some used a combination thereof. Moreover, the use of daily and monthly logs seemed to fit better to some participants than others. Only sometimes does organizing constancy in engagement in reflection seem to relate to the levels thereof. This study provides an example of how logs and RQA can be adopted to tap into professional learning as a dynamic and situated process in support of school improvement and educational change.

Learning in Collaboration: Exploring Processes and Outcomes

Moving towards school improvement requires coming to understand what it means for a teacher to engage in ongoing learning and how a professional community can contribute to that end. This mixed methods study first classifies 48 primary schools into clusters, based on the strength of three professional learning community (PLC) characteristics. This results in four meaningful categories of PLCs at different developmental stages. During a one-year project, teacher logs about a school-specific innovation were then collected in four primary schools belonging to two extreme clusters. This analysis focuses on contrasting the collaboration and resulting learning outcomes of experienced teachers in these high and low PLC schools. The groups clearly differed in the type, contents, and profoundness of their collaboration throughout the school year. While the contents of teachers’ learning outcomes show both differences and similarities between high and low PLC schools, outcomes were more diverse in high PLC schools, nurturing optimism about the learning potential in PLCs. The study has implications for systematically supporting teacher learning through PLCs.