A Reflection on Student Perceptions of Teaching Quality from Three Psychometric Perspectives: CCT, IRT and GT

This chapter discusses student perceptions in terms of three psychometric perspectives that dominate contemporary research on teaching quality, namely, Classical Test Theory (CTT), Item Response Theory (IRT) and Generalizability Theory (GT). These perspectives function as being exemplars for the connection between psychometric theories and the different perspectives on “what a perception is” as well as on how and for what purposes student perceptions should be used. The main message of the chapter is that the choice of a psychometric theory is not merely a technical matter, but also has implications for how the nature of perceptions is conceptualized. After presenting and linking each psychometric theory, their strengths and weaknesses in the context of student perceptions of teaching quality and issues on practical implementations are discussed.

Lessons Learned from Research on Student Evaluation of Teaching in Higher Education

In higher education, anonymous student evaluation of teaching (SET) ratings are used to measure faculty’s teaching effectiveness and to make high-stakes decisions about hiring, firing, promotion, merit pay, and teaching awards. SET have many desirable properties: SET are quick and cheap to collect, SET means and standard deviations give aura of precision and scientific validity, and SET provide tangible seemingly objective numbers for both high-stake decisions and public accountability purposes. Unfortunately, SET as a measure of teaching effectiveness are fatally flawed. First, experts cannot agree what effective teaching is. They only agree that effective teaching ought to result in learning. Second, SET do not measure faculty’s teaching effectiveness as students do not learn more from more highly rated professors. Third, SET depend on many teaching effectiveness irrelevant factors (TEIFs) not attributable to the professor (e.g., students’ intelligence, students’ prior knowledge, class size, subject). Fourth, SET are influenced by student preference factors (SPFs) whose consideration violates human rights legislation (e.g., ethnicity, accent). Fifth, SET are easily manipulated by chocolates, course easiness, and other incentives. However, student ratings of professors can be used for very limited purposes such as formative feedback and raising alarm about ineffective teaching practices.

Student Feedback on Teaching in Schools: Current State of Research and Future Perspectives

The aim of this volume was to give a comprehensive overview of the current state of the research on student perceptions of and student feedback on teaching. This chapter provides a resume of the important theoretical considerations and empirical evidence the authors contributed to this volume. First, evidence concerning the validity of student perceptions of teaching quality is discussed, highlighting the quality of the questionnaires used and accompanying materials provided by their authors. In the next step, empirical findings are summarized on student and teacher characteristics that can influence important processes within the feedback cycle. Subsequently, it is emphasized that the effectiveness of student feedback on teaching is significantly related to the nature of the individual school’s feedback culture. Furthermore, it is argued that the efficacy of student feedback depends on whether teachers are provided with a high level of support, when making use of the feedback information to improve their teaching practices. As the literature review impressively documents‚ teachers, teaching, and ultimately students can benefit substantially from student feedback on teaching in schools.

Effects of Student Feedback on Teaching and Classes: An Overview and Meta-Analysis of Intervention Studies

Based on a comprehensive literature review of student feedback intervention studies in schools, this chapter provides an overview of found effects on teachers and teaching. The first part summarizes the self-reported cognitive, affective, and motivational effects of student feedback on teachers, which can subsequently lead to behavioral changes in the classroom. In the second part, the focus is on the extent to which these behavioral changes are perceived by students. For the first time, a meta-analysis of changes in students’ perceptions of teaching was carried out for the 18 existing longitudinal studies for this purpose. A small but significant positive weighted mean effect size of d=0.21 for students’ perceived improvement of teaching quality was found, while more in-depth analyses pointed to a beneficial effect of individual support measures for teachers regarding reflection and subsequent development of teaching. Implications for further research and practical implementation of student feedback in schools are discussed.

Student Perceptions of Teaching Quality: Dimensionality and Halo Effects

This chapter deals with the factorial structure of survey instruments for student perception of teaching quality. Often, high intercorrelations occur between different theoretically postulated teaching quality dimensions; other analyses point to a single unified factor in student perceptions of teaching quality, seemingly reflecting a “general impression” instead of a differentiated judgment. At the same time, findings from research on social judgment processes and from classroom research indicate that the teachers’ communion (warmth or cooperation) as well as students’ general subject interest can be important biasing factors in the sense of halo effects in student ratings of teaching quality. After presenting an overview of studies on the dimensionality of various survey instruments, we discuss whether aggregated data is impacted by an overall “general impression”. We confirmed this hypothesis using a sample of N = 1056 students from 50 secondary school classes. Moreover, this general impression could be explained at student and class level to a large extent by students’ perception of the teacher’s communion. Student general subject interest showed a medium effect but only at the individual level. These findings indicate that student perceptions of teaching quality dimensions are indeed influenced by a general impression which can be explained largely by teacher's communion.

A Probabilistic Model for Feedback on Teachers’ Instructional Effectiveness: Its Potential and the Challenge of Combining Multiple Perspectives

This chapter describes research into the validity of a teacher evaluation framework that was applied between 2012 and 2016 to provide feedback to Dutch secondary school teachers concerning their instructional effectiveness. In this research project, the acquisition of instructional effectiveness was conceptualized as unfolding along a continuum ranging from ineffective novice to effective expert instructor. Using advanced statistical models, teachers’ current position on the continuum was estimated. This information was used to tailor feedback for professional development. Two instruments were applied to find teachers’ current position on the continuum, namely the International Comparative Assessment of Learning and Teaching (ICALT) observation instrument and the My Teacher–student questionnaire (MTQ). This chapter highlights background theory and central concepts behind the project and it introduces the logic behind the statistical methods that were used to operationalize the continuum of instructional effectiveness. Specific attention is given to differences between students and observers in how they experience teachers’ instructional effectiveness and the resulting disagreement in how they position teachers on the continuum. It is explained how this disagreement made feedback reports less actionable. The chapter then discusses evidence of two empirical studies that examined the disagreement from two methodological perspectives. Finally, it makes some tentative conclusions concerning the practical implications of the evidence.

The Quality of Student Perception Questionnaires: A Systematic Review

Student perceptions of teaching are promising for measuring the quality of teaching in primary and secondary education. However, generating valid and reliable measurements when using a student perception questionnaire (SPQ) is not self-evident. Many authors have pointed to issues that need to be taken into account when developing, selecting, and using an SPQ in order to generate valid and reliable scores. In this study, 22 SPQs that met the inclusion criteria used in the literature search were systematically evaluated by two reviewers. The reviewers were most positive about the theoretical basis of the SPQs and about the quality of the SPQ materials. According to their evaluation, most SPQs also had acceptable reliability and construct validity. However, norm information about the quality rating measures was often lacking and few sampling specifications were provided. Information about the features of the SPQs, if available, was also often not presented in an accessible way by the instrument developers (e.g., in a user manual), making it difficult for potential SPQ users to obtain an overview of the qualities of available SPQs in order to decide which SPQs best fit their own context and intended use. It is suggested to create an international database of SPQs and to develop a standardized evaluation framework to evaluate the SPQ qualities in order to provide potential users with the information they need to make a well-informed choice of an SPQ.

Student Ratings of Teaching Quality Dimensions: Empirical Findings and Future Directions

This chapter discusses current issues in research on the validity of student ratings of teaching quality. We first discuss the advantages and limitations of student ratings of teaching quality based on theoretical considerations regarding the teaching quality concept. Research reveals that the validity of student ratings differs depending on the aspect of teaching quality being rated (i.e., classroom management, cognitive activation, or student support). Extending this research, we propose that future studies on the validity of student ratings should take into account students’ cognitive processing while responding to survey items. We discuss three areas that seem promising for future research: the complexity and comprehensibility of survey items, the referent and addressee of items, and finally, the idiosyncratic nature of student ratings.

The Process Model of Student Feedback on Teaching (SFT): A Theoretical Framework and Introductory Remarks

Student feedback on teaching in schools, conceptualized as information on student perceptions of teaching, is described by many scholars as an effective instrument for the developmental use of teachers and teaching. Beyond that, various studies show that the productive use of this method is a very complex process in which a variety of aspects must be considered. As an introduction to this volume, this chapter presents a model based on findings from different research areas of feedback and school research, called Process Model of Student Feedback on Teaching (SFT). This model follows the steps of the student feedback process, starting with student perceptions of teaching, which must be professionally collected or measured. Subsequently, the teacher perceives and interprets this feedback information, which is linked to cognitive and affective reactions and processes. This can lead to an enhancement of teachers’ knowledge about their own teaching and to the initiation of improvement-oriented actions, finally resulting in improved teaching and development of the teachers’ professional competence. Thereby, characteristics of the organization, the students, and classes as well as the teachers need to be considered. This model serves as a framework for the subsequent overview of the contributions in this volume.

Student Voice and Student Feedback: How Critical Pragmatism Can Reframe Research and Practice

This chapter recognises the diverse definitions and practices of student feedback; focussing on how student feedback can facilitate dialogue and thus contribute to the development of schools as democratic communities. Student feedback is thus positioned as a part of student voice, which has its roots in the United Nations Convention on the Rights of the Child (UNICEF, 1989). We question the ways in which schools elicit the views of students and how students’ opinions are made use of, recognising the complexities arising from power relationships (Hart, 1992), the consumerisation of education (Whitty & Wisby, 2007) and the pressures of accountability. Furthermore, we consider ways in which researchers can address difficulties in the research-practice relationship (Chapman and Ainscow, 2019) and facilitate co-creation of research. We propose the perspective of critical pragmatism as a means to acknowledge the complexities of practice, whilst also highlighting the importance of critical reflection and dialogue. Critical pragmatism could move us from a “deconstructive scepticism toward a reconstructive imagination” (Forester, 2012, p. 6) in which schools and researchers collaborate to enable contextually rich practices of student feedback and student voice.