scholarly journals The Validity and Reliability of Student Evaluation of Teaching at the National University of Lesotho

2021 ◽  
Vol 8 (1) ◽  
pp. 139-157
Author(s):  
Regina Thetsane

Many Higher Education Institutions use the Student Evaluation of Teaching (SET) scale to evaluate the quality of instructors’ teaching. It includes students’ evaluation of the teacher, the teaching process, teaching approaches and the learning outcomes. Due to its reported dubious reliability and validity, and inherent bias in measuring the quality of teaching, SET remains a hotly debated and controversial instrument. This study evaluated thereliability and validity of the SET scale adopted by the National University of Lesotho. Self-administered SET questionnaires were distributed to 104 third- and fourth-year Bachelor of Commerce students to evaluate ten lecturers, resulting in 751 assessment records. The data were analysed using the Statistical Package for the Social Sciences (SPSS) and Partial Least Squares Structural Equation Modelling (PLS-SEM). While the findings suggest that the SET instrument used at the university is reasonably reliable and valid, minor concerns were raised with regard to discriminant validity, and serious concerns in relation to content validity. Based on the existing literature and the psychometric properties of this SET instrument, it is recommended that university management exercise caution in using its results to make evaluative personnel decisions such as promotions, confirmations, and dismissals. It is also recommended that the SET instrument should be revised and validated and be primarily used for formative purposes such as obtaining feedback for the development of individual instructors. Key words: formative assessment, reliability, student evaluation of teaching, summative assessment, validity

Retos ◽  
2019 ◽  
pp. 687-693
Author(s):  
António Carlos Rodrigues Sampaio ◽  
Nuno Pimenta ◽  
Mafalda Machado ◽  
Pedro Teques

Abstract. The purpose of the present research was to develop and provide initial validation of the Fitness Coaching Behavior Scale (FCBS-Fit) designed for assessing perceived the quality of instructor’s behaviors in fitness group classes through the theoretical adaptation to the Coaching Model (Côté et al., 1995), originally derived from sports. In total, 618 participants of fitness group classes accepted to participate in this investigation. The purpose of the research was achieved in three phases: (1) development of the item pool and content validation of the preliminary version of the FCBS-Fit (27 items); preliminary examination of the factorial structure through exploratory factor analysis (n1 = 185) that revealed a scale with 20 items distributed by 4 factors: Technical and Positive Rapport, Exercise Planning and Prescription, Negative Rapport, and Goal Setting; (2) confirmatory factor analysis to the 4-factor model indicated adequate fit model fit, reliability and convergent and discriminant validity. In addition, a multi-group CFA indicated measurement invariance of the factorial structure across samples (n2 = 210); (3) structural equation model examined criterion validity through analysis of the relationships between the 4-factors of EPCI-Fit and enjoyment in exercise (n3 = 223). Psychometric evidence suggests that the FCBS could be used as a reliable and valid measure to assess Portuguese participant`s perceptions of the quality of instructor’s behaviors in fitness group classes.Resumen. El propósito del presente estudio fue desarrollar y validar preliminarmente la Escala de Percepción del Comportamiento del Instructor de Fitness (EPCI-Fit) cuyo objetivo es evaluar la calidad de los comportamientos de los instructores de fitness en clases de grupo, a través de la adaptación teórica al Coaching Model (Côté et al., 1995), original del entrenamiento deportivo. En total, participaran 618 practicantes de fitness en clases de grupo de diversas áreas del litoral de Portugal continental. El propósito del estudio fue alcanzado en tres fases: (1) desarrollo del banco de ítems y validación inicial del contenido de la versión inicial de EPCI-Fit (27 ítems); la evaluación preliminar de la estructura factorial a través de análisis factorial exploratorio (n1 = 185) que resultó en una escala con 20 ítems distribuidos por 4 factores - Feedback técnico positivo, Feedback negativo, Formulación de objetivos, y Planificación y prescripción del ejercicio; (2) estimación del ajuste del modelo de 4-factores a través de análisis factorial confirmatorio, y análisis a la fiabilidad compuesta, y validez convergente y discriminante (n2 = 210); evaluación de la invariancia métrica del modelo de 4-factores en dos muestras independientes (n1 = 185; n2 = 210); y (3) estimación de la validez de criterio a través del análisis a las relaciones entre los 4-factores de la EPCI-Fit y el disfrute en la práctica de ejercicio (n3 = 223). En general, los resultados provenientes de las diferentes fases de análisis psicométrico a EPCI-Fit soportan la validez y la fiabilidad de los datos, indicando que esta escala podrá ser utilizada en el futuro para evaluar las percepciones de la calidad del comportamiento del instructor de fitness en clases de grupo. Resumo. O propósito do presente estudo foi desenvolver e validar preliminarmente a Escala de Perceção do Comportamento do Instrutor de Fitness (EPCI-Fit) cujo objetivo é avaliar a qualidade dos comportamentos dos instrutores de fitness em aulas de grupo, através da adaptação teórica ao Coaching Model (Côté et al., 1995), original do treino desportivo. No total, participaram 618 praticantes de fitness em aulas de grupo de várias zonas de Portugal continental. O propósito do estudo foi alcançado em três fases: (1) desenvolvimento da pool de itens e validação inicial do conteúdo da versão inicial da EPCI-Fit (27 itens); avaliação preliminar da estrutura fatorial através de análise fatorial exploratória (n1 = 185) que resultou numa escala com 20 itens distribuídos por 4 fatores – Feedback técnico positivo, Feedback negativo, Formulação de objetivos, e Planificação e prescrição do exercício; (2) estimação do ajustamento do modelo de 4-fatores através de analise fatorial confirmatória, e análise à fiabilidade compósita, e validade convergente e discriminante (n2 = 210); avaliação da invariância métrica do modelo de 4-fatores em duas amostras independentes (n1 = 185; n2 = 210); e (3) examinação da validade de critério através da análise às relações entre os 4-fatores da EPCI-Fit e o divertimento na prática de exercício (n3 = 223). No geral, os resultados oriundos das diferentes fases de análise psicométrica à EPCI-Fit suportam a validade e a fiabilidade dos dados, indicando que esta escala poderá ser utilizada futuramente para avaliar as perceções da qualidade do comportamento do instrutor de fitness em aulas de grupo.


2020 ◽  
Vol 2020 ◽  
pp. 1-8 ◽  
Author(s):  
Muhammad Nazir ◽  
Asim Al-Ansari ◽  
Khalifa AlKhalifa ◽  
Balgis Gaffar ◽  
Jehan AlHumaid

Leadership courses are being increasingly integrated into dental curricula. The study aimed to assess the validity and reliability of student evaluation of teaching (SET) instrument among dental students and to evaluate the effectiveness of teaching in a new leadership course over a period of three years. This cross-sectional study was conducted on fourth-year undergraduate dental students (N = 260) who took a practice management course over three consecutive years from 2014 to 2016. A 29-item SET questionnaire was administered among students who were willing to participate in the study. Out of 260 students, 185 returned completed surveys and the response rate was 71.15%. Factor analysis (principal component analysis) showed the validity of four dimensions of the SET instrument. Total variance explained by four dimensions was 62.80%. Cronbach’s alpha for the instrument was 0.95 and each dimension had fairly high internal consistency (>0.80). Treating students with respect (94%), accepting different viewpoints of students (94.1%), being flexible/open-minded (92.5%), and preparedness in the course (91.9%) were the most common effective teaching traits. Over the period of three years, 16 items showed improvement in teaching and there was a significant improvement in four items (P<0.05). In conclusion, it was found that SET is a valid instrument to evaluate the effectiveness of teaching in nonclinical courses in dentistry. This instrument should be used longitudinally to compare the effectiveness of teaching.


2012 ◽  
Vol 42 (1) ◽  
pp. 129-150 ◽  
Author(s):  
Eduardo de Carvalho Andrade ◽  
Bruno de Paula Rocha

We use a random-effects model to find the factors that affect the student evaluation of teaching (SET) scores. Dataset covers 6 semesters, 496 undergraduate courses related to 101 instructors and 89 disciplines. Our empirical findings are: (i) the class size affects negatively the SET score; (ii) instructors with more experience are better evaluated, but these gains reduce over time; (iii) participating in training programs, designed to improve the quality of teaching, did not increase the SET scores; (iv) instructors seem to be able to marginally 'buy' a better evaluation by inflating students' grade. Finally, there are significant changes in the rankings when we adjust the SET score to eliminate the effects of variables beyond instructors' control. Despite these changes, they are not statistically significant.


2004 ◽  
Vol 1 (6) ◽  
Author(s):  
Kim L. Chuah ◽  
Cynthia Hill

The student evaluation, used to measure students’ perceptions of teacher performance, has been increasingly used as the predominant component in assessing teaching effectiveness (Waters et al. 1988), and the widespread movement of outcomes assessment across the country makes this trend likely to continue in the future (McCoy et al. 1994, AACSB 1994, SACS 1995).  Substantial research has been conducted with regard to the reliability and accuracy of student evaluation of teaching quality, and a considerable number of uncontrollable factors are found to bias the results of the evaluation rating.  This paper identifies one more factor.  Each student has an “evaluator profile”, which decreases the reliability of the student evaluation.  An “evaluator profile” is a persistent pattern of evaluating behavior that may or may not be consistent with the quality of the characteristic being evaluated.  Each class of students consists of a random sample of different evaluator profiles.  A student evaluation rating of a teacher’s performance is biased up or down depending on the concentration of high or low evaluator profiles present.  This paper further shows through simulation the degree to which student “evaluator profiles” impact the overall student evaluation rating of teacher performance. We find that there is evidence to support the “evaluator profile” conjecture, and that these “evaluator profiles” do in fact have the potential to change overall student evaluation ratings substantially.


2017 ◽  
Vol 11 (1) ◽  
pp. 100-117
Author(s):  
Vladimír Jůva ◽  
Eva Valkounová

Importance of student evaluation of teaching (SET) growing currently as a response to demands of improving the quality of university education. The paper deals with SET aimed specifically at sports university education. The introduction briefly describes the concept, function and significance of SET. It also identifies selected specifics of student evaluation of university sports teaching. The empirical part of this text describes the methodological aspects and selected results of the evaluation carried out investigation and research aimed at the creation and verification tools for student evaluation of sports university teaching. The following discussion analyses selected issues that accompany this particular SET. Conclusion presents suggestions and asks other questions related to SET focused on university sports teaching.


Author(s):  
Nor Hapiza Mohd Ariffin ◽  
Siti Nor Hidayah Askol

<p>In higher education institutions, student evaluation is important to ensure that students are given the opportunity to attain a high quality of education. In Universiti Teknologi Mara (UiTM) Malaysia, student evaluation was implemented through a system called Student Feedback Online (SuFO). This study aims to investigate and analyze the perception of academicians towards the usage of online student evaluation using SUFO in UiTM.  The research employed the quantitative analysis and supported by Rasch measurement. The respondents are academicians in UiTM Shah Alam. A total of 152 academicians responded to the questionnaires.  By using a Likert scale, 25 items were designed in the questionnaire and distributed by official email to the academicians.  Data were analyzed using Rasch measurement to measure the validity and reliability of the items and the respondents involved in this research, while SPSS was used to analyze the quantitative data.  The results showed that the academicians accepted the outcome value of student evaluation and they agreed that results of student evaluation should be used for formative assessment. It is recommended for future research, an instrument using a method of multi-dimensional evaluation of teaching should be developed to evaluate the effectiveness of university teaching.</p>


Author(s):  
Tan Lay Khong

Most universities are using the Student Evaluation of Teaching (SET) as an instrument for students to assess a lecturer’s teaching performance. It is an essential instrument to reflect the feedback in enhancing the quality of teaching and learning. The purpose of this paper is to examine the validity and reliability of the SET as a valid instrument in evaluating teaching effectiveness in a private higher education institution in Malaysia. Exploratory Factor Analysis and Confirmatory Factor Analysis have validated all 10 items of SET whereby all items indicated high reliability and internal consistency. A Confirmatory Factor Analysis using AMOS software also confirmed that a single factor model was used to evaluate teaching effectiveness. The single factor model was further validated using 1000 repeated samples of Bootstrap method in AMOS.


Sign in / Sign up

Export Citation Format

Share Document