Common Errors in Teacher-Made Test Design

Teaching and test writing require different skill sets. While teachers are often required to create language tests, they often have inadequate training. This may lead to tests that lack validity and reliability, making tests unfair for test-takers. Tests may contain more general errors in their development, leading to construct underrepresentation or construct-irrelevant variance, decreasing reliability and validity of the entire test and rendering results meaningless. Well-designed tests may also contain items that are not well constructed, which may again lower overall validity and reliability. These item-related errors include word matches, testing single words, issues with phrasing, developing good sets and distractors, and testing outside the text (such as language in the item or math skills). Increasing awareness of these issues and improving teacher skills in test writing will ensure more fairness for students in decisions based on test results.

Summative Assessment in an IEP

The purpose of this chapter is to share an assessment model built specifically upon teacher collaboration and, more broadly, to encourage readers to consider the power of collaboration within an intensive English program (IEP). After examining traditional assessment challenges faced by IEPs, the chapter presents a collaborative assessment model, the basic premise of which is that teachers of the same students across three core courses within the same IEP level work together to create a common midterm and final exam. The model is examined in light of the five assessment principles of validity, reliability, practicality, authenticity, and washback. Finally, benefits and challenges of teacher collaboration are considered from the perspective of program administrators.

The Impact of the Social Dimension on Assessing Oral Proficiency

In contrast to the assessment of reading, listening, or writing skills, the assessment of oral proficiency has seen a growing preference for interactive task formats as the context in which to evaluate a learner's spoken ability. Tasks that require learners to engage in authentic, meaningful interactions with their peers or the teacher, however, deserve attention due to potential benefits as well as unintended consequences. This is because in interactive tasks a learner's performance is not the sole display of individual competence, as it is the case with writing or listening tests: The presence of an interlocutor introduces a social dimension, raising issues of test validity and reliability, and ultimately impacting test design and scoring systems. Therefore, the goal of this chapter is to discuss the challenges and implications of assessing oral proficiency when a social dimension is added to the picture and to contribute to a better understanding of how a joint construction of speaking performance can impact not only test development but also students' scores.

Connecting Teacher-Made Assessments to Course Learning Outcomes Through Learning Analytics

This chapter explores the possibility of adopting a data-driven approach to connecting teacher-made assessments with course learning outcomes. The authors begin by describing several key concepts, such as outcome-based education, curriculum alignment, and teacher-made assessments. Then, the context of the research site and the subject in question are described and the use of structural equation modeling (SEM) in this curriculum alignment study is explained. After that, the results of these SEM analyses are presented, and the various models derived from the analyses are discussed. In particular, the authors highlight how a data-driven curriculum model can benefit from input by curriculum leaders and how SEM provides insights into course development and enhancement. The chapter concludes with recommendations for curriculum leaders and front-line teachers to improve the quality of teacher-made assessments.

Aligning Curriculum, Instruction, and Classroom Assessment in a University English Language Program

Curriculum, instruction, and classroom assessments should all be aligned with each other in order to promote student learning. By achieving alignment, classroom assessments become integrated into the curriculum and guide what and how teachers teach and what and how students learn. This chapter describes the case of how one university English language program improved curricular alignment through the use of teacher alignment forms for student learning outcomes (SLOs), level meetings, the use of test specifications, a peer review process for test development, an assessment handbook, and assessment workshops for teachers. These practices ensure explicit and documented alignment among their curriculum, instruction, and assessments.

Ipsative Assessment of Essay Writing to Foster Reflection and Self-Awareness of Progress

Ipsative assessment to improve argumentative essay writing aimed at studying how an alternative type of assessment, which persuaded learners to reflect on the feedback received and involved them as active assessment participants, influenced the writing skill of 24 students from two Colombian universities. The queries addressed throughout the chapter were analyzing the way students structured their essays by means of the Ipsative feedback and the influence of the assessment approach on students at the self-management levels. Findings revealed that through Ipsative assessment students enhanced their argumentative essay writing as they grasped the structural and reflective nature of this skill. Furthermore, the comparison of various drafts allowed learners to reflect on their improvements at the same time they raised self-awareness of progress and the whole process took students to the realization that they were developing generic skills useful for academic discourse. This chapter was also concerned with the effects of Ipsative assessment on self-directedness and lifelong learning.

A Common Final Exams Project in an IEP

This chapter reports on a common assessment project in a university intensive English program (IEP). In the program, the authors developed, implemented, and supervised the common finals project (CFP) to standardize final exams in all courses through collaborative teacher-made test development. The report provides details regarding why and how the project was implemented, with test samples, and test development and review instruments provided to demonstrate the theoretical and practical issues involved. While still ongoing, this chapter covers the beginning stages of the project, covering a time frame from August 2015 to May 2016, and involved dozens of teachers and hundreds of students. After explaining the rationale for the CFP, and details about its methodology and implementation, this report shows that the standardization project has been successful in developing and administering better final exams for IEP students.

Instructor-Generated Orthographic Assessments in Intensive English Classes

Orthographic knowledge, or knowledge of spellings, word forms, and conventions of print, is a crucial skill underpinning a range of literacy skills. Despite its importance, orthographic knowledge receives relatively little attention in second language contexts, including in adult English as a second language (ESL) programs. This chapter provides an overview of orthographic knowledge, its development in first language (L1), and what is understood about orthography in second language (L2). The chapter then reports detailed results of a qualitative interview study in which current and former ESL instructors shared their experiences, practices, and perspectives on orthographic instruction and assessment, including the development and use of instructor-generated assessments. The chapter concludes with a summary of results and suggestions for ESL orthographic instruction and assessment.

Assessing Pre-Service Teachers' Assessment Literacy

Conducting classroom assessments is a regular part of teachers' daily work. Despite the centrality of tests in K-12 classrooms, teacher candidates consistently demonstrate fundamental weakness in their understanding and implementation of assessment. Student teaching has the potential to be an important training ground for teacher candidates to grow in their assessment practices, and by focusing on assessment during the student teaching experience, teacher candidates can more easily develop a deeper understanding of the myriad ways to evaluate student learning. This chapter explores the assessment relationships between teacher candidates and their mentors (i.e., cooperating teachers, student teaching supervisors, and seminar instructors) and provides a framework through which intentional and incidental classroom assessment can be considered. Ways to teach assessment through planning, debriefing, and raising awareness through noticing are discussed, and recommendations are made to help teacher candidates build the foundation of a strong assessment repertoire.

Developing Assessment Literacy Through Assessing Classroom Tests

While the importance and lack of assessment literacy (AL) has been widely reported in the education and language teaching literature, tools and methods to help teachers actually develop competence in classroom assessment are much more limited. This chapter seeks to help fill that gap and provide tools and procedures for developing AL through teachers formatively assessing classroom tests developed (or adapted) by their peers. This chapter reports on a test review process developed by the author (and Assessment Coordinator) in an English center at an American university. It details a process started in 2012 and currently in use to review teacher-made tests in various programs. Importantly, the instruments and procedures themselves, as well as samples of their use, are provided and shared for possible use in developing AL in other contexts and programs. This report is unique in detailing a process of actually using formative assessment, by and for teachers, to help develop teacher's assessment competence.