Developing individualized feedback for listening assessment: Combining standard setting and cognitive diagnostic assessment approaches

In this study, we present the development of individualized feedback for a large-scale listening assessment by combining standard setting and cognitive diagnostic assessment (CDA) approaches. We used the performance data from 3358 students’ item-level responses to a field test of a national EFL test primarily intended for tertiary-level EFL learners. The results showed that proficiency classifications and subskill mastery classifications were generally of acceptable reliability, and the two kinds of classifications were in alignment with each other at individual and group levels. The outcome of the study is a set of descriptors that describe each test taker’s ability to understand certain level of oral texts and his or her cognitive performance. The current study, by illustrating the feasibility of combining standard setting and CDA approaches to produce individualized feedback, contributes to the enhancement of score reporting and addresses the long-standing criticism that large-scale language assessments fail to provide individualized feedback to link assessment with instruction.

Meeting Expectations? An Empirical Investigation of a Standards-Based Assessment of Reading Comprehension

The present study was designed to examine the relationship between performance on a standards-based assessment of reading comprehension in fourth grade and performance on a diagnostic battery of component skills of reading for a cohort of children who were followed from kindergarten through fourth grade. The findings demonstrate that the relationship between performance on component skills of reading and the proficiency classifications from the standards-based assessment is generally weak. They also demonstrate that the relationship between the proficiency classifications and a norm-referenced diagnostic measure of reading comprehension is only moderate. Furthermore, the study shows that the broad proficiency-level classifications of the standards-based assessment mask significant heterogeneity in children’s performance on some of the component skills assessed, both within and across proficiency classifications. In turn, the ability of the standards-based assessment results to inform instructional interventions is low. The implications of these findings for instructional planning within standards-based accountability systems are discussed.