Comparative judgement and the hierarchy of students’ choice criteria

Author(s):  
Jennifer Palisse ◽  
Deborah Martina King ◽  
Mark MacLean
2021 ◽  
Vol 6 ◽  
Author(s):  
Tom Benton

This article describes an efficient way of using comparative judgement to calibrate scores from different educational assessments against one another (a task often referred to as test linking or equating). The context is distinct from other applications of comparative judgement as there is no need to create a new achievement scale using a Bradley-Terry model (or similar). The proposed method takes advantage of this fact to include evidence from the largest possible number of examples of students’ performances on the separate assessments whilst keeping the amount of time required from expert judges as low as possible. The paper describes the method and shows, via simulation, how it achieves greater accuracy than alternative approaches to the use of comparative judgement for test equating or linking.


2018 ◽  
Vol 18 (4) ◽  
pp. 366-391
Author(s):  
Stephen D. Holmes ◽  
Michelle Meadows ◽  
Ian Stockford ◽  
Qingping He

2020 ◽  
Vol 55 (1) ◽  
pp. 49-71
Author(s):  
Neil Marshall ◽  
Kirsten Shaw ◽  
Jodie Hunter ◽  
Ian Jones

Abstract There is growing interest in using comparative judgement to assess student work as an alternative to traditional marking. Comparative judgement requires no rubrics and is instead grounded in experts making pairwise judgements about the relative ‘quality’ of students’ work according to a high level criterion. The resulting decision data are fitted to a statistical model to produce a score for each student. Cited benefits of comparative judgement over traditional methods include increased reliability, validity and efficiency of assessment processes. We investigated whether such claims apply to summative statistics and English assessments in New Zealand. Experts comparatively judged students’ responses to two national assessment tasks, and the reliability and validity of the outcomes were explored using standard techniques. We present evidence that the comparative judgement process efficiently produced reliable and valid assessment outcomes. We consider the limitations of the study, and make suggestions for further research and potential applications.


2018 ◽  
Vol 27 (2) ◽  
pp. 248-261 ◽  
Author(s):  
Tanguy Coenen ◽  
Liesje Coertjens ◽  
Peter Vlerick ◽  
Marije Lesterhuis ◽  
Anneleen Viona Mortier ◽  
...  

Sign in / Sign up

Export Citation Format

Share Document