Evaluating Different Scoring Methods for Multiple Response Items Providing Partial Credit

2021 ◽  
pp. 001316442199463
Author(s):  
Joe Betts ◽  
William Muntean ◽  
Doyoung Kim ◽  
Shu-chuan Kao
Keyword(s):  
Irrelevant Information ◽  
Multiple Response ◽  
Instruction Set ◽  
Response Type ◽  
Scoring Methods ◽  
Partial Credit ◽  
Computer Based ◽  
Computer Based Testing ◽  
Item Types ◽  
True False

The multiple response structure can underlie several different technology-enhanced item types. With the increased use of computer-based testing, multiple response items are becoming more common. This response type holds the potential for being scored polytomously for partial credit. However, there are several possible methods for computing raw scores. This research will evaluate several approaches found in the literature using an approach that evaluates how the inclusion of scoring related to the selection/nonselection of both relevant and irrelevant information is incorporated extending Wilson’s approach. Results indicated all methods have potential, but the plus/minus and true/false methods seemed the most promising for items using the “select all that apply” instruction set. Additionally, these methods showed a large increase in information per time unit over the dichotomous method.

Comparison of the reliability of scoring methods of multiple-response items, matching items, and sequencing items

CADMO ◽  
2012 ◽  
pp. 85-104
Author(s):  
Theo J.H.M. Eggen ◽  
Tecla T.M. Lampe
Keyword(s):  
Classical Test Theory ◽  
Test Theory ◽  
Multiple Response ◽  
Response Patterns ◽  
Scoring Methods ◽  
Scoring Method ◽  
Knowledge Levels ◽  
Different Response ◽  
Dichotomous Scoring ◽  
Item Types

Multiple-response items, sequencing items, and matching items are three innovative item types often included in systems for computer-based assessment that offer the benefit of polytomous scoring and the possibility to measure partial knowledge. In the present study, different scoring methods of these three item types were compared. Based on the assumption that different response patterns to these item types represent different knowledge levels, these knowledge levels are described. Features of different scoring methods were studied to select the scoring methods included in this study. Subsequently, a probability distribution of scoring results for each knowledge level was derived and computed. Based on classical test theory, a measure for the reliability of the different scoring methods on the level of a single item was derived. To compare the results of the scoring methods selected, reliabilities were computed for several distributions of knowledge levels in a population. For a multiple-response item, when an examinee must select all the right options, the dichotomous scoring method resulted in higher reliabilities than scoring the response patterns polytomously. For matching items and for multiple-response items, when an examinee is asked to select fewer options than the total number of right options given, polytomous scoring methods gave higher reliabilities than the dichotomous scoring method. Simple polytomous scoring by counting the selected right options or relations is recommended instead of more complex polytomous scoring methods, for instance, using a correction for wrong answers or a so-called "floor". The results of scoring sequencing items were not as conclusive as for the other two item types explored.

Promoting positive washforward through personalised test feedback and other benefits: Piloting a computer-based testing system

2020 ◽  
Vol 10 (1) ◽  
pp. 235-244
Author(s):  
Elena A. M. Gandini ◽  
Tania Horák
Keyword(s):  
Foreign Language ◽  
Testing System ◽  
Test Results ◽  
Test Feedback ◽  
Teachers And Students ◽  
Computer Based ◽  
Computer Based Testing ◽  
The University ◽  
Future Learning ◽  
Personalised Feedback

AbstractThis contribution reports on the developing and piloting of a computer-based version of the test of English as a foreign language produced by the University of Central Lancashire (UCLan), where it is currently used for the admission of international students and the subsequent evaluation of their language progress. Among other benefits, computer-based testing allows for better and individualised feedback to both teachers and students, and it can provide a more authentic test experience in light of the current digital shift that UK universities are undergoing. In particular, the qualitative improvement in the feedback available for test-takers and teachers was for us a crucial factor. Providing students with personalised feedback, that is, directly linked to their performance, has positive washforward, because it means we can guide their future learning, highlighting the areas they need to work on to improve their language skills and giving them suggestions on how to succeed in academia. Furthermore, explaining the meaning of test results in detail improves transparency and ultimately washback, as teachers can use the more accessible marking criteria, together with information on how their students performed, to review plans and schemes of work for subsequent courses.

scholarly journals Introducing Computer-Based Testing in High-Stakes Exams in Higher Education: Results of a Field Experiment

PLoS ONE ◽  
2015 ◽  
Vol 10 (12) ◽  
pp. e0143616 ◽  
Author(s):  
Anja J. Boevé ◽  
Rob R. Meijer ◽  
Casper J. Albers ◽  
Yta Beetsma ◽  
Roel J. Bosker
Keyword(s):  
Higher Education ◽  
Field Experiment ◽  
High Stakes ◽  
Computer Based ◽  
Computer Based Testing ◽  
High Stakes Exams
Sign in / Sign up

Export Citation Format

Share Document