scholarly journals Improving Measures via Examining the Behavior of Distractors in Multiple-Choice Tests

2017 ◽  
Vol 77 (1) ◽  
pp. 82-103 ◽  
Author(s):  
Georgios Sideridis ◽  
Ioannis Tsaousis ◽  
Khaleel Al Harbi
Keyword(s):  
Present Article ◽  
Multiple Choice ◽  
Model Fit ◽  
Partial Credit Model ◽  
Partial Credit ◽  
Multiple Choice Tests ◽  
National Examination ◽  
Advantages And Disadvantages ◽  
Remedial Actions ◽  
Low Ability

The purpose of the present article was to illustrate, using an example from a national assessment, the value from analyzing the behavior of distractors in measures that engage the multiple-choice format. A secondary purpose of the present article was to illustrate four remedial actions that can potentially improve the measurement of the construct(s) under study. Participants were 2,248 individuals who took a national examination of chemistry. The behavior of the distractors was analyzed by modeling their behavior within the Rasch model. Potentially informative distractors were (a) further modeled using the partial credit model, (b) split onto separate items and retested for model fit and parsimony, (c) combined to form a “super” item or testlet, and (d) reexamined after deleting low-ability individuals who likely guessed on those informative, albeit erroneous, distractors. Results indicated that all but the item split strategies were associated with better model fit compared with the original model. The best fitted model, however, involved modeling and crediting informative distractors via the partial credit model or eliminating the responses of low-ability individuals who likely guessed on informative distractors. The implications, advantages, and disadvantages of modeling informative distractors for measurement purposes are discussed.

Abstract: An Option-Based Partial Credit IRT Model for Multiple-Choice Tests

2013 ◽  
Vol 48 (1) ◽  
pp. 146-147 ◽  
Author(s):  
Yuan-chao Bo ◽  
Charles Lewis ◽  
David V. Budescu
Keyword(s):  
Multiple Choice ◽  
Partial Credit ◽  
Multiple Choice Tests ◽  
Irt Model ◽  
Choice Tests

scholarly journals Students’ Perception Towards National Examination 2017: Computer-Based Test or Paper-Based Test

2017 ◽  
Vol 8 (4-1) ◽  
pp. 139-143
Author(s):  
◽  
Waode Ade Sarasmita Uke ◽  
Keyword(s):  
High School ◽  
Language Learning ◽  
Technological Development ◽  
Multiple Choice ◽  
Learning And Teaching ◽  
Negative Side ◽  
Multiple Choice Tests ◽  
National Examination ◽  
Choice Tests ◽  
Computer Based

Abstract Today, technological development has given a new lease of life in language learning and teaching field. Particularly in terms of evaluation, some schools in Indonesia conduct national examination using computer-based test. Traditionally, testing in a class involves paper-based test which is familiar to students. A substantial number of studies have been conducted to compare both of the tests. The researcher investigated what activities were performed by the students throughout the tests, what kind of multiple choice tests the students prefer to do, and what the positive and negative side of those tests. This study presents a wonder share quiz creator (WSQC) program to facilitate the teachers to design a test. The data was collected by questionnaire, interview, observation, and quiz. In this study, the authors concluded that students in senior high school of Kendari prefer paper-based test for national examination.

scholarly journals Implementation of Four-Tier Multiple-Choice Instruments Based on the Partial Credit Model in Evaluating Students’ Learning Progress

2021 ◽  
Vol 10 (2) ◽  
pp. 825-840
Author(s):  
Lukman Abdul ◽  
Syukrul Hamdi* ◽  
Masrid Pikoli ◽  
Romario Abdullah ◽  
Citra Panigoro
Keyword(s):  
Multiple Choice ◽  
Partial Credit Model ◽  
Partial Credit ◽  
Learning Progress

scholarly journals Review and Comparison of Four-Tier Multiple Choice and Five-Tier Multiple Choice Diagnostic Tests to Identify Mastery of Physics Concepts

2022 ◽  
Vol 8 (1) ◽  
pp. 1-12
Author(s):  
Hendra Musfa Dirman ◽  
Fatni Mufit ◽  
Festiyed Festiyed
Keyword(s):  
High School ◽  
Diagnostic Test ◽  
Diagnostic Tests ◽  
Multiple Choice ◽  
Optical Devices ◽  
Multiple Choice Tests ◽  
Advantages And Disadvantages ◽  
High School Physics ◽  
Choice Tests ◽  
Four Levels

Misconceptions of a concept in a lesson will have an influence in understanding the next concept. Having misconceptions that exist in learning can understand the nature of the misconceptions and consequently can help student learning progress. Therefore, a diagnostic test for misconceptions is needed, including the newest four-tier multiple choice and five-tier multiple choice. This research is a literature review that provides information systematically using the PRISMA method which often occurs in students' misconceptions in high school physics subjects. The data for this research are 60 selected articles from 2017-2021. The purpose of this study is to reveal the use of four-tier multiple choice and five-tier multiple choice diagnostic tests in physics and also provide a comparison of each instrument with the strengths and weaknesses of the four-tier multiple choice and five-tier multiple choice diagnostic tests. Furthermore, the use of multiple choice four levels (83.33%) and multiple choice five levels (16.67). %). In the use of the four-tier multiple choice physical material diagnostic test, which are often used in identifying misconceptions, are optical devices (12%) and energy businesses (10%). And the use of an additional five-tier multiple choice instrument diagnostic test that is often used is to present an overview or conclusion at the fifth level. However, each type of four-tier multiple choice and five-tier multiple choice tests has its own advantages and disadvantages in assessing students' conceptions

scholarly journals Item-Focused Trees for the Detection of Differential Item Functioning in Partial Credit Models

2017 ◽  
Vol 78 (5) ◽  
pp. 781-804 ◽  
Author(s):  
Stella Bollmann ◽  
Moritz Berger ◽  
Gerhard Tutz
Keyword(s):  
Differential Item Functioning ◽  
Present Article ◽  
Item Response ◽  
Partial Credit Model ◽  
Partial Credit ◽  
Response Models ◽  
Item Response Models ◽  
Item Functioning ◽  
Alternative Approaches ◽  
Ordered Response

Various methods to detect differential item functioning (DIF) in item response models are available. However, most of these methods assume that the responses are binary, and so for ordered response categories available methods are scarce. In the present article, DIF in the widely used partial credit model is investigated. An item-focused tree is proposed that allows the detection of DIF items, which might affect the performance of the partial credit model. The method uses tree methodology, yielding a tree for each item that is detected as DIF item. The visualization as trees makes the results easily accessible, as the obtained trees show which variables induce DIF and in which way. In the present paper, the new method is compared with alternative approaches and simulations demonstrate the performance of the method.

scholarly journals DIAGNOSIS KESALAHAN SISWA BERBASIS PENSKORAN POLITOMUS MODEL PARTIAL CREDIT PADA MATEMATIKA

2013 ◽  
Vol 15 (2) ◽  
pp. 308-325 ◽  
Author(s):  
Awal Isgiyanto
Keyword(s):  
High School Students ◽  
Junior High School ◽  
Diagnostic Information ◽  
Diagnostic Study ◽  
Partial Credit Model ◽  
Partial Credit ◽  
School Students ◽  
Test Items ◽  
National Examination ◽  
Post Hoc

Penelitian ini bertujuan untuk menemukan informasi diagnostik dari kesalahan jawaban peserta pada Ujian Nasional (UN) Matematika. Informasi diagnostik yang ditemukan meli-puti atribut yang mendasari butir soal, ketidaktuntasan atribut, dan jenis kesalahan yang dilakukan oleh peserta. Penelitian ini merupakan diagnosis post-hoc, yang digambarkan sebagai pende-katan retrofitting. Analisis butir soal dan respons butir pada UN mata pelajaran matematika untuk menemukan informasi diag-nostik pada kategori isi, proses, dan keterampilan siswa SMP di Bantul Yogyakarta 2007/2008. Hasil penelitian (1) atribut yang mendasari butir soal pada matematika ada 47 atribut, meliputi 4 atribut isi, 36 atribut proses, dan 7 atribut keterampilan, (2) ketidaktuntasan atribut isi, proses, dan keterampilan yang ter-tinggi pada geometri dan pengukuran, (3) jenis kesalahan ter-tinggi pada bilangan, aljabar, dan geometri dan pengukuran adalah kesalahan konsep, dan jenis kesalahan tertinggi pada sta-tistika dan peluang adalah kesalahan interpretasi bahasa, dan (4) penemuan informasi diagnostik data UN Matematika dapat dilakukan melalui mekanisme identifikasi atribut, pengembang-an rubrik penskoran politomus, perhitungan ketidaktuntasan atribut, dan diagnosis kesalahan peserta tes. Kata kunci: diagnostik, matematika, partial credit model ______________________________________________________________A DIAGNOSIS OF STUDENTS’ MISTAKES BASED ON POLYTOMOUS SCORNG PARTIAL CREDIT MODELS IN MATHEMATICS Abstract This study aims to reveal diagnostic information from the participants’ incorrect answers in the National Examination (NE) of Mathematics. The diagnostic information includes the attributes underlying test items, the attribute exhaustiveness, and the types of mistakes that the participants made. This study was a post-hoc diagnostic study, described as the retrofitting approach. The analysis of test items and item responses in the NE of Mathematics aimed to reveal the diagnostic information in the Junior High School students’ content, process, and skill categories. The results are that (1) there are 47 attributes underlying the mathematics test items, consisting of 4 content attributes, 36 process attributes, and 7 skill attributes, (2) the highest inexhaustiveness of the content, process, and skill attributes is in the topics of geometry and measurement, (3) most mistakes in the topics of numbers, algebra, geometry, and measurement are those of concepts, and most mistakes in statistics and probability are those of language interpretation, and (4) the findings of diagnostic information in the data on the NE of Mathematics can be revealed through the mechanism of identifying the attributes, developing a polytomous scoring rubric, finding out the attribute inexhaustiveness, and diagnosing mistakes the test participants.Keywords: diagnosis, mathematics, partial credit model

Sign in / Sign up

Export Citation Format

Share Document