Evaluating the Accuracy of the Empirical Item Characteristic Curve Preequating Method in the Presence of Test Speededness

This study aimed to assess the accuracy of the empirical item characteristic curve (EICC) preequating method given the presence of test speededness. The simulation design of this study considered the proportion of speededness, speededness point, speededness rate, proportion of missing on speeded items, sample size, and test length. After crossing all of the manipulated factors and then normalizing the evaluation criteria (bias and root mean square difference [ RMSD]) with regard to test length, the results revealed that (1) when test speededness was present, conversions from the EICC preequating method tended to be positively distorted; (2) no practically meaningful moderation effect associated with sample size was found on the relationship between test speededness and the accuracy of EICC preequating; and (3) the location of the speededness point was the driving factor in terms of its impact on the accuracy of EICC preequating. Implications and suggestions were discussed.

ДААЛГАВРЫН ХАРИУЛТЫН ОНОЛ (ITEM RESPONSE THEORY)-ЫН 3 ПАРАМЕТРТЭЙ ЗАГВАРЫГ АШИГЛАЖ ТЕСТИЙН ДААЛГАВРУУДАД ШИНЖИЛГЭЭ ХИЙХ НЬ

Энэхүү өгүүлэл нь Даалгаврын хариултын онол (Item Response Theory)-ын хэрэглээний ач холбогдол, тестийн даалгавар боловсруулах аргачлалыг тайлбарлахыг зорив. Ингэхдээ 7 дугаар ангийн сурагчдад зориулсан 4 сонголт бүхий математикийн тест боловсруулж, 480 сурагчаар 40 минутад ажиллуулж хариултуудыг цуглуулсан болно. Тестийн даалгавруудад өгсөн сурагчдын хариултуудад 3 параметртэй (a, b, c) IRT загвар, BILOG-MG программыг ашиглаж анализ хийв. Даалгаврын параметрүүдийг даалгаврын онцлогийн муруйг (Item Characteristic Curve) ашиглан тайлбарласан болно.

PENGEMBANGAN TES BERPIKIR KRITIS DENGAN PENDEKATAN ITEM RESPONSE THEORY

Penelitian ini bertujuan untuk menghasilkan sebuah alat ukur (tes) berpikir kritis yang valid dan reliabel untuk digunakan, baik dalam lingkup pendidikan maupun kerja di Indonesia. Tahapan penelitian dilakukan berdasarkan tahap pengembangan tes menurut Hambleton dan Jones (1993). Kisi-kisi dan pembuatan butir didasarkan pada konsep dalam tes Watson-Glaser Critical Thinking Appraisal (WGCTA). Pada WGCTA, berpikir kritis terdiri dari lima dimensi yaitu Inference, Recognition Assumption, Deduction, Interpretation dan Evaluation of arguments. Uji coba tes dilakukan pada 1.453 peserta tes seleksi karyawan di Surabaya, Gresik, Tuban, Bojonegoro, Rembang. Data dikotomi dianalisis dengan menggunakan model IRT dengan dua parameter yaitu daya beda dan tingkat kesulitan butir. Analisis dilakukan dengan menggunakan program statistik Mplus versi 6.11 Sebelum melakukan analisis dengan IRT, dilakukan pengujian asumsi yaitu uji unidimensionalitas, independensi lokal dan Item Characteristic Curve (ICC). Hasil analisis terhadap 68 butir menghasilkan 15 butir dengan daya beda yang cukup baik dan tingkat kesulitan butir yang berkisar antara –4 sampai dengan 2.448. Sedikitnya jumlah butir yang berkualitas baik disebabkan oleh kelemahan dalam menentukan subject matter experts di bidang berpikir kritis dan pemilihan metode skoring.Kata kunci: Pengembangan tes, berpikir kritis, item response theory DEVELOPING CRITICAL THINKING TEST UTILISING ITEM RESPONSE THEORYAbstractThe present study was aimed to develop a valid and reliable instrument in assesing critical thinking which can be implemented both in educational and work settings in Indonesia. Following the Hambleton and Jones’s (1993) procedures on test development, the study developed the instrument by employing the concept of critical thinking from Watson-Glaser Critical Thinking Appraisal (WGCTA). The study included five dimensions of critical thinking as adopted from the WGCTA: Inference, Recognition Assumption, Deduction, Interpretation dan Evaluation of arguments. 1453 respondents from Surabaya, Gresik, Tuban, Bojonegoro and Rembang were used for trailing the test. The dichotomous data were analized using the Item Response Theory with two parameter logistic model using statistical program Mplus ver. 6.11. Several assumptions were tested prior the IRT analysis; the test of unidimensionality, local independency and Item Characteristic Curve (ICC). Amongst 68 items only 15 items had good discrimination parameter. Difficulty item level ranged from – 4.95 to 2.448. The study was limited in producing high number of qualified items due to its failure in finding subject matter experts in critical thinking area and inadequate choice in scoring method.Keywords: test development, critical thinking, Item response theory

KARAKTERISTIK METODE PENYETARAAN SKOR TES UNTUK DATA DIKOTOMOS

Penelitian ini bertujuan untuk menemukan: 1) akurasi estimasi parameter item pada test equating menggunakan metode Item Characteristic Curve (ICC). 2) sensitivitas metode linear yang terdiri atas Tucker - Levine score method dan Levine true score method applied to observed scores serta metode equipercentile yang terdiri atas metode Braun-Holland linear dan chained equipercentile. Data empiris yang digunakan yaitu respons siswa peserta Ulangan Akhir Semester V Mata Pelajaran Ilmu Pengetahuan Alam (IPA) SMP Tahun Ajaran 2009/2010. Penyetaraan tes menggunakan anchor test design. Anchor test bersifat external, anchor test berfungsi sebagai pengait antara tes yang disetarakan. Item anchor berisi 10 item materi Fisika. Banyak item pada tes A 55 item, tes B 55 item dan tes C 50 item. Pola penyetaraan yang digunakan pola kelompok, sehingga banyak item hasil penyetaraan berjumlah 140 item terdiri atas 10 anchor item milik bersama, 45 item berasal dari tes A, 45 item berasal dari tes B, dan 40 item berasal dari tes C. Hasil penelitian menunjukkan bahwa: 1) Estimasi parameter item pada penyetaraan tes menggunakan metode Item Characteristic Curva (ICC) menghasilkan formula indeks kesulitan item, 2) urutan sensitivitas metode penyetaraan dari paling tinggi sampai paling rendah yaitu Tucker – Levine method, Levine method, Braun - Holland linear method. Chained Equipercentile Equating method.Kata kunci: Test equating, anchor test, external anchor test, RMSD, RMSE______________________________________________________________ THE CHARACTERISTICS OF TEST EQUATING METHODS FOR DICHOTOMOUS DATAAbstract This study aims: 1) to find out the accuracy of item parameter estimates in test equating by means of the Item Characteristic Curve (ICC) method, and 2) to find out the sensivity of the linear methods consisting of the Tucker-Levine score method and the Levine true score method applied to observed scores and the equipercentile methods consisting of the Braun-Holland linear method the chained equipercentile equating method. The data were empirical data obtained from the response patterns of the junior high school students taking the final test of Natural Sciences in the odd semester of the academic year of 2009/2010. The test equating employed the external anchor test design. The anchor test served to unite the equated tests. The anchor test consisted of 10 physics items. The test A had 55 items, the test B had 55 items, and the test C had 50 items. The equating pattern employed the group pattern, so that in the equating there were 140 items, consisting of 10 common anchor items, 45 items from tests A, 45 items from tests B, and 40 items from tests C. The results of the study are as follows. 1) The item parameter estimate in the test score equating by means of the Item Characteristic Curve (ICC) method yields the formula for the item difficulty index, and 2) urutan sensitivitas metode penyetaraan dari paling tinggi sampai paling rendah yaitu The order of the sensitivity of the equating methods from the highest to the lowest is Tucker- Levine method, Levine method, Braun-Holland linear method. Chained Equipercentile Equating method.Keywords: test equating, anchor test, external anchor test, RMSD, RMSE