A new approach to computerized adaptive testing

A new approach to computerized adaptive testing is presented on the basis of discrete-state discrete-time Markov processes. This approach is based on an extension of the G. Rasch model used in the Item Response Theory (IRT) and has decisive advantages over the adaptive IRT testing. This approach has a number of competitive advantages: takes into account all the observed history of performing test items that includes the distribution of successful and unsuccessful item solutions; incorporates time spent on performing test items; forecasts results in the future behavior of the subjects; allows for self-learning and changing subject abilities during a testing procedure; contains easily available model identification procedure based on simply accessible observation data. Markov processes and the adaptive transitions between the items remain hidden for the subjects who have access to the items only and do not know all the intrinsic mathematical details of a testing procedure. The developed model of adaptive testing is easily generalized for the case of polytomous items and multidimensional items and model structures.

Download Full-text

Development of Computerized Adaptive Testing for Emotion Regulation

Frontiers in Psychology ◽

10.3389/fpsyg.2020.561358 ◽

2020 ◽

Vol 11 ◽

Author(s):

Lingling Xu ◽

Ruyi Jin ◽

Feifei Huang ◽

Yanhui Zhou ◽

Zonglong Li ◽

...

Keyword(s):

Emotion Regulation ◽

Model Comparison ◽

Computerized Adaptive Testing ◽

Item Analysis ◽

Well Being ◽

Adaptive Testing ◽

Vital Role ◽

Test Items ◽

Item Fit ◽

Item Functioning

Emotion regulation (ER) plays a vital role in individuals’ well-being and successful functioning. In this study, we attempted to develop a computerized adaptive testing (CAT) to efficiently evaluate ER, namely the CAT-ER. The initial CAT-ER item bank comprised 154 items from six commonly used ER scales, which were completed by 887 participants recruited in China. We conducted unidimensionality testing, item response theory (IRT) model comparison and selection, and IRT item analysis including local independence, item fit, differential item functioning, and item discrimination. Sixty-three items with good psychometric properties were retained in the final CAT-ER. Then, two CAT simulation studies were implemented to assess the CAT-ER, which revealed that the CAT-ER developed in this study performed reasonably well, considering that it greatly lessened the test items and time without losing measurement accuracy.

Download Full-text

A computerized adaptive testing advancing the measurement of subjective well-being

Journal of Pacific Rim Psychology ◽

10.1017/prp.2019.6 ◽

2019 ◽

Vol 13 ◽

Author(s):

Yifang Wu ◽

Yan Cai ◽

Dongbo Tu

Keyword(s):

Computerized Adaptive Testing ◽

Simulated Data ◽

Real Data ◽

Well Being ◽

Adaptive Testing ◽

Test Time ◽

Subjective Well Being ◽

Test Items ◽

Item Fit ◽

Item Functioning

AbstractThis article aimed at developing an adaptive version of the subjective well-being (SWB) scale to measure a comprehensive concept of SWB among Chinese university students. Item response theory was employed to formulate the item bank of the SWB scale and computerized adaptive testing (CAT) for SWB (CAT-SWB), based on several commonly used SWB scales, after unidimensionality testing, model selection, local dependence testing, parameter estimation, item fit test and differential item functioning (DIF) analysis were performed. Finally, two CAT simulations using simulated-data and real-data were carried out to verify and evaluate the CAT-SWB. Results indicated that the proposed CAT-SWB had an excellent performance in that it largely reduces the number of test items and the length of test time without losing measurement precision.

Download Full-text

HASIL PENGUJIAN SOFTWARE COMPUTERIZED ADAPTIVE TESTING (CAT) MATA PELAJARAN PENDIDIKAN AGAMA ISLAM (PAI) DALAM MEMBUAT REKAM MEDIK PEMBELAJARAN (RMP) UNTUK MENDIAGNOSIS KEMAMPUAN SISWA DI SEKOLAH

INFERENSI ◽

10.18326/infsl3.v6i2.183-204 ◽

2012 ◽

Vol 6 (2) ◽

pp. 183

Author(s):

Winarno Winarno

Keyword(s):

Research And Development ◽

Religious Education ◽

Computerized Adaptive Testing ◽

Adaptive Testing ◽

Estimation Result ◽

Web Based ◽

Test Items ◽

Islamic Religion ◽

Development Approach ◽

Quantitative Descriptive

This research aims at producing a Computerized Adaptive Testing (CAT) software Islamic religion education subject (CAT PAI) for the year 7 students of public junior high schoo and to see the results of CAT software in making rekam medik pembelajaran (RMP) to diagnose student’s ability. This research used the research and development approach. The data was collected by doing observation and documentation. Analyzedto use quantitative descriptive technique. The findings of the research shows the CAT can be developed web-based, and The CAT is able to in estimating student’s ability correct be seen RMP. Overall, the CAT software can perform the tasks well in order to select test items and to measure student’s ability from correlation value between ability estimation result (è) in CAT with Islamic religious education subjects enough tall that is 0,69.

Download Full-text

A NEW APPROACH FOR IDENTIFYING LEARNING STYLES BASED ON COMPUTERIZED ADAPTIVE TESTING

EDULEARN17 Proceedings ◽

10.21125/edulearn.2017.1615 ◽

2017 ◽

Cited By ~ 1

Author(s):

Samia Rami ◽

Meriem Hnida ◽

Mohammed Khalidi Idrissi ◽

Samir Bennani

Keyword(s):

Learning Styles ◽

Computerized Adaptive Testing ◽

Adaptive Testing ◽

New Approach

Download Full-text

Students' Attitude toward and Acceptability of Computerized Adaptive Testing in Medical School and their Effect on the Examinees' Ability

Journal of Educational Evaluation for Health Professions ◽

10.3352/jeehp.2005.2.1.105 ◽

2005 ◽

Vol 2 ◽

pp. 105 ◽

Cited By ~ 3

Author(s):

Mee Young Kim ◽

Sun Huh

Keyword(s):

Computerized Adaptive Testing ◽

Adaptive Testing ◽

Test Items ◽

Medical College ◽

Ability Estimates ◽

Significant Difference ◽

Before And After ◽

The Mean ◽

Ability Estimate ◽

Senior Students

An examinee's ability can be evaluated precisely using computerized adaptive testing (CAT), which is shorter than written tests and more efficient in terms of the duration of the examination. We used CAT for the second General Examination of 98 senior students in medical college on November 27, 2004. We prepared 1,050 pre-calibrated test items according to item response theory, which had been used for the General Examination administered to senior students in 2003. The computer was programmed to pose questions until the standard error of the ability estimate was smaller than 0.01. To determine the students' attitude toward and evaluation of CAT, we conducted surveys before and after the examination, via the Web. The mean of the students' ability estimates was 0.3513 and its standard deviation was 0.9097 (range -2.4680 to +2.5310). There was no significant difference in the ability estimates according to the responses of students to items concerning their experience with CAT, their ability to use a computer, or their anxiety before and after the examination (p>0.05). Many students were unhappy that they could not recheck their responses (49%), and some stated that there were too few examination items (24%). Of the students, 79 % had no complaints concerning using a computer and 63% wanted to expand the use of CAT. These results indicate that CAT can be implemented in medical schools without causing difficulties for users.

Download Full-text

PENGEMBANGAN COMPUTERIZED ADAPTIVE TESTING (CAT) DENGAN ALGORITMA LOGIKA FUZZY

Jurnal Penelitian dan Evaluasi Pendidikan ◽

10.21831/pep.v15i1.1087 ◽

2013 ◽

Vol 15 (1) ◽

pp. 47-70

Author(s):

Haryanto Haryanto

Keyword(s):

Fuzzy Logic ◽

Research And Development ◽

Computerized Adaptive Testing ◽

Adaptive Testing ◽

Series Data ◽

Inference System ◽

Fuzzy Logic Algorithm ◽

Test Items ◽

Quantitative Descriptive ◽

User Friendly

Penelitian ini bertujuan menghasilkan: (1) model sistem inferensi dalam mengambil keputusan untuk memilih butir-butir tes yang tepat bagi siswa, (2) perangkat lunak Computerized Adaptive Testing (CAT) dengan algoritma logika fuzzy dalam mendeskripsikan kemampuan siswa. Penelitian dengan pen-dekatan Research and Development (R & D), terdiri atas dua bagian yaitu: (1) pengembangan program CAT, (2) pengujian program CAT pada siswa SMA Negeri 6 Yogyakarta kelas XII sebagai sampel, pada mata pelajaran Matematika dengan pokok bahasan Notasi sigma, barisan dan deret. Data dikumpulkan melalui observasi, dokumentasi, dan angket dan dianalisis secara deskriptif kuantitatif. Hasil penelitian menunjukkan program CAT: (1) mudah digunakan, tampilan interaktif, memiliki sistem keamanan, mudah diakses, dan mengacu standar kekinian, (2) dapat mengenali tiga macam pengguna saat proses login, (3) memiliki tiga macam basis data, (4) memiliki sembilan menu utama, (5) menggunakan model sistem inferensi algoritma logika fuzzy, (6) model tampilan program CAT, (7) dapat bekerja sesuai dengan fungsi dan jenis pengguna, dan (8) mampu mengelola: (a) bank soal, (b) mengemas butir-butir tes secara otomatis sesuai dengan kemampuan siswa, (c) pilihan jawaban dimunculkan secara acak, dan (d) menyimpan rekaman hasil tes secara individu maupun bersama-sama.Kata kunci: program CAT, bank soal______________________________________________________________DEVELOPING A COMPUTERIZED-ADAPTIVE TESTING WITH FUZZY LOGIC ALGORITHMAbstract The development of the Computerized-Adaptive Testing (CAT) aimed to investigate: 1) the performance of inference system in making decision to select the appropriate test items for the students, and 2) the performance of CAT program using fuzzy logic algorithm in describing the competence of students. This research and development (R&D) consists of two parts: (1) developing the CAT, (2) tested testing the CAT on mathematics at the topics of Sigma notation, sequence and series. Data were collected through observation, documentation and questionnaire and analyzed using quantitative descriptive technique. The result showed that: (1) the CAT user friendly, interactive, secure, accessible, and current and (2) able to recognize three types of users while logging-in three types of data bases, that nine options on the main menu it is able to work properly according to the function and the type of users, and administrered the bank and organize test items automatically, based on students’ competence, randomize the answer in the options, and record the result of the test simultaneously and individually.Keywords: CAT program

Download Full-text

Computerized Adaptive Testing With Polytomous Items

Applied Psychological Measurement ◽

10.1177/014662169501900103 ◽

1995 ◽

Vol 19 (1) ◽

pp. 5-22 ◽

Cited By ~ 75

Author(s):

Barbara G. Dodd ◽

R.J. De Ayala ◽

William R. Koch

Keyword(s):

Computerized Adaptive Testing ◽

Adaptive Testing ◽

Polytomous Items

Download Full-text

Computer Adaptive Test Development To Assess Students’ Psychology

JURNAL SERAMBI ILMU ◽

10.32672/si.v22i1.2760 ◽

2021 ◽

Vol 22 (1) ◽

pp. 139-149

Author(s):

Phan Thi Thu Nguyet ◽

Muslem Daud

Keyword(s):

Computerized Adaptive Testing ◽

Testing Procedure ◽

Multinomial Logit ◽

Adaptive Testing ◽

Random Coefficients ◽

Standard Errors ◽

Multidimensional Model ◽

Good Reliability ◽

Computer Adaptive Test ◽

Simulated Experiment

Stress becomes a significantly serious issue among university students and we need efficient tools to understand it more. The aim of present study is to develop a Computerized Adaptive Testing (CAT) to measure the mentioned stress, as pioneer project in Vietnam. In this vein, an item bank of 68 items has been constructed, which is based on Likert Polytomous Scales through five subdomains: behavior, academic performance, family, lecturer and finance. The sampling of the survey is large. It has assessed 2,085 students (704 males and 1,381 females). Multidimensional Random Coefficients Multinomial Logit (MRCML) Model is applied to develop Multidimensional Stress Scales and Computerized Adaptive Testing procedure. The result findings indicate that Multidimensional Random Coefficients Multinomial Logit (MRCML) can be used to develop new scale with psychometric properties. Indicated by various fit criteria MNSQ, standard errors, Z (t-test) implemented in software ConQuest. The subdomain has a good reliability (from .857 to .798). As respect to CATs, a simulated experiment based on the empirical data is applied to evaluate the performance of the proposed computerized adaptive testing. The standard error of the estimated stress proficiencies are reported in this study. The 68 items stress data appropriate fit the Multidimensional model applied.

Download Full-text

EFISIENSI DAN AKURASI COMPUTERIZED ADAPTIVE TESTING PADA SISTEM UJIAN AKHIR SEMESTER UNIVERSITAS TERBUKA

Jurnal Pendidikan ◽

10.33830/jp.v11i1.527.2010 ◽

2010 ◽

Vol 11 (1) ◽

pp. 1-9

Author(s):

Agus Santoso

Keyword(s):

Standard Error ◽

Computerized Adaptive Testing ◽

Adaptive Testing ◽

Theory Model ◽

Test Design ◽

Examination System ◽

Test Items ◽

Conventional Test ◽

Standard Error Of Measurement ◽

Error Of Measurement

Universitas Terbuka (UT) applied online examination system (sistem ujian online SUO) for end of semester examination (ujian akhir semester-UAS), beside the paper and pencil test (P & P test). In order to improve efficiency, adaptive test application should be analyzed, as an alternative to present UAS system. The aim of the research was to compare the efficiency and accuracy level of the computerized adaptive testing (CAT) design and conventional test using both P & P test and SUO. The research was conducted by simulation procedure. The item bank for the simulation used calibrated 404 test items using item response theory model. In the research, CAT and P & P test algorithm was developed. To measure efficiency, the required number of the CAT design was analyzed, while to measure accuracy of the estimation, the bias and standard error of measurement of both design were compared. The simulation result showed that (1) CAT design was more efficient, since it required only half of the number of item which was used in P & P test, to estimate the ability of examinee, (2) CAT design was more accurate in estimating ability of examinee, compared to P & P test design, since it resulted lower bias and standard error of measurement compared to conventional test design. Therefore, CAT design could be applied in UTs UAS system, while considering the balance of content for each modules.

Download Full-text