Applicability and Efficiency of a Computerized Adaptive Test for the Washington Assessment of the Risks and Needs of Students

Assessment ◽  
2021 ◽  
pp. 107319112110478
Author(s):  
Cihan Demir ◽  
Brian F. French
Keyword(s):  
Service Providers ◽  
Accurate Estimate ◽  
Theory Model ◽  
Drop Out ◽  
Future Directions ◽  
Computerized Adaptive Test ◽  
Adaptive Test ◽  
Youth Service ◽  
Administration Time ◽  
Computer Based

The Washington Assessment of the Risks and Needs of Students (WARNS) is a computer-based assessment created to help courts, schools, and youth service providers determine an adolescent’s risks and needs that may lead to truancy, drop out, or delinquency from school. Users are advised to consider the WARNS total score to work with youth. A total score estimate based on fewer items than the full item set may result in less respondent burden, administration time, and fatigue, while not hindering accurate decisions. This simulation study examined the applicability and efficiency of a computerized adaptive test (CAT) to estimate a WARNS total score under a unidimensional item response theory model. The results demonstrate that the CAT provides an accurate estimate of students’ risks and needs and reduces the number of items administered for each examinee compared with the existing version. Future directions and limitations of CAT development with the WARNS are discussed.

scholarly journals MEMBANDINGKAN IRT MODEL 3PL DENGAN IRT MODEL 4PL UNTUK PENILAIAN MENGGUNAKAN COMPUTERIZED ADAPTIVE TEST

SAINTEKBU ◽  
2016 ◽  
Vol 9 (1) ◽  
Author(s):  
Aslam Fatkhudin ◽  
M. Fikri Hidayatullah
Keyword(s):  
Testing System ◽  
Computerized Adaptive Test ◽  
Test Items ◽  
Irt Model ◽  
Adaptive Test ◽  
Irt Models ◽  
Best Value ◽  
Computer Based ◽  
Computer Based Testing ◽  
Better Than

One of the computer-based testing is the Computerized Adaptive Test (CAT), which is a computer-based testing system where the items were given to the participants adapted to test the ability of the participants. Assessment methods are usually applied in CAT is Item Response Theory (IRT). IRT models are most commonly used today is the model 3 Parameter Logistic (3PL), which is about the discrimination, difficulty and guessing. However 3PL IRT models have not provided information more objectively test the ability of participants. The opinion of the test participants were tested items were also to be considered. In this study using CAT in combination with IRT model of 4PL.In this research, the development of CAT which uses about 4 parameters, namely the discrimination, difficulty, guessing and questionnaires. The questions used were about UAS 1 English subjects. Samples were taken from 40 students answer with the best value of the total 172 students spread across 6 classes to measure the parameter estimation problem. Further testing using CAT application 4PL IRT models compared to CAT 3PL IRT models.From research done shows that the CAT application combined with IRT models 4PL can measure the ability of the test taker shorter or faster and also opportunities participants correctly answered the test items was done tend to be better than the 3PL IRT models. Keywords : Ability, CAT, IRT, 3PL, 4PL, Probability, Test 

scholarly journals Academic English Proficiency Assessment Using a Computerized Adaptive Test

2019 ◽  
Vol 20 (2) ◽  
pp. 381 ◽  
Author(s):  
Mariana Cúri ◽  
Vanessa Silva
Keyword(s):  
Evaluation System ◽  
English Language ◽  
Graduate Program ◽  
English Proficiency ◽  
Latent Trait ◽  
Estimation Method ◽  
Information Criterion ◽  
Theory Model ◽  
Computerized Adaptive Test ◽  
Adaptive Test

This paper describes the steps to convert a paper-and-pencil English proficiency test for academic purposes, consisting of multiple choice items administered following the Admissible Probability Measurement Procedure [24], adopted by the graduate program at the Institute of Mathematics and Computer Sciences at the University of São Paulo (ICMC-USP), Brazil, to a computerized adaptive test (CAT) based on an Item Response Theory Model (IRT). Despite the fact that the program accepts various internationally recognized tests that attest non-native speakers English proficiency, such as the Test of English as a Foreign Language (TOEFL), the International English Language Testing System (IELTS) and the Cambridge English: Proficiency (CPE), for instance, its requirement is incoherent in public universities in Brazil due to the cost, which ranges from US$ 200.00 to US$ 300.00 per exam. The TAI-PI software (Computerized Adaptive Test for English Proficiency), which was developed in Java language and SQLite, started to be used to assess the English pro?ciency of students on the program from October, 2013. The statistical methodology used was defined considering the history and aims of the test and adopted Samejima's Graded Response Model [21], the Kullback-Leibler information criterion for item selection, the a posteriori estimation method for latent trait [2] and the Shadow Test approach [29] to impose restrictions (content and test length) on the test composition of each individual. A description of the test design, the statistical methods used, and the results of a real application of TAI-PI for graduate students are presented in this paper, as well as the validation studies of the new methodology for pass or fail classification, showing the good quality of the new evaluation system and examination of improvement using the IRT and CAT methods.

Developing and Validating a Computerized Adaptive Test to Measure Broad and Specific Factors of Internalizing in a Community Sample

Assessment ◽  
2017 ◽  
Vol 26 (6) ◽  
pp. 1030-1045 ◽  
Author(s):  
Matthew Sunderland ◽  
Philip Batterham ◽  
Natacha Carragher ◽  
Alison Calear ◽  
Tim Slade
Keyword(s):  
Online Survey ◽  
Community Sample ◽  
Theory Model ◽  
Item Bank ◽  
Obsessive Compulsive ◽  
Computerized Adaptive Test ◽  
Compulsive Disorder ◽  
Adaptive Test ◽  
Convergent And Divergent Validity ◽  
Anxiety Depression

Highly efficient assessments that better account for comorbidity between mood and anxiety disorders (internalizing) are required to identify individuals who are most at risk of psychopathology in the community. The current study examined the efficiency and validity associated with a multidimensional computerized adaptive test (CAT) to measure broad and specific levels of internalizing psychopathology. The sample comprised 3,175 respondents to an online survey. Items from five banks (generalized anxiety, depression, obsessive–compulsive disorder, panic disorder, social anxiety disorder) were jointly calibrated using a bifactor item response theory model. Simulations indicated that an adaptive algorithm could accurately ( rs ≥ 0.90) estimate general internalizing and specific disorder scores using on average 44 items in comparison with the full 133-item bank (67% reduction in items). Scores on the CAT demonstrate convergent and divergent validity with previously validated short severity scales and could significantly differentiate cases of DSM-5 disorder. As such, the CAT validly measures both broad and specific constructs of internalizing disorders in a manner similar to the full item bank and a static brief form but with greater gains in efficiency and, therefore, a reduced degree of respondent burden.

Penn Reading Assessment Computerized Adaptive Test

2019 ◽  
Author(s):  
Mikhal A. Yudien ◽  
Tyler M. Moore ◽  
Allison M. Port ◽  
Kosha Ruparel ◽  
Raquel E. Gur ◽  
...  
Keyword(s):  
Reading Assessment ◽  
Computerized Adaptive Test ◽  
Adaptive Test

scholarly journals A Generic Model for Identifying QoS Parameters Interrelations in Cloud Services Selection Ontology during Runtime

Symmetry ◽  
2021 ◽  
Vol 13 (4) ◽  
pp. 563
Author(s):  
Babu Rajendiran ◽  
Jayashree Kanniappan
Keyword(s):  
Service Providers ◽  
Cloud Service ◽  
Cloud Services ◽  
Operating Costs ◽  
Generic Model ◽  
Business Organizations ◽  
Cloud Environment ◽  
Cloud Service Provider ◽  
Qos Parameters ◽  
Computer Based

Nowadays, many business organizations are operating on the cloud environment in order to diminish their operating costs and to select the best service from many cloud providers. The increasing number of Cloud Services available on the market encourages the cloud consumer to be conscious in selecting the most apt Cloud Service Provider that satisfies functionality, as well as QoS parameters. Many disciplines of computer-based applications use standardized ontology to represent information in their fields that indicate the necessity of an ontology-based representation. The proposed generic model can help service consumers to identify QoS parameters interrelations in the cloud services selection ontology during run-time, and for service providers to enhance their business by interpreting the various relations. The ontology has been developed using the intended attributes of QoS from various service providers. A generic model has been developed and it is tested with the developed ontology.

scholarly journals Measurement Precision and Efficiency of Computerized Adaptive Testing for the Activities-specific Balance Confidence Scale in People With Stroke

2021 ◽  
Author(s):  
Bryant A Seamon ◽  
Steven A Kautz ◽  
Craig A Velozo
Keyword(s):  
Rasch Model ◽  
Computerized Adaptive Testing ◽  
Adaptive Testing ◽  
Measurement Precision ◽  
Strongly Correlated ◽  
Computerized Adaptive Test ◽  
Balance Confidence ◽  
Adaptive Test ◽  
The Rasch Model ◽  
Confidence Scale

Abstract Objective Administrative burden often prevents clinical assessment of balance confidence in people with stroke. A computerized adaptive test (CAT) version of the Activities-specific Balance Confidence Scale (ABC CAT) can dramatically reduce this burden. The objective of this study was to test balance confidence measurement precision and efficiency in people with stroke with an ABC CAT. Methods We conducted a retrospective cross-sectional simulation study with data from 406 adults approximately 2-months post-stroke in the Locomotor-Experience Applied Post-Stroke (LEAPS) trial. Item parameters for CAT calibration were estimated with the Rasch model using a random sample of participants (n = 203). Computer simulation was used with response data from remaining 203 participants to evaluate the ABC CAT algorithm under varying stopping criteria. We compared estimated levels of balance confidence from each simulation to actual levels predicted from the Rasch model (Pearson correlations and mean standard error (SE)). Results Results from simulations with number of items as a stopping criterion strongly correlated with actual ABC scores (full item, r = 1, 12-item, r = 0.994; 8-item, r = 0.98; 4-item, r = 0.929). Mean SE increased with decreasing number of items administered (full item, SE = 0.31; 12-item, SE = 0.33; 8-item, SE = 0.38; 4-item, SE = 0.49). A precision-based stopping rule (mean SE = 0.5) also strongly correlated with actual ABC scores (r = .941) and optimized the relationship between number of items administrated with precision (mean number of items 4.37, range [4–9]). Conclusions An ABC CAT can determine accurate and precise measures of balance confidence in people with stroke with as few as 4 items. Individuals with lower balance confidence may require a greater number of items (up to 9) and attributed to the LEAPS trial excluding more functionally impaired persons. Impact Statement Computerized adaptive testing can drastically reduce the ABC’s test administration time while maintaining accuracy and precision. This should greatly enhance clinical utility, facilitating adoption of clinical practice guidelines in stroke rehabilitation. Lay Summary If you have had a stroke, your physical therapist will likely test your balance confidence. A computerized adaptive test version of the ABC scale can accurately identify balance with as few as 4 questions, which takes much less time.

Using a Response Time–Based Expected A Posteriori Estimator to Control for Differential Speededness in Computerized Adaptive Test

2021 ◽  
pp. 014662162110146
Author(s):  
Justin L. Kern ◽  
Edison Choe
Keyword(s):  
Response Times ◽  
Real Data ◽  
Joint Estimation ◽  
Item Selection ◽  
A Posteriori ◽  
Computerized Adaptive Test ◽  
Maximum Information ◽  
Adaptive Test ◽  
Estimation And Control ◽  
And Control

This study investigates using response times (RTs) with item responses in a computerized adaptive test (CAT) setting to enhance item selection and ability estimation and control for differential speededness. Using van der Linden’s hierarchical framework, an extended procedure for joint estimation of ability and speed parameters for use in CAT is developed following van der Linden; this is called the joint expected a posteriori estimator (J-EAP). It is shown that the J-EAP estimate of ability and speededness outperforms the standard maximum likelihood estimator (MLE) of ability and speededness in terms of correlation, root mean square error, and bias. It is further shown that under the maximum information per time unit item selection method (MICT)—a method which uses estimates for ability and speededness directly—using the J-EAP further reduces average examinee time spent and variability in test times between examinees above the resulting gains of this selection algorithm with the MLE while maintaining estimation efficiency. Simulated test results are further corroborated with test parameters derived from a real data example.

Sign in / Sign up

Export Citation Format

Share Document