Development of an item bank for computerized adaptive test (CAT) measurement of pain

Background: Contemporary clinical assessments of activity are needed across the age span for children with cerebral palsy (CP). Computerized adaptive testing (CAT) has the potential to efficiently administer items for children across wide age spans and functional levels. Objective: The objective of this study was to examine the psychometric properties of a new item bank and simulated computerized adaptive test to assess activity level abilities in children with CP. Design: This was a cross-sectional item calibration study. Methods: The convenience sample consisted of 308 children and youth with CP, aged 2 to 20 years (X=10.7, SD=4.0), recruited from 4 pediatric hospitals. We collected parent-report data on an initial set of 45 activity items. Using an Item Response Theory (IRT) approach, we compared estimated scores from the activity item bank with concurrent instruments, examined discriminate validity, and developed computer simulations of a CAT algorithm with multiple stop rules to evaluate scale coverage, score agreement with CAT algorithms, and discriminant and concurrent validity. Results: Confirmatory factor analysis supported scale unidimensionality, local item dependence, and invariance. Scores from the computer simulations of the prototype CATs with varying stop rules were consistent with scores from the full item bank (r=.93–.98). The activity summary scores discriminated across levels of upper-extremity and gross motor severity and were correlated with the Pediatric Outcomes Data Collection Instrument (PODCI) physical function and sports subscale (r=.86), the Functional Independence Measure for Children (Wee-FIM) (r=.79), and the Pediatric Quality of Life Inventory–Cerebral Palsy version (r=.74). Limitations: The sample size was small for such IRT item banks and CAT development studies. Another limitation was oversampling of children with CP at higher functioning levels. Conclusions: The new activity item bank appears to have promise for use in a CAT application for the assessment of activity abilities in children with CP across a wide age range and different levels of motor severity.

Download Full-text

Development of a Computerized Adaptive Test for Anxiety Based on the Dutch–Flemish Version of the PROMIS Item Bank

Assessment ◽

10.1177/1073191117746742 ◽

2017 ◽

Vol 26 (7) ◽

pp. 1362-1374 ◽

Cited By ~ 2

Author(s):

Gerard Flens ◽

Niels Smits ◽

Caroline B. Terwee ◽

Joost Dekker ◽

Irma Huijbrechts ◽

...

Keyword(s):

General Population ◽

Clinical Sample ◽

Population Sample ◽

Mental Health Disorder ◽

Item Bank ◽

General Population Sample ◽

Computerized Adaptive Test ◽

Adaptive Test ◽

Anxiety Item ◽

The Usa

We used the Dutch–Flemish version of the USA PROMIS adult V1.0 item bank for Anxiety as input for developing a computerized adaptive test (CAT) to measure the entire latent anxiety continuum. First, psychometric analysis of a combined clinical and general population sample ( N = 2,010) showed that the 29-item bank has psychometric properties that are required for a CAT administration. Second, a post hoc CAT simulation showed efficient and highly precise measurement, with an average number of 8.64 items for the clinical sample, and 9.48 items for the general population sample. Furthermore, the accuracy of our CAT version was highly similar to that of the full item bank administration, both in final score estimates and in distinguishing clinical subjects from persons without a mental health disorder. We discuss the future directions and limitations of CAT development with the Dutch–Flemish version of the PROMIS Anxiety item bank.

Download Full-text

Developing and Validating a Computerized Adaptive Test to Measure Broad and Specific Factors of Internalizing in a Community Sample

Assessment ◽

10.1177/1073191117707817 ◽

2017 ◽

Vol 26 (6) ◽

pp. 1030-1045 ◽

Cited By ~ 7

Author(s):

Matthew Sunderland ◽

Philip Batterham ◽

Natacha Carragher ◽

Alison Calear ◽

Tim Slade

Keyword(s):

Online Survey ◽

Community Sample ◽

Theory Model ◽

Item Bank ◽

Obsessive Compulsive ◽

Computerized Adaptive Test ◽

Compulsive Disorder ◽

Adaptive Test ◽

Convergent And Divergent Validity ◽

Anxiety Depression

Highly efficient assessments that better account for comorbidity between mood and anxiety disorders (internalizing) are required to identify individuals who are most at risk of psychopathology in the community. The current study examined the efficiency and validity associated with a multidimensional computerized adaptive test (CAT) to measure broad and specific levels of internalizing psychopathology. The sample comprised 3,175 respondents to an online survey. Items from five banks (generalized anxiety, depression, obsessive–compulsive disorder, panic disorder, social anxiety disorder) were jointly calibrated using a bifactor item response theory model. Simulations indicated that an adaptive algorithm could accurately ( rs ≥ 0.90) estimate general internalizing and specific disorder scores using on average 44 items in comparison with the full 133-item bank (67% reduction in items). Scores on the CAT demonstrate convergent and divergent validity with previously validated short severity scales and could significantly differentiate cases of DSM-5 disorder. As such, the CAT validly measures both broad and specific constructs of internalizing disorders in a manner similar to the full item bank and a static brief form but with greater gains in efficiency and, therefore, a reduced degree of respondent burden.

Download Full-text

Rotating Item Banks versus Restriction of Maximum Exposure Rates in Computerized Adaptive Testing

The Spanish Journal of Psychology ◽

10.1017/s1138741600004625 ◽

2008 ◽

Vol 11 (2) ◽

pp. 618-625 ◽

Cited By ~ 4

Author(s):

Juan Ramón Barrada ◽

Julio Olea ◽

Francisco José Abad

Keyword(s):

Measurement Accuracy ◽

Computerized Adaptive Testing ◽

Item Bank ◽

Exposure Rate ◽

Exposure Control ◽

Computerized Adaptive Test ◽

Adaptive Test ◽

Item Banks ◽

Overlap Rate ◽

Maximum Exposure

If examinees were to know, beforehand, part of the content of a computerized adaptive test, their estimated trait levels would then have a marked positive bias. One of the strategies to avoid this consists of dividing a large item bank into several sub-banks and rotating the sub-bank employed (Ariel, Veldkamp & van der Linden, 2004). This strategy permits substantial improvements in exposure control at little cost to measurement accuracy. However, we do not know whether this option provides better results than using the master bank with greater restriction in the maximum exposure rates (Sympson & Hetter, 1985). In order to investigate this issue, we worked with several simulated banks of 2100 items, comparing them, for RMSE and overlap rate, with the same banks divided in two, three… up to seven sub-banks. By means of extensive manipulation of the maximum exposure rate in each bank, we found that the option of rotating banks slightly outperformed the option of restricting maximum exposure rate of the master bank by means of the Sympson-Hetter method.

Download Full-text

The EORTC emotional functioning computerized adaptive test: phases I–III of a cross‐cultural item bank development

Psycho-Oncology ◽

10.1002/pon.3427 ◽

2013 ◽

Vol 23 (4) ◽

pp. 397-403 ◽

Cited By ~ 17

Author(s):

Eva‐Maria Gamper ◽

Mogens Groenvold ◽

Morten Aa Petersen ◽

Teresa Young ◽

Anna Costantini ◽

...

Keyword(s):

Emotional Functioning ◽

Item Bank ◽

Cross Cultural ◽

Computerized Adaptive Test ◽

Adaptive Test

Download Full-text

Supplemental Material for Efficiently Measuring Dimensions of the Externalizing Spectrum Model: Development of the Externalizing Spectrum Inventory-Computerized Adaptive Test (ESI-CAT)

Psychological Assessment ◽

10.1037/pas0000384.supp ◽

2016 ◽

Keyword(s):

Model Development ◽

Computerized Adaptive Test ◽

Adaptive Test ◽

Spectrum Model

Download Full-text

Penn Reading Assessment Computerized Adaptive Test

PsycTESTS Dataset ◽

10.1037/t72729-000 ◽

2019 ◽

Author(s):

Mikhal A. Yudien ◽

Tyler M. Moore ◽

Allison M. Port ◽

Kosha Ruparel ◽

Raquel E. Gur ◽

...

Keyword(s):

Reading Assessment ◽

Computerized Adaptive Test ◽

Adaptive Test

Download Full-text

Computerized Adaptive Test to Assess Stress

PsycTESTS Dataset ◽

10.1037/t78303-000 ◽

2020 ◽

Author(s):

Xueyin Tian ◽

Buyun Dai

Keyword(s):

Computerized Adaptive Test ◽

Adaptive Test

Download Full-text

Measurement Precision and Efficiency of Computerized Adaptive Testing for the Activities-specific Balance Confidence Scale in People With Stroke

Physical Therapy ◽

10.1093/ptj/pzab020 ◽

2021 ◽

Author(s):

Bryant A Seamon ◽

Steven A Kautz ◽

Craig A Velozo

Keyword(s):

Rasch Model ◽

Computerized Adaptive Testing ◽

Adaptive Testing ◽

Measurement Precision ◽

Strongly Correlated ◽

Computerized Adaptive Test ◽

Balance Confidence ◽

Adaptive Test ◽

The Rasch Model ◽

Confidence Scale

Abstract Objective Administrative burden often prevents clinical assessment of balance confidence in people with stroke. A computerized adaptive test (CAT) version of the Activities-specific Balance Confidence Scale (ABC CAT) can dramatically reduce this burden. The objective of this study was to test balance confidence measurement precision and efficiency in people with stroke with an ABC CAT. Methods We conducted a retrospective cross-sectional simulation study with data from 406 adults approximately 2-months post-stroke in the Locomotor-Experience Applied Post-Stroke (LEAPS) trial. Item parameters for CAT calibration were estimated with the Rasch model using a random sample of participants (n = 203). Computer simulation was used with response data from remaining 203 participants to evaluate the ABC CAT algorithm under varying stopping criteria. We compared estimated levels of balance confidence from each simulation to actual levels predicted from the Rasch model (Pearson correlations and mean standard error (SE)). Results Results from simulations with number of items as a stopping criterion strongly correlated with actual ABC scores (full item, r = 1, 12-item, r = 0.994; 8-item, r = 0.98; 4-item, r = 0.929). Mean SE increased with decreasing number of items administered (full item, SE = 0.31; 12-item, SE = 0.33; 8-item, SE = 0.38; 4-item, SE = 0.49). A precision-based stopping rule (mean SE = 0.5) also strongly correlated with actual ABC scores (r = .941) and optimized the relationship between number of items administrated with precision (mean number of items 4.37, range [4–9]). Conclusions An ABC CAT can determine accurate and precise measures of balance confidence in people with stroke with as few as 4 items. Individuals with lower balance confidence may require a greater number of items (up to 9) and attributed to the LEAPS trial excluding more functionally impaired persons. Impact Statement Computerized adaptive testing can drastically reduce the ABC’s test administration time while maintaining accuracy and precision. This should greatly enhance clinical utility, facilitating adoption of clinical practice guidelines in stroke rehabilitation. Lay Summary If you have had a stroke, your physical therapist will likely test your balance confidence. A computerized adaptive test version of the ABC scale can accurately identify balance with as few as 4 questions, which takes much less time.

Download Full-text

Using a Response Time–Based Expected A Posteriori Estimator to Control for Differential Speededness in Computerized Adaptive Test

Applied Psychological Measurement ◽

10.1177/01466216211014601 ◽

2021 ◽

pp. 014662162110146

Author(s):

Justin L. Kern ◽

Edison Choe

Keyword(s):

Response Times ◽

Real Data ◽

Joint Estimation ◽

Item Selection ◽

A Posteriori ◽

Computerized Adaptive Test ◽

Maximum Information ◽

Adaptive Test ◽

Estimation And Control ◽

And Control

This study investigates using response times (RTs) with item responses in a computerized adaptive test (CAT) setting to enhance item selection and ability estimation and control for differential speededness. Using van der Linden’s hierarchical framework, an extended procedure for joint estimation of ability and speed parameters for use in CAT is developed following van der Linden; this is called the joint expected a posteriori estimator (J-EAP). It is shown that the J-EAP estimate of ability and speededness outperforms the standard maximum likelihood estimator (MLE) of ability and speededness in terms of correlation, root mean square error, and bias. It is further shown that under the maximum information per time unit item selection method (MICT)—a method which uses estimates for ability and speededness directly—using the J-EAP further reduces average examinee time spent and variability in test times between examinees above the resulting gains of this selection algorithm with the MLE while maintaining estimation efficiency. Simulated test results are further corroborated with test parameters derived from a real data example.

Download Full-text