Computerized Adaptive Testing for Sleep Disorders: Development of An Item Bank and Validation in A Simulated Study

2020 ◽  
Author(s):  
Menghua She ◽  
Yaling Li ◽  
Dongbo Tu ◽  
Yan Cai
Keyword(s):  
Sleep Disorders ◽  
Computerized Adaptive Testing ◽  
Assessment Tool ◽  
Adaptive Testing ◽  
Item Bank ◽  
Accurate Assessment ◽  
Item Pool ◽  
Psychometric Characteristics ◽  
Predictive Utility ◽  
Item Fit

Abstract Background: As more and more people suffer from sleep disorders, developing an efficient, cheap and accurate assessment tool for screening sleep disorders is becoming more urgent. This study developed a computerized adaptive testing for sleep disorders (CAT-SD). Methods: A large sample of 1,304 participants was recruited to construct the item pool of CAT-SD and to investigate the psychometric characteristics of CAT-SD. More specifically, firstly the analyses of unidimensionality, model fit, item fit, item discrimination parameter and differential item functioning (DIF) were conducted to construct a final item pool which meets the requirements of item response theory (IRT) measurement. In addition, a simulated CAT study with real response data of participants was performed to investigate the psychometric characteristics of CAT-SD, including reliability, validity and predictive utility (sensitivity and specificity). Results: The final unidimensional item bank of the CAT-SD not only had good item fit, high discrimination and no DIF; Moreover, it had acceptable reliability, validity and predictive utility. Conclusions: The CAT-SD could be used as an effective and accurate assessment tool for measuring individuals' severity of the sleep disorders and offers a bran-new perspective for screening of sleep disorders with psychological scales.

Computerized Adaptive Testing for Sleep Disorders

2021 ◽  
pp. 1-12
Author(s):  
Menghua She ◽  
Yaling Li ◽  
Dongbo Tu ◽  
Yan Cai
Keyword(s):  
Sleep Disorders ◽  
Sensitivity And Specificity ◽  
Computerized Adaptive Testing ◽  
Assessment Tool ◽  
Adaptive Testing ◽  
Item Bank ◽  
Accurate Assessment ◽  
Psychometric Characteristics ◽  
Predictive Utility ◽  
Item Fit

Abstract. Background: As more and more people suffer from sleep disorders, the need to develop an efficient, inexpensive, and accurate assessment tool for screening sleep disorders has become more urgent. Aim: The aim of the current study was to develop a system allowing computerized adaptive testing for sleep disorders (CAT-SD). Methods: A large sample ( N = 1,304) was recruited to construct an item bank for CAT-SD and to investigate the psychometric characteristics of CAT-SD. First, analyses of unidimensionality, model fit, item fit, item discrimination parameters, and differential item functioning (DIF) were conducted to construct a final item pool to meet the requirements of item response theory measurement. Then, a simulated CAT study with real data was performed to investigate the psychometric characteristics of CAT-SD, including the reliability, validity, and predictive utility (sensitivity and specificity). Results: The final unidimensional item bank of the CAT-SD had good item fit, high discrimination, and no DIF. Moreover, it had acceptable reliability, validity, and predictive utility. Limitations: Non-statistical assembly constraints, execution environment, construction of item bank, criterion-related validity, and predictive utility (sensitivity and specificity) of CAT-SD, and sample representativeness are discussed. Conclusions: The CAT-SD could be used as an effective and accurate assessment tool for measuring the sleep disorders in individuals and offers a novel approach to the screening of sleep disorders utilizing psychological scales.

Evaluating a Computerized Adaptive Testing Version of a Cognitive Ability Test Using a Simulation Study

2021 ◽  
pp. 073428292110277
Author(s):  
Ioannis Tsaousis ◽  
Georgios D. Sideridis ◽  
Hannan M. AlGhamdi
Keyword(s):  
Cognitive Ability ◽  
Simulation Study ◽  
Computerized Adaptive Testing ◽  
Adaptive Testing ◽  
Estimation Methods ◽  
Item Pool ◽  
Sequential Approach ◽  
Ability Test ◽  
Promising Alternative ◽  
Item Exposure

This study evaluated the psychometric quality of a computerized adaptive testing (CAT) version of the general cognitive ability test (GCAT), using a simulation study protocol put forth by Han, K. T. (2018a). For the needs of the analysis, three different sets of items were generated, providing an item pool of 165 items. Before evaluating the efficiency of the GCAT, all items in the final item pool were linked (equated), following a sequential approach. Data were generated using a standard normal for 10,000 virtual individuals ( M = 0 and SD = 1). Using the measure’s 165-item bank, the ability value (θ) for each participant was estimated. maximum Fisher information (MFI) and maximum likelihood estimation with fences (MLEF) were used as item selection and score estimation methods, respectively. For item exposure control, the fade away method (FAM) was preferred. The termination criterion involved a minimum SE ≤ 0.33. The study revealed that the average number of items administered for 10,000 participants was 15. Moreover, the precision level in estimating the participant’s ability score was very high, as demonstrated by the CBIAS, CMAE, and CRMSE). It is concluded that the CAT version of the test is a promising alternative to administering the corresponding full-length measure since it reduces the number of administered items, prevents high rates of item exposure, and provides accurate scores with minimum measurement error.

scholarly journals Calibration of the food parenting practice (FPP) item bank: tools for improving the measurement of food parenting practices of parents of 5–12-year-old children

2020 ◽  
Vol 17 (1) ◽  
Author(s):  
Louise C. Mâsse ◽  
Teresia M. O’Connor ◽  
Yingyi Lin ◽  
Sheryl O. Hughes ◽  
Claire N. Tugault-Lafleur ◽  
...  
Keyword(s):  
Conceptual Framework ◽  
Parenting Practices ◽  
Computerized Adaptive Testing ◽  
Short Form ◽  
Adaptive Testing ◽  
Item Bank ◽  
Short Version ◽  
Parenting Practice ◽  
Practice Item ◽  
Improve Measurement

Abstract Purpose There has been a call to improve measurement rigour and standardization of food parenting practices measures, as well as aligning the measurement of food parenting practices with the parenting literature. Drawing from an expert-informed conceptual framework assessing three key domains of food parenting practices (autonomy promotion, control, and structure), this study combined factor analytic methods with Item Response Modeling (IRM) methodology to psychometrically validate responses to the Food Parenting Practice item bank. Methods A sample of 799 Canadian parents of 5–12-year-old children completed the Food Parenting Practice item bank (129 items measuring 17 constructs). The factorial structure of the responses to the item bank was assessed with confirmatory factor analysis (CFA), confirmatory bi-factor item analysis, and IRM. Following these analyses, differential Item Functioning (DIF) and Differential Response Functioning (DRF) analyses were then used to test invariance properties by parents’ sex, income and ethnicity. Finally, the efficiency of the item bank was examined using computerized adaptive testing simulations to identify the items to include in a short form. Results Overall, the expert-informed conceptual framework was predominantly supported by the CFA as it retained the same 17 constructs included in the conceptual framework with the exception of the access/availability and permissive constructs which were respectively renamed covert control and accommodating the child to better reflect the content of the final solution. The bi-factor item analyses and IRM analyses revealed that the solution could be simplified to 11 unidimensional constructs and the full item bank included 86-items (empirical reliability from 0.78 to 0.96, except for 1 construct) and the short form had 48 items. Conclusion Overall the food parenting practice item bank has excellent psychometric properties. The item bank includes an expanded version and short version to meet various study needs. This study provides more efficient tools for assessing how food parenting practices influence child dietary behaviours. Next steps are to use the IRM calibrated item bank and draw on computerized adaptive testing methodology to administer the item bank and provide flexibility in item selection.

scholarly journals Development of Computerized Adaptive Testing for Emotion Regulation

2020 ◽  
Vol 11 ◽  
Author(s):  
Lingling Xu ◽  
Ruyi Jin ◽  
Feifei Huang ◽  
Yanhui Zhou ◽  
Zonglong Li ◽  
...  
Keyword(s):  
Emotion Regulation ◽  
Model Comparison ◽  
Computerized Adaptive Testing ◽  
Item Analysis ◽  
Well Being ◽  
Adaptive Testing ◽  
Vital Role ◽  
Test Items ◽  
Item Fit ◽  
Item Functioning

Emotion regulation (ER) plays a vital role in individuals’ well-being and successful functioning. In this study, we attempted to develop a computerized adaptive testing (CAT) to efficiently evaluate ER, namely the CAT-ER. The initial CAT-ER item bank comprised 154 items from six commonly used ER scales, which were completed by 887 participants recruited in China. We conducted unidimensionality testing, item response theory (IRT) model comparison and selection, and IRT item analysis including local independence, item fit, differential item functioning, and item discrimination. Sixty-three items with good psychometric properties were retained in the final CAT-ER. Then, two CAT simulation studies were implemented to assess the CAT-ER, which revealed that the CAT-ER developed in this study performed reasonably well, considering that it greatly lessened the test items and time without losing measurement accuracy.

scholarly journals A computerized adaptive testing advancing the measurement of subjective well-being

2019 ◽  
Vol 13 ◽  
Author(s):  
Yifang Wu ◽  
Yan Cai ◽  
Dongbo Tu
Keyword(s):  
Computerized Adaptive Testing ◽  
Simulated Data ◽  
Real Data ◽  
Well Being ◽  
Adaptive Testing ◽  
Test Time ◽  
Subjective Well Being ◽  
Test Items ◽  
Item Fit ◽  
Item Functioning

AbstractThis article aimed at developing an adaptive version of the subjective well-being (SWB) scale to measure a comprehensive concept of SWB among Chinese university students. Item response theory was employed to formulate the item bank of the SWB scale and computerized adaptive testing (CAT) for SWB (CAT-SWB), based on several commonly used SWB scales, after unidimensionality testing, model selection, local dependence testing, parameter estimation, item fit test and differential item functioning (DIF) analysis were performed. Finally, two CAT simulations using simulated-data and real-data were carried out to verify and evaluate the CAT-SWB. Results indicated that the proposed CAT-SWB had an excellent performance in that it largely reduces the number of test items and the length of test time without losing measurement precision.

The Implementation of a Server-based Computerized Adaptive Testing on Mobile Devices (CAT-MD)

2010 ◽  
pp. 316-326
Author(s):  
Evangelos Triantafillou ◽  
Elissavet Georgiadou ◽  
Anastasios A. Economides
Keyword(s):  
Mobile Devices ◽  
Formative Evaluation ◽  
Computerized Adaptive Testing ◽  
Assessment Tool ◽  
Adaptive Testing ◽  
Educational Process ◽  
Second Phase ◽  
Computer Lab ◽  
E Learning ◽  
Two Phases

The introduction of mobile devices into the learning pedagogy can compliment e-learning and e-testing by creating an additional channel of assessment with mobile devices. The current study describes the design issues that were considered for the development and the implementation of a CAT on mobile devices, the CAT-MD (Computerized Adaptive Testing on Mobile Devices). The system was implemented in two phases, where initially, a standalone prototype application was developed in order to implement the architecture of the CAT-MD. After a formative evaluation, the second phase took place, where a server-based application was developed in order to add new functionalities to the system so that CAT-MD can be an effective and efficient assessment tool that can add value to the educational process. The mobility of the CAT-MD eliminates the need for a specialized computer lab, as it can be used anywhere, including a traditional classroom.

scholarly journals Optimal Number of Strata for the Stratified Methods in Computerized Adaptive Testing

2014 ◽  
Vol 17 ◽  
Author(s):  
Juan Ramón Barrada ◽  
Francisco José Abad ◽  
Julio Olea
Keyword(s):  
Computerized Adaptive Testing ◽  
Maximum Level ◽  
Adaptive Testing ◽  
Optimal Number ◽  
Item Bank ◽  
Test Security ◽  
Constant Measurement ◽  
The Mean ◽  
Share Information ◽  
Do So

AbstractTest security can be a major problem in computerized adaptive testing, as examinees can share information about the items they receive. Of the different item selection rules proposed to alleviate this risk, stratified methods are among those that have received most attention. In these methods, only low discriminative items can be presented at the beginning of the test and the mean information of the items increases as the test goes on. To do so, the item bank must be divided into several strata according to the information of the items. To date, there is no clear guidance about the optimal number of strata into which the item bank should be split. In this study, we will simulate conditions with different numbers of strata, from 1 (no stratification) to a number of strata equal to test length (maximum level of stratification) while manipulating the maximum exposure rate that no item should surpass (rmax) in its whole domain. In this way, we can plot the relation between test security and accuracy, making it possible to determine the number of strata that leads to better security while holding constant measurement accuracy. Our data indicates that the best option is to stratify into as many strata as possible.

Sign in / Sign up

Export Citation Format

Share Document