Test Equating with the Rasch Model to Compare Pre-test and Post-test Measurements

Abstract Objective Administrative burden often prevents clinical assessment of balance confidence in people with stroke. A computerized adaptive test (CAT) version of the Activities-specific Balance Confidence Scale (ABC CAT) can dramatically reduce this burden. The objective of this study was to test balance confidence measurement precision and efficiency in people with stroke with an ABC CAT. Methods We conducted a retrospective cross-sectional simulation study with data from 406 adults approximately 2-months post-stroke in the Locomotor-Experience Applied Post-Stroke (LEAPS) trial. Item parameters for CAT calibration were estimated with the Rasch model using a random sample of participants (n = 203). Computer simulation was used with response data from remaining 203 participants to evaluate the ABC CAT algorithm under varying stopping criteria. We compared estimated levels of balance confidence from each simulation to actual levels predicted from the Rasch model (Pearson correlations and mean standard error (SE)). Results Results from simulations with number of items as a stopping criterion strongly correlated with actual ABC scores (full item, r = 1, 12-item, r = 0.994; 8-item, r = 0.98; 4-item, r = 0.929). Mean SE increased with decreasing number of items administered (full item, SE = 0.31; 12-item, SE = 0.33; 8-item, SE = 0.38; 4-item, SE = 0.49). A precision-based stopping rule (mean SE = 0.5) also strongly correlated with actual ABC scores (r = .941) and optimized the relationship between number of items administrated with precision (mean number of items 4.37, range [4–9]). Conclusions An ABC CAT can determine accurate and precise measures of balance confidence in people with stroke with as few as 4 items. Individuals with lower balance confidence may require a greater number of items (up to 9) and attributed to the LEAPS trial excluding more functionally impaired persons. Impact Statement Computerized adaptive testing can drastically reduce the ABC’s test administration time while maintaining accuracy and precision. This should greatly enhance clinical utility, facilitating adoption of clinical practice guidelines in stroke rehabilitation. Lay Summary If you have had a stroke, your physical therapist will likely test your balance confidence. A computerized adaptive test version of the ABC scale can accurately identify balance with as few as 4 questions, which takes much less time.

Download Full-text

Futuristic Trends and Innovations for Examining the Performance of Course Learning Outcomes Using the Rasch Analytical Model

Electronics ◽

10.3390/electronics10060727 ◽

2021 ◽

Vol 10 (6) ◽

pp. 727

Author(s):

Moustafa M. Nasralla ◽

Basiem Al-Shattarat ◽

Dhafer J. Almakhles ◽

Abdelhakim Abdelhadi ◽

Eman S. Abowardah

Keyword(s):

Learning Outcomes ◽

Analytical Model ◽

Rasch Model ◽

Teaching Techniques ◽

Business Courses ◽

Learning And Teaching ◽

Association Pattern ◽

Engineering Education Research ◽

Academic Year ◽

The Rasch Model

The literature on engineering education research highlights the relevance of evaluating course learning outcomes (CLOs). However, generic and reliable mechanisms for evaluating CLOs remain challenges. The purpose of this project was to accurately assess the efficacy of the learning and teaching techniques through analysing the CLOs’ performance by using an advanced analytical model (i.e., the Rasch model) in the context of engineering and business education. This model produced an association pattern between the students and the overall achieved CLO performance. The sample in this project comprised students who are enrolled in some nominated engineering and business courses over one academic year at Prince Sultan University, Saudi Arabia. This sample considered several types of assessment, such as direct assessments (e.g., quizzes, assignments, projects, and examination) and indirect assessments (e.g., surveys). The current research illustrates that the Rasch model for measurement can categorise grades according to course expectations and standards in a more accurate manner, thus differentiating students by their extent of educational knowledge. The results from this project will guide the educator to track and monitor the CLOs’ performance, which is identified in every course to estimate the students’ knowledge, skills, and competence levels, which will be collected from the predefined sample by the end of each semester. The Rasch measurement model’s proposed approach can adequately assess the learning outcomes.

Download Full-text

Changing the Order of Factors Does Not Change the Product but Does Affect Students’ Answers, Especially Girls’ Answers

Education Sciences ◽

10.3390/educsci11050201 ◽

2021 ◽

Vol 11 (5) ◽

pp. 201

Author(s):

Clelia Cascella ◽

Chiara Giberti ◽

Giorgio Bolondi

Keyword(s):

Experimental Design ◽

Rasch Model ◽

Achievement Tests ◽

Mathematical Content ◽

Probability Sample ◽

Decimal Numbers ◽

Common Items ◽

The Rasch Model ◽

Grade 8

This study is aimed at exploring how different formulations of the same mathematical item may influence students’ answers, and whether or not boys and girls are equally affected by differences in presentation. An experimental design was employed: the same stem-items (i.e., items with the same mathematical content and question intent) were formulated differently and administered to a probability sample of 1647 students (grade 8). All the achievement tests were anchored via a set of common items. Students’ answers, equated and then analysed using the Rasch model, confirmed that different formulations affect students’ performances and thus the psychometric functionality of items, with discernible differences according to gender. In particular, we explored students’ sensitivity to the effect of a typical misconception about multiplication with decimal numbers (often called “multiplication makes bigger”) and tested the hypothesis that girls are more prone than boys to be negatively affected by misconception.

Download Full-text

An item analysis according to the Rasch model of the German 12-item WHO Disability Assessment Schedule (WHODAS 2.0)

Quality of Life Research ◽

10.1007/s11136-021-02872-8 ◽

2021 ◽

Author(s):

Lusine Vaganian ◽

Sonja Bussmann ◽

Maren Boecker ◽

Michael Kusch ◽

Hildegard Labouvie ◽

...

Keyword(s):

Cancer Patients ◽

Rasch Model ◽

Item Analysis ◽

Classical Test Theory ◽

Health Condition ◽

Test Theory ◽

World Health ◽

Good Reliability ◽

Whodas 2.0 ◽

The Rasch Model

Abstract Purpose The World Health Organization Disability Assessent Schedule 2.0 (WHODAS 2.0) assesses disability in individuals irrespective of their health condition. Previous studies validated the usefulness of the WHODAS 2.0 using classical test theory. This study is the first investigating the psychometric properties of the 12-items WHODAS 2.0 in patients with cancer using item analysis according to the Rasch model. Methods In total, 350 cancer patients participated in the study. Rasch analysis of the 12-items version of the WHODAS 2.0 was conducted and included testing unidimensionality, local independence, and testing for differential item functioning (DIF) with regard to age, gender, type of cancer, presence of metastases, psycho-oncological support, and duration of disease. Results After accounting for local dependence, which was mainly found across items of the same WHODAS domain, satisfactory overall fit to the Rasch model was established (χ2 = 36.14, p = 0.07) with good reliability (PSI = 0.82) and unidimensionality of the scale. DIF was found for gender (testlet ‘Life activities’) and age (testlet ‘Getting around/Self-care’), but the size of DIF was not substantial. Conclusion Overall, the analysis results according to the Rasch model support the use of the WHODAS 2.0 12-item version as a measure of disability in cancer patients.

Download Full-text

Statistical and practical significance of the Likelihood Ratio Test of the linear logistic test model versus the Rasch model

Educational Research and Evaluation ◽

10.1080/13803611.2011.630522 ◽

2011 ◽

Vol 17 (5) ◽

pp. 335-350 ◽

Cited By ~ 1

Author(s):

Rainer W. Alexandrowicz

Keyword(s):

Likelihood Ratio ◽

Likelihood Ratio Test ◽

Rasch Model ◽

Practical Significance ◽

Ratio Test ◽

Test Model ◽

Linear Logistic Test Model ◽

Model Versus ◽

The Rasch Model

Download Full-text

Controlling response dependence in the measurement of change using the Rasch model

Statistical Methods in Medical Research ◽

10.1177/0962280217710834 ◽

2017 ◽

Vol 27 (12) ◽

pp. 3709-3725 ◽

Cited By ~ 1

Author(s):

David Andrich

Keyword(s):

Rasch Model ◽

Real Data ◽

Repeated Measurements ◽

Measurement Of Change ◽

Location Parameters ◽

Response Dependence ◽

The Individual ◽

First Time ◽

The Rasch Model ◽

Research And Policy

The advantages of using person location estimates from the Rasch model over raw scores for the measurement of change using a common test include the linearization of scores and the automatic handling of statistical properties of repeated measurements. However, the application of the model requires that the responses to the items are statistically independent in the sense that the specific responses to the items on the first time of testing do not affect the responses at a second time. This requirement implies that the responses to the items at both times of assessment are governed only by the invariant location parameters of the items at the two times of testing and the location parameters of each person each time. A specific form of dependence that is pertinent when the same items are used is when the observed response to an item at the second time of testing is affected by the response to the same item at the first time, a form of dependence which has been referred to as response dependence. This paper presents the logic of applying the Rasch model to quantify, control and remove the effect of response dependence in the measurement of change when the same items are used on two occasions. The logic is illustrated with four sets of simulation studies with dichotomous items and with a small example of real data. It is shown that the presence of response dependence can reduce the evidence of change, a reduction which may impact interpretations at the individual, research, and policy levels.

Download Full-text