scholarly journals The direction of the response scale matters – accounting for the unit of measurement

2019 ◽  
Vol 53 (5) ◽  
pp. 871-891
Author(s):  
Thomas Salzberger ◽  
Monika Koller
Keyword(s):  
Rasch Model ◽  
Measurement Theory ◽  
Measurement Precision ◽  
Spatial Proximity ◽  
Validity And Reliability ◽  
Measurement Instruments ◽  
Response Scale ◽  
Content Type ◽  
Unit Of Measurement ◽  
The Rasch Model

Purpose Psychometric analyses of self-administered questionnaire data tend to focus on items and instruments as a whole. The purpose of this paper is to investigate the functioning of the response scale and its impact on measurement precision. In terms of the response scale direction, existing evidence is mixed and inconclusive. Design/methodology/approach Three experiments are conducted to examine the functioning of response scales of different direction, ranging from agree to disagree versus from disagree to agree. The response scale direction effect is exemplified by two different latent constructs by applying the Rasch model for measurement. Findings The agree-to-disagree format generally performs better than the disagree-to-agree variant with spatial proximity between the statement and the agree-pole of the scale appearing to drive the effect. The difference is essentially related to the unit of measurement. Research limitations/implications A careful investigation of the functioning of the response scale should be part of every psychometric assessment. The framework of Rasch measurement theory offers unique opportunities in this regard. Practical implications Besides content, validity and reliability, academics and practitioners utilising published measurement instruments are advised to consider any evidence on the response scale functioning that is available. Originality/value The study exemplifies the application of the Rasch model to assess measurement precision as a function of the design of the response scale. The methodology raises the awareness for the unit of measurement, which typically remains hidden.

scholarly journals Measurement Precision and Efficiency of Computerized Adaptive Testing for the Activities-specific Balance Confidence Scale in People With Stroke

2021 ◽  
Author(s):  
Bryant A Seamon ◽  
Steven A Kautz ◽  
Craig A Velozo
Keyword(s):  
Rasch Model ◽  
Computerized Adaptive Testing ◽  
Adaptive Testing ◽  
Measurement Precision ◽  
Strongly Correlated ◽  
Computerized Adaptive Test ◽  
Balance Confidence ◽  
Adaptive Test ◽  
The Rasch Model ◽  
Confidence Scale

Abstract Objective Administrative burden often prevents clinical assessment of balance confidence in people with stroke. A computerized adaptive test (CAT) version of the Activities-specific Balance Confidence Scale (ABC CAT) can dramatically reduce this burden. The objective of this study was to test balance confidence measurement precision and efficiency in people with stroke with an ABC CAT. Methods We conducted a retrospective cross-sectional simulation study with data from 406 adults approximately 2-months post-stroke in the Locomotor-Experience Applied Post-Stroke (LEAPS) trial. Item parameters for CAT calibration were estimated with the Rasch model using a random sample of participants (n = 203). Computer simulation was used with response data from remaining 203 participants to evaluate the ABC CAT algorithm under varying stopping criteria. We compared estimated levels of balance confidence from each simulation to actual levels predicted from the Rasch model (Pearson correlations and mean standard error (SE)). Results Results from simulations with number of items as a stopping criterion strongly correlated with actual ABC scores (full item, r = 1, 12-item, r = 0.994; 8-item, r = 0.98; 4-item, r = 0.929). Mean SE increased with decreasing number of items administered (full item, SE = 0.31; 12-item, SE = 0.33; 8-item, SE = 0.38; 4-item, SE = 0.49). A precision-based stopping rule (mean SE = 0.5) also strongly correlated with actual ABC scores (r = .941) and optimized the relationship between number of items administrated with precision (mean number of items 4.37, range [4–9]). Conclusions An ABC CAT can determine accurate and precise measures of balance confidence in people with stroke with as few as 4 items. Individuals with lower balance confidence may require a greater number of items (up to 9) and attributed to the LEAPS trial excluding more functionally impaired persons. Impact Statement Computerized adaptive testing can drastically reduce the ABC’s test administration time while maintaining accuracy and precision. This should greatly enhance clinical utility, facilitating adoption of clinical practice guidelines in stroke rehabilitation. Lay Summary If you have had a stroke, your physical therapist will likely test your balance confidence. A computerized adaptive test version of the ABC scale can accurately identify balance with as few as 4 questions, which takes much less time.

scholarly journals Evaluation of Hajj Instrument (HAJI) Psychometric Properties Using Rasch Measurement

2016 ◽  
Vol 12 (8) ◽  
pp. 212
Author(s):  
Muhammad Iqbal Tariq Idris ◽  
Abdul Hafidz Omar ◽  
Dayang Hjh Tiawa Awang Hj Hamid ◽  
Fahmi Bahri Sulaiman
Keyword(s):  
Physical Activity ◽  
Psychometric Properties ◽  
Healthy Eating ◽  
Rasch Model ◽  
Rasch Measurement ◽  
Mental Toughness ◽  
Validity And Reliability ◽  
Item Fit ◽  
The Rasch Model ◽  
Physical Care

<p>Hajj Instrument (HAJI) was developed to determine hajj pilgrim’s wellness. This study used Rasch measurement to evaluate the psychometric properties including validity and reliability of the HAJI. The respondents involved in this study were 300 comprised of Malaysian hajj pilgrims. HAJI consists of eight constructs namely physical care, physical activity, healthy eating, knowledge, mental toughness, intrapersonal, interpersonal and relationship with Creator and natures. Validity of each construct and content was determined through dimensionality, item fit and item polarity while the reliability was achieved by administered person and item separation. The results showed that the reliability for both item and person were 0.99 and 0.96 respectively. Besides, there were no items need to be dropped based on PTMEA CORR and INFIT MNSQ results. The study revealed that the items of HAJI fit the Rasch model as well as able to measure hajj pilgrim’s wellness. </p>

scholarly journals THE USE OF THE RASCH MODEL IN ANALYSING THE VALIDITY AND RELIABILITY OF STUDENTS’ COPING STRATEGY SCALE

2021 ◽  
Vol 7 (2) ◽  
pp. 110-119
Author(s):  
Maya Oktaviani ◽  
Uswatun Hasanah ◽  
Muhammad Faesal ◽  
Elmanora ◽  
Vania Zulfa
Keyword(s):  
Coping Strategy ◽  
Rasch Model ◽  
Content Validity ◽  
Three Dimensions ◽  
Validity And Reliability ◽  
Empirical Validity ◽  
Validity Analysis ◽  
Validity Coefficient ◽  
The Rasch Model ◽  
Model Approach

This research is part of the standardization of the Students' Coping Strategy Scale (SCSS). This study aims to reveal the content validity, empirical validity, and reliability of the SCSS. The non-test instrument consisting of 42 statements from three dimensions: problem-focused coping, emotion-focused coping, and seeking social support. Content validity analysis was carried out by experts to obtain Gregory's content validity coefficient, while the empirical validity and reliability analysis by empirically testing the instrument on 211 respondents and analyzed using the Rasch model approach. The results show that Gregory's content validity coefficient is 0.714, so it needs improvement on some items that are considered unsuitable. Empirical validity analysis using WINSTEPS software shows that three items do not fit the model. The reliability coefficient of the SCSS is 0.99. Thus, the remaining 39 items are appropriate to be used to measure students' coping strategies.

scholarly journals A Competency-Based Approach to Pass/Fail Decisions: An Observational Study

2021 ◽  
Author(s):  
Nazdar E. Alkhateeb ◽  
Ali Al-Dabbagh ◽  
Yaseen Mohammed ◽  
Mohammed Ibrahim
Keyword(s):  
Observational Study ◽  
Rasch Model ◽  
Standard Setting ◽  
Careful Consideration ◽  
Validity And Reliability ◽  
High Stakes ◽  
High Stakes Assessment ◽  
Sufficient Degree ◽  
Competency Based ◽  
The Rasch Model

Any high-stakes assessment that leads to an important decision requires careful consideration in determining whether a student passes or fails. Despite the implementation of many standard-setting methods in clinical examinations, concerns remain about the reliability of pass/fail decisions in high stakes assessment, especially clinical assessment. This observational study proposes a defensible pass/fail decision based on the number of failed competencies. In the study conducted in Erbil, Iraq, in June 2018, results were obtained for 150 medical students on their final objective structured clinical examination. Cutoff scores and pass/fail decisions were calculated using the modified Angoff, borderline, borderline-regression, and holistic methods. The results were compared with each other and with a new competency method using Cohen’s kappa. Rasch analysis was used to compare the consistency of competency data with Rasch model estimates. The competency method resulted in 40 (26.7%) students failing, compared with 76 (50.6%), 37 (24.6%), 35 (23.3%), and 13 (8%) for the modified Angoff, borderline, borderline regression, and holistic methods, respectively. The competency method demonstrated a sufficient degree of fit to the Rasch model (mean outfit and infit statistics of 0.961 and 0.960, respectively). In conclusion, the competency method was more stringent in determining pass/fail, compared with other standard-setting methods, except for the modified Angoff method. The fit of competency data to the Rasch model provides evidence for the validity and reliability of pass/fail decisions.

scholarly journals Rasch analysis of the living with chronic illness scale in Parkinson’s disease

BMC Neurology ◽  
2020 ◽  
Vol 20 (1) ◽  
Author(s):  
Leire Ambrosio ◽  
Carmen Rodriguez-Blazquez ◽  
Alba Ayala ◽  
Maria João Forjaz
Keyword(s):  
Parkinson’S Disease ◽  
Parkinson's Disease ◽  
Chronic Illness ◽  
Latin American ◽  
Rasch Model ◽  
Rasch Analysis ◽  
Response Scale ◽  
Clinical Tool ◽  
Cross Sectional ◽  
The Rasch Model

Abstract Background Neurologists play an essential role in facilitating the patient’s process of living with Parkinson’s disease (PD). The Living with Chronic Illness Scale-PD (LW-CI-PD) is a unique available clinical tool that evaluates how the patient is living with PD. The objective of the study was to analyse the LW-CI-PD properties according to the Rasch model. Methods An open, international, cross-sectional study was carried out in 324 patients with Parkinson’s disease from four Latin American countries and Spain. Psychometric properties of the LW-CI-PD were tested using Rasch analysis: fit to the Rasch model, item local independency, unidimensionality, reliability, and differential item functioning by age and gender. Results Original LW-CI-PD do not fit Rasch model. Modifications emerged included simplifying the response scale and deleting misfit items, the dimensions Acceptance, Coping and Integration showed a satisfactory fit to the Rasch model, with reliability indices greater than 0.70. The dimensions Self-management and Adjustment to the disease did not reach fit to the Rasch model. Conclusion Suggestions for improving the LW-CI-PD include a multidimensional and shorter scale with 12 items grouped in three subscales with a simpler response scheme. The final LW-CI-PD Scale version is a reliable scale, with good internal construct validity, that provides Rasch transformed results on linear metric scale.

Validity and Reliability of an Arabic Modified Version of the Otis-Lennon Mental Ability Test

1997 ◽  
Vol 80 (3) ◽  
pp. 947-954 ◽  
Author(s):  
Abdelfattah I. Elkorashy
Keyword(s):  
Secondary School ◽  
Rasch Model ◽  
Classical Model ◽  
Mental Ability ◽  
Validity And Reliability ◽  
Ability Test ◽  
Arab Culture ◽  
The Rasch Model ◽  
Adequate Evidence

An Arabic modified version of the Otis-Lennon Mental Ability Test was administered to 400 boys and girls in secondary school. Their ages ranged from 15 to 23 years ( M = 16.2 yr., SD = 1.4). The validity and reliability of the test were estimated according to the classical model and the Rasch model and adequate evidence of validity and reliability were obtained, supporting the efficiency and the suitability of the test as a measure of mental ability in the Arab culture.

scholarly journals Rasch analysis of the Living with Chronic Illness Scale in Parkinson’s disease

2020 ◽  
Author(s):  
Leire Ambrosio ◽  
Carmen Rodriguez-Blazquez ◽  
Alba Ayala ◽  
Maria João Forjaz
Keyword(s):  
Parkinson’S Disease ◽  
Parkinson's Disease ◽  
Chronic Illness ◽  
Latin American ◽  
Rasch Model ◽  
Rasch Analysis ◽  
Response Scale ◽  
Clinical Tool ◽  
Cross Sectional ◽  
The Rasch Model

Abstract Background Neurologists play an essential role in facilitating the patient’s process of living with Parkinson’s disease (PD). The Living with Chronic Illness Scale-PD (LW-CI-PD) is a unique available clinical tool that evaluates how the patient is living with PD. The objective of the study was to analyse the LW-CI-PD properties according to the Rasch model.Methods An open, international, cross-sectional study was carried out in 324 patients with Parkinson’s disease from four Latin American countries and Spain. Psychometric properties of the LW-CI-PD were tested using Rasch analysis: fit to the Rasch model, item local independency, unidimensionality, reliability, and differential item functioning by age and gender.Results Original LW-CI-PD do not fit Rasch model. Modifications emerged included simplifying the response scale and deleting misfit items, the dimensions Acceptance, Coping and Integration showed a satisfactory fit to the Rasch model, with reliability indices greater than 0.70. The dimensions Self-management and Adjustment to the disease did not reach fit to the Rasch model.Conclusion Suggestions for improving the LW-CI-PD include a multidimensional and shorter scale with 12 items grouped in three subscales with a simpler response scheme. The final LW-CI-PD Scale version is a reliable scale, with good internal construct validity, that provides Rasch transformed results on linear metric scale.

scholarly journals Evaluating Construct Validity and Reliability of Intention to Transfer Training Conduct Instrument Using Rasch Model Analysis

2021 ◽  
Vol 29 (2) ◽  
Author(s):  
Shaliza Shafie ◽  
Faizah Abd Majid ◽  
Teoh Sian Hoon ◽  
Siti Maftuhah Damio
Keyword(s):  
Construct Validity ◽  
Rasch Model ◽  
Mean Square ◽  
Validity And Reliability ◽  
Self Assessment ◽  
New Instrument ◽  
Transfer Training ◽  
Point Measure ◽  
The Impact ◽  
The Rasch Model

The impact of the Industry Revolution 4.0 (IR4.0) in the workplace requires organisations to ensure clerical employees can effectively transfer their newly acquired knowledge and skills learned in training back into the workplace. Hence, an instrument is required to identify factors influencing the intention to transfer training conduct amongst clerical employees. Thus, this paper presents the evaluation of construct validity and reliability of the new instrument to confirm its objectivity and clarity in measuring the constructs under study as intended. This four-point Likert-type scale instrument consists of 72 self-assessment items that represent 12 constructs. The Rasch Model was then employed to analyse the construct validity and reliability by evaluating the suitability of items in the respective constructs on the instrument. The item and person reliability and strata indices, point-measure correlation, and outfit mean square values were examined. The analysis found that three constructs in the item and person reliability index and eight constructs in the item and person reliability strata index were low but adequate and met the Rasch Model measurement acceptable level. Meanwhile, point-measure correlation values for all constructs fulfilled the criteria. Finally, the outfit mean square values established that 65 items in the constructs were found to be fit, whereas seven items were misfits which require improvement. Subsequently, the seven misfit items were improved as the item and person reliability values could be increased, thus the items were retained. Thereafter, the instrument was ready to be used for data collection in the actual study.

Sign in / Sign up

Export Citation Format

Share Document