scholarly journals A competency-based approach to pass/fail decisions in an objective structured clinical examination: An observational study

2020 ◽  
Author(s):  
Nazdar Ezzaddin Alkhateeb ◽  
Ali Al-Dabbagh ◽  
Yaseen Omar Mohammed ◽  
Mohammed Ibrahim
Keyword(s):  
Observational Study ◽  
Clinical Examination ◽  
Rasch Model ◽  
Careful Consideration ◽  
Objective Structured Clinical Examination ◽  
Validity And Reliability ◽  
High Stakes ◽  
Sufficient Degree ◽  
Competency Based ◽  
The Rasch Model

BackgroundAny high-stakes assessment that leads to an important decision requires careful consideration in determining whether a student passes or fails. This observational study conducted in Erbil, Iraq, in June 2018 proposes a defensible pass/fail decision based on the number of failed competencies.MethodsResults were obtained for 150 medical students on their final objective structured clinical examination. Cutoff scores and pass/fail decisions were calculated using the modified Angoff, borderline, borderline-regression and holistic methods. The results were compared with each other and with a new competency method using Cohen’s kappa. Rasch analysis was used to compare the consistency of competency data with Rasch model estimates.ResultsThe competency method resulted in 40 (26.7%) students failing, compared with 76 (50.6%), 37 (24.6%), 35 (23.3%) and 13 (8%) for the modified Angoff, borderline, borderline regression and holistic methods, respectively. The competency method demonstrated a sufficient degree of fit to the Rasch model (mean outfit and infit statistics of 0.961 and 0.960, respectively).Conclusionsthe competency method was more stringent in determining pass/fail, compared with other standard-setting methods, except for the modified Angoff method. The fit of competency data to the Rasch model provides evidence for the validity and reliability of pass/fail decisions.

scholarly journals A Competency-Based Approach to Pass/Fail Decisions: An Observational Study

2021 ◽  
Author(s):  
Nazdar E. Alkhateeb ◽  
Ali Al-Dabbagh ◽  
Yaseen Mohammed ◽  
Mohammed Ibrahim
Keyword(s):  
Observational Study ◽  
Rasch Model ◽  
Standard Setting ◽  
Careful Consideration ◽  
Validity And Reliability ◽  
High Stakes ◽  
High Stakes Assessment ◽  
Sufficient Degree ◽  
Competency Based ◽  
The Rasch Model

Any high-stakes assessment that leads to an important decision requires careful consideration in determining whether a student passes or fails. Despite the implementation of many standard-setting methods in clinical examinations, concerns remain about the reliability of pass/fail decisions in high stakes assessment, especially clinical assessment. This observational study proposes a defensible pass/fail decision based on the number of failed competencies. In the study conducted in Erbil, Iraq, in June 2018, results were obtained for 150 medical students on their final objective structured clinical examination. Cutoff scores and pass/fail decisions were calculated using the modified Angoff, borderline, borderline-regression, and holistic methods. The results were compared with each other and with a new competency method using Cohen’s kappa. Rasch analysis was used to compare the consistency of competency data with Rasch model estimates. The competency method resulted in 40 (26.7%) students failing, compared with 76 (50.6%), 37 (24.6%), 35 (23.3%), and 13 (8%) for the modified Angoff, borderline, borderline regression, and holistic methods, respectively. The competency method demonstrated a sufficient degree of fit to the Rasch model (mean outfit and infit statistics of 0.961 and 0.960, respectively). In conclusion, the competency method was more stringent in determining pass/fail, compared with other standard-setting methods, except for the modified Angoff method. The fit of competency data to the Rasch model provides evidence for the validity and reliability of pass/fail decisions.

scholarly journals Evaluation of Hajj Instrument (HAJI) Psychometric Properties Using Rasch Measurement

2016 ◽  
Vol 12 (8) ◽  
pp. 212
Author(s):  
Muhammad Iqbal Tariq Idris ◽  
Abdul Hafidz Omar ◽  
Dayang Hjh Tiawa Awang Hj Hamid ◽  
Fahmi Bahri Sulaiman
Keyword(s):  
Physical Activity ◽  
Psychometric Properties ◽  
Healthy Eating ◽  
Rasch Model ◽  
Rasch Measurement ◽  
Mental Toughness ◽  
Validity And Reliability ◽  
Item Fit ◽  
The Rasch Model ◽  
Physical Care

<p>Hajj Instrument (HAJI) was developed to determine hajj pilgrim’s wellness. This study used Rasch measurement to evaluate the psychometric properties including validity and reliability of the HAJI. The respondents involved in this study were 300 comprised of Malaysian hajj pilgrims. HAJI consists of eight constructs namely physical care, physical activity, healthy eating, knowledge, mental toughness, intrapersonal, interpersonal and relationship with Creator and natures. Validity of each construct and content was determined through dimensionality, item fit and item polarity while the reliability was achieved by administered person and item separation. The results showed that the reliability for both item and person were 0.99 and 0.96 respectively. Besides, there were no items need to be dropped based on PTMEA CORR and INFIT MNSQ results. The study revealed that the items of HAJI fit the Rasch model as well as able to measure hajj pilgrim’s wellness. </p>

scholarly journals THE USE OF THE RASCH MODEL IN ANALYSING THE VALIDITY AND RELIABILITY OF STUDENTS’ COPING STRATEGY SCALE

2021 ◽  
Vol 7 (2) ◽  
pp. 110-119
Author(s):  
Maya Oktaviani ◽  
Uswatun Hasanah ◽  
Muhammad Faesal ◽  
Elmanora ◽  
Vania Zulfa
Keyword(s):  
Coping Strategy ◽  
Rasch Model ◽  
Content Validity ◽  
Three Dimensions ◽  
Validity And Reliability ◽  
Empirical Validity ◽  
Validity Analysis ◽  
Validity Coefficient ◽  
The Rasch Model ◽  
Model Approach

This research is part of the standardization of the Students' Coping Strategy Scale (SCSS). This study aims to reveal the content validity, empirical validity, and reliability of the SCSS. The non-test instrument consisting of 42 statements from three dimensions: problem-focused coping, emotion-focused coping, and seeking social support. Content validity analysis was carried out by experts to obtain Gregory's content validity coefficient, while the empirical validity and reliability analysis by empirically testing the instrument on 211 respondents and analyzed using the Rasch model approach. The results show that Gregory's content validity coefficient is 0.714, so it needs improvement on some items that are considered unsuitable. Empirical validity analysis using WINSTEPS software shows that three items do not fit the model. The reliability coefficient of the SCSS is 0.99. Thus, the remaining 39 items are appropriate to be used to measure students' coping strategies.

scholarly journals The direction of the response scale matters – accounting for the unit of measurement

2019 ◽  
Vol 53 (5) ◽  
pp. 871-891
Author(s):  
Thomas Salzberger ◽  
Monika Koller
Keyword(s):  
Rasch Model ◽  
Measurement Theory ◽  
Measurement Precision ◽  
Spatial Proximity ◽  
Validity And Reliability ◽  
Measurement Instruments ◽  
Response Scale ◽  
Content Type ◽  
Unit Of Measurement ◽  
The Rasch Model

Purpose Psychometric analyses of self-administered questionnaire data tend to focus on items and instruments as a whole. The purpose of this paper is to investigate the functioning of the response scale and its impact on measurement precision. In terms of the response scale direction, existing evidence is mixed and inconclusive. Design/methodology/approach Three experiments are conducted to examine the functioning of response scales of different direction, ranging from agree to disagree versus from disagree to agree. The response scale direction effect is exemplified by two different latent constructs by applying the Rasch model for measurement. Findings The agree-to-disagree format generally performs better than the disagree-to-agree variant with spatial proximity between the statement and the agree-pole of the scale appearing to drive the effect. The difference is essentially related to the unit of measurement. Research limitations/implications A careful investigation of the functioning of the response scale should be part of every psychometric assessment. The framework of Rasch measurement theory offers unique opportunities in this regard. Practical implications Besides content, validity and reliability, academics and practitioners utilising published measurement instruments are advised to consider any evidence on the response scale functioning that is available. Originality/value The study exemplifies the application of the Rasch model to assess measurement precision as a function of the design of the response scale. The methodology raises the awareness for the unit of measurement, which typically remains hidden.

An App for Comparing the Epidemic Impacts in Continents and Trends of the Confirmed Cases on COVID-19 using the Online Rasch Model: An Observational Study (Preprint)

2020 ◽  
Author(s):  
CHIEN WEI
Keyword(s):  
Observational Study ◽  
Rasch Model ◽  
Online Algorithm ◽  
Forest Tree ◽  
Google Maps ◽  
Line Chart ◽  
Weighted Score ◽  
Novel Coronavirus ◽  
The Rasch Model ◽  
Outbreak Situation

BACKGROUND When a novel coronavirus (e.g., COVID-19) starts to spread, two of the most frequently asked questions are about (1) the overall trend of daily confirmed cases increasing or decreasing during the on-going outbreak epidemic and (2) the worst-hit continents for COVID-19 in the recent weeks. Finding the trend of the outbreak spread and the epidemic impacts on continents amid COVID-19 is continuously an urgent concern. OBJECTIVE This study aims to (1) inspect the epidemic trend over days, (2) develop an online algorithm to draw the epidemic impacts for COVID-19 among continents, and (3) design an app for a better understanding of the outbreak situation on Google Maps. METHODS We downloaded the COVID-19 outbreak numbers from Jun 24 to July 13, 2020, from Github that contains the number of confirmed cases in countries/regions. Three methods were used to compare differences in COVID-19-struck measures, including (1)the traditional summation score, (2) the Rasch logit score, and (3) the weighted score(i.e., adjusted by the estimated variance). Rasch model was applied to estimate the overall item (i.e., day) difficulties and the COVID-19-struck measures for all countries/areas. The epidemic trend was assessed by the correlation coefficient (CC) computed by the item difficulties over the observed days. An online algorithm based on the Rasch model was built for displaying the outbreak trend and the epidemic effects in comparison for continents using the forest tree plot and the analysis of variance(ANOVA). An app was developed to understand the daily epidemic trends on Google Maps. RESULTS The three methods used for comparing differences in COVID-19-struck measures were displayed somewhat different. A line chart was drawn online to present the trend measured by item(i.e., day) difficulties approaching stability with CC=-0.07. Differences in COVID-19-struck impacts were observed among continents using ANOVA(p<0.001= Chidist(160.31, 5)) and the forest tree plot. A dashboard was created to present the COVID-19 situation on Google Maps. CONCLUSIONS The three methods used for comparing differences in COVID-19-struck measures were displayed somewhat different. A line chart was drawn online to present the trend measured by item(i.e., day) difficulties approaching stability with CC=-0.07. Differences in COVID-19-struck impacts were observed among continents using ANOVA(p<0.001= Chidist(160.31, 5)) and the forest tree plot. A dashboard was created to present the COVID-19 situation on Google Maps. CLINICALTRIAL Nil

Sign in / Sign up

Export Citation Format

Share Document