Psychometric evaluation of the Relative Mastery Scale: An Occupational Adaptation instrument

2021 ◽  
pp. 153944922110608
Author(s):  
Lorrie George-Paschal ◽  
Nancy E. Krusen ◽  
Chia-Wei Fan

This study evaluated the psychometric properties of the Relative Mastery Scale (RMS). Valid and reliable client-centered instruments support practice in value-based health care and community-based settings. Participants were 368 community-dwelling adults aged 18 to 95 years. Researchers conducted validity and reliability examinations of the RMS using classical test theory and Rasch measurement model. A partial credit model allowed exploration of individual scale properties. Spearman’s correlation coefficients between items were statistically significant at the .01 level. Cronbach’s alpha coefficient was .94 showing strong internal consistency. In exploratory factor analysis, Factor 1 accounted for 71% of variance with an eigenvalue of 4.26. In Rasch analysis, the 5-point rating scale demonstrated adequate functioning, confirmed unidimensionality, and person/item separation. The RMS instrument demonstrates sound psychometric characteristics. A valid and reliable measure of internal occupational adaptation supports application to monitor progress of internal occupational adaptation across a variety of individuals.

2019 ◽  
Vol 60 (3) ◽  
pp. e117-e126
Author(s):  
Katarzyna Zawisza ◽  
Aleksander Galas ◽  
Beata Tobiasz-Adamczyk ◽  
Tomasz Grodzicki

Abstract Background and Objectives The purpose of the study was to create and validate a tool that could be implemented easily to recognize the presence and assess the level of neglect in community-dwelling older adults, and to provide information about the prevalence of the phenomenon in different subgroups of older adults in Poland. Research Design and Methods The cross-sectional study of elder neglect and self-neglect was conducted in Lesser Poland in 2017. It included 2,443 face-to-face interviews with randomly selected community-dwelling individuals from among the general population (1,635), social service users (280), and hospital patients (528). Classical Test Theory and Item Response Theory (IRT) were used to build the scale, and its content and construct validity and reliability were assessed. Results The Self-Reported Neglect Scale (SRNS) with a 2-factor structure (basic needs and psychological needs dimensions) was created. Results of the IRT analysis showed high item discrimination (2.7–4.8 for the first factor, 0.8–3.2 for the second). The 1-year prevalence of neglect as a percentage of nonzero values of the SRNS was estimated at 11.4%. Discussion and Implications The SRNS exhibited good psychometric properties. It may have promise as a tool for the assessment of neglect in epidemiological studies as well in the everyday practice of medical professionals and social workers.


2016 ◽  
Vol 3 (4) ◽  
Author(s):  
Károly Zerényi

A pedagógia és pszichológia területén a kutatók a kérdőíves felmérésekben előszeretettel használnak Likert-skálás attitűdkérdéseket is, amelyek segítségével információhoz lehet jutni a vizsgált személyek adott állítással kapcsolatos pozitív vagy negatív attitűdjéről. Szakirodalmi kutatások alapján elmondható, hogy a kutatók a skálaértékeket többek között átlagolják, így élnek azzal a feltételezéssel, hogy intervallumváltozókról van szó, hiszen ebben az esetben van értelme az átlagszámításnak (Brown, 2011; Kehl, 2012). Ugyanakkor a klasszikus tesztelmélet keretei között a Likert-skálás kérdések esetében a skálaértékek átlagolására nincs mód, hiszen azok nem intervallumskálán helyezkednek el, így legfeljebb az ordinális skála alapján megengedhető statisztikai műveletek elvégzésére és vizsgálati módszerek alkalmazására nyílik lehetőség. A fennálló problémára egy lehetséges megoldást nyújthat a modern tesztelméleti modellek alkalmazása, amely egyre több nemzetközi és hazai kutatásban megjelenik (Schulz és Sibberns, 2004; Kontra, 2008). Napjainkban számos olyan ingyenesen is elérhető szoftver áll rendelkezésre (például Winstep, ConQuest), amelyek kezelni tudják a Likert-skálás adatok esetén használható rangskálás és parciális kredit modellt. A Likert-skálás adatok elemzésére képes rangskálás és parciális kredit modell segítségével olyan intervallumskálán elhelyezkedő attitűderősségek határozhatók meg, amelyekkel már elvégezhetők a megbízhatóbb matematikai-statisztikai vizsgálatok is. Mindazonáltal az IRT modellek alkalmazásának van egy komoly akadálya, hogy azok komplex jelenségeknél nem működnek. Ebben az esetben nem marad más választás, mint a Likert-skálás adatok legfeljebb ordinális skálán történő kezelése.*AbstractOpportunities and limitations of the Likert scale Researchers of education and psychology frequently use attitude questions on the Likert scale in surveys to collect sufficient information regarding positive or negative attitudes of the examined persons. Based on the research, the scale values are averaged among others and they are assumed as interval variables, so it makes sense to calculate the average (Brown, 2011; Kehl, 2012). However, according to classical test theory it is not possible to calculate mean from the Likert scale data, because they are not interpreted on interval scale, and therefore solely operations based on ordinal scale statistical operations and test methods are possible. This problem can be solved with models of the item response theory, which have been presented in some international and Hungarian research (Schulz és Sibberns, 2004; Kontra, 2008). Nowadays some software (e.g. Winstep, ConQuest) is available which can provide rating scale and partial credit model to the analysis of the Likert scale data. Such attitude strengths can be determined with these models which are located on interval scale and offer more reliable mathematical and statistical testing opportunities. Nevertheless, the use of the IRT models has the serious limitation that these models are not applicable where complex phenomena are investigated. In this case there is no other option than treating the Likert scale data on ordinal scale.


2011 ◽  
Vol 27 (3) ◽  
pp. 164-170 ◽  
Author(s):  
Anna Sundström

This study evaluated the psychometric properties of a self-report scale for assessing perceived driver competence, labeled the Self-Efficacy Scale for Driver Competence (SSDC), using item response theory analyses. Two samples of Swedish driving-license examinees (n = 795; n = 714) completed two versions of the SSDC that were parallel in content. Prior work, using classical test theory analyses, has provided support for the validity and reliability of scores from the SSDC. This study investigated the measurement precision, item hierarchy, and differential functioning for males and females of the items in the SSDC as well as how the rating scale functions. The results confirmed the previous findings; that the SSDC demonstrates sound psychometric properties. In addition, the findings showed that measurement precision could be increased by adding items that tap higher self-efficacy levels. Moreover, the rating scale can be improved by reducing the number of categories or by providing each category with a label.


Assessment ◽  
2021 ◽  
pp. 107319112199416
Author(s):  
Desirée Blázquez-Rincón ◽  
Juan I. Durán ◽  
Juan Botella

A reliability generalization meta-analysis was carried out to estimate the average reliability of the seven-item, 5-point Likert-type Fear of COVID-19 Scale (FCV-19S), one of the most widespread scales developed around the COVID-19 pandemic. Different reliability coefficients from classical test theory and the Rasch Measurement Model were meta-analyzed, heterogeneity among the most reported reliability estimates was examined by searching for moderators, and a predictive model to estimate the expected reliability was proposed. At least one reliability estimate was available for a total of 44 independent samples out of 42 studies, being that Cronbach’s alpha was most frequently reported. The coefficients exhibited pooled estimates ranging from .85 to .90. The moderator analyses led to a predictive model in which the standard deviation of scores explained 36.7% of the total variability among alpha coefficients. The FCV-19S has been shown to be consistently reliable regardless of the moderator variables examined.


Author(s):  
Mehmet Barış Horzum ◽  
Gülden Kaya Uyanik

The aim of this study is to examine validity and reliability of Community of Inquiry Scale commonly used in online learning by the means of Item Response Theory. For this purpose, Community of Inquiry Scale version 14 is applied on 1,499 students of a distance education center’s online learning programs at a Turkish state university via internet. The collected data is analyzed by using a statistical software package. Research data is analyzed in three aspects, which are checking model assumptions, checking model-data fit and item analysis. Item and test features of the scale are examined by the means of Graded Response Theory. In order to use this model of IRT, after testing the assumptions out of the data gathered from 1,499 participants, data model compliance was examined. Following the affirmative results gathered from the examinations, all data is analyzed by using GRM. As a result of the study, the Community of Inquiry Scale adapted to Turkish by Horzum (in press) is found to be reliable and valid by the means of Classical Test Theory and Item Response Theory.


2016 ◽  
Vol 33 (1) ◽  
pp. 74 ◽  
Author(s):  
Alejandro Veas ◽  
Juan Luis Castejón ◽  
Raquel Gilar ◽  
Pablo Miñano

<p>The School Attitude Assessment Survey-Revised (SAAS-R) was developed by McCoach &amp; Siegle (2003b) and validated in Spain by Author (2014) using Classical Test Theory. The objective of the current research is to validate SAAS-R using multidimensional Rasch analysis. Data were collected from 1398 students attending different high schools. Principal Component Analysis supported the multidimensional SAAS-R. The item difficulty and person ability were calibrated along the same latent trait scale. 10 items were removed from the scale due to misfit with the Rasch model. Differential Item Functioning revealed no significant differences across gender for the remaining 25 items. The 7-category rating scale structure did not function well, and the subscale goal valuation obtained low reliability values. The multidimensional Rasch model supported 25 item-scale SAAS-R measures from five latent factors. Therefore, the advantages of multidimensional Rasch analysis are demonstrated in this study.</p>


Nutrients ◽  
2020 ◽  
Vol 12 (11) ◽  
pp. 3561
Author(s):  
Karla Vázquez-Espino ◽  
Carles Fernández-Tena ◽  
Maria Antonia Lizarraga-Dallo ◽  
Andreu Farran-Codina

Weak evidence exists on the relationship between nutritional knowledge and diet quality. Many researchers claim that this could be in part because of inadequate validation of the questionnaires used. The aim of this study was to develop a compact reliable questionnaire on nutrition knowledge for young and adult athletes (NUKYA). Researchers and the sport clubs medical staff developed the questionnaire by taking into consideration the latest athlete dietary guidelines. The questionnaire content was validated by a panel of 12 nutrition experts, and finally tested by 445 participants including athletes (n = 264), nutrition students (n = 49) and non-athletes with no formal nutrition knowledge (n = 132). After consulting the experts, 59 of the 64 initial items remained in the questionnaire. To collect the evaluation of experts, we used the content validity index, obtaining high indices for relevance and ambiguity (0.99) as well as for clarity and simplicity (0.98). The final questionnaire included 24 questions with 59 items. We ensured construct validity and reliability through psychometric validation based on the Classical Test Theory and the Item–Response Theory (Rasch model). We found significant statistical differences comparing the groups of nutrition knowledgeable participants with the rest of the groups (ANOVA p < 0.001). We verified the questionnaire for test–retest reliability (R = 0.895, p < 0.001) and internal consistency (Cronbach’s α=0.849). We successfully fit the questionnaire data to a rating scale model (global separation reliability of 0.861) and examined discrimination and difficulty indices for items. Finally, we validated the NUKYA questionnaire as an effective tool to appraise nutrition knowledge in athletes. This questionnaire can be used for guiding in educational interventions, studying the influence of nutrition knowledge on nutrient intake and assessing/monitoring sport nutritional knowledge in large groups.


Author(s):  
Phoom Praraksa ◽  
Wanida Simpol

Life and career skills are essential attributes for living in the 21st century because they are important to both learning and working in local and international workplaces. This study tried to create a measurement model of life and career skills and develop an online scale for investigating the psychometric property of the scale. The participants consisted of 646 primary students in Northern, Central, Southern and North eastern regions of Thailand. Then, the classical test theory, the multidimensional item response theory and the confirmatory factor analysis (CFA) were used for data analysis. The analysis of data appeared as item of difficulty index, discrimination power index and reliability of the scale. In addition, the multidimensional analysis and the CFA of data showed item fit and construct validity of the scale. This may lead to the development of clear and correct measure of students’ life and career skills structure. Policy implications are discussed. Keywords: Life and career skills, tentative model, online scale, construct validity.


2016 ◽  
Vol 76 (6) ◽  
pp. 976-985 ◽  
Author(s):  
Leanne M. Stanley ◽  
Michael C. Edwards

The purpose of this article is to highlight the distinction between the reliability of test scores and the fit of psychometric measurement models, reminding readers why it is important to consider both when evaluating whether test scores are valid for a proposed interpretation and/or use. It is often the case that an investigator judges both the reliability of scores and the fit of a corresponding measurement model to be either acceptable or unacceptable for a given situation, but these are not the only possible outcomes. This article focuses on situations in which model fit is deemed acceptable, but reliability is not. Data were simulated based on the item characteristics of the PROMIS (Patient Reported Outcomes Measurement Information System) anxiety item bank and analyzed using methods from classical test theory, factor analysis, and item response theory. Analytic techniques from different psychometric traditions were used to illustrate that reliability and model fit are distinct, and that disagreement among indices of reliability and model fit may provide important information bearing on a particular validity argument, independent of the data analytic techniques chosen for a particular research application. We conclude by discussing the important information gleaned from the assessment of reliability and model fit.


2019 ◽  
Author(s):  
◽  
Andrew Judson Womack

[ACCESS RESTRICTED TO THE UNIVERSITY OF MISSOURI AT AUTHOR'S REQUEST.] Scientific literacy (SL) as an idea is considered by many to be one of the main goals of science education. This study focuses on Roberts' (2007) second vision of SL, the ability of students to competently navigate "situations with a scientific component..." (p. 730). Assessments centered on Vision-II-SL could be refined to better capture student's competency. To this end, this study focuses on the development of an open- ended Vision-II aligned assessment (QuASSR-oe2). To make this assessment usable in large populations this study utilizes automated scoring, which is the synergy of natural- language-processing and supervised-machine-learning. This study establishes the validity and reliability of the QuASSR-oe2 using psychometric techniques from classical-test theory and Rasch analysis. The study also establishes that accurate automated-scoring models (QWK > .7) can be generated, thus allowing for efficient scoring of open-ended student responses. Ultimately, this allows for the QuASSR-oe2 to be easily utilized in large populations.


Sign in / Sign up

Export Citation Format

Share Document