Three Conceptual Impediments to Developing Scale Theory for Formative Scales

Methodology ◽  
2018 ◽  
Vol 14 (4) ◽  
pp. 156-164 ◽  
Author(s):  
Keith A. Markus
Keyword(s):  
Scale Development ◽  
Item Analysis ◽  
Model Fitting ◽  
Item Bias ◽  
Validity Evidence ◽  
Scale Theory ◽  
Underlying Theory ◽  
Standard Scale ◽  
Item Scores

Abstract. Bollen and colleagues have advocated the use of formative scales despite the fact that formative scales lack an adequate underlying theory to guide development or validation such as that which underlies reflective scales. Three conceptual impediments impede the development of such theory: the redefinition of measurement restricted to the context of model fitting, the inscrutable notion of conceptual unity, and a systematic conflation of item scores with attributes. Setting aside these impediments opens the door to progress in developing the needed theory to support formative scale use. A broader perspective facilitates consideration of standard scale development concerns as applied to formative scales including scale development, item analysis, reliability, and item bias. While formative scales require a different pattern of emphasis, all five of the traditional sources of validity evidence apply to formative scales. Responsible use of formative scales requires greater attention to developing the requisite underlying theory.

scholarly journals Development of A Preschool Teacher’s Guidance Qualifications Scale and Its Psychometric Properties

2017 ◽  
Vol 5 (9) ◽  
pp. 83
Author(s):  
Hülya Bilgin
Keyword(s):  
Factor Analysis ◽  
Interpersonal Relationships ◽  
Internal Consistency ◽  
Item Analysis ◽  
Individual Development ◽  
Total Variance ◽  
Measurement Tool ◽  
Validity And Reliability ◽  
Item Scores ◽  
Test Retest Reliability

The goal of this study is to develop a Guidance Qualification Scale for Preschool Teachers and examine the validity and reliability of the scale. The research was conducted with 374 teachers working in preschools in Istanbul and they were chosen by the random sampling method. Factor analysis was carried out for the construct validity of the scale. Internal consistency coefficients and test-retest reliabilities were examined and item analysis was performed for reliability studies. As a result of the factor analysis, 59 items which explained 55,989% of the total variance (Total Variance Explained 45,050%), were obtained under four factors. When the factor weights of the scale are examined, items were found to have weight values ranging from .746 to .453. Cronbach’s Alpha internal consistency coefficients of the “Preschool Teacher’s Guidance Qualifications Scale” were found to be .977 for the total scale, .954 for the "Supporting Educational Development and Interpersonal Relationships" sub-dimension, .943 for the "Supporting Individual Development and Orientation" sub-dimension, .904 for the "Self-acceptance, Support for Self and Social Development" sub-dimension and .909 for the "Recognizing Individuals, Identifying Problems and Supporting Parents" sub-dimension. Also, the test-retest reliability study of the scale was conducted with 44 teachers. According to the results of the t-test applied for differences between the average item scores of upper 27% and lower 27% groups of items on (in?) the scale, all differences were found to be significantly in favor of the upper 27% group. In light of these findings, it can be said that the “Preschool Teacher’s Guidance Qualifications Scale” can be used as a valid and reliable measurement tool in the field of education.

Global Academic Interest Scale for Undergraduate and Graduate Students

2020 ◽  
pp. 073428292094614
Author(s):  
Jihyun Lee ◽  
Tracy L. Durksen
Keyword(s):  
Graduate Students ◽  
Undergraduate Students ◽  
Career Aspiration ◽  
Validity Evidence ◽  
Factor Scores ◽  
Academic Interest ◽  
Student Groups ◽  
The Future ◽  
Item Scores ◽  
Future Work

Using an established global academic interest scale, academic interest dimensions of undergraduate ( n = 326) and graduate ( n = 401) students were compared. The four-factor structure, consisting of passion for learning, confidence in the future, career aspiration, and self-expression, held for both student groups. However, the item scores and mean factor scores as well as the pattern of factor correlations differed between the two groups. While the sample of graduate students exhibited greater levels of passion for learning, the undergraduate students showed higher levels of confidence in the future. Overall, a broad range of correlates of academic interest employed in this study provided construct validity evidence of academic interest, supporting the main tenants of the motivational theory of life-span development. Limitations and directions for future work are discussed.

Idea Adoption in Workplace Innovation: Scale Development and Initial Validity Evidence

2018 ◽  
Vol 2018 (1) ◽  
pp. 15998
Author(s):  
Karen Van Dam ◽  
Marjolein C.J. Caniels ◽  
Gladys Cools-Tummers ◽  
Heidi Lenearts
Keyword(s):  
Scale Development ◽  
Validity Evidence ◽  
Workplace Innovation

scholarly journals A proposal of a measure of societal sexual double standard

2018 ◽  
Author(s):  
Carmen Gómez-Berrocal ◽  
Pablo Vallejo-Medina ◽  
Nieves Moyano ◽  
Juan Carlos Sierra
Keyword(s):  
Factor Analysis ◽  
Factor Structure ◽  
Predictive Validity ◽  
Temporal Stability ◽  
Double Standard ◽  
Sexual Double Standard ◽  
Validity Evidence ◽  
Standard Scale ◽  
Order Factor ◽  
The Individual

A version of the Double Standard Scale (SDSS) is presented to assess the subjectiveperception of society's support for the Sexual Double Standard (SDS). We examine itspsychometric properties in a sample of Spanish population heterosexual of 1,206individuals (50% males), distributed across three groups (18-34, 35-49, and 50 yearsold or older).We performed Exploratory Factor Analysis and Confirmatory. The foundversion consists of 18 item distributed into three factors (Acceptance for Male SexualShyness,Acceptance for Female Sexual Freedom and Acceptance for TraditionalDouble Standard). A second-order factor structure was also adequate. Internalconsistency, temporal stability and validity evidence are reported. This version of theSDSS is reliable and valid. It allows to assess the perception of societal sexual doublestandard. Its usefulness to improve the predictive validity of the standardized scalesthat evaluate the individual adhesion to SDS is discussed.

Multidimensionafity of Leader-Member Exchange: An Empirical Assessment through Scale Development

1998 ◽  
Vol 24 (1) ◽  
pp. 43-72 ◽  
Author(s):  
Robert C. Liden ◽  
John M. Maslyn
Keyword(s):  
Scale Development ◽  
Item Analysis ◽  
Fourth Dimension ◽  
Leader Member Exchange ◽  
Multidimensional Construct ◽  
Empirical Assessment ◽  
Multidimensional Measure ◽  
Working Students ◽  
Member Exchange ◽  
Development And Validation

Whether Leader-Member Exchange (LMX) is a unidimensional or a multidimensional construct was assessed through the development and validation of a multidimensional measure. Item analysis involving 302 working students, followed by construct and criterion-related validation using 249 employees representing two organizations resulted in a multidimensional LMX scale. The results provided support for the affect, loyalty, and contribution dimensions identified by Dienesch and Liden (1986), as well as a fourth dimension, professional respect.

scholarly journals Item-Score Reliability in Empirical-Data Sets and Its Relationship With Other Item Indices

2017 ◽  
Vol 78 (6) ◽  
pp. 998-1020 ◽  
Author(s):  
Eva A. O. Zijlmans ◽  
Jesper Tijmstra ◽  
L. Andries van der Ark ◽  
Klaas Sijtsma
Keyword(s):  
Empirical Data ◽  
Item Analysis ◽  
Factor Loading ◽  
Data Sets ◽  
Individual Item ◽  
Item Discrimination ◽  
Item Score ◽  
Score Reliability ◽  
Reliability Methods ◽  
Item Scores

Reliability is usually estimated for a total score, but it can also be estimated for item scores. Item-score reliability can be useful to assess the repeatability of an individual item score in a group. Three methods to estimate item-score reliability are discussed, known as method MS, method [Formula: see text], and method CA. The item-score reliability methods are compared with four well-known and widely accepted item indices, which are the item-rest correlation, the item-factor loading, the item scalability, and the item discrimination. Realistic values for item-score reliability in empirical-data sets are monitored to obtain an impression of the values to be expected in other empirical-data sets. The relation between the three item-score reliability methods and the four well-known item indices are investigated. Tentatively, a minimum value for the item-score reliability methods to be used in item analysis is recommended.

scholarly journals Direct Observation of Clinical Skills Feedback Scale: Development and Validity Evidence

2016 ◽  
Vol 28 (4) ◽  
pp. 385-394 ◽  
Author(s):  
Samantha Halman ◽  
Nancy Dudek ◽  
Timothy Wood ◽  
Debra Pugh ◽  
Claire Touchie ◽  
...  
Keyword(s):  
Scale Development ◽  
Direct Observation ◽  
Clinical Skills ◽  
Validity Evidence

scholarly journals Development and Psychometric Properties of the Questionnaire for Assessing Educational Podcasts (QAEP)

2020 ◽  
Vol 11 ◽  
Author(s):  
Rafael Alarcón ◽  
María J. Blanca
Keyword(s):  
Psychometric Properties ◽  
Factor Structure ◽  
Cross Validation ◽  
Item Analysis ◽  
Evidence Based ◽  
Validity Evidence ◽  
Four Dimensions ◽  
Confirmatory Factor ◽  
Direct Student ◽  
Teaching Learning

The aim of this research was to develop and validate the Questionnaire for Assessing Educational Podcasts (QAEP), an instrument designed to gather students’ views about four dimensions of educational podcasts: access and use, design and structure, content adequacy, and value as an aid to learning. In study 1 we gathered validity evidence based on test content by asking a panel of experts to rate the clarity and relevance of items. Study 2 examined the psychometric properties of the QAEP, including confirmatory factor analysis with cross-validation to test the factor structure of the questionnaire, as well as item and reliability analysis. The results from study 1 showed that the experts considered the items to be clearly worded and relevant in terms of their content. The results from study 2 showed a factor structure consistent with the underlying dimensions, as well as configural and metric invariance across groups. The item analysis and internal consistency for scores on each factor and for total scores were also satisfactory. The scores obtained on the QAEP provide teachers with direct student feedback and highlight those aspects that need to be enhanced in order to improve the teaching/learning process.

scholarly journals The Logic of Latent Variable Analysis as Validity Evidence in Psychological Measurement

2016 ◽  
Vol 9 (1) ◽  
pp. 168-175 ◽  
Author(s):  
Purya Baghaei ◽  
Mona Tabatabaee Yazdi
Keyword(s):  
Latent Variable ◽  
Latent Trait ◽  
Model Fitting ◽  
Validity Evidence ◽  
Latent Trait Models ◽  
Test Items ◽  
Latent Variable Analysis ◽  
Latent Trait Model ◽  
Trait Model ◽  
Number Of Publications

Background:Validity is the most important characteristic of tests and social science researchers have a general consensus of opinion that the trustworthiness of any substantive research depends on the validity of the instruments employed to gather the data.Objective:It is a common practice among psychologists and educationalists to provide validity evidence for their instruments by fitting a latent trait model such as exploratory and confirmatory factor analysis or the Rasch model. However, there has been little discussion on the rationale behind model fitting and its use as validity evidence. The purpose of this paper is to answer the question: why the fit of data to a latent trait model counts as validity evidence for a test?Method:To answer this question latent trait theory and validity concept as delineated by Borsboom and his colleagues in a number of publications between 2003 to 2013 is reviewed.Results:Validating psychological tests employing latent trait models rests on the assumption of conditional independence. If this assumption holds it means that there is a ‘common cause’ underlying the co-variation among the test items, which hopefully is our intended construct.Conclusion:Providing validity evidence by fitting latent trait models is logistically easy and straightforward. However, it is of paramount importance that researchers appreciate what they do and imply about their measures when they demonstrate that their data fit a model. This helps them to avoid unforeseen pitfalls and draw logical conclusions.

Sign in / Sign up

Export Citation Format

Share Document