Pitfalls and Challenges in Constructing Short Forms of Cognitive Ability Measures

2014 ◽  
Vol 35 (4) ◽  
pp. 190-200 ◽  
Author(s):  
Stefan Schipolowski ◽  
Ulrich Schroeders ◽  
Oliver Wilhelm
Keyword(s):  
Cognitive Abilities ◽  
Large Scale ◽  
Measurement Model ◽  
Measurement Precision ◽  
Individual Level ◽  
Crystallized Intelligence ◽  
Large Scale Assessment ◽  
Short Forms ◽  
Cost Efficient ◽  
The Individual

Especially in survey research and large-scale assessment there is a growing interest in short scales for the cost-efficient measurement of psychological constructs. However, only relatively few standardized short forms are available for the measurement of cognitive abilities. In this article we point out pitfalls and challenges typically encountered in the construction of cognitive short forms. First we discuss item selection strategies, the analysis of binary response data, the problem of floor and ceiling effects, and issues related to measurement precision and validity. We subsequently illustrate these challenges and how to deal with them based on an empirical example, the development of short forms for the measurement of crystallized intelligence. Scale shortening had only small effects on associations with covariates. Even for an ultra-short six-item scale, a unidimensional measurement model showed excellent fit and yielded acceptable reliability. However, measurement precision on the individual level was very low and the short forms were more likely to produce skewed score distributions in ability-restricted subpopulations. We conclude that short scales may serve as proxies for cognitive abilities in typical research settings, but their use for decisions on the individual level should be discouraged in most cases.

Scale Shortening and Decrease in Measurement Precision: Analysis of the Pain Self-Efficacy Questionnaire and Its Short Forms in an Italian-Speaking Population With Neck Pain Disorders

2021 ◽  
Author(s):  
Marco Monticone ◽  
Andrea Giordano ◽  
Franco Franchignoni
Keyword(s):  
Neck Pain ◽  
Self Efficacy ◽  
Clinical Decision Making ◽  
Short Form ◽  
Measurement Precision ◽  
Psychometric Characteristics ◽  
Individual Level ◽  
Short Forms ◽  
Wide Range ◽  
The Individual

ABSTRACT Objective Short (2- and 4-item) forms of the Pain Self-Efficacy Questionnaire (PSEQ) have been proposed, but their measurement precision at the individual level is unclear. The purpose of this study was to analyze the Rasch psychometric characteristics of PSEQ and its 3 short forms (one 4-item and two 2-item versions) in an Italian-speaking population with neck pain disorders and compare their measurement precision at the individual level through calculation of the test information function (TIF). Methods Secondary analysis of data from a prospective single-group observational study was conducted. In 161 consecutive participants (mean age = 45 y (SD = 14); 104 women) with neck pain disorders, a Rasch analysis was performed on each version of PSEQ (full scale plus 3 short forms), and the TIF was calculated to examine the degree of measurement precision in estimating person ability over the whole measured construct (pain self-efficacy). Results In all versions of PSEQ, the rating scale fulfilled the category functioning criteria, and all items showed an adequate fit to the Rasch model. The TIF showed a bell-shaped distribution of information, with an acceptable measurement precision (standard error < 0.5) for persons with a wide range of ability; conversely, measurement precision was unacceptably low in each short form (particularly the two 2-item versions). Conclusions The results confirm and expand reports on the sound psychometric characteristics of PSEQ, showing for the first time its conditional precision in estimating pain self-efficacy measures in Italian individuals with neck pain disorders. The study cautions against use of the 3 PSEQ short forms for individual-level clinical decision making. Impact Short scales are popular in rehabilitation settings largely because they can save assessment time and related costs. The psychometric characteristics of the 10-item PSEQ were confirmed and deepened, including its precision in estimating individual pain self-efficacy at different levels of this latent variable. On the other hand, low measurement precision of the 3 PSEQ short forms caution against their use for individual judgments.

scholarly journals Large-Scale Assessments, Personalized Learning, and Creativity: Paradoxes and Possibilities

2019 ◽  
Vol 2 (3) ◽  
pp. 311-327 ◽  
Author(s):  
Ronald A. Beghetto
Keyword(s):  
Large Scale ◽  
Personalized Learning ◽  
Individual Level ◽  
Large Scale Assessment ◽  
Working Definition ◽  
Creative Learning ◽  
Additional Process ◽  
Large Scale Assessments ◽  
Definition Of ◽  
The Individual

Purpose: This article, based on an invited talk, aims to explore the relationship among large-scale assessments, creativity and personalized learning. Design/Approach/Methods: Starting with the working definition of large-scale assessments, creativity, and personalized learning, this article identified the paradox of combining these three components together. As a consequence, a logic mode of large-scale assessment and creativity expressions is illustrated, along with an exploration of new possibilities. Findings: Smarter design of large-scale assessments is needed. Firstly, we need to assess creative learning at the individual level, so complex tasks with high uncertainty should be presented to students. Secondly, additional process and experiential data while students are working on problems need to be captured. Thirdly, the human-artificial intelligence (AI) augmented scoring should be explored, developed, and refined. Originality/Value: This article addresses the drawbacks of current large-scale assessments and explores possibilities for combining assessment with creativity and personalized learning. A logic model illustrating variations necessary for creative learning and considerations and cautions for designing large-scale assessments are also provided.

Perceived Trust and Confidence for Cryptocurrency Adoption

2022 ◽  
pp. 250-279
Author(s):  
Ewilly Jie Ying Liew ◽  
Wei Li Peh ◽  
Zhuan Kee Leong
Keyword(s):  
Large Scale ◽  
Social Impact ◽  
Public Perceptions ◽  
Middle Income ◽  
Individual Level ◽  
Perceived Trust ◽  
Middle Income Countries ◽  
Regulatory Bodies ◽  
The Individual ◽  
National Databases

This chapter seeks to examine the influence of public perceptions of trust in people and confidence in institutions on cryptocurrency adoption, taking into account the individual-level demographic factors and the regional-level contextual factors. Data is obtained from three large-scale international surveys and national databases and analyzed using R software. The multivariate results demonstrate that individuals' public perceptions of trust and confidence significantly contribute to cryptocurrency adoption. Lower perceived trust in people and higher perceived confidence in civil service and international regulatory bodies increase cryptocurrency adoption, while perceived confidence in political and financial institutions discourages cryptocurrency adoption. Additionally, the univariate results find significant comparisons of gender and perceived trust differences on the predictors of cryptocurrency adoption. This chapter discusses and provides insights on the social impact and future of cryptocurrency adoption, particularly among the upper- and lower-middle-income countries.

scholarly journals The measurement of populist attitudes: Testing cross-national scales using item response theory

Politics ◽  
2019 ◽  
Vol 40 (1) ◽  
pp. 3-21 ◽  
Author(s):  
Steven M Van Hauwaert ◽  
Christian H Schimpf ◽  
Flavio Azevedo
Keyword(s):  
Item Response Theory ◽  
Item Response ◽  
Empirical Evaluation ◽  
Measurement Model ◽  
Future Research ◽  
Response Theory ◽  
Individual Level ◽  
National Measurement ◽  
The Individual ◽  
Cross National

Recent research in the populism literature has devoted considerable efforts to the conceptualisation and examination of populism on the individual level, that is, populist attitudes. Despite rapid progress in the field, questions of adequate measurement and empirical evaluation of measures of populist attitudes remain scarce. Seeking to remedy these shortcomings, we apply a cross-national measurement model, using item response theory, to six established and two new populist indicators. Drawing on a cross-national survey (nine European countries, n = 18,368), we engage in a four-folded analysis. First, we examine the commonly used 6-item populism scale. Second, we expand the measurement with two novel items. Third, we use the improved 8-item populism scale to further refine equally comprehensive but more concise and parsimonious populist measurements. Finally, we externally validate these sub-scales and find that some of the proposed sub-scales outperform the initial 6- and 8-item scales. We conclude that existing measures of populism capture moderate populist attitudes, but face difficulties measuring more extreme levels, while the individual information of some of the populist items remains limited. Altogether, this provides several interesting routes for future research, both within and between countries.

Children’s Height and Parental Unemployment: A Large-Scale Anthropometric Study on Eastern Germany, 1994–2006

2010 ◽  
Vol 11 (1) ◽  
pp. 1-24 ◽  
Author(s):  
Jörg Baten ◽  
Andreas Böhm
Keyword(s):  
Employment Status ◽  
Large Scale ◽  
Psychological Effects ◽  
Average Height ◽  
Panel Analysis ◽  
Individual Level ◽  
Parental Unemployment ◽  
The Individual ◽  
Anthropometric Study

Abstract The average height of children is an indicator of the quality of nutrition and healthcare. In this study, we assess the effect of unemployment and other factors on this variable. In the Eastern German Land of Brandenburg, a dataset of 253,050 preschool height measurements was compiled and complemented with information on parents’ schooling and employment status. Unemployment might have negative psychological effects, with an impact on parental care. Both a panel analysis of districts and an assessment at the individual level yield the result that increasing unemployment, net out-migration and fertility were in fact reducing height.

The Effect of Public Service Motivation at Individual, Group, and Organisational Levels of Citizenship Behaviour

2020 ◽  
Vol 33 (1) ◽  
pp. 39-58
Author(s):  
Kuo-Tai Cheng ◽  
Yuan-Chieh Chang ◽  
Changyen Lee
Keyword(s):  
Public Service ◽  
Organizational Citizenship Behavior ◽  
Large Scale ◽  
Public Service Motivation ◽  
Employee Attitudes ◽  
Public Utility ◽  
Citizenship Behavior ◽  
Perceived Effectiveness ◽  
Individual Level ◽  
The Individual

This study conceptualizes and empirically investigates how dimensions of public service motivation affect perceived citizenship behaviour in the context of government-owned utilities. This study used a large-scale questionnaire survey from four utility sectors in Taiwan (N = 1,087). The emergent model suggests that compassion (COM) and self-sacrifice (SS) affect the perceived effectiveness of individual-level Organizational Citizenship Behavior (OCB). Commitment to the Public Interest (CPI) and Attraction to Public Policy making (APP) affect perceived effectiveness of OCB at the group and organisational levels, respectively. The results support the expected contribution of OCB, from the individual to the group levels, and from the group level to the organisational level. Public utility managers should strive to improve employee attitudes and motivation towards greater levels of OCB.

NOFO Oil spill response “The way we train”

2017 ◽  
Vol 2017 (1) ◽  
pp. 431-446
Keyword(s):  
Oil Spill ◽  
Large Scale ◽  
Performance Requirement ◽  
Incident Command ◽  
Large Scale Field ◽  
Oil Spill Response ◽  
Cost Efficient ◽  
The Individual ◽  
Different Response ◽  
Norwegian Continental Shelf

ABSTRACT In a situation where oil is spilled on the Norwegian Continental Shelf (NCS) the operator is responsible for the oil spill response. To do this in a robust and efficient way Norwegian Clean Seas Association for Operating Companies (NOFO) handles the oil spill response on behalf of all member companies. Handling an oil spill response situation in all its forms from offshore incident to beach restoration involves many different resources, skills and people. Introducing Incident Command System (ICS) as the command system for this task even increases the amount of training we need to do. How can NOFO achieve the optimal training of our common and shared response resources in a time where focus is on an effective and robust response? Having an overview of the different response needs and response plans NOFO coordinates activity, training and exercises in an efficient way. This is done with the aid of NOFO’s operative plan. The plan describes every resource with a performance requirement and puts it in to a response context. This gives NOFO a foundation to build a response that is structured and cost efficient for our members. Furthermore, this enables NOFO to tailor our training and exercises from the individual responder/resource to the complex large-scale field exercise which involves typically 250–350 people from numerous different operating companies, municipalities, governmental and private responders. This paper will describe how we plan, train and exercise on the NCS in order to be prepared for response in an efficient and robust way.

scholarly journals Reflections on Analytical Choices in the Scaling Model for Test Scores in International Large-Scale Assessment Studies

2021 ◽  
Author(s):  
Alexander Robitzsch ◽  
Oliver Lüdtke
Keyword(s):  
Large Scale ◽  
International Student ◽  
Student Assessment ◽  
Individual Student ◽  
Scaling Model ◽  
Large Scale Assessment ◽  
International Student Assessment ◽  
Wide Range ◽  
Analytical Strategies ◽  
The Individual

International large-scale assessments (LSAs) such as the Programme for International Student Assessment (PISA) provide important information about the distribution of student proficiencies across a wide range of countries. The repeated assessments of these content domains offer policymakers important information for evaluating educational reforms and received considerable attention from the media. Furthermore, the analytical strategies employed in LSAs often define methodological standards for applied researchers in the field. Hence, it is vital to critically reflect the conceptual foundations of analytical choices in LSA studies. This article discusses methodological challenges in selecting and specifying the scaling model used to obtain proficiency estimates from the individual student responses in LSA studies. We distinguish design-based inference from model-based inference. It is argued that for the official reporting of LSA results, design-based inference should be preferred because it allows for a clear definition of the target of inference (e.g., country mean achievement) and is less sensitive to specific modeling assumptions. More specifically, we discuss five analytical choices in the specification of the scaling model: (1) Specification of the functional form of item response functions, (2) the treatment of local dependencies and multidimensionality, (3) the consideration of test-taking behavior for estimating student ability, and the role of country differential items functioning (DIF) for (4) cross-country comparisons, and (5) trend estimation. This article's primary goal is to stimulate discussion about recently implemented changes and suggested refinements of the scaling models in LSA studies.

scholarly journals A difference that matters: comparisons of structured and semi-structured psychiatric diagnostic interviews in the general population

1999 ◽  
Vol 29 (5) ◽  
pp. 1013-1020 ◽  
Author(s):  
T. S. BRUGHA ◽  
P. E. BEBBINGTON ◽  
R. JENKINS
Keyword(s):  
General Population ◽  
Psychiatric Disorders ◽  
Large Scale ◽  
Population Level ◽  
Individual Level ◽  
Case Identification ◽  
Diagnostic Interviews ◽  
Health Project ◽  
The Individual ◽  
General Populations

Psychiatric case-identification in general populations allows us to study both individuals with functional psychiatric disorders and the populations from which they come. The individual level of analysis permits disorders to be related to factors of potential aetiological significance and the study of attributes of the disorders that need to be assessed in non-referred populations (an initially scientific endeavour). At the population level valid case identification can be used to evaluate needs for treatment and the utilization of service resources (a public health project). Thus, prevalence is of interest both to scientists and to those responsible for commissioning and planning services (Brugha et al. 1997; Regier et al. 1998). The quality of case identification techniques and of estimates of prevalence is thus of general concern (Bartlett & Coles, 1998).Structured diagnostic interviews were introduced into general population surveys in the 1970s as a method ‘to enable interviewers to obtain psychiatric diagnoses comparable to those a psychiatrist would obtain’ (Robins et al. 1981). The need to develop reliable standardized measures was partly driven by an earlier generation of prevalence surveys showing rates ranging widely from 10·9% (Pasamanick et al. 1956) to 55% (Leighton et al. 1963) in urban and rural North American communities respectively. If the success of large scale psychiatric epidemiological enquiries using structured diagnostic interviews and standardized classifications is measured in terms of citation rates it would seem difficult to question. But the development of standardized interviews of functional psychiatric disorders has not solved this problem of variability: the current generation of large scale surveys, using structured diagnostic interviews and serving strictly defined classification rules, have generated, for example, 12-month prevalence rates of major depression in the US of 4·2% (Robins & Regier, 1991) and 10·1% (Kessler et al. 1994). This calls into question the validity of the assessments, such that we must reopen the question of what they should be measuring and how they should do it.

Incidents in high-volume elongated crowd facilities: A simulation-based study

SIMULATION ◽  
2018 ◽  
Vol 95 (9) ◽  
pp. 823-843
Author(s):  
Ahmed Abdelghany ◽  
Hani Mahmassani ◽  
Khaled Abdelghany ◽  
Hasan Al-Ahmadi ◽  
Wael Alhalabi
Keyword(s):  
Large Scale ◽  
Real Life ◽  
Linear Structure ◽  
High Volume ◽  
Heterogeneous Traffic ◽  
Individual Level ◽  
Stop And Go ◽  
Simulation Based ◽  
The Individual ◽  
Inefficient Operation

This paper presents the main findings of a simulation-based study to evaluate incidents in pedestrian/crowd tunnels and similar elongated confined facilities, with high-volume heterogeneous traffic. These incidents, when occur, imposes hazardous conditions that always result in significant number of fatalities. The aim of this study is to understand how these facilities perform under different irregular scenarios and possibly identify potential causes of accidents. The problem of studying incidents in large-scale high-volume pedestrian facilities is that these incidents are difficult to expect or replicate. Thus, studying these facilities through real-life scenarios is almost impossible. Accordingly, a micro-simulation assignment model for multidirectional pedestrian movement is used for this purpose. The model adopts a Cellular Automata (CA) discrete system, which allows detailed representation of the pedestrians’ walkways in the tunnel. The modeling approach captures crowd dynamics through representation of behavioral decisions of heterogeneous pedestrians at the individual level. Several experiments are conducted to study the pedestrian flow in the proposed tunnel considering different operational scenarios including demand levels, heterogeneous traffic, evacuation scenario, and tunnel blockage. Results show that flow of large pedestrian volumes through a long confined linear structure, such as a tunnel, are subject to the same flow dynamics as we observe with vehicular traffic. In particular, they are subject to the formation of “clumps” and shock waves that can rapidly propagate and lead to inefficient operation, including flow breakdown with stop-and-go waves.

Sign in / Sign up

Export Citation Format

Share Document