How Representational Pictures Enhance Students’ Performance and Test-Taking Pleasure in Low-Stakes Assessment

2018 ◽  
Vol 34 (6) ◽  
pp. 376-385 ◽  
Author(s):  
Marlit A. Lindner ◽  
Jan M. Ihme ◽  
Steffani Saß ◽  
Olaf Köller
Keyword(s):  
Cognitive Abilities ◽  
Large Scale ◽  
Item Difficulty ◽  
Large Scale Assessment ◽  
Test Taking ◽  
Irt Model ◽  
Reading Abilities ◽  
Test Parameters ◽  
Potential Interactions ◽  
Representational Pictures

Abstract. Pictures are often used in standardized educational large-scale assessment (LSA), but their impact on test parameters has received little attention up until now. Even less is known about pictures’ affective effects on students in testing (i.e., test-taking pleasure and motivation). However, such knowledge is crucial for a focused application of multiple representations in LSA. Therefore, this study investigated how adding representational pictures (RPs) to text-based item stems affects (1) item difficulty and (2) students’ test-taking pleasure. An experimental study with N = 305 schoolchildren was conducted, using 48 manipulated parallel science items (text-only vs. text-picture) in a rotated multimatrix design to realize within-subject measures. Students’ general cognitive abilities, reading abilities, and background variables were assessed to consider potential interactions between RPs’ effects and students’ performance. Students also rated their item-solving pleasure for each item. Results from item-response theory (IRT) model comparisons showed that RPs only reduced item difficulty when pictures visualized information mandatory for solving the task, while RPs substantially enhanced students’ test-taking pleasure even when they visualized optional context information. Overall, our findings suggest that RPs have a positive cognitive and affective influence on students’ performance in LSA (i.e., multimedia effect in testing) and should be considered more frequently.

Representation of Competencies in Multidimensional IRT Models with Within-Item and Between-Item Multidimensionality

2008 ◽  
Vol 216 (2) ◽  
pp. 89-101 ◽  
Author(s):  
Johannes Hartig ◽  
Jana Höhler
Keyword(s):  
Cognitive Abilities ◽  
Listening Comprehension ◽  
Large Scale ◽  
Factor Model ◽  
Model Performance ◽  
Multidimensional Item Response ◽  
Large Scale Assessment ◽  
Irt Model ◽  
Irt Models ◽  
Psychometric Models

Multidimensional item response theory (MIRT) holds considerable promise for the development of psychometric models of competence. It provides an ideal foundation for modeling performance in complex domains, simultaneously taking into account multiple basic abilities. The aim of this paper is to illustrate the relations between a two-dimensional IRT model with between-item multidimensionality and a nested-factor model with within-item multidimensionality, and the different substantive meanings of the ability dimensions in the two models. Both models are applied to empirical data from a large-scale assessment of reading and listening comprehension in a foreign language. In the between-item model, performance in the reading and listening items is modeled by two separate dimensions. In the within-item model, one dimension represents the abilities common to both tests, and a second dimension represents abilities specific to listening comprehension. Distinct relations of external variables, such as gender and cognitive abilities, with ability scores demonstrate that the alternative models have substantively different implications.

On the Impact of the Response Options’ Position on Item Difficulty in Multiple-Choice-Items

2020 ◽  
pp. 1-10
Author(s):  
Bettina Hagenmüller
Keyword(s):  
Large Scale ◽  
Item Difficulty ◽  
Multiple Choice ◽  
Test Construction ◽  
Test Model ◽  
Group Setting ◽  
Response Options ◽  
Large Scale Assessment ◽  
Multiple Choice Item ◽  
The Impact

Abstract. The multiple-choice item format is widely used in test construction and Large-Scale Assessment. So far, there has been little research on the impact of the position of the solution among the response options and the few existing results are even inconsistent. Since it would be an easy way to create parallel items for group setting by altering the response options, the influence of the response options’ position on item difficulty should be examined. The Linear Logistic Test Model ( Fischer, 1972 ) was used to analyze the data of 829 students aged 8–20 years, who worked on general knowledge items. It was found that the position of the solution among the response options has an influence on item difficulty. Items are easiest when the solution is in first place and more difficult when the solution is placed in a middle position or at the end of the set of response options.

scholarly journals The Need for Self-Control in Achievement Tests: Changes in Students’ State Self-Control Capacity and Effort Investment

2018 ◽  
Author(s):  
Christoph Lindner ◽  
Gabriel Nagy ◽  
Jan Retelsdorf
Keyword(s):  
Process Model ◽  
Large Scale ◽  
Achievement Test ◽  
Self Control ◽  
Large Scale Assessment ◽  
Test Taking ◽  
Alternative Approach ◽  
Major Predictor ◽  
Underlying Mechanisms ◽  
Mathematics And Science

In the present research, we investigated the relation between changes in students’ state self-control capacity and their motivational test-taking effort over the course of an achievement test. Thereby, we considered trait self-control as a major predictor of achievement-related behavior as a covariate. N = 1,840 apprentices repeatedly rated their state self-control capacity and the test-taking effort they invested while working on a 140-minute achievement test in mathematics and science. Using growth curve analyses, our results revealed correlated decreases in state self-control capacity and the test-taking effort invested over the course of the test. Furthermore, we found that trait self-control helped to keep state self-control capacity and test-taking effort at a higher level over the course of the test. Our results support the assumption of the process model of self-control that waning self-control capacity is reflected in reduced motivational effort. Furthermore, our findings provide evidence that self-control might play a crucial role in students’ test-taking behavior in large-scale assessment studies. By modeling changes in state self-control capacity and effort investment while considering trait self-control, we provide an alternative approach for investigating self-control-dependent processes and the underlying mechanisms of self-control in achievement situations.

scholarly journals Reading Comprehension in Problems Solving in Mathematics Test of Prova Brasil

2019 ◽  
Vol 22 (1) ◽  
pp. 88
Author(s):  
Alessandra Pereira Gomes Machado ◽  
Andréa Maria dos Santos Matos
Keyword(s):  
Reading Comprehension ◽  
Problem Solving ◽  
Large Scale ◽  
Standardized Test ◽  
Mathematical Problem ◽  
Large Scale Assessment ◽  
Reading Abilities ◽  
Mathematics Test ◽  
Comprehension Skills ◽  
Mathematical Problems

Difficulty in reading comprehension and problem solving presented in the results of large- scale assessment may be in basic cognitive processes of reading. Researches suggest the relation between reading comprehension skills and mathematical problem solving. The aim of this article is to present the articulation between reading and problem solving, considering the cognitive dimension, and to accentuate the importance of automaticity and reading comprehension for solving problems in the Mathematics test of Prova Brasil. We analysed the descriptors of three topics of the reference matrix of Portuguese Language that indicate the reading abilities in items of Prova Brasil of a standardized test, elaborated in order to emphasize the need for the student to have developed the processes of reading comprehension to be successful in solving mathematical problems.

An Investigation of Examinee Test-Taking Effort on a Large-Scale Assessment

2013 ◽  
Vol 26 (1) ◽  
pp. 34-49 ◽  
Author(s):  
J. Carl Setzer ◽  
Steven L. Wise ◽  
Jill R. van den Heuvel ◽  
Guangming Ling
Keyword(s):  
Large Scale ◽  
Large Scale Assessment ◽  
Test Taking ◽  
Scale Assessment

Pitfalls and Challenges in Constructing Short Forms of Cognitive Ability Measures

2014 ◽  
Vol 35 (4) ◽  
pp. 190-200 ◽  
Author(s):  
Stefan Schipolowski ◽  
Ulrich Schroeders ◽  
Oliver Wilhelm
Keyword(s):  
Cognitive Abilities ◽  
Large Scale ◽  
Measurement Model ◽  
Measurement Precision ◽  
Individual Level ◽  
Crystallized Intelligence ◽  
Large Scale Assessment ◽  
Short Forms ◽  
Cost Efficient ◽  
The Individual

Especially in survey research and large-scale assessment there is a growing interest in short scales for the cost-efficient measurement of psychological constructs. However, only relatively few standardized short forms are available for the measurement of cognitive abilities. In this article we point out pitfalls and challenges typically encountered in the construction of cognitive short forms. First we discuss item selection strategies, the analysis of binary response data, the problem of floor and ceiling effects, and issues related to measurement precision and validity. We subsequently illustrate these challenges and how to deal with them based on an empirical example, the development of short forms for the measurement of crystallized intelligence. Scale shortening had only small effects on associations with covariates. Even for an ultra-short six-item scale, a unidimensional measurement model showed excellent fit and yielded acceptable reliability. However, measurement precision on the individual level was very low and the short forms were more likely to produce skewed score distributions in ability-restricted subpopulations. We conclude that short scales may serve as proxies for cognitive abilities in typical research settings, but their use for decisions on the individual level should be discouraged in most cases.

scholarly journals Item Position Effects Are Moderated by Changes in Test-Taking Effort

2016 ◽  
Vol 41 (2) ◽  
pp. 115-129 ◽  
Author(s):  
Sebastian Weirich ◽  
Martin Hecht ◽  
Christiane Penk ◽  
Alexander Roppelt ◽  
Katrin Böhme
Keyword(s):  
Large Scale ◽  
Context Effects ◽  
Item Difficulty ◽  
Position Effect ◽  
Reliability And Validity ◽  
Position Effects ◽  
Test Taking ◽  
Current Test ◽  
Item Position ◽  
Large Scale Assessments

This article examines the interdependency of two context effects that are known to occur regularly in large-scale assessments: item position effects and effects of test-taking effort on the probability of correctly answering an item. A microlongitudinal design was used to measure test-taking effort over the course of a large-scale assessment of 60 min. Two components of test-taking effort were investigated: initial effort and change in effort. Both components of test-taking effort significantly affected the probability to solve an item. In addition, it was found that participants’ current test-taking effort diminished considerably across the course of the test. Furthermore, a substantial linear position effect was found, which indicated that item difficulty increased during the test. This position effect varied considerably across persons. Concerning the interplay of position effects and test-taking effort, it was found that only the change in effort moderates the position effect and that persons differ with respect to this moderation effect. The consequences of these results concerning the reliability and validity of large-scale assessments are discussed.

Using Web-based Testing for Large-Scale Assessment

2013 ◽  
Author(s):  
Laura S. Hamilton ◽  
Stephen P. Klein ◽  
William Lorie
Keyword(s):  
Large Scale ◽  
Web Based ◽  
Large Scale Assessment ◽  
Scale Assessment

Large-Scale Assessment of Binding Free Energy Calculations in Active Drug Discovery Projects

2020 ◽  
Author(s):  
Christina Schindler ◽  
Hannah Baumann ◽  
Andreas Blum ◽  
Dietrich Böse ◽  
Hans-Peter Buchstaller ◽  
...  
Keyword(s):  
Free Energy ◽  
Drug Discovery ◽  
Large Scale ◽  
Active Drug ◽  
Binding Free Energy ◽  
Free Energy Calculations ◽  
Energy Calculation ◽  
Large Scale Assessment ◽  
New Public ◽  
Binding Free Energy Calculations

Here we present an evaluation of the binding affinity prediction accuracy of the free energy calculation method FEP+ on internal active drug discovery projects and on a large new public benchmark set.<br>

Sign in / Sign up

Export Citation Format

Share Document