scholarly journals Validation of Cognitive Load During Inquiry-Based Learning With Multimedia Scaffolds Using Subjective Measurement and Eye Movements

2021 ◽  
Vol 12 ◽  
Author(s):  
Marit Kastaun ◽  
Monique Meier ◽  
Stefan Küchemann ◽  
Jochen Kuhn
Keyword(s):  
Eye Tracking ◽  
Cognitive Load ◽  
Test Scores ◽  
Scientific Inquiry ◽  
Reading Skills ◽  
Measurement Instrument ◽  
Theoretical Framework ◽  
Causal Factors ◽  
Verbal Abilities ◽  
Subjective Measurement

Subject-method barriers and cognitive load (CL) of students have a particular importance in the complex learning process of scientific inquiry. In this work, we investigate the valid measurement of CL as well as different scaffolds to reduce it during experimentation. Specifically, we examine the validity of a subjective measurement instrument to assess CL [in extraneous cognitive load (ECL), intrinsic cognitive load, and germane cognitive load (GCL)] during the use of multimedia scaffolds in the planning phase of the scientific inquiry process based on a theoretical framework of the CL theory. The validity is analyzed by investigating possible relationships between causal (e.g., cognitive abilities) and assessment (e.g., eye-tracking metrics) factors in relation to the obtained test scores of the adapted subjective measurement instrument. The study aims to elucidate possible relationships of causal factors that have not yet been adequately investigated in relation to CL. Furthermore, a possible, still inconclusive convergence between subjective test scores on CL and objectively measured indicators will be tested using different eye-tracking metrics. In two studies (n=250), 9th and 11th grade students experimentally investigated a biological phenomenon. At the beginning of the planning phase, students selected one of four multimedia scaffolds using a tablet (Study I: n=181) or a computer with a stationary eye-tracking device (Study II: n=69). The subjective cognitive load was measured via self-reports using a standardized questionnaire. Additionally, we recorded students’ gaze data during learning with the scaffolds as objective measurements. Besides the causal factors of cognitive-visual and verbal abilities, reading skills and spatial abilities were quantified using established test instruments and the learners indicated their representation preference by selecting the scaffolds. The results show that CL decreases substantially with higher grade level. Regarding the causal factors, we observed that cognitive-visual and verbal abilities have a significant influence on the ECL and GCL in contrast to reading skills. Additionally, there is a correlation between the representation preference and different types of CL. Concerning the objective measurement data, we found that the absolute fixation number is predictive for the ECL. The results are discussed in the context of the overall methodological research goal and the theoretical framework of CL.

scholarly journals What can gaze behaviors, neuroimaging data, and test scores tell us about test method effects and cognitive load in listening assessments?

2021 ◽  
pp. 026553222110268
Author(s):  
Vahid Aryadoust ◽  
Stacy Foo ◽  
Li Ying Ng
Keyword(s):  
Eye Tracking ◽  
Cognitive Load ◽  
Test Scores ◽  
Test Validity ◽  
Test Methods ◽  
Load Test ◽  
Test Method ◽  
Functional Near Infrared Spectroscopy ◽  
Listening Test ◽  
The Impact

The aim of this study was to investigate how test methods affect listening test takers’ performance and cognitive load. Test methods were defined and operationalized as while-listening performance (WLP) and post-listening performance (PLP) formats. To achieve the goal of the study, we examined test takers’ ( N = 80) brain activity patterns (measured by functional near-infrared spectroscopy (fNIRS)), gaze behaviors (measured by eye-tracking), and listening performance (measured by test scores) across the two test methods. We found that the test takers displayed lower activity levels across brain regions supporting comprehension during the WLP tests relative to the PLP tests. Additionally, the gaze behavioral patterns exhibited during the WLP tests suggested that the test takers adopted keyword matching and “shallow listening.” Together, the neuroimaging and gaze behavioral data indicated that the WLP tests imposed a lower cognitive load on the test takers than the PLP tests. However, the test takers performed better with higher test scores for one of two WLP tests compared with the PLP tests. By incorporating eye-tracking and neuroimaging in this exploration, this study has advanced the current knowledge on cognitive load and the impact imposed by different listening test methods. To advance our knowledge of test validity, other researchers could adopt our research protocol and focus on extending the test method framework used in this study.

scholarly journals Analyzing Relationships Between Causal and Assessment Factors of Cognitive Load: Associations Between Objective and Subjective Measures of Cognitive Load, Stress, Interest, and Self-Concept

2021 ◽  
Vol 6 ◽  
Author(s):  
Nina Minkley ◽  
Kate M. Xu ◽  
Moritz Krell
Keyword(s):  
Heart Rate ◽  
Molecular Biology ◽  
Cognitive Load ◽  
Task Performance ◽  
Perceived Stress ◽  
Mental Effort ◽  
Theoretical Framework ◽  
Self Concept ◽  
Mental Load ◽  
Causal Factors

The present study is based on a theoretical framework of cognitive load that distinguishes causal factors (learner characteristics affecting cognitive load e.g., self-concept; interest; perceived stress) and assessment factors (indicators of cognitive load e.g., mental load; mental effort; task performance) of cognitive load. Various assessment approaches have been used in empirical research to measure cognitive load during task performance. The most common methods are subjective self-reported questionnaires; only occasionally objective physiological measures such as heart rates are used. However, the convergence of subjective and objective approaches has not been extensively investigated yet, leaving unclear the meaning of each kind of measure and its validity. This study adds to this body of research by analyzing the relationship between these causal and assessment (subjective and objective) factors of cognitive load. The data come from three comparable studies in which high school students (N = 309) participated in a one-day out of school molecular biology project and completed different tasks about molecular biology structures and procedures. Heart rate variability (objective cognitive load) was measured via a chest belt. Subjective cognitive load (i.e., mental load and mental effort) and causal factors including self-concept, interest, and perceived stress were self-reported by participants on questionnaires. The findings show that a) objective heart rate measures of cognitive load are related to subjective measures of self-reported mental effort but not of mental load; b) self-reported mental effort and mental load are better predictors of task performance than objective heart rate measures of cognitive load; c) self-concept, interest and perceived stress are associated with self-reported measures of mental load and mental effort, and self-concept is associated with one of the objective heart rate measures. The findings are discussed based on the theoretical framework of cognitive load and implications for the validity of each measure are proposed.

scholarly journals The impact of translation modality on user experience: an eye-tracking study of the Microsoft Word user interface

2021 ◽  
Author(s):  
Ana Guerberof Arenas ◽  
Joss Moorkens ◽  
Sharon O’Brien
Keyword(s):  
User Interface ◽  
Eye Tracking ◽  
Cognitive Load ◽  
Task Completion ◽  
Eye Tracker ◽  
Software Products ◽  
Microsoft Word ◽  
Set Up ◽  
The Impact ◽  
The Given

AbstractThis paper presents results of the effect of different translation modalities on users when working with the Microsoft Word user interface. An experimental study was set up with 84 Japanese, German, Spanish, and English native speakers working with Microsoft Word in three modalities: the published translated version, a machine translated (MT) version (with unedited MT strings incorporated into the MS Word interface) and the published English version. An eye-tracker measured the cognitive load and usability according to the ISO/TR 16982 guidelines: i.e., effectiveness, efficiency, and satisfaction followed by retrospective think-aloud protocol. The results show that the users’ effectiveness (number of tasks completed) does not significantly differ due to the translation modality. However, their efficiency (time for task completion) and self-reported satisfaction are significantly higher when working with the released product as opposed to the unedited MT version, especially when participants are less experienced. The eye-tracking results show that users experience a higher cognitive load when working with MT and with the human-translated versions as opposed to the English original. The results suggest that language and translation modality play a significant role in the usability of software products whether users complete the given tasks or not and even if they are unaware that MT was used to translate the interface.

An eye-tracking study on the effect of infographic structures on viewer’s comprehension and cognitive load

2017 ◽  
Vol 17 (3) ◽  
pp. 257-266 ◽  
Author(s):  
Azam Majooni ◽  
Mona Masood ◽  
Amir Akhavan
Keyword(s):  
Eye Tracking ◽  
Cognitive Load ◽  
Pupil Diameter ◽  
Information Graphics ◽  
Basic Premise ◽  
Quantitative Evidence ◽  
The Gaze ◽  
Movement Data ◽  
Graphical Design ◽  
Tracking Devices

The basic premise of this research is investigating the effect of layout on the comprehension and cognitive load of the viewers in the information graphics. The term ‘Layout’ refers to the arrangement and organization of the visual and textual elements in a graphical design. The experiment conducted in this study is designed based on two stories and each one of these stories is presented with two different layouts. During the experiment, eye-tracking devices are applied to collect the gaze data including the eye movement data and pupil diameter fluctuation. In the research on the modification of the layouts, contents of each story are narrated using identical visual and textual elements. The analysis of eye-tracking data provides quantitative evidence concerning the change of layout in each story and its effect on the comprehension of participants and variation of their cognitive load. In conclusion, it can be claimed that the comprehension from the zigzag form of the layout was higher with a less imposed cognitive load.

Reconstructing Validity

2007 ◽  
Vol 36 (8) ◽  
pp. 470-476 ◽  
Author(s):  
Pamela A. Moss
Keyword(s):  
Construct Validity ◽  
Test Scores ◽  
Scientific Inquiry ◽  
Educational Measurement ◽  
Validity Theory ◽  
Validity Research ◽  
Unitary Concept

In response to Lissitz and Samuelsen (2007) , the author reconstructs the historical arguments for the more comprehensive unitary concept of validity and the principles of scientific inquiry underlying it. Her response is organized in terms of four questions: (a) How did validity in educational measurement come to be conceptualized as unitary, and why? (b) What is construct validity, and how does it provide the basis for a unitary concept of validity? (c) Why has the focus of validity been on the interpretations and uses of test scores rather than on the test itself? and (d) What sort of guidance for test developers and evaluators has been provided within a unitary concept of validity, and how might it be enhanced? The author highlights the role that cases of programmatic validity research can play in representing validity theory and guiding validity inquiry.

The Eyes Chico, They Never Lie – Using Eye-Tracking to Examine Faking on the Conditional Reasoning Test for Aggression

2021 ◽  
Vol 20 (2) ◽  
pp. 84-96
Author(s):  
Mitja Ružojčić ◽  
Zvonimir Galić ◽  
Antun Palanović ◽  
Maja Parmač Kovačić ◽  
Andreja Bubić
Keyword(s):  
Eye Movements ◽  
Eye Tracking ◽  
Personnel Selection ◽  
Test Scores ◽  
Conditional Reasoning ◽  
Reasoning Test ◽  
Test Study ◽  
True Purpose ◽  
Response Alternatives

Abstract. To better understand the process of responding to the Conditional Reasoning Test for Aggression (CRT-A) and its implication for the test's use in personnel selection, we conducted two lab studies in which we compared test scores and eye movements of participants responding honestly and faking the test. Study 1 results showed that, although participants might try to respond differently to the CRT-A while faking, it remains an indirect and unfakeable measure as long as the test's purpose is undisclosed. Study 2 showed that revealing the true purpose of the CRT-A diminishes the test's indirect nature so the test becomes fakeable, solving it requires less attention and participants direct their eyes more to response alternatives congruent with the presentational demands.

Measuring Cognitive Load

2007 ◽  
pp. 188-194
Author(s):  
N. Hogg
Keyword(s):  
Working Memory ◽  
Cognitive Load ◽  
Rating Scale ◽  
Processing System ◽  
Reliability And Validity ◽  
Measurement Instrument ◽  
Subjective Rating ◽  
Subjective Rating Scale ◽  
Very High ◽  
The Web

Cognitive load theory describes learning in terms of a processing system when all too often working memory is overloaded and learning is impeded. Measuring cognitive load is an important component of research in the area of information processing. The design, delivery, and administration of an instrument, as well as its reliability and validity are discussed as a solution to the measurement of cognitive load. A nine-point subjective rating scale ranging from very, very low (1) to very, very high (9) measures cognitive load, defined as the mental effort needed to complete a task. It is a replica of the instrument used by Paas (1992) and Paas and van Merriënboer (1994). The measurement instrument can be used both on paper and on the Web.

scholarly journals A cognitive load delays predictive eye movements similarly during L1 and L2 comprehension

2017 ◽  
Vol 21 (2) ◽  
pp. 251-264 ◽  
Author(s):  
AINE ITO ◽  
MARTIN CORLEY ◽  
MARTIN J. PICKERING
Keyword(s):  
Working Memory ◽  
Eye Movements ◽  
Eye Tracking ◽  
Cognitive Load ◽  
Memory Task ◽  
Target Object ◽  
Cognitive Resources ◽  
Visual World ◽  
L1 And L2

We used the visual world eye-tracking paradigm to investigate the effects of cognitive load on predictive eye movements in L1 (Experiment 1) and L2 (Experiment 2) speakers. Participants listened to sentences whose verb was predictive or non-predictive towards one of four objects they were viewing. They then clicked on a mentioned object. Half the participants additionally performed a working memory task of remembering words. Both L1 and L2 speakers looked more at the target object predictively in predictable- than in non-predictable sentences when they performed the listen-and-click task only. However, this predictability effect was delayed in those who performed the concurrent memory task. This pattern of results was similar in L1 and L2 speakers. L1 and L2 speakers make predictions, but cognitive resources are required for making predictive eye movements. The findings are compatible with the claim that L2 speakers use the same mechanisms as L1 speakers to make predictions.

The Expertise Reversal Effect for Sequential Presentation in Dynamic Soccer Visualizations

2013 ◽  
Vol 35 (3) ◽  
pp. 260-269 ◽  
Author(s):  
Aïmen Khacharem ◽  
Bachir Zoudji ◽  
Slava Kalyuga ◽  
Hubert Ripoll
Keyword(s):  
Cognitive Load ◽  
Learning Outcomes ◽  
Mental Effort ◽  
Visual Presentation ◽  
Theoretical Framework ◽  
Soccer Players ◽  
Sequential Presentation ◽  
Expertise Reversal Effect ◽  
Reversal Effect ◽  
Presentation Formats

Cognitive load perspective was used as a theoretical framework to investigate effects of expertise and type of presentation of interacting elements of information in learning from dynamic visualizations. Soccer players (N = 48) were required to complete a recall reconstruction test and to rate their invested mental effort after studying a concurrent or sequential presentation of the elements of play. The results provided evidence for an expertise reversal effect. For novice players, the sequential presentation produced better learning outcomes. In contrast, expert players performed better after studying the concurrent presentation. The findings suggest that the effectiveness of different visual presentation formats depend on levels of learner expertise.

Sign in / Sign up

Export Citation Format

Share Document